Postulo/postulo

Fork 0

Capturing the same advert twice makes two captures and two listings, silently #178

New issue

Closed

opened 2026-09-12 08:50:23 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-09-12 08:50:23 +00:00

Owner

Observation

Nothing anywhere checks whether a posting has been captured before. Capture the same advert
on Monday and again on Thursday — or press the button twice — and you get two captures, and
after reviewing both, two listings for one job.

What exists

The extension has half of it, locally:

/** A capture of this address that has not reached Postulo, if there is one. */
async unsentFor(url) { ... record.url === url && UNSENT.has(record.state) ... }

That is exact string equality, in one browser, and only for captures not yet sent. Once a
capture reaches Postulo the extension forgets it, so the second capture of the same advert is
the ordinary case rather than the awkward one.

The server has none of it. Capture.Meta carries:

indexes = [models.Index(fields=("owner", "status"))]

and no constraint. JobPosting.url has none either — the only unique constraints in
jobs/models.py are on industry slugs, company names, company identifiers and department
names. Two listings for one advert is a state the schema permits and nothing reports.

Not a unique constraint

A constraint would refuse the second capture, and refusing is the wrong behaviour:

The same address genuinely carries a different advert over time. Boards reuse paths.
Somebody may deliberately re-capture an advert that has been edited, to see what changed.
A refusal at the database is a 500 or an ugly form error, arriving after the work, rather
than an answer to a question somebody would have asked first.

It should tell, not refuse. "You captured this on 3 September; it is in your listings as
Research Engineer" — with a link — and let the person decide.

What matching on

url normalised, not compared raw. _same_url() in plugins/builtin/__init__.py already
strips the scheme, a leading www. and a trailing slash and lowercases the rest; it was
written for #176 and is the right shape to reuse rather than write a second one.

Worth deciding, and not obvious: a board that mints a fresh address per visit — tracking
parameters, a session id in the path — defeats URL matching entirely, and LinkedIn's own
og:url for a job differs from the address you were on. A second match on
(company, title) catches those and will also produce false positives for a company running
two genuinely different openings with the same title. Probably: match on URL, and warn on
(company, title) more softly.

Roughly, the work

A cheap way to ask, for the extension and for the web form. It has to work before the
capture is made, so the person is told rather than shown a duplicate afterwards.
The extension asks it when the popup opens, and says so in the popup rather than after
sending. It must degrade quietly: offline, or an older Postulo, means no answer and the
capture proceeds exactly as today.
The review screen says it too, since a capture can be reviewed long after it arrived.
store.unsentFor stays for what it is good at — the capture you made a minute ago and have
not sent — and stops being the only check there is.

This gets more valuable, not less, once a results page can produce forty captures at once:
the second visit to a board's search page is where duplicates arrive in bulk.

## Observation Nothing anywhere checks whether a posting has been captured before. Capture the same advert on Monday and again on Thursday — or press the button twice — and you get two captures, and after reviewing both, two listings for one job. ## What exists The extension has half of it, locally: /** A capture of this address that has not reached Postulo, if there is one. */ async unsentFor(url) { ... record.url === url && UNSENT.has(record.state) ... } That is exact string equality, in one browser, and only for captures **not yet sent**. Once a capture reaches Postulo the extension forgets it, so the second capture of the same advert is the ordinary case rather than the awkward one. The server has none of it. `Capture.Meta` carries: indexes = [models.Index(fields=("owner", "status"))] and no constraint. `JobPosting.url` has none either — the only unique constraints in `jobs/models.py` are on industry slugs, company names, company identifiers and department names. Two listings for one advert is a state the schema permits and nothing reports. ## Not a unique constraint A constraint would refuse the second capture, and refusing is the wrong behaviour: - The same address genuinely carries a different advert over time. Boards reuse paths. - Somebody may deliberately re-capture an advert that has been edited, to see what changed. - A refusal at the database is a 500 or an ugly form error, arriving after the work, rather than an answer to a question somebody would have asked first. It should **tell**, not refuse. "You captured this on 3 September; it is in your listings as *Research Engineer*" — with a link — and let the person decide. ## What matching on `url` normalised, not compared raw. `_same_url()` in `plugins/builtin/__init__.py` already strips the scheme, a leading `www.` and a trailing slash and lowercases the rest; it was written for #176 and is the right shape to reuse rather than write a second one. Worth deciding, and not obvious: a board that mints a fresh address per visit — tracking parameters, a session id in the path — defeats URL matching entirely, and LinkedIn's own `og:url` for a job differs from the address you were on. A second match on `(company, title)` catches those and will also produce false positives for a company running two genuinely different openings with the same title. Probably: match on URL, and warn on `(company, title)` more softly. ## Roughly, the work - A cheap way to ask, for the extension and for the web form. It has to work **before** the capture is made, so the person is told rather than shown a duplicate afterwards. - The extension asks it when the popup opens, and says so in the popup rather than after sending. It must degrade quietly: offline, or an older Postulo, means no answer and the capture proceeds exactly as today. - The review screen says it too, since a capture can be reviewed long after it arrived. - `store.unsentFor` stays for what it is good at — the capture you made a minute ago and have not sent — and stops being the only check there is. This gets more valuable, not less, once a results page can produce forty captures at once: the second visit to a board's search page is where duplicates arrive in bulk.

tiagoagueda added the

bug

enhancement

labels

2026-09-12 08:50:24 +00:00

tiagoagueda referenced this issue

2026-09-12 09:07:42 +00:00

A candidate's own record cannot be taken out on its own, and nothing can be put back #181

tiagoagueda added the

tier/4

label

2026-09-12 10:02:05 +00:00

tiagoagueda referenced this issue from a commit

2026-09-12 19:45:36 +00:00

Say when an advert has been captured before, and refuse nothing

tiagoagueda closed this issue

2026-09-12 19:45:36 +00:00

tiagoagueda referenced this issue from a commit

2026-09-12 19:45:45 +00:00

Say "captured before" in the popup, before anything is sent

tiagoagueda commented

2026-09-12 19:46:24 +00:00

Author

Owner

Landed on three sides, told and never refused:

core 27d514c89 — jobs/known.py answers the question: a listing at this address
however it was spelled, a capture of it still waiting for review, and, more softly, a
listing with this title at this company at another address. same_url moved from the
builtin plugin to core/addresses.py so both can use it; the address is narrowed in SQL by
its path and decided in Python. The web form says this address is already in your
listings, since … with a link before anything is fetched, and the same submit again is the
answer; the review screen says it too, with the softer match; and
POST /api/v1/captures/known answers for up to a hundred postings at once.
postulo-chromium 259dcd0 — once the form is up the popup asks and says Captured
before: in your listings since … as …, or the waiting-capture or same-title variant, with
a link into Postulo; on a results page the postings already held carry a captured before
tag. It degrades to silence — not set up, the testing switch, no permission, unreachable,
an older Postulo — and never prompts for permission. store.unsentFor stays for what it is
good at.
wiki — the API page documents the call and lists it in Every call.

As the issue asked: no constraint, no refusal; a second capture is the person's to want.

Landed on three sides, told and never refused: - **core `27d514c89`** — `jobs/known.py` answers the question: a listing at this address however it was spelled, a capture of it still waiting for review, and, more softly, a listing with this title at this company at another address. `same_url` moved from the builtin plugin to `core/addresses.py` so both can use it; the address is narrowed in SQL by its path and decided in Python. The **web form** says *this address is already in your listings, since …* with a link before anything is fetched, and the same submit again is the answer; the **review screen** says it too, with the softer match; and **`POST /api/v1/captures/known`** answers for up to a hundred postings at once. - **postulo-chromium `259dcd0`** — once the form is up the popup asks and says *Captured before: in your listings since … as …*, or the waiting-capture or same-title variant, with a link into Postulo; on a results page the postings already held carry a *captured before* tag. It degrades to silence — not set up, the testing switch, no permission, unreachable, an older Postulo — and never prompts for permission. `store.unsentFor` stays for what it is good at. - **wiki** — the API page documents the call and lists it in *Every call*. As the issue asked: no constraint, no refusal; a second capture is the person's to want.

tiagoagueda referenced this issue

2026-09-17 20:01:12 +00:00

Bind a listing to what it came from and what arrives about it: a capture, an email, a document, a message #270