schema.org source: on a page with several postings, an identifier matches by the employer's name and picks the wrong one #590

New issue

Closed

opened 2026-10-01 10:48:06 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:48:06 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading how the schema.org source chooses among several postings and running it on a page that embeds three.

Checked again on 2026-10-01 against main at 0a5cad587: still present.

_matches_url reads a posting's identifier with _text, which prefers a dict's name over its value. schema.org and Google's documented JobPosting example write the identifier as {"@type": "PropertyValue", "name": "<employer>", "value": "<id>"}, so what is compared with the page's address is the employer's name. The comparison is a substring test, and _best_posting returns the first posting that matches by any rule, so an earlier posting whose employer's name appears in the host wins over a later one whose url is exactly the page.

Related to #176, which introduced choosing the posting that names the page.

Where: plugins/builtin/__init__.py:251-253 (identifier = _text(posting.get("identifier")), then identifier.lower() in here), :110-117 (_text: value.get("name") or value.get("value")) and :267-275 (_best_posting returns the first posting matching by any rule). The extension does the same: postulo-chromium src/lib/parse.js:403-411 (text), :556-570 (matchesUrl) and :581-587 (bestPosting).
What happens:
1. A person captures https://careers.acme.com/jobs/3. The page embeds JSON-LD for jobs 1, 2 and 3 (the advert and two "similar jobs"), each with its own url and identifier: {"@type": "PropertyValue", "name": "Acme", "value": "n000N"} for job N.
2. Posting 1's identifier reads as "Acme", which is in the host, so posting 1 matches first; SchemaOrgSource().parse returns "Job 1".
3. With a plain-string identifier the substring test misfires too: identifiers 1000 and 10001 on /jobs/10001 choose the posting with 1000.
  The project's own fixture tests/fixtures/postings/weworkremotely.html:42-46 has exactly this identifier shape, with the employer's name in name.
Why it matters: the capture arrives with another advert's title, location, salary and description for the page the person was reading. On similar adverts from the same employer the review screen gives little sign of it, and the wrong posting is saved.
Fix: choose in two passes: first a posting whose url, @id or sameAs equals the page; only if none does, fall back to identifiers. Read an identifier's value (or the plain string), never its name, and match it as a whole path segment or query value rather than as a substring. Mirror the change in postulo-chromium's parse.js.

Done when

A fixture in tests/fixtures/postings/ with three postings that share an employer and carry PropertyValue identifiers, read by tests/test_source_corpus.py as the posting whose url is the page; today it reads as the first.
A test in tests/test_capture.py that identifiers 1000 and 10001 on /jobs/10001 choose 10001.
The fixture passes in postulo-chromium's test/corpus.test.mjs.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading how the schema.org source chooses among several postings and running it on a page that embeds three. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. `_matches_url` reads a posting's `identifier` with `_text`, which prefers a dict's `name` over its `value`. schema.org and Google's documented JobPosting example write the identifier as `{"@type": "PropertyValue", "name": "<employer>", "value": "<id>"}`, so what is compared with the page's address is the employer's name. The comparison is a substring test, and `_best_posting` returns the first posting that matches by any rule, so an earlier posting whose employer's name appears in the host wins over a later one whose `url` is exactly the page. Related to #176, which introduced choosing the posting that names the page. - **Where:** `plugins/builtin/__init__.py:251-253` (`identifier = _text(posting.get("identifier"))`, then `identifier.lower() in here`), `:110-117` (`_text`: `value.get("name") or value.get("value")`) and `:267-275` (`_best_posting` returns the first posting matching by any rule). The extension does the same: `postulo-chromium` `src/lib/parse.js:403-411` (`text`), `:556-570` (`matchesUrl`) and `:581-587` (`bestPosting`). - **What happens:** 1. A person captures `https://careers.acme.com/jobs/3`. The page embeds JSON-LD for jobs 1, 2 and 3 (the advert and two "similar jobs"), each with its own `url` and `identifier: {"@type": "PropertyValue", "name": "Acme", "value": "n000N"}` for job N. 2. Posting 1's identifier reads as "Acme", which is in the host, so posting 1 matches first; `SchemaOrgSource().parse` returns "Job 1". 3. With a plain-string identifier the substring test misfires too: identifiers `1000` and `10001` on `/jobs/10001` choose the posting with `1000`. The project's own fixture `tests/fixtures/postings/weworkremotely.html:42-46` has exactly this identifier shape, with the employer's name in `name`. - **Why it matters:** the capture arrives with another advert's title, location, salary and description for the page the person was reading. On similar adverts from the same employer the review screen gives little sign of it, and the wrong posting is saved. - **Fix:** choose in two passes: first a posting whose `url`, `@id` or `sameAs` equals the page; only if none does, fall back to identifiers. Read an identifier's `value` (or the plain string), never its `name`, and match it as a whole path segment or query value rather than as a substring. Mirror the change in `postulo-chromium`'s `parse.js`. ## Done when - A fixture in `tests/fixtures/postings/` with three postings that share an employer and carry `PropertyValue` identifiers, read by `tests/test_source_corpus.py` as the posting whose `url` is the page; today it reads as the first. - A test in `tests/test_capture.py` that identifiers `1000` and `10001` on `/jobs/10001` choose `10001`. - The fixture passes in `postulo-chromium`'s `test/corpus.test.mjs`.

tiagoagueda added this to the 0.5.0 milestone

2026-10-01 10:48:06 +00:00

tiagoagueda added the

bug

label

2026-10-01 10:48:06 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 10:05:01 +00:00

Choose the posting by its address before its identifier

tiagoagueda closed this issue

2026-10-04 10:05:01 +00:00