Follow the server's reading of pages with no standard: the shared corpus, the title split, and remembered fields in the popup #3

New issue

Open

opened 2026-09-29 17:31:16 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-09-29 17:31:16 +00:00

Owner

Postulo now reads pages with no structured data further, and remembers where a person's corrections showed a field to be on a site (postulo/postulo#267, landed as f39f14d0e). Three things follow for the extension.

1. The shared posting corpus changed

tests/fixtures/postings/plain-board.json now expects the company "Aperture Science", read from a title line "Job Application for … at Aperture Science". The extension's mirror of that corpus must follow, or the two disagree about the same page.

2. Port the title and salary reading to `parse.js`, so the preview agrees

The server's page-metadata source now:

reads og:title, twitter:title and <title>, and splits a title line on hyphen, en dash, em dash and middle dot (with spaces around), pipe, and "at". A part is named only when something vouches for it: the page's one <h1> names the job; the part after " at" names the employer; a part is a place only when the offline city table knows it and the line names just one; the site's own name (from og:site_name or its address) is taken out. Anything else stays empty.
reads a salary and a closing date only just after the words the page's language uses for them, with a currency beside the figure and a date written in full; ambiguous dates, a bare $ on a page that names no country, and disagreeing salaries give nothing.

The server's code is src/postulo/plugins/builtin/titles.py and patterns.py.

POST /api/v1/captures/preview now answers hinted: the fields of data that came from a place the person's own corrections showed on this site before, rather than from what the page states. Mark them in the popup in words (not colour alone), as the review screen does: "Read where your own corrections showed it on this site before." Corrections the popup sends with a capture already teach the server; nothing new is needed for that.

Optional, later: for a page where a corrected value appears twice, the popup could send the DOM place of each corrected field in the shapes the server uses (id, itemprop, data-*, a label beside the value, a unique class, a heading's position). The server would check it against the HTML it received before remembering it. Not needed for learning to work.

Postulo now reads pages with no structured data further, and remembers where a person's corrections showed a field to be on a site (postulo/postulo#267, landed as `f39f14d0e`). Three things follow for the extension. ## 1. The shared posting corpus changed `tests/fixtures/postings/plain-board.json` now expects the company **"Aperture Science"**, read from a title line "Job Application for … at Aperture Science". The extension's mirror of that corpus must follow, or the two disagree about the same page. ## 2. Port the title and salary reading to `parse.js`, so the preview agrees The server's page-metadata source now: - reads `og:title`, `twitter:title` and `<title>`, and splits a title line on hyphen, en dash, em dash and middle dot (with spaces around), pipe, and "at". A part is named only when something vouches for it: the page's one `<h1>` names the job; the part after "<job> at" names the employer; a part is a place only when the offline city table knows it and the line names just one; the site's own name (from `og:site_name` or its address) is taken out. Anything else stays empty. - reads a salary and a closing date only just after the words the page's language uses for them, with a currency beside the figure and a date written in full; ambiguous dates, a bare `$` on a page that names no country, and disagreeing salaries give nothing. The server's code is `src/postulo/plugins/builtin/titles.py` and `patterns.py`. ## 3. Show remembered fields in the popup `POST /api/v1/captures/preview` now answers `hinted`: the fields of `data` that came from a place the person's own corrections showed on this site before, rather than from what the page states. Mark them in the popup in words (not colour alone), as the review screen does: "Read where your own corrections showed it on this site before." Corrections the popup sends with a capture already teach the server; nothing new is needed for that. **Optional, later:** for a page where a corrected value appears twice, the popup could send the DOM place of each corrected field in the shapes the server uses (id, itemprop, `data-*`, a label beside the value, a unique class, a heading's position). The server would check it against the HTML it received before remembering it. Not needed for learning to work.

tiagoagueda added the

enhancement

label

2026-09-29 17:31:16 +00:00

tiagoagueda referenced this issue

2026-10-01 10:47:15 +00:00

Reading a page: a JobPosting's identifier is matched by its name, so every posting on a careers site counts as this page's #15

tiagoagueda referenced this issue from a commit

2026-10-04 16:23:09 +00:00

Read titles, pay and closing date on pages with no standard, as Postulo does

tiagoagueda commented

2026-10-04 16:28:09 +00:00

Author

Owner

Landed in part (3c7d787): the title split and the salary and closing-date reading are ported (src/lib/titles.js, patterns.js) and the corpus mirror follows the server's. Left, and it is a decision: part 3, marking remembered fields in the popup. The popup never calls POST /captures/preview, and calling it would send the page to the server before the person presses Send, which PRIVACY.md says never happens. Either accept that (and say so in PRIVACY.md and ask first), or leave the marking to the review screen on the server. The optional DOM-place hints sent with corrections are not done either.

Landed in part (`3c7d787`): the title split and the salary and closing-date reading are ported (`src/lib/titles.js`, `patterns.js`) and the corpus mirror follows the server's. Left, and it is a decision: **part 3, marking remembered fields in the popup.** The popup never calls `POST /captures/preview`, and calling it would send the page to the server before the person presses Send, which PRIVACY.md says never happens. Either accept that (and say so in PRIVACY.md and ask first), or leave the marking to the review screen on the server. The optional DOM-place hints sent with corrections are not done either.