Reference source: a <br>, <img> or unclosed <p> inside a field leaves it open, and later fields are swallowed into it #9

New issue

Closed

opened 2026-10-01 10:48:14 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:48:14 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo-helloworld at 19f0bcc29, running HelloWorldSource.parse from the snapshot on pages with line breaks and images inside the marked elements.

_PostingReader tracks nesting with a counter: every start tag inside a field adds one, every end tag takes one away, and the field ends on an end tag at depth zero. An element with no end tag leaves the counter one too high, so the field's own closing tag only brings it back to zero and the field stays open until some later, unrelated end tag. Everything in between, including the elements that should have started the next fields, is appended to it. This is the file the README invites people to copy.

Where: src/postulo_helloworld/source.py:59-81 (handle_starttag, handle_endtag): if self._inside: self._depth += 1; return runs for every start tag. tests/test_helloworld.py:24-32 has one page, whose only inline element is an <em>, which has an end tag.
What happens: run against the snapshot:
- <div data-description>We need someone.<br>Apply now.</div><p data-location>Lyon</p> gives location='' and description='We need someone.Apply now.Lyon'.
- <h1>Engineer<br>Research</h1> followed by the other fields gives title='EngineerResearchBlack MesaLyonWe need someone.' and every other field empty.
- <p data-company><img src=logo.png>Black Mesa</p> gives a company holding the rest of the article.
- <p> elements without end tags inside the description, which HTML allows, do the same.
- A site <h1> outside the article is joined to the posting's: title='Hello JobsEngineer'. Every <h1> on the page is read, inside the data-posting element or not.
- With enough text swallowed into the title, JobPostingData raises on its 500-character limit instead of parse returning a posting or None.
The self-closing spelling <br/> is not affected: HTMLParser reports it as a start tag and an end tag.
Why it matters: the result is a confidently wrong value on the review screen, which is what the README's first rule ("Leave a field empty rather than guessing", README.md:71) exists to prevent. The module says its parsing is not the point, but a nesting counter is the part of it most likely to be kept by somebody adapting the file to a real board, where descriptions with <br> are the norm.
Fix: do not count elements that have no end tag: skip the increment for the HTML void elements (br, img, hr, input, meta, link, wbr and the rest), and add a space for <br> so the text either side is not joined. Close a field on the end tag that matches the tag which opened it, not on any end tag at depth zero, so an unclosed <p> cannot hold it open. Read fields only inside the data-posting element, and take the title from its first <h1>.

Done when

A test in tests/test_helloworld.py parses a page with <br> and <img> inside each field, an unclosed <p> in the description and a site-level <h1> outside the article, and gets the four fields exactly. It fails today.

Found on 2026-09-29 in a read-only review of postulo-helloworld at 19f0bcc29, running `HelloWorldSource.parse` from the snapshot on pages with line breaks and images inside the marked elements. `_PostingReader` tracks nesting with a counter: every start tag inside a field adds one, every end tag takes one away, and the field ends on an end tag at depth zero. An element with no end tag leaves the counter one too high, so the field's own closing tag only brings it back to zero and the field stays open until some later, unrelated end tag. Everything in between, including the elements that should have started the next fields, is appended to it. This is the file the README invites people to copy. - **Where:** `src/postulo_helloworld/source.py:59-81` (`handle_starttag`, `handle_endtag`): `if self._inside: self._depth += 1; return` runs for every start tag. `tests/test_helloworld.py:24-32` has one page, whose only inline element is an `<em>`, which has an end tag. - **What happens:** run against the snapshot: - `<div data-description>We need someone.<br>Apply now.</div><p data-location>Lyon</p>` gives `location=''` and `description='We need someone.Apply now.Lyon'`. - `<h1>Engineer<br>Research</h1>` followed by the other fields gives `title='EngineerResearchBlack MesaLyonWe need someone.'` and every other field empty. - `<p data-company><img src=logo.png>Black Mesa</p>` gives a company holding the rest of the article. - `<p>` elements without end tags inside the description, which HTML allows, do the same. - A site `<h1>` outside the article is joined to the posting's: `title='Hello JobsEngineer'`. Every `<h1>` on the page is read, inside the `data-posting` element or not. - With enough text swallowed into the title, `JobPostingData` raises on its 500-character limit instead of `parse` returning a posting or `None`. The self-closing spelling `<br/>` is not affected: `HTMLParser` reports it as a start tag and an end tag. - **Why it matters:** the result is a confidently wrong value on the review screen, which is what the README's first rule ("Leave a field empty rather than guessing", `README.md:71`) exists to prevent. The module says its parsing is not the point, but a nesting counter is the part of it most likely to be kept by somebody adapting the file to a real board, where descriptions with `<br>` are the norm. - **Fix:** do not count elements that have no end tag: skip the increment for the HTML void elements (`br`, `img`, `hr`, `input`, `meta`, `link`, `wbr` and the rest), and add a space for `<br>` so the text either side is not joined. Close a field on the end tag that matches the tag which opened it, not on any end tag at depth zero, so an unclosed `<p>` cannot hold it open. Read fields only inside the `data-posting` element, and take the title from its first `<h1>`. ## Done when - A test in `tests/test_helloworld.py` parses a page with `<br>` and `<img>` inside each field, an unclosed `<p>` in the description and a site-level `<h1>` outside the article, and gets the four fields exactly. It fails today.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:48:14 +00:00

tiagoagueda added the

bug

label

2026-10-01 10:48:14 +00:00

tiagoagueda referenced this issue from a commit

2026-10-19 07:08:42 +00:00

Reference source: stop a <br>, <img> or unclosed <p> from holding a field open

tiagoagueda referenced this issue from a pull request that will close it,

2026-10-19 07:09:04 +00:00

Fix the open bug and security issues: source parser, release gate, notifier path, lock and CI #15