PST-049: The capture parser does quadratic work on a hostile page, with no time bound in the errand worker #778

New issue

Closed

opened 2026-10-10 10:56:22 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-10-10 10:56:22 +00:00

Owner

Found in the security audit of 2026-10-10 (SECURITY-AUDIT-2026-10-10.md), run against main at c30851b from 8 to 10 October 2026, as PST-049. Severity: Low.

The text below is the audit's entry, unchanged apart from links to the other issues it names. The finding was produced by automated review and checked by a second automated pass; re-check it before acting on it.

Severity: Low
Category: denial-of-service / algorithmic complexity
Location: src/postulo/plugins/builtin/htmlutil.py (185-196,223-227,482-497)
Found by: completeness review: shipped plugins

Who can do it, and what they need. Two attackers. (a) The in-scope 'malicious page that is captured': any third party whose job page a member captures by URL, with no account on the instance. (b) Any member, or the holder of a captures-only API token, who sends their own html to POST /api/v1/captures or the paste form.

How it works.

Attacker serves or submits an HTML page of about 150-300 KB of deeply nested, unclosed or mismatched tags. Examples: '<div>'*20000 followed by '</x>'*20000, or '<article>'*8000 around some text.
The page reaches registry.parse_page, via jobs/slow.py:65 (errand worker) or api/api.py:640 (_read, in the web request). With a supplied html body, api/api.py:629 does not charge the capture allowance, so only the 600/h token rate applies.
_TreeBuilder.handle_endtag and _close_up_to scan the whole open-element stack for every unmatched end tag or implied close. main_text() calls text_of() on every nested <article>/<main>. hints._labels() calls text_of() on every nested <span>/<b>/<td>. All are O(n^2).
Measured with scratch scripts: tree build 45 KB = 2.5 s, 90 KB = 10.4 s, 180 KB = 48 s; nested <article> main_text 152 KB = 52 s; nested <span> _labels 152 KB = 17 s. Extrapolated to the 2 MB fetch cap (plugins/fetching.py MAX_BYTES), a single page is about 1.5 hours or more of CPU.
Via the API (cap 2.5 MB body), 3 concurrent requests of about 300 KB pin all 3 sync gunicorn workers (docker/Dockerfile: --workers 3, --timeout 120) until each is killed at 120 s. The attacker repeats every 2 minutes. Via a captured URL, the single errand worker, which has no handler time limit, is stuck for hours.

Impact. Availability for every account on a multi-user instance: web workers repeatedly killed, or the background worker (exports, PDFs, notifications, logo and capture fetches) stalled for hours by one hostile page. No data exposure. The threat model puts DoS against a personal instance out of scope, but it lists a captured page as an attacker and bounds the PDF renderer with a CPU limit; the HTML parser has no equivalent.

Evidence.

htmlutil.py:223-227 `def handle_endtag(self, tag): for index in range(len(self._open) - 1, 0, -1): if self._open[index].tag == tag: del self._open[index:]; return`. htmlutil.py:189 `for index in range(len(self._open) - 1, 0, -1):` in _close_up_to, run for every <li>/<p>/<td>. htmlutil.py:483-493 `for candidate in body.iter(): ... text = text_of(candidate, drop)`. hints.py:285-291 `for element in _elements(root): if element.tag not in LABELS: continue; own = _plain(text_of(element))`. api/api.py:628-629 `if not payload.html: throttle.capture(owner)`. Dockerfile CMD `--workers 3`, ENV `--timeout 120`.

Verification (verdict: confirmed; severity after verification: Low). Precise conditions, corrected where the auditor's description was loose.

(1) The nested-<article>/<main> cost in main_text only appears when the outer landmarks fail the early return, i.e. the text is under 200 chars or under 25% of the page. With ordinary long text the outermost article returns at once (0.03 s measured). With short text it is 41 s at 54 KB.

(2) hints._labels is reached through remembered.after_capture (api.py:525, jobs/slow.py) only after the page parsed into a capture. The attacker therefore puts a valid JSON-LD JobPosting first and the nesting after it. A page that parses to nothing returns 422 before _labels runs (measured: 422 in under 4 s at 80 KB). The tree-build and main_text costs apply to any page.

(3) Three concurrent API posts of about 70 KB each (about 70 s apiece) occupy all 3 sync workers. Posts of about 100 KB or more are killed at the 120 s timeout. I inferred this from the measurements and the Dockerfile and did not run a concurrent load test.

(4) Via captured URL (up to 2 MB), a member has to capture the hostile page. The single errand worker then stays busy for as long as the parse runs, since nothing ever interrupts it. …

Suggested remediation. Cap the number of open elements and total nodes in _TreeBuilder (stop or flatten past a depth such as 200 and a node budget). Make end-tag matching O(1) or bounded with a per-name index. Compute subtree text once bottom-up, as _link_farms already does, instead of calling text_of() per candidate. Run parse_page under a wall-clock budget (a child process or resource limit like the capture renderer's). Count own-html captures against an allowance, or bound html length well below 2.5 MB.

Found in the security audit of 2026-10-10 (`SECURITY-AUDIT-2026-10-10.md`), run against `main` at `c30851b` from 8 to 10 October 2026, as **PST-049**. Severity: **Low**. The text below is the audit's entry, unchanged apart from links to the other issues it names. The finding was produced by automated review and checked by a second automated pass; re-check it before acting on it. --- - **Severity:** Low - **Category:** denial-of-service / algorithmic complexity - **Location:** `src/postulo/plugins/builtin/htmlutil.py` (185-196,223-227,482-497) - **Found by:** completeness review: shipped plugins **Who can do it, and what they need.** Two attackers. (a) The in-scope 'malicious page that is captured': any third party whose job page a member captures by URL, with no account on the instance. (b) Any member, or the holder of a captures-only API token, who sends their own html to POST /api/v1/captures or the paste form. **How it works.** 1. Attacker serves or submits an HTML page of about 150-300 KB of deeply nested, unclosed or mismatched tags. Examples: '<div>'*20000 followed by '</x>'*20000, or '<article>'*8000 around some text. 2. The page reaches registry.parse_page, via jobs/slow.py:65 (errand worker) or api/api.py:640 (_read, in the web request). With a supplied html body, api/api.py:629 does not charge the capture allowance, so only the 600/h token rate applies. 3. _TreeBuilder.handle_endtag and _close_up_to scan the whole open-element stack for every unmatched end tag or implied close. main_text() calls text_of() on every nested <article>/<main>. hints._labels() calls text_of() on every nested <span>/<b>/<td>. All are O(n^2). 4. Measured with scratch scripts: tree build 45 KB = 2.5 s, 90 KB = 10.4 s, 180 KB = 48 s; nested <article> main_text 152 KB = 52 s; nested <span> _labels 152 KB = 17 s. Extrapolated to the 2 MB fetch cap (plugins/fetching.py MAX_BYTES), a single page is about 1.5 hours or more of CPU. 5. Via the API (cap 2.5 MB body), 3 concurrent requests of about 300 KB pin all 3 sync gunicorn workers (docker/Dockerfile: --workers 3, --timeout 120) until each is killed at 120 s. The attacker repeats every 2 minutes. Via a captured URL, the single errand worker, which has no handler time limit, is stuck for hours. **Impact.** Availability for every account on a multi-user instance: web workers repeatedly killed, or the background worker (exports, PDFs, notifications, logo and capture fetches) stalled for hours by one hostile page. No data exposure. The threat model puts DoS against a personal instance out of scope, but it lists a captured page as an attacker and bounds the PDF renderer with a CPU limit; the HTML parser has no equivalent. **Evidence.** ```text htmlutil.py:223-227 `def handle_endtag(self, tag): for index in range(len(self._open) - 1, 0, -1): if self._open[index].tag == tag: del self._open[index:]; return`. htmlutil.py:189 `for index in range(len(self._open) - 1, 0, -1):` in _close_up_to, run for every <li>/<p>/<td>. htmlutil.py:483-493 `for candidate in body.iter(): ... text = text_of(candidate, drop)`. hints.py:285-291 `for element in _elements(root): if element.tag not in LABELS: continue; own = _plain(text_of(element))`. api/api.py:628-629 `if not payload.html: throttle.capture(owner)`. Dockerfile CMD `--workers 3`, ENV `--timeout 120`. ``` **Verification** (verdict: confirmed; severity after verification: Low). Precise conditions, corrected where the auditor's description was loose. (1) The nested-`<article>`/`<main>` cost in `main_text` only appears when the outer landmarks fail the early return, i.e. the text is under 200 chars or under 25% of the page. With ordinary long text the outermost article returns at once (0.03 s measured). With short text it is 41 s at 54 KB. (2) `hints._labels` is reached through `remembered.after_capture` (api.py:525, jobs/slow.py) only after the page parsed into a capture. The attacker therefore puts a valid JSON-LD JobPosting first and the nesting after it. A page that parses to nothing returns 422 before `_labels` runs (measured: 422 in under 4 s at 80 KB). The tree-build and `main_text` costs apply to any page. (3) Three concurrent API posts of about 70 KB each (about 70 s apiece) occupy all 3 sync workers. Posts of about 100 KB or more are killed at the 120 s timeout. I inferred this from the measurements and the Dockerfile and did not run a concurrent load test. (4) Via captured URL (up to 2 MB), a member has to capture the hostile page. The single errand worker then stays busy for as long as the parse runs, since nothing ever interrupts it. … **Suggested remediation.** Cap the number of open elements and total nodes in _TreeBuilder (stop or flatten past a depth such as 200 and a node budget). Make end-tag matching O(1) or bounded with a per-name index. Compute subtree text once bottom-up, as _link_farms already does, instead of calling text_of() per candidate. Run parse_page under a wall-clock budget (a child process or resource limit like the capture renderer's). Count own-html captures against an allowance, or bound html length well below 2.5 MB.

tiagoagueda added the

bug

security

labels

2026-10-10 10:56:22 +00:00

tiagoagueda added this to the 0.6.0 milestone

2026-10-10 10:57:51 +00:00

tiagoagueda referenced this issue

2026-10-10 10:58:40 +00:00

Security audit of 2026-10-10: index of PST-001 to PST-098 (#730–#827) #828

tiagoagueda self-assigned this

2026-10-11 10:54:31 +00:00