Tables: a number filter in scientific notation with a huge exponent holds a worker for minutes #499

New issue

Closed

opened 2026-10-01 10:47:07 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:47:07 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the table filters and timing them with the snapshot's interpreter.

Checked again on 2026-10-01 against main at 0a5cad587: still present.

A table's min/max filter accepts any finite Decimal, including one written as a short exponent. The bound is then handed to a lookup on an integer count, and Django turns it into a Python int, which for a value like 1e1000000 takes tens of seconds and grows with the square of the exponent. One GET of a few characters ties up a gunicorn worker until it is killed.

Where: core/tables.py:729-735, _number, whose only check is value.is_finite(); core/tables.py:582-588, Table.filter, which passes it to {lookup}__gte / __lte; the number columns in jobs/tables.py:68 (posting_count), :78 (application_count), :105 (contact_count) and :287 (listings' application_count).
What happens: postings_min (or any *_min / *_max of a number column) arrives from request.GET through Table.given, _number parses it as Decimal, and the integer field's get_prep_value calls int() on it. Measured: int(Decimal("1e100000")) takes 0.35 s, 1e300000 3.2 s, and 1e1000000 about 35 s; the time is all in that conversion. A nine-character value runs past the image's --timeout 120 (docker/Dockerfile:354), and the image runs three sync workers (docker/Dockerfile:362).
Who can trigger it: any signed-in account holder, on the companies or listings table; on an instance with open registration, anybody who signs up. A handful of such requests every two minutes keeps all three workers busy, so every other account on the instance gets no answer.
Why it matters: a denial of service against the whole instance from one ordinary account, with no rate limit in the way and nothing in the logs but slow requests.
Fix: bound the value before it reaches the queryset. Every number column today is a count, so _number can accept only an optional sign and at most, say, 12 digits (re.fullmatch(r"[+-]?\d{1,12}", text)) and return an int; if decimals are wanted later, refuse a Decimal whose adjusted() exceeds 18. What is refused narrows nothing, as an unparsable value does today.

Done when

tests/test_tables.py has a test that ?postings_min=1e1000000 on the companies table answers within a second and leaves the rows unfiltered, and one that ?postings_min=3 still narrows.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the table filters and timing them with the snapshot's interpreter. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. A table's *min*/*max* filter accepts any finite `Decimal`, including one written as a short exponent. The bound is then handed to a lookup on an integer count, and Django turns it into a Python `int`, which for a value like `1e1000000` takes tens of seconds and grows with the square of the exponent. One GET of a few characters ties up a gunicorn worker until it is killed. - **Where:** `core/tables.py:729-735`, `_number`, whose only check is `value.is_finite()`; `core/tables.py:582-588`, `Table.filter`, which passes it to `{lookup}__gte` / `__lte`; the number columns in `jobs/tables.py:68` (`posting_count`), `:78` (`application_count`), `:105` (`contact_count`) and `:287` (listings' `application_count`). - **What happens:** `postings_min` (or any `*_min` / `*_max` of a number column) arrives from `request.GET` through `Table.given`, `_number` parses it as `Decimal`, and the integer field's `get_prep_value` calls `int()` on it. Measured: `int(Decimal("1e100000"))` takes 0.35 s, `1e300000` 3.2 s, and `1e1000000` about 35 s; the time is all in that conversion. A nine-character value runs past the image's `--timeout 120` (`docker/Dockerfile:354`), and the image runs three sync workers (`docker/Dockerfile:362`). - **Who can trigger it:** any signed-in account holder, on the companies or listings table; on an instance with open registration, anybody who signs up. A handful of such requests every two minutes keeps all three workers busy, so every other account on the instance gets no answer. - **Why it matters:** a denial of service against the whole instance from one ordinary account, with no rate limit in the way and nothing in the logs but slow requests. - **Fix:** bound the value before it reaches the queryset. Every number column today is a count, so `_number` can accept only an optional sign and at most, say, 12 digits (`re.fullmatch(r"[+-]?\d{1,12}", text)`) and return an `int`; if decimals are wanted later, refuse a `Decimal` whose `adjusted()` exceeds 18. What is refused narrows nothing, as an unparsable value does today. ## Done when - `tests/test_tables.py` has a test that `?postings_min=1e1000000` on the companies table answers within a second and leaves the rows unfiltered, and one that `?postings_min=3` still narrows.

tiagoagueda added this to the 0.5.0 milestone

2026-10-01 10:47:07 +00:00

tiagoagueda added the

security

tier

bug

labels

2026-10-01 10:47:07 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 10:04:58 +00:00

Refuse a number filter bound that is not a plain, short number

tiagoagueda closed this issue