Capture API: an Idempotency-Key longer than 200 characters answers 500 on PostgreSQL #433

New issue

Closed

opened 2026-10-01 10:46:40 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:46:40 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the capture call's idempotency handling.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The code is now at api/api.py:425-433 (the Idempotency-Key header of create_capture); api/idempotency.py and api/models.py have not moved.

POST /api/v1/captures accepts an Idempotency-Key of any length and stores it in a 200-character column. PostgreSQL refuses the over-long value with DataError, which idempotency.once does not catch (it catches IntegrityError only), so the request fails as an unhandled 500 before any capture is made. SQLite does not enforce varchar lengths, so the default install and the SQLite test run never see it.

Where: api/api.py:423-431 (create_capture, idempotency_key: str | None = Header(None, alias="Idempotency-Key", ...), no max_length); api/idempotency.py:78-89 (once: answer.key = key.strip(), then IdempotentRequest.objects.create(...) inside transaction.atomic(), except IntegrityError); api/models.py:183 (key = models.CharField(_("key"), max_length=200, editable=False)).
What happens: a client on a PostgreSQL instance (docker/compose.postgres.yml, POSTULO_DATABASE_URL=postgres://…) sends a capture with a 201-character key, for example one built from the posting's URL. The insert raises DataError: value too long for type character varying(200); none of the handlers in api/problems.py or api/api.py catches it, and the client gets a 500 rather than a problem document. Every retry does the same. The same client works on SQLite. The wiki's The capture API says the key is "any string of your own invention" and names no limit.
Why it matters: a documented, supported database turns a client-side choice into a server error with no hint of the cause, and the 500 lands in the operator's error log for every such capture.
Fix: bound the header where it is declared, Header(None, alias="Idempotency-Key", max_length=200, ...). Checked against the installed django-ninja: a 201-character value then answers 422 (a validation-failed problem document naming the header) and a 200-character one passes. State the limit in the header's description and in the wiki's Sending the same posting twice paragraph.

Done when

A test in tests/test_capture_api.py that sends a capture with a 201-character Idempotency-Key gets a 422 naming the header, and one with 200 characters is accepted; it fails today on the SQLite run too, which answers 201.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the capture call's idempotency handling. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The code is now at `api/api.py:425-433` (the `Idempotency-Key` header of `create_capture`); `api/idempotency.py` and `api/models.py` have not moved. `POST /api/v1/captures` accepts an `Idempotency-Key` of any length and stores it in a 200-character column. PostgreSQL refuses the over-long value with `DataError`, which `idempotency.once` does not catch (it catches `IntegrityError` only), so the request fails as an unhandled 500 before any capture is made. SQLite does not enforce `varchar` lengths, so the default install and the SQLite test run never see it. - **Where:** `api/api.py:423-431` (`create_capture`, `idempotency_key: str | None = Header(None, alias="Idempotency-Key", ...)`, no `max_length`); `api/idempotency.py:78-89` (`once`: `answer.key = key.strip()`, then `IdempotentRequest.objects.create(...)` inside `transaction.atomic()`, `except IntegrityError`); `api/models.py:183` (`key = models.CharField(_("key"), max_length=200, editable=False)`). - **What happens:** a client on a PostgreSQL instance (`docker/compose.postgres.yml`, `POSTULO_DATABASE_URL=postgres://…`) sends a capture with a 201-character key, for example one built from the posting's URL. The insert raises `DataError: value too long for type character varying(200)`; none of the handlers in `api/problems.py` or `api/api.py` catches it, and the client gets a 500 rather than a problem document. Every retry does the same. The same client works on SQLite. The wiki's *The capture API* says the key is "any string of your own invention" and names no limit. - **Why it matters:** a documented, supported database turns a client-side choice into a server error with no hint of the cause, and the 500 lands in the operator's error log for every such capture. - **Fix:** bound the header where it is declared, `Header(None, alias="Idempotency-Key", max_length=200, ...)`. Checked against the installed django-ninja: a 201-character value then answers 422 (a `validation-failed` problem document naming the header) and a 200-character one passes. State the limit in the header's description and in the wiki's *Sending the same posting twice* paragraph. ## Done when - A test in `tests/test_capture_api.py` that sends a capture with a 201-character `Idempotency-Key` gets a 422 naming the header, and one with 200 characters is accepted; it fails today on the SQLite run too, which answers 201.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:46:40 +00:00

tiagoagueda added the

bug

tier

labels

2026-10-01 10:46:40 +00:00

tiagoagueda referenced this issue from a commit

2026-10-03 22:56:00 +00:00

Refuse a capture Idempotency-Key longer than 200 characters

tiagoagueda closed this issue

2026-10-03 22:56:00 +00:00