The API: pages built after loading everything, an unpaginated /captures, captures that duplicate on retry, a public schema, no change feed #230

New issue

Closed

opened 2026-09-15 21:23:28 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-09-15 21:23:28 +00:00

Owner

The capture API is what the extensions and postulo-mcp are built on. Found in the 2026-09-15 code audit.

1. List endpoints serialise every row before `@paginate` takes a page

api/routers/applications.py:57 does return [application_out(request, a) for a in applications]. The same pattern is in companies.py:43, listings.py:56, interviews.py:61, reminders.py:28 and documents.py:64,75.
Given a list, django-ninja counts it with len() and slices it in Python.
So every page loads the whole queryset, its subqueries and the tag prefetch, and builds an absolute URL for every row. With 1,000 applications a page of 100 still serialises 1,000, and an agent paging through everything does quadratic work.

Fix: return the queryset and map rows in schema resolvers (resolve_*), or subclass the paginator to map rows only after slicing.

2. `GET /captures` is not paginated, though the wiki says lists are

api/api.py:352-362 returns a bare list cut at 50, with no explicit order. The capture API promises limit/offset and {"items", "count"}.

Fix: @paginate with an explicit ordering, plus a test that every list operation in the OpenAPI schema is paginated. test_wiki_surface.py checks paths and scopes, not response shape.

3. Capturing is not idempotent

api/api.py:271-279 creates a capture every time. The code's own comment expects retries (:285-286), and /captures/known is only advisory. A client retrying after a lost 201 creates a duplicate capture and a duplicate notification.

Fix: honour an Idempotency-Key header (owner + key + body hash, kept 24 h, replaying the first response), or return the existing pending capture for the same owner and URL within a short window.

4. `/api/v1/openapi.json` answers without a token

api/api.py:54-65 sets docs_url=None but leaves openapi_url at its default, and django-ninja protects that view only when docs_decorator is set. This contradicts THREAT-MODEL.md:13 ("the API answers 401 to everything without a live token"), and makes Postulo instances easy to fingerprint.

Fix: a docs_decorator requiring a live token or a staff session, or openapi_url=None with the schema published in the repository or wiki.

5. No change feed

List endpoints filter by applied date only; there is no updated_since. An agent or extension has to poll everything and diff.

Fix: an updated_since cursor on the list endpoints. The outbound webhooks that pair with this are a separate feature issue (#240).

The capture API is what the extensions and `postulo-mcp` are built on. Found in the 2026-09-15 code audit. ## 1. List endpoints serialise every row before `@paginate` takes a page - `api/routers/applications.py:57` does `return [application_out(request, a) for a in applications]`. The same pattern is in `companies.py:43`, `listings.py:56`, `interviews.py:61`, `reminders.py:28` and `documents.py:64,75`. - Given a list, django-ninja counts it with `len()` and slices it in Python. - So every page loads the whole queryset, its subqueries and the tag prefetch, and builds an absolute URL for every row. With 1,000 applications a page of 100 still serialises 1,000, and an agent paging through everything does quadratic work. **Fix:** return the queryset and map rows in schema resolvers (`resolve_*`), or subclass the paginator to map rows only after slicing. ## 2. `GET /captures` is not paginated, though the wiki says lists are `api/api.py:352-362` returns a bare list cut at 50, with no explicit order. *The capture API* promises `limit`/`offset` and `{"items", "count"}`. **Fix:** `@paginate` with an explicit ordering, plus a test that every list operation in the OpenAPI schema is paginated. `test_wiki_surface.py` checks paths and scopes, not response shape. ## 3. Capturing is not idempotent `api/api.py:271-279` creates a capture every time. The code's own comment expects retries (`:285-286`), and `/captures/known` is only advisory. A client retrying after a lost `201` creates a duplicate capture and a duplicate notification. **Fix:** honour an `Idempotency-Key` header (owner + key + body hash, kept 24 h, replaying the first response), or return the existing pending capture for the same owner and URL within a short window. ## 4. `/api/v1/openapi.json` answers without a token `api/api.py:54-65` sets `docs_url=None` but leaves `openapi_url` at its default, and django-ninja protects that view only when `docs_decorator` is set. This contradicts `THREAT-MODEL.md:13` ("the API answers 401 to everything without a live token"), and makes Postulo instances easy to fingerprint. **Fix:** a `docs_decorator` requiring a live token or a staff session, or `openapi_url=None` with the schema published in the repository or wiki. ## 5. No change feed List endpoints filter by applied date only; there is no `updated_since`. An agent or extension has to poll everything and diff. **Fix:** an `updated_since` cursor on the list endpoints. The outbound webhooks that pair with this are a separate feature issue (#240).

tiagoagueda added the

bug

enhancement

labels

2026-09-15 21:23:28 +00:00

tiagoagueda referenced this issue

2026-09-15 21:24:50 +00:00

Outbound events: a signed webhook notifier and more events than three #240

tiagoagueda referenced this issue

2026-09-15 21:25:14 +00:00

Query performance: N+1 in search, one-join aggregates, closed cards loaded, uncached site settings, missing composite indexes #231

tiagoagueda added this to the 0.4.0 milestone