Catching up: the wiki's recipe still omits `after_id`, and `GET /captures` does not take it #439

New issue

Closed

opened 2026-10-01 10:46:43 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:46:43 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, checking every list call against api/paging.py and the wiki's The capture API.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The code is now at api/api.py:633-654 (list_captures); the wiki page has not changed.

#245 added after_id because a cursor made of updated_at alone cannot get past a run of rows that share one timestamp, and it said the handbook's recipe would change to "remember the last updated_at and id". The recipe did not change, so a client written from the wiki still has the loop #245 fixed. And the one list the wiki names as readable with the captures scope alone, the capture queue, never gained the parameter.

Where:
- postulo.wiki/The-capture-API.md:277-284 (Catching up): "Every row carries an updated_at, and that is the cursor: read a page, remember the updated_at of the last row on it, and ask again from there." The example URL carries only updated_since. after_id is named once on the page, for /listings/choices (line 330).
- postulo.wiki/The-capture-API.md:60-62 and :273-275 say every list takes updated_since, and say nothing of after_id.
- api/api.py:631-652, list_captures: takes updated_since only and returns changed_since(captures, updated_since). Every other queryset list (applications, companies, CVs, letters, interviews, listings, listing choices, offers, reminders) takes after_id and passes it on; /documents is the one deliberate exception, and its docstring explains why.
What happens: an import or a bulk edit writes a hundred applications within one microsecond (updated_at is auto_now). A client following the wiki asks GET /applications?updated_since=…&limit=50, remembers the updated_at of the fiftieth row and asks again from it. changed_since without after_id filters updated_at__gte=since, so the client is handed the same fifty rows every time: it loops, or stops and never reads the other fifty. after_id would fix this, but nothing on the page tells the client it exists. A client catching up on /captures has no after_id to send at all.
Why it matters: the fix for #245 is in the code and the tests (tests/test_api_lists.py walks with both halves), but not in the documentation a client is written from, and not on one of the lists. The mirror a client keeps comes up short with no error.
Fix:
- Rewrite Catching up so the cursor is the pair: send updated_since and after_id from the same last row, with an example URL carrying both, and say what goes wrong without after_id. Mention after_id wherever the page says every list takes updated_since, and note that /documents does not take it.
- Add after_id: int | None = Query(None, description=AFTER_ID) to list_captures and pass it to changed_since.

Related to #245 (closed), which promised the recipe change and added after_id to the other lists.

Done when

tests/test_api_lists.py has a test that walks a run of pending captures sharing one updated_at through GET /api/v1/captures?updated_since=…&after_id=…&limit=… and reaches every one of them; it fails today because after_id is ignored.
The wiki's Catching up section and its example use updated_since together with after_id.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, checking every list call against `api/paging.py` and the wiki's *The capture API*. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The code is now at `api/api.py:633-654` (`list_captures`); the wiki page has not changed. #245 added `after_id` because a cursor made of `updated_at` alone cannot get past a run of rows that share one timestamp, and it said the handbook's recipe would change to "remember the last `updated_at` **and** `id`". The recipe did not change, so a client written from the wiki still has the loop #245 fixed. And the one list the wiki names as readable with the `captures` scope alone, the capture queue, never gained the parameter. - **Where:** - `postulo.wiki/The-capture-API.md:277-284` (*Catching up*): "Every row carries an `updated_at`, and that is the cursor: read a page, remember the `updated_at` of the last row on it, and ask again from there." The example URL carries only `updated_since`. `after_id` is named once on the page, for `/listings/choices` (line 330). - `postulo.wiki/The-capture-API.md:60-62` and `:273-275` say every list takes `updated_since`, and say nothing of `after_id`. - `api/api.py:631-652`, `list_captures`: takes `updated_since` only and returns `changed_since(captures, updated_since)`. Every other queryset list (applications, companies, CVs, letters, interviews, listings, listing choices, offers, reminders) takes `after_id` and passes it on; `/documents` is the one deliberate exception, and its docstring explains why. - **What happens:** an import or a bulk edit writes a hundred applications within one microsecond (`updated_at` is `auto_now`). A client following the wiki asks `GET /applications?updated_since=…&limit=50`, remembers the `updated_at` of the fiftieth row and asks again from it. `changed_since` without `after_id` filters `updated_at__gte=since`, so the client is handed the same fifty rows every time: it loops, or stops and never reads the other fifty. `after_id` would fix this, but nothing on the page tells the client it exists. A client catching up on `/captures` has no `after_id` to send at all. - **Why it matters:** the fix for #245 is in the code and the tests (`tests/test_api_lists.py` walks with both halves), but not in the documentation a client is written from, and not on one of the lists. The mirror a client keeps comes up short with no error. - **Fix:** - Rewrite *Catching up* so the cursor is the pair: send `updated_since` and `after_id` from the same last row, with an example URL carrying both, and say what goes wrong without `after_id`. Mention `after_id` wherever the page says every list takes `updated_since`, and note that `/documents` does not take it. - Add `after_id: int | None = Query(None, description=AFTER_ID)` to `list_captures` and pass it to `changed_since`. Related to #245 (closed), which promised the recipe change and added `after_id` to the other lists. ## Done when - `tests/test_api_lists.py` has a test that walks a run of pending captures sharing one `updated_at` through `GET /api/v1/captures?updated_since=…&after_id=…&limit=…` and reaches every one of them; it fails today because `after_id` is ignored. - The wiki's *Catching up* section and its example use `updated_since` together with `after_id`.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:46:43 +00:00

tiagoagueda added the

documentation

tier

bug

labels

2026-10-01 10:46:43 +00:00

tiagoagueda referenced this issue from a commit

2026-10-03 22:56:00 +00:00

Let GET /captures take after_id like every other list

tiagoagueda closed this issue