Scheduler: finished background task rows are kept for ever #1121

New issue

Closed

opened 2026-10-31 03:04:09 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-31 03:04:09 +00:00

Owner

What exists today

django_tasks_db 0.13.0 ships manage.py prune_db_task_results (.venv/Lib/site-packages/django_tasks_db/management/commands/prune_db_task_results.py; --min-age-days default 14, --failed-min-age-days, --queue-name default default, * for all). Nothing in src/postulo calls it (grep for prune_db_task_results finds nothing), the scheduler pass does not do the equivalent, and the wiki does not tell operators to run it.

Every task enqueued (core/errands.py:106 perform_errand.enqueue, core/backups.py:279 / core/offsite_queue.py:25) leaves a DBTaskResult row with args_kwargs, return_value, worker_ids and, when it failed, exception_class_path and the whole traceback (django_tasks_db/models.py:152-169). The only reader is the staff overview's count of READY rows (core/server_views.py:201-205).

The scheduler already forgets the errand rows these tasks belong to after seven days (core/errands.py:211-221, called from send_due_reminders.py:215), so a task result outlives the errand that gave it meaning.

What would change

In the scheduler pass, beside errands.forget_old(), delete finished task results older than the same seven days, for every queue:

DBTaskResult.objects.finished().filter(finished_at__lte=cutoff).delete()

which is exactly the query upstream's command runs (lines 111-122), or call_command("prune_db_task_results", queue_name="*", min_age_days=7). Calling the upstream command keeps the library's definition of "finished"; the direct query avoids its logging setup. Either is a few lines; the house rule (library first) favours the command or its finished() manager method rather than a hand-written status list. Add the count to the pass's reaped total. A test: a successful and a failed result eight days old are gone after a pass, a READY one and a fresh one stay.

What it saves or protects

small: the django_tasks_db result table stops growing by one row per errand, archive copy and webhook task for the life of the instance; failed rows carry a full traceback each

Accessibility

none

Decisions

How long to keep task results? Recommended: seven days, matching errands; a failed task's traceback is already in the log.

Found by the efficiency and reliability audit of 2026-10-30 (finding B5; index postulo/postulo#1108). Checked against main at daee30b15 (2026-10-30).

## What exists today `django_tasks_db` 0.13.0 ships `manage.py prune_db_task_results` (`.venv/Lib/site-packages/django_tasks_db/management/commands/prune_db_task_results.py`; `--min-age-days` default 14, `--failed-min-age-days`, `--queue-name` default `default`, `*` for all). Nothing in `src/postulo` calls it (grep for `prune_db_task_results` finds nothing), the scheduler pass does not do the equivalent, and the wiki does not tell operators to run it. Every task enqueued (`core/errands.py:106` `perform_errand.enqueue`, `core/backups.py:279` / `core/offsite_queue.py:25`) leaves a `DBTaskResult` row with `args_kwargs`, `return_value`, `worker_ids` and, when it failed, `exception_class_path` and the whole `traceback` (`django_tasks_db/models.py:152-169`). The only reader is the staff overview's count of `READY` rows (`core/server_views.py:201-205`). The scheduler already forgets the errand rows these tasks belong to after seven days (`core/errands.py:211-221`, called from `send_due_reminders.py:215`), so a task result outlives the errand that gave it meaning. ## What would change In the scheduler pass, beside `errands.forget_old()`, delete finished task results older than the same seven days, for every queue: `DBTaskResult.objects.finished().filter(finished_at__lte=cutoff).delete()` which is exactly the query upstream's command runs (lines 111-122), or `call_command("prune_db_task_results", queue_name="*", min_age_days=7)`. Calling the upstream command keeps the library's definition of "finished"; the direct query avoids its logging setup. Either is a few lines; the house rule (library first) favours the command or its `finished()` manager method rather than a hand-written status list. Add the count to the pass's `reaped` total. A test: a successful and a failed result eight days old are gone after a pass, a `READY` one and a fresh one stay. ## What it saves or protects small: the `django_tasks_db` result table stops growing by one row per errand, archive copy and webhook task for the life of the instance; failed rows carry a full traceback each ## Accessibility none ## Decisions - **How long to keep task results?** Recommended: seven days, matching errands; a failed task's traceback is already in the log. --- Found by the efficiency and reliability audit of 2026-10-30 (finding B5; index postulo/postulo#1108). Checked against `main` at `daee30b15` (2026-10-30).

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 03:04:09 +00:00

tiagoagueda added the

performance

enhancement

labels

2026-10-31 03:04:09 +00:00

tiagoagueda referenced this issue

2026-10-31 03:04:40 +00:00

Efficiency and reliability audit of 2026-10-30: index #1108

tiagoagueda referenced this issue from a pull request that will close it,

2026-11-11 10:38:54 +00:00