Tests: every local run and every plugin suite migrates Postulo's 231 migrations before its first test #1144

New issue

Closed

opened 2026-10-31 03:04:20 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-31 03:04:20 +00:00

Owner

What exists today

src/postulo/*/migrations hold 231 migration files, 57 of them with RunPython. src/postulo/config/settings/test.py:13 uses in-memory SQLite, so pytest-django's --reuse-db has nothing to keep and every process migrates from zero.

Measured with uv run pytest tests/test_career_preview.py -q -p no:cacheprovider -p no:xdist --durations=5 (one test, Windows, other agents running): setup 55.89 s, call 2.50 s, total 58.72 s. The same with --no-migrations: setup 2.14 s, total 5.11 s. So about 54 s of a one-test run is migrations on this machine; #1033 measured 32-35 s per process on the CI host.

Plugin suites: postulo-dav, -imap, -apprise, -paperless, -helloworld, -templates and tiagoagueda/postulo-francetravail each have a tests/settings.py that imports postulo.config.settings.base with in-memory SQLite (postulo-dav/tests/settings.py:16), and run uv run pytest in one process (postulo-dav/.forgejo/workflows/ci.yml:84). Each run therefore migrates all of core before its first test, which for a small plugin suite is most of the test step.

#1033 item 3 (and order-of-work item 9) covers the core CI angle: per-worker migration on CI, a file-backed database copied per worker (3b), or squashing (3c). It does not mention local runs or the plugin suites, which are the new part here.

What would change

A django_db_setup override, written once in core and published through postulo.plugins.testing (#229's published test helpers) so plugin repos use the same one: when POSTULO_TEST_DB_CACHE is set (or by default outside CI), migrate a file-backed SQLite once, store it under a cache directory named by a hash of every migration file of the installed apps, and on later runs restore it into the in-memory connection with SQLite's backup API (sqlite3.Connection.backup, standard library) instead of migrating. A change to any migration changes the hash and the next run migrates afresh.
Plugin CI caches that directory with actions/cache keyed on the hash of the core version in uv.lock, so a plugin run migrates only when its core pin moves.
Core CI keeps migrating per process, as now, until #1033 item 9 decides; the migration tests (tests/test_migration_*.py) need the graph and keep it either way.
--no-migrations is not the fix: the 57 RunPython migrations seed rows some tests rely on, and it would stop testing the plugin's own migrations.

What it saves or protects

medium: measured locally, one one-test file takes 58.7 s of which 53.7 s is building the test database by migration and 2.1 s without; that cost is paid on every local pytest of a single file, and once per CI run of each of the eight plugin repos that test against core

Accessibility

none

Decided by the maintainer (2026-10-31)

cached migrated test database keyed by migration hash, in core, exposed through postulo.plugins.testing; core CI unchanged until #1033 decides

Decisions

Reuse a cached, migrated test database for local runs and plugin suites? Recommended: yes, keyed by the migration files' hash, off on core CI until #1033 item 9 is decided.
Where it lives. Recommended: in core, exposed through postulo.plugins.testing, so seven plugin repos gain it with their next repin (#1090) instead of seven copies.

Found by the efficiency and reliability audit of 2026-10-30 (finding T1; index postulo/postulo#1108). Checked against main at daee30b15 (2026-10-30).

## What exists today `src/postulo/*/migrations` hold 231 migration files, 57 of them with `RunPython`. `src/postulo/config/settings/test.py:13` uses in-memory SQLite, so pytest-django's `--reuse-db` has nothing to keep and every process migrates from zero. Measured with `uv run pytest tests/test_career_preview.py -q -p no:cacheprovider -p no:xdist --durations=5` (one test, Windows, other agents running): setup 55.89 s, call 2.50 s, total 58.72 s. The same with `--no-migrations`: setup 2.14 s, total 5.11 s. So about 54 s of a one-test run is migrations on this machine; #1033 measured 32-35 s per process on the CI host. Plugin suites: `postulo-dav`, `-imap`, `-apprise`, `-paperless`, `-helloworld`, `-templates` and `tiagoagueda/postulo-francetravail` each have a `tests/settings.py` that imports `postulo.config.settings.base` with in-memory SQLite (`postulo-dav/tests/settings.py:16`), and run `uv run pytest` in one process (`postulo-dav/.forgejo/workflows/ci.yml:84`). Each run therefore migrates all of core before its first test, which for a small plugin suite is most of the test step. #1033 item 3 (and order-of-work item 9) covers the core CI angle: per-worker migration on CI, a file-backed database copied per worker (3b), or squashing (3c). It does not mention local runs or the plugin suites, which are the new part here. ## What would change - A `django_db_setup` override, written once in core and published through `postulo.plugins.testing` (#229's published test helpers) so plugin repos use the same one: when `POSTULO_TEST_DB_CACHE` is set (or by default outside CI), migrate a file-backed SQLite once, store it under a cache directory named by a hash of every migration file of the installed apps, and on later runs restore it into the in-memory connection with SQLite's backup API (`sqlite3.Connection.backup`, standard library) instead of migrating. A change to any migration changes the hash and the next run migrates afresh. - Plugin CI caches that directory with `actions/cache` keyed on the hash of the core version in `uv.lock`, so a plugin run migrates only when its core pin moves. - Core CI keeps migrating per process, as now, until #1033 item 9 decides; the migration tests (`tests/test_migration_*.py`) need the graph and keep it either way. - `--no-migrations` is not the fix: the 57 `RunPython` migrations seed rows some tests rely on, and it would stop testing the plugin's own migrations. ## What it saves or protects medium: measured locally, one one-test file takes 58.7 s of which 53.7 s is building the test database by migration and 2.1 s without; that cost is paid on every local `pytest` of a single file, and once per CI run of each of the eight plugin repos that test against core ## Accessibility none ## Decided by the maintainer (2026-10-31) cached migrated test database keyed by migration hash, in core, exposed through postulo.plugins.testing; core CI unchanged until #1033 decides ## Decisions - **Reuse a cached, migrated test database for local runs and plugin suites?** Recommended: yes, keyed by the migration files' hash, off on core CI until #1033 item 9 is decided. - **Where it lives.** Recommended: in core, exposed through `postulo.plugins.testing`, so seven plugin repos gain it with their next repin (#1090) instead of seven copies. --- Found by the efficiency and reliability audit of 2026-10-30 (finding T1; index postulo/postulo#1108). Checked against `main` at `daee30b15` (2026-10-30).

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 03:04:20 +00:00

tiagoagueda added the

performance

enhancement

labels

2026-10-31 03:04:20 +00:00

tiagoagueda referenced this issue

2026-10-31 03:04:41 +00:00

Efficiency and reliability audit of 2026-10-30: index #1108

tiagoagueda referenced this issue from a pull request that will close it,

2026-11-11 20:36:51 +00:00