Metrics: every scrape builds a migration graph to say whether migrations are applied #1113

New issue

Closed

opened 2026-10-31 03:04:06 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-31 03:04:06 +00:00

Owner

What exists today

src/postulo/core/metrics.py:93-101 _migrations_are_applied() creates a MigrationExecutor(connection) and computes migration_plan(leaf_nodes()) on every call; collect() (:104) calls it at :152 on every scrape, then runs about 15 COUNTs (:163-259).

Measured with CaptureQueriesContext on the second GET of /metrics with the endpoint on and a token, under postulo.config.settings.test (probe audit/m_scripts/test_probe_queries.py): 37 queries and 62 ms for the request; _migrations_are_applied() alone 52 ms (the migration modules already imported). Of the 37 queries, 8 are the database cache used by the throttle and the rest the counts and the migration table read.

What changed from the candidate: the counts are not the cost worth fixing. They are what the endpoint exists to report, each is a single COUNT over one table, and caching them for less than a scrape interval saves nothing. The migration graph is the part that is pure overhead: it is rebuilt from every migration file of every app on every scrape, and its answer cannot change while the process lives, because new migrations arrive only with new code and new code means a new process. The endpoint is off by default (POSTULO_METRICS_ENABLED), so only operators who turned it on pay.

What would change

Remember a true answer for the life of the process: a module-level flag set the first time _migrations_are_applied() returns true, after which it returns true without building anything. A false answer is still checked again at each scrape, so an instance started before migrate finished reports the change. Leave the counts as they are.

What it saves or protects

small: about 52 ms of the scrape's CPU (measured on a desktop, several times that on a Pi) at every Prometheus scrape, typically every 15 to 60 s

Accessibility

none

Decisions

Also fold the counts into one statement? Recommended: no; one query per table is clearer and costs little next to the graph.

Found by the efficiency and reliability audit of 2026-10-30 (finding Q2; index postulo/postulo#1108). Checked against main at daee30b15 (2026-10-30).

## What exists today `src/postulo/core/metrics.py:93-101` `_migrations_are_applied()` creates a `MigrationExecutor(connection)` and computes `migration_plan(leaf_nodes())` on every call; `collect()` (`:104`) calls it at `:152` on every scrape, then runs about 15 `COUNT`s (`:163-259`). Measured with `CaptureQueriesContext` on the second GET of `/metrics` with the endpoint on and a token, under `postulo.config.settings.test` (probe `audit/m_scripts/test_probe_queries.py`): 37 queries and 62 ms for the request; `_migrations_are_applied()` alone 52 ms (the migration modules already imported). Of the 37 queries, 8 are the database cache used by the throttle and the rest the counts and the migration table read. What changed from the candidate: the counts are not the cost worth fixing. They are what the endpoint exists to report, each is a single `COUNT` over one table, and caching them for less than a scrape interval saves nothing. The migration graph is the part that is pure overhead: it is rebuilt from every migration file of every app on every scrape, and its answer cannot change while the process lives, because new migrations arrive only with new code and new code means a new process. The endpoint is off by default (`POSTULO_METRICS_ENABLED`), so only operators who turned it on pay. ## What would change Remember a true answer for the life of the process: a module-level flag set the first time `_migrations_are_applied()` returns true, after which it returns true without building anything. A false answer is still checked again at each scrape, so an instance started before `migrate` finished reports the change. Leave the counts as they are. ## What it saves or protects small: about 52 ms of the scrape's CPU (measured on a desktop, several times that on a Pi) at every Prometheus scrape, typically every 15 to 60 s ## Accessibility none ## Decisions - **Also fold the counts into one statement?** Recommended: no; one query per table is clearer and costs little next to the graph. --- Found by the efficiency and reliability audit of 2026-10-30 (finding Q2; index postulo/postulo#1108). Checked against `main` at `daee30b15` (2026-10-30).

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 03:04:06 +00:00

tiagoagueda added the

performance

enhancement

labels

2026-10-31 03:04:06 +00:00

tiagoagueda referenced this issue

2026-10-31 03:04:40 +00:00

Efficiency and reliability audit of 2026-10-30: index #1108

tiagoagueda referenced this issue from a pull request that will close it,

2026-11-10 23:08:20 +00:00