Release gate: a tag passes with the PostgreSQL, security audit and stylesheet jobs red #419

New issue

Closed

opened 2026-10-01 10:46:34 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-10-01 10:46:34 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, comparing the release gate's required jobs in scripts/release_tools.py with the jobs in .forgejo/workflows/ci.yml. Related to #233 (closed), which added the gate and proposed requiring only test* and browser, at a time when the PostgreSQL job did not exist; this issue adds the three jobs the list was never revisited for. Also related to #219 (closed), which added the PostgreSQL job, and #319 (closed), which renamed the jobs and noted that a job split out of the test legs should be required too.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The code is now at .forgejo/workflows/ci.yml:264-335 (security) and .forgejo/workflows/ci.yml:337-375 (styles), and the paragraph about the gate at CONTRIBUTING.md:349-353; the other locations are unchanged.

The release gate refuses a tag unless named CI jobs passed on its commit, and it names only the unit-test legs and the browser job. Three other jobs check things a release ships and are not required: Database tests on PostgreSQL, Security audit and Stylesheet, icons and scripts current. A tag whose commit failed any of them is released, and its image built, as if CI were green.

Where:
- REQUIRED_JOBS in scripts/release_tools.py:114-120 holds ^CI / Unit tests and ^CI / Browser tests only; its comment (:109-113) explains why the dev image job is excluded and says nothing of the others. ci_problems (:123-143) reads nothing else.
- The gate is called from .forgejo/workflows/release.yml:59 and .forgejo/workflows/image.yml:67.
- The jobs it skips, all of which run on every push with no condition: postgres (.forgejo/workflows/ci.yml:148-262, the real backup and restore at :234-262), security (:264-326: pip-audit, npm audit, zizmor) and styles (:328-366).
- tests/test_release_tools.py:242-251: the GREEN fixture holds a green Security audit and no PostgreSQL or stylesheet context, so no test shows what happens when any of the three is red or missing.
- CONTRIBUTING.md:347-351 and the comment at .forgejo/workflows/release.yml:52-54 describe the gate as "any test leg or the browser job".
What happens:
- A change breaks migrations on PostgreSQL, or the PostgreSQL half of core/backup.py (pg_dump/pg_restore). The PostgreSQL job is the only place either runs for real, so it goes red while the SQLite legs and the browser stay green. The maintainer tags. release_tools.py check --ci reports nothing, release.yml publishes the sdist and the wheel, image.yml passes the same check and pushes the image, and PostgreSQL operators upgrade into the failure #219 was opened for.
- A locked dependency gets a published advisory, or zizmor finds a fault in a workflow. Only Security audit turns red, and the release goes out the same way.
- The committed static/css/app.css, icons or vendored scripts are stale. The wheel and the image ship the committed files, so the release carries assets that do not match their sources.
Why it matters: docker/compose.postgres.yml and the wiki's Installing Postulo page offer PostgreSQL as a deployment, and backup and restore are how an operator keeps their users' data. SECURITY.md:36 says dependencies are "audited against known vulnerabilities in CI on every run". These jobs fail because of what the commit ships, not because of the runner, which is the reason the comment gives for excluding the dev image job; the one check meant to stop such a release does not consult them.
Fix: add to REQUIRED_JOBS:
- ("the PostgreSQL job", re.compile(r"^CI / Database tests on PostgreSQL"));
- ("the stylesheet check", re.compile(r"^CI / Stylesheet"));
- ("the security audit", re.compile(r"^CI / Security audit")), decided deliberately. Requiring it means a newly published advisory holds a release until it is dealt with, which matches SECURITY.md. If it stays optional (an advisory with no fix available should not block a release), say why in the comment above REQUIRED_JOBS instead of leaving the gap unexplained.
Update that comment, the one at .forgejo/workflows/release.yml:52-54 and CONTRIBUTING.md:347-351 to name the required jobs.

Done when

tests/test_release_tools.py: with every unit leg and the browser green and CI / Database tests on PostgreSQL (push) recorded as failure, ci_problems names it; with that context missing, ci_problems reports that no status was found for it. The same two cases hold for CI / Stylesheet, icons and scripts current (push), and for CI / Security audit (push) if the audit is required. All fail today.
GREEN holds every newly required context, and test_no_ci_at_all_is_refused_rather_than_waved_through counts the new entries.
If the audit is left optional, the comment above REQUIRED_JOBS says why.
CONTRIBUTING.md names the jobs a release must have passed.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, comparing the release gate's required jobs in `scripts/release_tools.py` with the jobs in `.forgejo/workflows/ci.yml`. Related to #233 (closed), which added the gate and proposed requiring only `test*` and `browser`, at a time when the PostgreSQL job did not exist; this issue adds the three jobs the list was never revisited for. Also related to #219 (closed), which added the PostgreSQL job, and #319 (closed), which renamed the jobs and noted that a job split out of the test legs should be required too. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The code is now at `.forgejo/workflows/ci.yml:264-335` (`security`) and `.forgejo/workflows/ci.yml:337-375` (`styles`), and the paragraph about the gate at `CONTRIBUTING.md:349-353`; the other locations are unchanged. The release gate refuses a tag unless named CI jobs passed on its commit, and it names only the unit-test legs and the browser job. Three other jobs check things a release ships and are not required: *Database tests on PostgreSQL*, *Security audit* and *Stylesheet, icons and scripts current*. A tag whose commit failed any of them is released, and its image built, as if CI were green. - **Where:** - `REQUIRED_JOBS` in `scripts/release_tools.py:114-120` holds `^CI / Unit tests` and `^CI / Browser tests` only; its comment (`:109-113`) explains why the dev image job is excluded and says nothing of the others. `ci_problems` (`:123-143`) reads nothing else. - The gate is called from `.forgejo/workflows/release.yml:59` and `.forgejo/workflows/image.yml:67`. - The jobs it skips, all of which run on every push with no condition: `postgres` (`.forgejo/workflows/ci.yml:148-262`, the real backup and restore at `:234-262`), `security` (`:264-326`: pip-audit, npm audit, zizmor) and `styles` (`:328-366`). - `tests/test_release_tools.py:242-251`: the `GREEN` fixture holds a green *Security audit* and no PostgreSQL or stylesheet context, so no test shows what happens when any of the three is red or missing. - `CONTRIBUTING.md:347-351` and the comment at `.forgejo/workflows/release.yml:52-54` describe the gate as "any test leg or the browser job". - **What happens:** - A change breaks migrations on PostgreSQL, or the PostgreSQL half of `core/backup.py` (`pg_dump`/`pg_restore`). The PostgreSQL job is the only place either runs for real, so it goes red while the SQLite legs and the browser stay green. The maintainer tags. `release_tools.py check --ci` reports nothing, `release.yml` publishes the sdist and the wheel, `image.yml` passes the same check and pushes the image, and PostgreSQL operators upgrade into the failure #219 was opened for. - A locked dependency gets a published advisory, or zizmor finds a fault in a workflow. Only *Security audit* turns red, and the release goes out the same way. - The committed `static/css/app.css`, icons or vendored scripts are stale. The wheel and the image ship the committed files, so the release carries assets that do not match their sources. - **Why it matters:** `docker/compose.postgres.yml` and the wiki's *Installing Postulo* page offer PostgreSQL as a deployment, and backup and restore are how an operator keeps their users' data. `SECURITY.md:36` says dependencies are "audited against known vulnerabilities in CI on every run". These jobs fail because of what the commit ships, not because of the runner, which is the reason the comment gives for excluding the dev image job; the one check meant to stop such a release does not consult them. - **Fix:** add to `REQUIRED_JOBS`: - `("the PostgreSQL job", re.compile(r"^CI / Database tests on PostgreSQL"))`; - `("the stylesheet check", re.compile(r"^CI / Stylesheet"))`; - `("the security audit", re.compile(r"^CI / Security audit"))`, decided deliberately. Requiring it means a newly published advisory holds a release until it is dealt with, which matches `SECURITY.md`. If it stays optional (an advisory with no fix available should not block a release), say why in the comment above `REQUIRED_JOBS` instead of leaving the gap unexplained. Update that comment, the one at `.forgejo/workflows/release.yml:52-54` and `CONTRIBUTING.md:347-351` to name the required jobs. ## Done when - `tests/test_release_tools.py`: with every unit leg and the browser green and `CI / Database tests on PostgreSQL (push)` recorded as `failure`, `ci_problems` names it; with that context missing, `ci_problems` reports that no status was found for it. The same two cases hold for `CI / Stylesheet, icons and scripts current (push)`, and for `CI / Security audit (push)` if the audit is required. All fail today. - `GREEN` holds every newly required context, and `test_no_ci_at_all_is_refused_rather_than_waved_through` counts the new entries. - If the audit is left optional, the comment above `REQUIRED_JOBS` says why. - `CONTRIBUTING.md` names the jobs a release must have passed.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:46:34 +00:00

tiagoagueda added the

enhancement

tier

labels

2026-10-01 10:46:34 +00:00

tiagoagueda referenced this issue from a commit

2026-10-06 06:21:08 +00:00

Require the PostgreSQL job and the security audit for a release

tiagoagueda commented

2026-10-06 09:32:17 +00:00