Name every CI job for what it checks: three are called test and two image #319

New issue

Closed

opened 2026-09-29 13:36:52 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-09-29 13:36:52 +00:00

Owner

Asked for on 2026-09-29: several CI jobs are called test; name each for what it checks, so that a failed one says what failed.

What is there today

No job in .forgejo/workflows/ has a name:, so Forgejo shows each by its id:

Workflow	Job id	Shown as	What it runs
`ci.yml`	`test`	`test` three times, one per Python in the matrix (`CI / test (3.12) (push)` in the commit status)	ruff, the migrations check, the catalogue checks, the unit suite on 3.12 and 3.13, the unit suite with coverage and its floor on 3.14, and the production deployment check
`ci.yml`	`postgres`	`postgres`	the database-facing tests against PostgreSQL, and a real backup and restore
`ci.yml`	`security`	`security`	pip-audit, npm audit, zizmor
`ci.yml`	`styles`	`styles`	that the committed stylesheet, icons and vendored scripts are current
`ci.yml`	`browser`	`browser`	the browser suite in Chromium, axe included
`dev-image.yml`	`image`	`image`	build, scan and push the dev image
`image.yml`	`image`	`image`	build, scan and push a release image
`dependencies.yml`	`upgrade`	`upgrade`	the monthly lock-file upgrade and a run of the suite against it
`release.yml`	`release`	`release`	the release itself

A red test can mean a lint error, a missing migration, a stale catalogue, a failing test, the coverage floor or the deployment check, and it does not say which Python it was without opening it. Two jobs are called image.

What to do

Give every job a name: that says what it checks, keeping the ids (other files read them by id). A proposal:

Job	Name
`ci.yml` `test`	`Unit tests (Python ${{ matrix.python-version }})`, and `Unit tests and coverage (Python 3.14)` for the leg that measures it
`ci.yml` `postgres`	`Database tests on PostgreSQL`
`ci.yml` `security`	`Security audit`
`ci.yml` `styles`	`Stylesheet, icons and scripts current`
`ci.yml` `browser`	`Browser tests (Chromium)`
`dev-image.yml` `image`	`Dev image: build, scan, push`
`image.yml` `image`	`Release image: build, scan, push`
`dependencies.yml` `upgrade`	`Monthly dependency upgrade`
`release.yml` `release`	`Publish the release`

Worth deciding with it: the checks that are not tests. ruff, the migrations check and the catalogue checks run inside every test leg, three times, and a failure in one of them is reported as a failed test on every Python at once. A job of their own, Lint, migrations and translations, run once, would name that failure for what it is and save two runs of each on a runner the three legs already share (#233).

What depends on the names

The release gate. scripts/release_tools.py::REQUIRED_JOBS finds the jobs a release must have passed by the commit status's name: ^CI / test \( for every test leg and ^CI / browser for the browser. The release and image workflows refuse a tag unless it finds them green, so renaming the jobs without updating those patterns refuses every tag. tests/test_release_tools.py pins the contexts it expects; they change with it. If the checks move to a job of their own, the gate should require it too.
The commit statuses become CI / Unit tests (Python 3.12) (push) and so on. Nothing else here reads them: there is no branch protection rule on main today.
Forgejo and matrix values in a name. GitHub evaluates ${{ matrix.* }} in jobs.<id>.name; confirm that this Forgejo does (the first run's statuses show it), or name the legs another way.
CONTRIBUTING.md and any wiki page that names a job.

Done when

Every job has a name that says what it checks, and no two jobs share one.
A failed run names the failing check without opening its log.
The release gate finds the renamed jobs, with its tests updated, and a dispatch of Image for a tag whose CI passed is not refused.

Asked for on 2026-09-29: several CI jobs are called `test`; name each for what it checks, so that a failed one says what failed. ## What is there today No job in `.forgejo/workflows/` has a `name:`, so Forgejo shows each by its id: | Workflow | Job id | Shown as | What it runs | | --- | --- | --- | --- | | `ci.yml` | `test` | `test` three times, one per Python in the matrix (`CI / test (3.12) (push)` in the commit status) | ruff, the migrations check, the catalogue checks, the unit suite on 3.12 and 3.13, the unit suite with coverage and its floor on 3.14, and the production deployment check | | `ci.yml` | `postgres` | `postgres` | the database-facing tests against PostgreSQL, and a real backup and restore | | `ci.yml` | `security` | `security` | pip-audit, npm audit, zizmor | | `ci.yml` | `styles` | `styles` | that the committed stylesheet, icons and vendored scripts are current | | `ci.yml` | `browser` | `browser` | the browser suite in Chromium, axe included | | `dev-image.yml` | `image` | `image` | build, scan and push the dev image | | `image.yml` | `image` | `image` | build, scan and push a release image | | `dependencies.yml` | `upgrade` | `upgrade` | the monthly lock-file upgrade and a run of the suite against it | | `release.yml` | `release` | `release` | the release itself | A red `test` can mean a lint error, a missing migration, a stale catalogue, a failing test, the coverage floor or the deployment check, and it does not say which Python it was without opening it. Two jobs are called `image`. ## What to do Give every job a `name:` that says what it checks, keeping the ids (other files read them by id). A proposal: | Job | Name | | --- | --- | | `ci.yml` `test` | `Unit tests (Python ${{ matrix.python-version }})`, and `Unit tests and coverage (Python 3.14)` for the leg that measures it | | `ci.yml` `postgres` | `Database tests on PostgreSQL` | | `ci.yml` `security` | `Security audit` | | `ci.yml` `styles` | `Stylesheet, icons and scripts current` | | `ci.yml` `browser` | `Browser tests (Chromium)` | | `dev-image.yml` `image` | `Dev image: build, scan, push` | | `image.yml` `image` | `Release image: build, scan, push` | | `dependencies.yml` `upgrade` | `Monthly dependency upgrade` | | `release.yml` `release` | `Publish the release` | **Worth deciding with it: the checks that are not tests.** ruff, the migrations check and the catalogue checks run inside every `test` leg, three times, and a failure in one of them is reported as a failed test on every Python at once. A job of their own, `Lint, migrations and translations`, run once, would name that failure for what it is and save two runs of each on a runner the three legs already share (#233). ## What depends on the names - **The release gate.** `scripts/release_tools.py::REQUIRED_JOBS` finds the jobs a release must have passed by the commit status's name: `^CI / test \(` for every test leg and `^CI / browser ` for the browser. The release and image workflows refuse a tag unless it finds them green, so **renaming the jobs without updating those patterns refuses every tag**. `tests/test_release_tools.py` pins the contexts it expects; they change with it. If the checks move to a job of their own, the gate should require it too. - **The commit statuses** become `CI / Unit tests (Python 3.12) (push)` and so on. Nothing else here reads them: there is no branch protection rule on `main` today. - **Forgejo and matrix values in a name.** GitHub evaluates `${{ matrix.* }}` in `jobs.<id>.name`; confirm that this Forgejo does (the first run's statuses show it), or name the legs another way. - `CONTRIBUTING.md` and any wiki page that names a job. ## Done when - Every job has a name that says what it checks, and no two jobs share one. - A failed run names the failing check without opening its log. - The release gate finds the renamed jobs, with its tests updated, and a dispatch of *Image* for a tag whose CI passed is not refused.

tiagoagueda added this to the 0.5.0 milestone

2026-09-29 13:36:52 +00:00

tiagoagueda added the

enhancement

label

2026-09-29 13:36:52 +00:00

tiagoagueda referenced this issue from a commit

2026-09-29 16:46:42 +00:00

Name every CI job for what it checks, and have the release gate follow

tiagoagueda closed this issue

2026-09-29 16:46:42 +00:00

tiagoagueda referenced this issue