Release gate: nothing runs `pytest -m release`, and it fails today on 21 of the 23 catalogues #11

New issue

Closed

opened 2026-10-01 10:48:15 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:48:15 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo-helloworld at 19f0bcc29, comparing its workflows and catalogue tests with those of postulo-apprise and postulo-paperless, and running the catalogue check against the snapshot.

The README says uv run pytest -m release "holds the twenty-four European Union catalogues complete before a release". The marker is deselected by default and no workflow selects it, so the gate runs only if somebody remembers to. It has already drifted: 21 of the 23 European Union catalogues are incomplete, and CI is green.

Where: pyproject.toml:58-64 (addopts = "--strict-markers -m 'not release'"); .forgejo/workflows/ci.yml:14-17 (on: push: branches: [main] and pull_request, no tags) and .forgejo/workflows/ci.yml:67-68 (uv run pytest); there is no release.yml; tests/test_catalogues.py:61-65 (test_every_european_union_language_stays_complete); tests/test_catalogues.py:15-17 (pytest.importorskip); README.md:55-56.
What happens:
1. ci.yml runs uv run pytest, which deselects the release test. No other workflow exists, and ci.yml does not run on a tag. postulo-apprise, postulo-paperless, postulo-imap and postulo-dav each have a release.yml triggered by tags: ["v*"] that runs uv run pytest -m release; this repository was not given one.
2. Parsing the catalogues in the snapshot with the core's messages_tool shows every European Union language except fr-fr and pt-pt with three untranslated entries: "File name", "A name, not a path: …" and "Give the name of a file on its own, not a path: %(name)s is not one." These are the strings the file-name field brought in (#1). pyproject.toml already says version = "0.3.0".
3. tests/test_catalogues.py:15 still opens with pytest.importorskip("postulo.core.messages_tool", …), the pattern postulo/postulo-apprise#1 and postulo/postulo-paperless#2 replaced because a core without the tool turned the release tests into a skip. Here that cannot pass unnoticed as the repository stands: tests/test_surface.py:27 imports postulo.plugins.testing at module level, which is newer than the catalogue tool, so such a core fails collection. It is still the wrong shape to hand to people who copy the file, and a copy that drops the surface test inherits a gate that skips.
Why it matters: a version of the reference plugin can be tagged with most of its catalogues incomplete and nothing is red. Authors copy this repository, so they copy a gate nobody runs.
Fix: add .forgejo/workflows/release.yml as in postulo-paperless (trigger tags: ["v*"], run uv run pytest -m release). Replace the importorskip with the paperless shape (postulo-paperless/tests/test_catalogues.py:58-71): import in a try, and in the tool fixture call pytest.fail under the release marker and pytest.skip otherwise. Fill the three entries in the 21 catalogues. If the reference plugin is deliberately never tagged, say so in the README instead of promising the gate.

Done when

uv run pytest -m release passes: tests/test_catalogues.py::test_every_european_union_language_stays_complete fails today for 21 language codes.
.forgejo/workflows/release.yml runs that command on a v* tag.
With messages_tool set to None, the tool fixture in tests/test_catalogues.py fails a test carrying the release marker and skips one without it.

Found on 2026-09-29 in a read-only review of postulo-helloworld at 19f0bcc29, comparing its workflows and catalogue tests with those of postulo-apprise and postulo-paperless, and running the catalogue check against the snapshot. The README says `uv run pytest -m release` "holds the twenty-four European Union catalogues complete before a release". The marker is deselected by default and no workflow selects it, so the gate runs only if somebody remembers to. It has already drifted: 21 of the 23 European Union catalogues are incomplete, and CI is green. - **Where:** `pyproject.toml:58-64` (`addopts = "--strict-markers -m 'not release'"`); `.forgejo/workflows/ci.yml:14-17` (`on: push: branches: [main]` and `pull_request`, no tags) and `.forgejo/workflows/ci.yml:67-68` (`uv run pytest`); there is no `release.yml`; `tests/test_catalogues.py:61-65` (`test_every_european_union_language_stays_complete`); `tests/test_catalogues.py:15-17` (`pytest.importorskip`); `README.md:55-56`. - **What happens:** 1. `ci.yml` runs `uv run pytest`, which deselects the release test. No other workflow exists, and `ci.yml` does not run on a tag. postulo-apprise, postulo-paperless, postulo-imap and postulo-dav each have a `release.yml` triggered by `tags: ["v*"]` that runs `uv run pytest -m release`; this repository was not given one. 2. Parsing the catalogues in the snapshot with the core's `messages_tool` shows every European Union language except `fr-fr` and `pt-pt` with three untranslated entries: "File name", "A name, not a path: …" and "Give the name of a file on its own, not a path: %(name)s is not one." These are the strings the file-name field brought in (#1). `pyproject.toml` already says `version = "0.3.0"`. 3. `tests/test_catalogues.py:15` still opens with `pytest.importorskip("postulo.core.messages_tool", …)`, the pattern postulo/postulo-apprise#1 and postulo/postulo-paperless#2 replaced because a core without the tool turned the release tests into a skip. Here that cannot pass unnoticed as the repository stands: `tests/test_surface.py:27` imports `postulo.plugins.testing` at module level, which is newer than the catalogue tool, so such a core fails collection. It is still the wrong shape to hand to people who copy the file, and a copy that drops the surface test inherits a gate that skips. - **Why it matters:** a version of the reference plugin can be tagged with most of its catalogues incomplete and nothing is red. Authors copy this repository, so they copy a gate nobody runs. - **Fix:** add `.forgejo/workflows/release.yml` as in postulo-paperless (trigger `tags: ["v*"]`, run `uv run pytest -m release`). Replace the `importorskip` with the paperless shape (`postulo-paperless/tests/test_catalogues.py:58-71`): import in a `try`, and in the `tool` fixture call `pytest.fail` under the `release` marker and `pytest.skip` otherwise. Fill the three entries in the 21 catalogues. If the reference plugin is deliberately never tagged, say so in the README instead of promising the gate. ## Done when - `uv run pytest -m release` passes: `tests/test_catalogues.py::test_every_european_union_language_stays_complete` fails today for 21 language codes. - `.forgejo/workflows/release.yml` runs that command on a `v*` tag. - With `messages_tool` set to `None`, the `tool` fixture in `tests/test_catalogues.py` fails a test carrying the `release` marker and skips one without it.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:48:15 +00:00

tiagoagueda added the

bug

label

2026-10-01 10:48:15 +00:00

tiagoagueda referenced this issue

2026-10-17 10:39:57 +00:00

SEC2-065: The dev lock pins a core commit 211 behind and carries pyjwt 2.13.0 and urllib3 2.7.0 with published advisories #14

tiagoagueda referenced this issue from a commit

2026-10-19 07:08:42 +00:00

Run the release gate on a tag, and make a missing catalogue tool fail it