A browser smoke test of the critical path in CI, with Playwright #36

New issue

Closed

opened 2026-09-05 14:04:21 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-09-05 14:04:21 +00:00

Owner

Why

CI (.forgejo/workflows/ci.yml) runs lint, a missing-migrations check, the test suite with coverage on three Python versions, the production deployment check and the stylesheet freshness check. It never opens a browser. The 0.2.0 milestone rewrites the header (#10), adds a settings area (#22), a new stage in the workflow (#25), table behaviour driven by htmx (#20) and a theme switch in JavaScript (#9): exactly the changes that unit tests pass through and a person notices in ten seconds.

Playwright is already an optional dependency (chromium extra, for the PDF fallback), and pytest-django provides live_server.

Shape

tests/e2e/, marked e2e, excluded from the default run (-m "not e2e") so uv run pytest stays fast; run with -m e2e locally after playwright install chromium.
One path, end to end, against live_server with the test settings: sign in → capture by pasting a page's HTML → review → the application appears → change its status on the board → record what was sent (a WeasyPrint render) → download the export archive and check its manifest. Assertions on visible text and roles, not on CSS classes, so a restyle does not fail it.
A CI job e2e on one Python version, in the same node:22-bookworm container, with playwright install --with-deps chromium (Debian bookworm is a supported target). About two minutes. On failure it uploads the screenshot and the Playwright trace — with actions/upload-artifact@v3, which works on Forgejo Actions; v4 depends on GitHub's own backend and does not.
Accessibility as a second step: an axe run (axe-playwright-python) on the dashboard, the table, the board and the settings pages, failing on serious violations. The interface is server-rendered and already careful about focus and labels; a check keeps it that way when nobody is looking.
seed_demo gives the test a populated instance for the board and table steps instead of a dozen fixtures.

Classification

Enhancement, to the test suite. Not breaking.

Open questions

Run the browser job on every push, or only on pull requests and tags? Proposal: every push to main and every pull request; two minutes is cheap next to a broken header on the demo instance.
Firefox as a second browser in the matrix, given #18? Chromium only until the extension exists.

## Why CI (`.forgejo/workflows/ci.yml`) runs lint, a missing-migrations check, the test suite with coverage on three Python versions, the production deployment check and the stylesheet freshness check. It never opens a browser. The 0.2.0 milestone rewrites the header (#10), adds a settings area (#22), a new stage in the workflow (#25), table behaviour driven by htmx (#20) and a theme switch in JavaScript (#9): exactly the changes that unit tests pass through and a person notices in ten seconds. Playwright is already an optional dependency (`chromium` extra, for the PDF fallback), and `pytest-django` provides `live_server`. ## Shape - **`tests/e2e/`**, marked `e2e`, excluded from the default run (`-m "not e2e"`) so `uv run pytest` stays fast; run with `-m e2e` locally after `playwright install chromium`. - **One path, end to end**, against `live_server` with the test settings: sign in → capture by pasting a page's HTML → review → the application appears → change its status on the board → record what was sent (a WeasyPrint render) → download the export archive and check its manifest. Assertions on visible text and roles, not on CSS classes, so a restyle does not fail it. - **A CI job `e2e`** on one Python version, in the same `node:22-bookworm` container, with `playwright install --with-deps chromium` (Debian bookworm is a supported target). About two minutes. On failure it uploads the screenshot and the Playwright trace — with `actions/upload-artifact@v3`, which works on Forgejo Actions; v4 depends on GitHub's own backend and does not. - **Accessibility as a second step**: an axe run (`axe-playwright-python`) on the dashboard, the table, the board and the settings pages, failing on serious violations. The interface is server-rendered and already careful about focus and labels; a check keeps it that way when nobody is looking. - **`seed_demo`** gives the test a populated instance for the board and table steps instead of a dozen fixtures. ## Classification Enhancement, to the test suite. Not breaking. ## Open questions 1. Run the browser job on every push, or only on pull requests and tags? Proposal: every push to `main` and every pull request; two minutes is cheap next to a broken header on the demo instance. 2. Firefox as a second browser in the matrix, given #18? Chromium only until the extension exists.

tiagoagueda added this to the 0.2.0 milestone

2026-09-05 14:04:21 +00:00

tiagoagueda added the

enhancement

tier

labels

2026-09-05 14:04:21 +00:00

tiagoagueda referenced this issue from a commit

2026-09-05 14:24:01 +00:00

Drive the critical path in a browser on every push

tiagoagueda closed this issue

2026-09-05 14:24:01 +00:00