Contributing: say what belongs in the browser suite and how to keep it fast

tiagoagueda commented

2026-10-04 20:43:15 +00:00

Owner

Measured from Forgejo's task API and job logs, runs 561–710 (24 Sep – 4 Oct 2026): wall time per push about 52–55 min, up from about 15; the browser suite grew from 149 to 645 tests in five days and runs serially (29 min); the runner takes two jobs at a time, in the order the workflow file lists them, so the browser job starts 19–22 min after the push; and Forgejo cancels the older push run when a newer push lands, which stopped 18 of 56 browser jobs before they gave a verdict.

The browser suite grew more than fourfold in five days because every change added browser tests, many of which check something the Django test client could check without a browser. CONTRIBUTING.md:39-45 and CLAUDE.md:61 still describe it as "the critical path".

Proposal

A Browser tests paragraph in CONTRIBUTING.md, and the matching line in CLAUDE.md:

A browser test is for what needs a browser: scripts, layout, focus, the content security policy. Markup, permissions and redirects belong in the unit suite.
A new page joins the walk (signed_in_paths()), and a new per-page check joins the walk's passes instead of walking the site again.
Sign in with the shared helper; fill the form only in a test about the form.
Wait for state with expect; a fixed wait needs a comment saying what time it measures.
The CI log ends with the 25 slowest tests: read it when adding to the suite.

Then, with about fifteen pushes measured after these changes, the runner's capacity (two jobs at a time today) is decided.

Done when

Both files describe the browser suite as it is, and say how to add to it.

Measured from Forgejo's task API and job logs, runs 561–710 (24 Sep – 4 Oct 2026): wall time per push about 52–55 min, up from about 15; the browser suite grew from 149 to 645 tests in five days and runs serially (29 min); the runner takes **two jobs at a time, in the order the workflow file lists them**, so the browser job starts 19–22 min after the push; and Forgejo cancels the older push run when a newer push lands, which stopped 18 of 56 browser jobs before they gave a verdict. The browser suite grew more than fourfold in five days because every change added browser tests, many of which check something the Django test client could check without a browser. `CONTRIBUTING.md:39-45` and `CLAUDE.md:61` still describe it as "the critical path". ## Proposal A *Browser tests* paragraph in `CONTRIBUTING.md`, and the matching line in `CLAUDE.md`: - A browser test is for what needs a browser: scripts, layout, focus, the content security policy. Markup, permissions and redirects belong in the unit suite. - A new page joins the walk (`signed_in_paths()`), and a new per-page check joins the walk's passes instead of walking the site again. - Sign in with the shared helper; fill the form only in a test about the form. - Wait for state with `expect`; a fixed wait needs a comment saying what time it measures. - The CI log ends with the 25 slowest tests: read it when adding to the suite. Then, with about fifteen pushes measured after these changes, the runner's capacity (two jobs at a time today) is decided. ## Done when - Both files describe the browser suite as it is, and say how to add to it.

tiagoagueda added the

documentation

label

2026-10-04 20:43:15 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 23:41:37 +00:00

Run the browser tests on one process again, and log the host's load

tiagoagueda referenced this issue from a commit

2026-10-05 00:00:26 +00:00

Say what belongs in the browser suite and how to keep it fast

tiagoagueda added this to the 0.5.0 milestone

2026-10-25 17:04:12 +00:00