Unit tests: three tests pass only when the machine is idle #719

New issue

Closed

opened 2026-10-04 20:43:13 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-04 20:43:13 +00:00

Owner

robots.txt (tests/security/test_outbound.py:1135-1146). The deadline is patched to 0.15 s and the test asserts asked == ["/robots.txt"], but the server thread appends to asked only after it has read the whole request (:1058-1065). On a loaded machine the client gives up first, the fetch allows, and asked is still empty. Seen failing while other suites ran (0.6.0 bug sweep). The server should say when it has the request, and the test wait for that rather than race it.
TOTP (tests/test_mfa.py:19-34). The helper accepts only the current code and sleeps up to 5.5 s to keep clear of a 30-second boundary. That is slow on every run and still fails if more than the margin passes between making and posting the code. It builds the code from a clock the test controls instead.
A page in under a second (tests/test_tables.py:689-691). A full GET must finish in 1 s, on the leg that measures coverage, on a shared host. What it protects is that the page does not do work per row; a bound on the number of queries says that without depending on the machine.

Done when

None of the three reads the wall clock to decide whether it passed.

Measured from Forgejo's task API and job logs, runs 561–710 (24 Sep – 4 Oct 2026): wall time per push about 52–55 min, up from about 15; the browser suite grew from 149 to 645 tests in five days and runs serially (29 min); the runner takes **two jobs at a time, in the order the workflow file lists them**, so the browser job starts 19–22 min after the push; and Forgejo cancels the older push run when a newer push lands, which stopped 18 of 56 browser jobs before they gave a verdict. - **robots.txt** (`tests/security/test_outbound.py:1135-1146`). The deadline is patched to 0.15 s and the test asserts `asked == ["/robots.txt"]`, but the server thread appends to `asked` only after it has read the whole request (`:1058-1065`). On a loaded machine the client gives up first, the fetch allows, and `asked` is still empty. Seen failing while other suites ran (0.6.0 bug sweep). The server should say when it has the request, and the test wait for that rather than race it. - **TOTP** (`tests/test_mfa.py:19-34`). The helper accepts only the current code and sleeps up to 5.5 s to keep clear of a 30-second boundary. That is slow on every run and still fails if more than the margin passes between making and posting the code. It builds the code from a clock the test controls instead. - **A page in under a second** (`tests/test_tables.py:689-691`). A full GET must finish in 1 s, on the leg that measures coverage, on a shared host. What it protects is that the page does not do work per row; a bound on the number of queries says that without depending on the machine. ## Done when - None of the three reads the wall clock to decide whether it passed.

tiagoagueda added the

bug

label

2026-10-04 20:43:13 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 22:21:24 +00:00

Let three unit tests pass on a loaded machine as on an idle one

tiagoagueda closed this issue

2026-10-04 22:21:24 +00:00