Image: no workflow starts the image before publishing it, and the start check depends on GeoNames answering #1143

New issue

Closed

opened 2026-10-31 03:04:19 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-31 03:04:19 +00:00

Owner

What exists today

.forgejo/workflows/dev-image.yml:134-162 builds, scans with scripts/scan-image.sh, then builds both platforms and pushes; image.yml (build, scan, release_tools.py check, push, verify-image) is the same shape. Neither ever runs a container. The gates are CI on the commit, the scan and, for a release, that the registry serves the digest. Ragnar follows :dev, so the first start of a new dev image happens there.

scripts/check-image.sh already does the boot test: it reads the compose files (:37-40), builds (:46), starts the image with a published port and waits 60 s for /healthz (:50-73), then starts it as a deployment does and waits up to 180 s for Docker's own health check (:82-109). image.yml:5 and CONTRIBUTING.md:797 describe it as something run by hand where a daemon exists; the docker runner that both image workflows use is exactly such a place. The script builds its own image, so a workflow would need it to accept an existing tag (it takes TAG as $1 but always builds).

The GeoNames part of the candidate holds: docker/entrypoint.sh:31-36 runs python manage.py fetch_geonames before exec gunicorn whenever geonames-cities1000.txt is missing from the volume, which on a fresh container it always is. check-image.sh sets neither POSTULO_SKIP_GEONAMES=1 nor a volume, so each of its two containers downloads and imports the city table before it answers. That puts download.geonames.org inside the check and eats into the first container's 60 s budget; a slow mirror fails the check for a reason that has nothing to do with the image.

What would change

check-image.sh: pass -e POSTULO_SKIP_GEONAMES=1 to both docker run calls, and take an option (for example CHECK_IMAGE_BUILD=0 or a --no-build flag) to check a tag that is already built.
dev-image.yml and image.yml: a step after the scan, "Start it and wait for it to be healthy", running scripts/check-image.sh --no-build postulo:dev-scan (or postulo:scan), before the multi-platform push. It checks the amd64 image only; the arm64 build is the same Dockerfile, so this catches what the candidate is about. With I1 (one builder, #1033 item 5) the checked image is also the published amd64 one.
tests/test_image_build.py asserts that the step exists and sits before the push.

What it saves or protects

medium (reliability): an image that builds and scans clean but cannot start (an entrypoint, a settings check, a migration on an empty volume) is stopped before :dev or a release tag moves, for about a minute of the docker runner

Accessibility

none

Decisions

Boot test in both workflows? Recommended: yes; the dev one matters most, since Ragnar follows :dev and is otherwise where a start-up failure is first seen (#580, an unhealthy container behind a 400 on its own health check, is the kind check-image.sh was extended to catch).

Found by the efficiency and reliability audit of 2026-10-30 (finding I8; index postulo/postulo#1108). Checked against main at daee30b15 (2026-10-30).

## What exists today `.forgejo/workflows/dev-image.yml:134-162` builds, scans with `scripts/scan-image.sh`, then builds both platforms and pushes; `image.yml` (build, scan, `release_tools.py check`, push, `verify-image`) is the same shape. Neither ever runs a container. The gates are CI on the commit, the scan and, for a release, that the registry serves the digest. Ragnar follows `:dev`, so the first start of a new dev image happens there. `scripts/check-image.sh` already does the boot test: it reads the compose files (`:37-40`), builds (`:46`), starts the image with a published port and waits 60 s for `/healthz` (`:50-73`), then starts it as a deployment does and waits up to 180 s for Docker's own health check (`:82-109`). `image.yml:5` and `CONTRIBUTING.md:797` describe it as something run by hand where a daemon exists; the `docker` runner that both image workflows use is exactly such a place. The script builds its own image, so a workflow would need it to accept an existing tag (it takes `TAG` as `$1` but always builds). The GeoNames part of the candidate holds: `docker/entrypoint.sh:31-36` runs `python manage.py fetch_geonames` before `exec gunicorn` whenever `geonames-cities1000.txt` is missing from the volume, which on a fresh container it always is. `check-image.sh` sets neither `POSTULO_SKIP_GEONAMES=1` nor a volume, so each of its two containers downloads and imports the city table before it answers. That puts download.geonames.org inside the check and eats into the first container's 60 s budget; a slow mirror fails the check for a reason that has nothing to do with the image. ## What would change - `check-image.sh`: pass `-e POSTULO_SKIP_GEONAMES=1` to both `docker run` calls, and take an option (for example `CHECK_IMAGE_BUILD=0` or a `--no-build` flag) to check a tag that is already built. - `dev-image.yml` and `image.yml`: a step after the scan, "Start it and wait for it to be healthy", running `scripts/check-image.sh --no-build postulo:dev-scan` (or `postulo:scan`), before the multi-platform push. It checks the amd64 image only; the arm64 build is the same Dockerfile, so this catches what the candidate is about. With I1 (one builder, #1033 item 5) the checked image is also the published amd64 one. - `tests/test_image_build.py` asserts that the step exists and sits before the push. ## What it saves or protects medium (reliability): an image that builds and scans clean but cannot start (an entrypoint, a settings check, a migration on an empty volume) is stopped before `:dev` or a release tag moves, for about a minute of the docker runner ## Accessibility none ## Decisions - **Boot test in both workflows?** Recommended: yes; the dev one matters most, since Ragnar follows `:dev` and is otherwise where a start-up failure is first seen (#580, an unhealthy container behind a 400 on its own health check, is the kind `check-image.sh` was extended to catch). --- Found by the efficiency and reliability audit of 2026-10-30 (finding I8; index postulo/postulo#1108). Checked against `main` at `daee30b15` (2026-10-30).

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 03:04:19 +00:00

tiagoagueda added the

performance

bug

labels

2026-10-31 03:04:19 +00:00

tiagoagueda referenced this issue

2026-10-31 03:04:41 +00:00

Efficiency and reliability audit of 2026-10-30: index #1108

tiagoagueda referenced this issue from a commit

2026-11-10 11:31:09 +00:00