Image: a README change re-installs every dependency, and each re-install downloads every wheel again #1140

New issue

Closed

opened 2026-10-31 03:04:18 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-31 03:04:18 +00:00

Owner

What exists today

docker/Dockerfile:61 copies pyproject.toml uv.lock README.md before the first uv sync at :90, so the dependency layer's cache key includes README.md. pyproject.toml:5 names it as the readme, which only the project build needs, and that is the second sync at :95, after COPY src. .forgejo/workflows/readme.yml (#1044) rewrites README.md whenever translations land.

Measured with git log --since=2026-09-30: 33 commits touched README.md, uv.lock or pyproject.toml; 11 of them touched README.md and neither of the other two. Each of those rebuilt the dependency layer for both platforms in both builders for nothing. Checked locally that the first sync does not need the README: in a scratch directory holding only pyproject.toml and uv.lock, uv sync --locked --no-default-groups --extra server --extra postgres --no-install-project --dry-run resolves the full set without complaint.

:90 and :95 pass --no-cache (the comment at :71-75 gives the reason: the 296 MB cache of #154). The build stage no longer ships, so the flag now only means each rebuild of the layer downloads every wheel again.

What would change

Move README.md from :61 to just before the second sync at :95 (with COPY src). This needs no decision; no test reads that COPY line.
Replace --no-cache on both syncs with RUN --mount=type=cache,target=/root/.cache/uv (uv's own Docker guide shows this form; UV_LINK_MODE=copy is already set, which a cache mount needs). The cache lives in the builder's storage, never in a layer, so the image is unchanged; the runner keeps one more cache directory per builder. Same for the uv pip install of POSTULO_EXTRA_PACKAGES at :131. tests/test_image_build.py:116 asserts --no-cache on every sync line and would be rewritten to assert the cache mount instead (and that no cache directory reaches the runtime stage).

The candidate holds on both counts; the change is that the README part is now measured and stands alone, and only the cache mount needs the user.

What it saves or protects

small-medium: 11 of the 33 commits in the last month that invalidated the dependency layer touched only README.md (the uv sync layer is 162 s of arm64 under QEMU per #1033, plus its amd64 copy in both builders); a uv cache mount would make the lock changes that remain mostly local copies

Accessibility

none

Decided by the maintainer (2026-10-31)

README out of the dependency layer AND a BuildKit uv cache mount instead of --no-cache, one PR

Decisions

One pull request for both? Recommended: yes; both touch the same two lines.
Cache mount instead of --no-cache? Recommended: yes. The reason --no-cache was added (a cache shipped in the image) no longer applies since the multi-stage build; the comment at :71-75 would be rewritten to say so.

Found by the efficiency and reliability audit of 2026-10-30 (finding I4; index postulo/postulo#1108). Checked against main at daee30b15 (2026-10-30).

## What exists today `docker/Dockerfile:61` copies `pyproject.toml uv.lock README.md` before the first `uv sync` at `:90`, so the dependency layer's cache key includes README.md. `pyproject.toml:5` names it as the readme, which only the project build needs, and that is the second sync at `:95`, after `COPY src`. `.forgejo/workflows/readme.yml` (#1044) rewrites README.md whenever translations land. Measured with `git log --since=2026-09-30`: 33 commits touched README.md, uv.lock or pyproject.toml; 11 of them touched README.md and neither of the other two. Each of those rebuilt the dependency layer for both platforms in both builders for nothing. Checked locally that the first sync does not need the README: in a scratch directory holding only `pyproject.toml` and `uv.lock`, `uv sync --locked --no-default-groups --extra server --extra postgres --no-install-project --dry-run` resolves the full set without complaint. `:90` and `:95` pass `--no-cache` (the comment at `:71-75` gives the reason: the 296 MB cache of #154). The build stage no longer ships, so the flag now only means each rebuild of the layer downloads every wheel again. ## What would change 1. Move `README.md` from `:61` to just before the second sync at `:95` (with `COPY src`). This needs no decision; no test reads that COPY line. 2. Replace `--no-cache` on both syncs with `RUN --mount=type=cache,target=/root/.cache/uv` (uv's own Docker guide shows this form; `UV_LINK_MODE=copy` is already set, which a cache mount needs). The cache lives in the builder's storage, never in a layer, so the image is unchanged; the runner keeps one more cache directory per builder. Same for the `uv pip install` of `POSTULO_EXTRA_PACKAGES` at `:131`. `tests/test_image_build.py:116` asserts `--no-cache` on every sync line and would be rewritten to assert the cache mount instead (and that no cache directory reaches the runtime stage). The candidate holds on both counts; the change is that the README part is now measured and stands alone, and only the cache mount needs the user. ## What it saves or protects small-medium: 11 of the 33 commits in the last month that invalidated the dependency layer touched only README.md (the `uv sync` layer is 162 s of arm64 under QEMU per #1033, plus its amd64 copy in both builders); a uv cache mount would make the lock changes that remain mostly local copies ## Accessibility none ## Decided by the maintainer (2026-10-31) README out of the dependency layer AND a BuildKit uv cache mount instead of --no-cache, one PR ## Decisions - **One pull request for both?** Recommended: yes; both touch the same two lines. - **Cache mount instead of `--no-cache`?** Recommended: yes. The reason `--no-cache` was added (a cache shipped in the image) no longer applies since the multi-stage build; the comment at `:71-75` would be rewritten to say so. --- Found by the efficiency and reliability audit of 2026-10-30 (finding I4; index postulo/postulo#1108). Checked against `main` at `daee30b15` (2026-10-30).

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 03:04:18 +00:00

tiagoagueda added the

performance

enhancement

labels

2026-10-31 03:04:18 +00:00

tiagoagueda referenced this issue

2026-10-31 03:04:41 +00:00

Efficiency and reliability audit of 2026-10-30: index #1108

tiagoagueda referenced this issue from a commit

2026-11-10 11:31:09 +00:00