ESCO classification: after fetch_esco writes a new revision, every listing save in a fresh process fails #535

New issue

Closed

opened 2026-10-01 10:47:27 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:47:27 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading jobs/esco.py and fetch_esco, and checking the loader against two revisions in one directory.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The callers are now at jobs/models.py:974 (JobPosting.save), jobs/models.py:993 (isco_name) and jobs/forms.py:749 (esco.suggestions()); jobs/esco.py, fetch_esco.py and the other locations have not moved.

manage.py fetch_esco --revision <new> writes the new revision's two files beside the old ones and then only prints a warning asking for the old ones to be deleted. From that moment the loader refuses to read either, and because the refusal is an exception, lru_cache does not keep it: every call in a process that had not already loaded the classification raises again. The upgrade the wiki describes, run with docker compose exec against a running instance, therefore breaks saving listings until somebody deletes the old files, and nothing reports the state at start-up.

Where: jobs/esco.py:108-122 (data_file) and jobs/esco.py:296-310 (skills_file), which raise RuntimeError on two files; jobs/esco.py:125-126 (classification, lru_cache) and jobs/esco.py:313-322 (_member, which calls skills_file() on every read, uncached); jobs/management/commands/fetch_esco.py:182-193, which lists the old files in others and only warns; the callers jobs/models.py:971 (JobPosting.save → esco.code_for), jobs/models.py:990 (isco_name, shown by templates/jobs/posting_detail.html:122), resume/models.py:184 (Skill → esco.skill_for), jobs/forms.py:752 and applications/forms.py:177 (esco.suggestions()).
What happens: with esco-1.2.1.json and esco-1.2.2.json both in data/, esco.code_for("Software developer") raises RuntimeError: two ESCO classifications in … delete the one being replaced before starting Postulo on every call, and so does esco.suggestions() (checked against the snapshot). Gunicorn retires each worker after about 500 requests (docker/Dockerfile:354), so within the window new workers answer 500 on adding or editing a listing, on the listing and application forms that offer titles, on the detail page of a listing that carries a code, and on the API and capture paths that save a listing. Already-running processes keep the old revision, except that _member checks skills_file() on every read, so they too fail on the first skill lookup in a language they have not read yet. The management commands and the scheduler fail the same way once restarted.
Why it matters: an ordinary upgrade step leaves the instance half-broken, and the error text ("before starting Postulo") speaks of a start that is not happening. The wiki (Installing Postulo, "For a later revision ... delete the old files once the new ones are there") documents the deletion, but the command itself already knows exactly which files to delete and asks the operator to "check" new files that Postulo cannot read while the old ones remain.
Fix: have fetch_esco delete the files in others itself once both new files are written and validated (it already validates both halves before writing anything), so the directory goes from one revision to one revision with no window; keep the loader's refusal for a directory somebody filled by hand. Add a Django system check that reports two revisions of either file, so check --deploy at start-up names the state instead of the first save.

Done when

tests/test_fetch_esco.py: a run with --revision over a directory holding the previous revision's two files leaves only the new two.
tests/test_esco.py: a system check reports an error when two esco-*.json (or two esco-skills-*.zip) files are present.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading `jobs/esco.py` and `fetch_esco`, and checking the loader against two revisions in one directory. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The callers are now at `jobs/models.py:974` (`JobPosting.save`), `jobs/models.py:993` (`isco_name`) and `jobs/forms.py:749` (`esco.suggestions()`); `jobs/esco.py`, `fetch_esco.py` and the other locations have not moved. `manage.py fetch_esco --revision <new>` writes the new revision's two files beside the old ones and then only prints a warning asking for the old ones to be deleted. From that moment the loader refuses to read either, and because the refusal is an exception, `lru_cache` does not keep it: every call in a process that had not already loaded the classification raises again. The upgrade the wiki describes, run with `docker compose exec` against a running instance, therefore breaks saving listings until somebody deletes the old files, and nothing reports the state at start-up. - **Where:** `jobs/esco.py:108-122` (`data_file`) and `jobs/esco.py:296-310` (`skills_file`), which raise `RuntimeError` on two files; `jobs/esco.py:125-126` (`classification`, `lru_cache`) and `jobs/esco.py:313-322` (`_member`, which calls `skills_file()` on every read, uncached); `jobs/management/commands/fetch_esco.py:182-193`, which lists the old files in `others` and only warns; the callers `jobs/models.py:971` (`JobPosting.save` → `esco.code_for`), `jobs/models.py:990` (`isco_name`, shown by `templates/jobs/posting_detail.html:122`), `resume/models.py:184` (`Skill` → `esco.skill_for`), `jobs/forms.py:752` and `applications/forms.py:177` (`esco.suggestions()`). - **What happens:** with `esco-1.2.1.json` and `esco-1.2.2.json` both in `data/`, `esco.code_for("Software developer")` raises `RuntimeError: two ESCO classifications in … delete the one being replaced before starting Postulo` on every call, and so does `esco.suggestions()` (checked against the snapshot). Gunicorn retires each worker after about 500 requests (`docker/Dockerfile:354`), so within the window new workers answer 500 on adding or editing a listing, on the listing and application forms that offer titles, on the detail page of a listing that carries a code, and on the API and capture paths that save a listing. Already-running processes keep the old revision, except that `_member` checks `skills_file()` on every read, so they too fail on the first skill lookup in a language they have not read yet. The management commands and the scheduler fail the same way once restarted. - **Why it matters:** an ordinary upgrade step leaves the instance half-broken, and the error text ("before starting Postulo") speaks of a start that is not happening. The wiki (*Installing Postulo*, "For a later revision ... delete the old files once the new ones are there") documents the deletion, but the command itself already knows exactly which files to delete and asks the operator to "check" new files that Postulo cannot read while the old ones remain. - **Fix:** have `fetch_esco` delete the files in `others` itself once both new files are written and validated (it already validates both halves before writing anything), so the directory goes from one revision to one revision with no window; keep the loader's refusal for a directory somebody filled by hand. Add a Django system check that reports two revisions of either file, so `check --deploy` at start-up names the state instead of the first save. ## Done when - `tests/test_fetch_esco.py`: a run with `--revision` over a directory holding the previous revision's two files leaves only the new two. - `tests/test_esco.py`: a system check reports an error when two `esco-*.json` (or two `esco-skills-*.zip`) files are present.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:47:27 +00:00

tiagoagueda added the

tier

bug

labels

2026-10-01 10:47:27 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 05:28:45 +00:00

Delete the replaced ESCO files in fetch_esco and check for two revisions

tiagoagueda closed this issue

2026-10-04 05:28:45 +00:00