Repository: the GeoNames lookup databases are not ignored, so one `git add -A` commits 30 MB #1161

New issue

Closed

opened 2026-10-31 03:04:26 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-31 03:04:26 +00:00

Owner

What exists today

manage.py fetch_geonames downloads the GeoNames text files into src/postulo/jobs/data/ and builds a SQLite lookup beside them (LOOKUP_FILE, jobs/places.py:55). .gitignore ignores /src/postulo/jobs/data/geonames-*.txt only (line 55). git check-ignore matches neither geonames-cities1000.sqlite nor geonames-postcodes.sqlite. In a developer checkout that has fetched the data, git status --short lists ?? src/postulo/jobs/data/geonames-cities1000.sqlite (30 MB). One git add -A puts it in a commit; the workflow's recipes use git add -A after extract.

What would change

Add /src/postulo/jobs/data/geonames-*.sqlite (and any -journal/-wal siblings) to .gitignore beside the text-file rule, with a line in the comment saying the lookup is derived. A test that every file fetch_geonames and fetch_esco write is ignored would keep the next one from slipping through.

What it saves or protects

small: keeps a 30 MB derived file (and the postcode one) out of a commit, the pack and every clone

Accessibility

none

Decisions

Guard with a test. Recommended: yes, a small test that asks git check-ignore about each path the fetch commands write, skipped where git is absent.

Found by the efficiency and reliability audit of 2026-10-30 (finding X3; index postulo/postulo#1108). Checked against main at daee30b15 (2026-10-30).

## What exists today `manage.py fetch_geonames` downloads the GeoNames text files into `src/postulo/jobs/data/` and builds a SQLite lookup beside them (`LOOKUP_FILE`, `jobs/places.py:55`). `.gitignore` ignores `/src/postulo/jobs/data/geonames-*.txt` only (line 55). `git check-ignore` matches neither `geonames-cities1000.sqlite` nor `geonames-postcodes.sqlite`. In a developer checkout that has fetched the data, `git status --short` lists `?? src/postulo/jobs/data/geonames-cities1000.sqlite` (30 MB). One `git add -A` puts it in a commit; the workflow's recipes use `git add -A` after `extract`. ## What would change Add `/src/postulo/jobs/data/geonames-*.sqlite` (and any `-journal`/`-wal` siblings) to `.gitignore` beside the text-file rule, with a line in the comment saying the lookup is derived. A test that every file `fetch_geonames` and `fetch_esco` write is ignored would keep the next one from slipping through. ## What it saves or protects small: keeps a 30 MB derived file (and the postcode one) out of a commit, the pack and every clone ## Accessibility none ## Decisions - **Guard with a test.** Recommended: yes, a small test that asks `git check-ignore` about each path the fetch commands write, skipped where git is absent. --- Found by the efficiency and reliability audit of 2026-10-30 (finding X3; index postulo/postulo#1108). Checked against `main` at `daee30b15` (2026-10-30).

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 03:04:26 +00:00

tiagoagueda added the

performance

bug

labels

2026-10-31 03:04:26 +00:00

tiagoagueda referenced this issue

2026-10-31 03:04:41 +00:00

Efficiency and reliability audit of 2026-10-30: index #1108

tiagoagueda referenced this issue from a commit

2026-11-10 11:31:09 +00:00