Postulo/postulo

Watch

Fork 0

Pin an address at its street with the geocoder the operator chooses (#913) #1056

Merged

tiagoagueda merged 1 commit from 913-street-geocoding into main

2026-10-27 15:07:15 +00:00

tiagoagueda commented

2026-10-27 14:35:55 +00:00

Owner

Closes #913

Pins an address that has a street at that street, using a geocoding service the operator chooses: Nominatim, Photon or addok, each at a base address the operator can point at their own server.

Decisions (maintainer, 2026-10-27)

What is sent: every eligible address, once the operator gives consent. That means every company's address, plus a person's own address or a contact's when it is shown on the map. People's and contacts' addresses are personal data. OpenStreetMap's public Nominatim asks not to be sent any, so on the page the setting needs a consent box ticked; setting it in the environment counts as consent. The consent text, the explanation card on the page and docs/THREAT-MODEL.md all say this.
Default: off. Choosing Nominatim with no address fills in https://nominatim.openstreetmap.org.
Existing addresses: the scheduler works through them gradually, one a second.

Where this differs from the issue's sketch

The issue proposed looking the street up as the address is saved. Here it happens in the scheduler instead (send_due_reminders → geocoding.run()):

The offline placement doesn't change. Saving still places the address offline, at once.
Saving never waits on another server. No form that saves an address makes a request to a stranger's server.
The one-a-second limit holds for the whole instance. Only one scheduler pass runs at a time, so it holds without a lock.
Old addresses need no separate tool. They are reached the same way: at most 60 s of look-ups per pass.

The address page says "Its street will be looked up with {service} shortly." until the look-up happens.

Behaviour

A street-level answer pins the address (placed_by="street"). Street level means Nominatim place_rank ≥ 26, Photon house/street, or addok housenumber/street. street_from records the address as it was looked up, so an unchanged address is not asked again.
A town-level answer, or no answer, leaves the offline point. The address is not asked again until it changes.
429, 5xx, timeouts and network errors are tried again after an hour (street_tried_at).
401/403 records geocoder_problem. That stops every look-up and shows an error on the settings page; saving the page lifts it.
A pin placed by hand or from Wikidata is never looked up or overwritten. Take off clears the street state, so the address is looked up again.
Editing the street drops the old street pin at once, back to the offline point, and the scheduler looks up the new street.
Requests go through postulo.plugins.http: destination policy, 10 s/15 s deadlines, no redirects, and a 256 KiB body cap.
The companies map credits street pins with the service's attribution.
The record of processing lists the service as a recipient.
Start-up checks: postulo.E043 for an unknown POSTULO_GEOCODER, postulo.E044 for a malformed POSTULO_GEOCODER_URL.

Changes

Models: migration 0038, with PostalAddress.street_from / street_tried_at, Placed.STREET, and SiteSettings.geocoder / geocoder_url / geocoder_consent / geocoder_problem.
Settings: POSTULO_GEOCODER / POSTULO_GEOCODER_URL, both overriding the page.
New module: jobs/geocoding.py. jobs/placing.py drops a stale street pin and resets the street state on take off.
Settings page: MapsForm and the Maps page.
Address pages: the status line under an address, and the map credit.
Data protection: GDPR record of processing.
Docs: threat model and CHANGELOG.
Tests: tests/test_geocoding.py (23 tests, network faked with httpx.MockTransport).

New strings reach Weblate after the merge.

🤖 Generated with Claude Code

Closes #913 Pins an address that has a street at that street, using a geocoding service the operator chooses: **Nominatim**, **Photon** or **addok**, each at a base address the operator can point at their own server. ## Decisions (maintainer, 2026-10-27) - **What is sent:** every eligible address, once the operator gives consent. That means every company's address, plus a person's own address or a contact's when it is shown on the map. People's and contacts' addresses are personal data. OpenStreetMap's public Nominatim asks not to be sent any, so on the page the setting needs a consent box ticked; setting it in the environment counts as consent. The consent text, the explanation card on the page and `docs/THREAT-MODEL.md` all say this. - **Default:** off. Choosing Nominatim with no address fills in `https://nominatim.openstreetmap.org`. - **Existing addresses:** the scheduler works through them gradually, one a second. ## Where this differs from the issue's sketch The issue proposed looking the street up as the address is saved. Here it happens in the scheduler instead (`send_due_reminders` → `geocoding.run()`): - **The offline placement doesn't change.** Saving still places the address offline, at once. - **Saving never waits on another server.** No form that saves an address makes a request to a stranger's server. - **The one-a-second limit holds for the whole instance.** Only one scheduler pass runs at a time, so it holds without a lock. - **Old addresses need no separate tool.** They are reached the same way: at most 60 s of look-ups per pass. The address page says "Its street will be looked up with {service} shortly." until the look-up happens. ## Behaviour - **A street-level answer pins the address** (`placed_by="street"`). Street level means Nominatim `place_rank ≥ 26`, Photon `house`/`street`, or addok `housenumber`/`street`. `street_from` records the address as it was looked up, so an unchanged address is not asked again. - **A town-level answer, or no answer,** leaves the offline point. The address is not asked again until it changes. - **429, 5xx, timeouts and network errors** are tried again after an hour (`street_tried_at`). - **401/403** records `geocoder_problem`. That stops every look-up and shows an error on the settings page; saving the page lifts it. - **A pin placed by hand or from Wikidata** is never looked up or overwritten. *Take off* clears the street state, so the address is looked up again. - **Editing the street drops the old street pin at once,** back to the offline point, and the scheduler looks up the new street. - **Requests go through `postulo.plugins.http`:** destination policy, 10 s/15 s deadlines, no redirects, and a 256 KiB body cap. - **The companies map credits street pins** with the service's attribution. - **The record of processing** lists the service as a recipient. - **Start-up checks:** `postulo.E043` for an unknown `POSTULO_GEOCODER`, `postulo.E044` for a malformed `POSTULO_GEOCODER_URL`. ## Changes - **Models:** migration `0038`, with `PostalAddress.street_from` / `street_tried_at`, `Placed.STREET`, and `SiteSettings.geocoder` / `geocoder_url` / `geocoder_consent` / `geocoder_problem`. - **Settings:** `POSTULO_GEOCODER` / `POSTULO_GEOCODER_URL`, both overriding the page. - **New module:** `jobs/geocoding.py`. `jobs/placing.py` drops a stale street pin and resets the street state on *take off*. - **Settings page:** `MapsForm` and the Maps page. - **Address pages:** the status line under an address, and the map credit. - **Data protection:** GDPR record of processing. - **Docs:** threat model and CHANGELOG. - **Tests:** `tests/test_geocoding.py` (23 tests, network faked with `httpx.MockTransport`). New strings reach Weblate after the merge. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

tiagoagueda added 1 commit

2026-10-27 14:35:55 +00:00

Pin an address at its street with the geocoder the operator chooses (#913 )

CI / Unit tests and coverage (Python 3.14) (push) Successful in 12m31s

Details

CI / Security audit (push) Successful in 46s

Details

CI / Browser tests (Chromium) (pull_request) Successful in 31m40s

Details

CI / Checks: lint, migrations, catalogues, settings, built files (push) Successful in 2m38s

Details

CI / Database tests on PostgreSQL (push) Successful in 3m41s

Details

CI / Browser tests (Chromium) (push) Successful in 30m24s

Details

Every Python / Unit tests (Python 3.12) (push) Successful in 12m21s

Details

Every Python / Unit tests (Python 3.13) (push) Successful in 14m4s

Details

CI / Unit tests and coverage (Python 3.14) (pull_request) Successful in 12m29s

Details

CI / Checks: lint, migrations, catalogues, settings, built files (pull_request) Successful in 2m42s

Details

CI / Database tests on PostgreSQL (pull_request) Successful in 3m25s

Details

CI / Security audit (pull_request) Successful in 41s

Details

be4786789f

Nominatim, Photon or addok, at a base address of the operator's choosing; off by
default, and switched on from the Maps settings with the operator's consent, or by
POSTULO_GEOCODER. The scheduler looks up one address a second, for at most a minute
a pass, which also works through the addresses saved before: every company's, and a
person's own or a contact's only where it is shown on the map. Only the address's
parts are sent. A pin placed by hand or from Wikidata is never looked up; a busy
service is tried again an hour later; a refusal stops every look-up until the setting
is saved. The record of processing names the service, and the threat model says
what the choice costs.

Closes #913

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

tiagoagueda merged commit 527d87d5a9 into main

2026-10-27 15:07:15 +00:00

tiagoagueda referenced this pull request from a commit

2026-10-27 15:07:16 +00:00

Pin an address at its street with the geocoder the operator chooses (#913) (#1056)

tiagoagueda deleted branch 913-street-geocoding

2026-10-27 15:07:17 +00:00