PST-053: The GeoNames download and the update check are read without a size bound, and GeoNames has no integrity check #782

New issue

Closed

opened 2026-10-10 10:56:25 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-10-10 10:56:25 +00:00

Owner

Found in the security audit of 2026-10-10 (SECURITY-AUDIT-2026-10-10.md), run against main at c30851b from 8 to 10 October 2026, as PST-053. Severity: Low.

The text below is the audit's entry, unchanged apart from links to the other issues it names. The finding was produced by automated review and checked by a second automated pass; re-check it before acting on it.

Severity: Low
Category: supply chain / resource exhaustion
Location: src/postulo/jobs/management/commands/fetch_geonames.py (82-96); src/postulo/core/updates.py (47-55; 52-55)
Found by: SSRF and outbound traffic, configuration and dependencies
Related: PST-052 (#781)

Who can do it, and what they need. Whoever can make download.geonames.org answer differently: a compromise of that host, its DNS plus a CA-valid certificate, or a hostile mirror if an operator ever redirects the URL. The request itself goes through public_only_client with certificate verification, so a plain on-path attacker cannot do this.

How it works.

The container entrypoint runs manage.py fetch_geonames on first start when the table is absent (docker/entrypoint.sh:31-35).
client.get(url) reads the whole response body into memory with no limit (every other fetch in the code base uses http.read_body(limit=...)).
zipfile.ZipFile(io.BytesIO(response.content)) then archive.read(name) inflates the single member fully in memory, so a small zip bomb becomes gigabytes.
The entrypoint continues on failure (|| echo), so the effect is a killed provisioning step and a map without dots; alternatively a tampered but shape-valid table (>=10000 rows, the five required cities present) is accepted and placed on the operator's map.

Impact. Memory exhaustion of the provisioning command at first start, or silent substitution of the city/country table that drives company placement. Low impact because the data only feeds map coordinates and the request has nobody's data in it.

Evidence.

fetch_geonames.py:83 `response = client.get(url)`; :88 `with zipfile.ZipFile(io.BytesIO(response.content)) as archive:`; :91 `return archive.read(archive.namelist()[0]).decode("utf-8")`; validation at :98-115 is shape-only.

Verification (verdict: confirmed; severity after verification: Low). Keep it low. Precondition: control of the answers from download.geonames.org (host compromise, or a CA-valid on-path attacker); nothing a user or stranger can trigger. Effect: memory exhaustion of the first-start fetch_geonames command, retried on every start while the table is absent. The container does not stop, because the entrypoint continues past the failure. The other effect is a shape-valid table that puts companies at wrong coordinates. No user data is involved, no markup is rendered from the table, and the other effects are not confidentiality or integrity of any account's records. Lines are correct (83, 88, 91). Remediation: stream the response with http.read_body(limit=...) and a deadline, cap the inflated size by reading the zip member in bounded pieces, and optionally pin a checksum or compare against the previous table.

Suggested remediation. Read the body through postulo.plugins.http.read_body with a ceiling (a few tens of MB), check ZipInfo.file_size against a ceiling before archive.read, and either pin a published checksum per image build or ship the dataset with the image/release so start-up fetches nothing.

Also covered by this issue.

Update-check answer is parsed without a size limit (src/postulo/core/updates.py (47-55); Low). A daily memory spike or crash of the scheduler container driven by a remote release feed. Narrow and opt-in; recorded for completeness because the project's own rule is to bound every download.
Operator-configured fetches read unbounded bodies (update check; plugin catalogue index checked against its cap only after it was read in full) (src/postulo/core/updates.py (52-55); Informational). Availability of the scheduler or a worker, from a source the operator chose to trust. No data exposure. Listed because the project's own standard (#321) is that a stranger's body is read streamed and capped, and these two callers of the capped clients bypass the cap.

Found in the security audit of 2026-10-10 (`SECURITY-AUDIT-2026-10-10.md`), run against `main` at `c30851b` from 8 to 10 October 2026, as **PST-053**. Severity: **Low**. The text below is the audit's entry, unchanged apart from links to the other issues it names. The finding was produced by automated review and checked by a second automated pass; re-check it before acting on it. --- - **Severity:** Low - **Category:** supply chain / resource exhaustion - **Location:** `src/postulo/jobs/management/commands/fetch_geonames.py` (82-96); `src/postulo/core/updates.py` (47-55; 52-55) - **Found by:** SSRF and outbound traffic, configuration and dependencies - **Related:** PST-052 (#781) **Who can do it, and what they need.** Whoever can make download.geonames.org answer differently: a compromise of that host, its DNS plus a CA-valid certificate, or a hostile mirror if an operator ever redirects the URL. The request itself goes through public_only_client with certificate verification, so a plain on-path attacker cannot do this. **How it works.** 1. The container entrypoint runs `manage.py fetch_geonames` on first start when the table is absent (docker/entrypoint.sh:31-35). 2. `client.get(url)` reads the whole response body into memory with no limit (every other fetch in the code base uses http.read_body(limit=...)). 3. `zipfile.ZipFile(io.BytesIO(response.content))` then `archive.read(name)` inflates the single member fully in memory, so a small zip bomb becomes gigabytes. 4. The entrypoint continues on failure (`|| echo`), so the effect is a killed provisioning step and a map without dots; alternatively a tampered but shape-valid table (>=10000 rows, the five required cities present) is accepted and placed on the operator's map. **Impact.** Memory exhaustion of the provisioning command at first start, or silent substitution of the city/country table that drives company placement. Low impact because the data only feeds map coordinates and the request has nobody's data in it. **Evidence.** ```text fetch_geonames.py:83 `response = client.get(url)`; :88 `with zipfile.ZipFile(io.BytesIO(response.content)) as archive:`; :91 `return archive.read(archive.namelist()[0]).decode("utf-8")`; validation at :98-115 is shape-only. ``` **Verification** (verdict: confirmed; severity after verification: Low). Keep it low. Precondition: control of the answers from download.geonames.org (host compromise, or a CA-valid on-path attacker); nothing a user or stranger can trigger. Effect: memory exhaustion of the first-start `fetch_geonames` command, retried on every start while the table is absent. The container does not stop, because the entrypoint continues past the failure. The other effect is a shape-valid table that puts companies at wrong coordinates. No user data is involved, no markup is rendered from the table, and the other effects are not confidentiality or integrity of any account's records. Lines are correct (83, 88, 91). Remediation: stream the response with http.read_body(limit=...) and a deadline, cap the inflated size by reading the zip member in bounded pieces, and optionally pin a checksum or compare against the previous table. **Suggested remediation.** Read the body through postulo.plugins.http.read_body with a ceiling (a few tens of MB), check ZipInfo.file_size against a ceiling before archive.read, and either pin a published checksum per image build or ship the dataset with the image/release so start-up fetches nothing. **Also covered by this issue.** - **Update-check answer is parsed without a size limit** (`src/postulo/core/updates.py` (47-55); Low). A daily memory spike or crash of the scheduler container driven by a remote release feed. Narrow and opt-in; recorded for completeness because the project's own rule is to bound every download. - **Operator-configured fetches read unbounded bodies (update check; plugin catalogue index checked against its cap only after it was read in full)** (`src/postulo/core/updates.py` (52-55); Informational). Availability of the scheduler or a worker, from a source the operator chose to trust. No data exposure. Listed because the project's own standard (#321) is that a stranger's body is read streamed and capped, and these two callers of the capped clients bypass the cap.

tiagoagueda added the

bug

security

labels

2026-10-10 10:56:26 +00:00

tiagoagueda referenced this issue

2026-10-10 10:57:53 +00:00

PST-052: Outbound responses are read with no size cap or total deadline, and name resolution has no timeout #781

tiagoagueda added this to the 0.6.0 milestone

2026-10-10 10:57:53 +00:00

tiagoagueda referenced this issue

2026-10-10 10:58:39 +00:00