An address is checked and then resolved a second time, so DNS can change in between #58

New issue

Closed

opened 2026-09-06 16:05:53 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-09-06 16:05:53 +00:00

Owner

What happens

validate_public_url in plugins/fetching.py resolves a hostname, checks every address it
answers with, and returns the URL as a string:

addresses = _addresses_for(parts.hostname)
if not all(address.is_global for address in addresses):
    raise UnsafeURL(...)
return urlunparse(parts)

The caller then hands that string to httpx, which resolves the name again before
connecting. Between the two lookups the answer is free to change, and a record with a
one-second time to live is free to give a public address to the first query and
127.0.0.1 to the second. The check passes and the connection goes somewhere else.

This is the standard way a resolve-then-connect check is defeated, and it applies to every
caller: capture (fetch_page), company logos, portfolio links, and every plugin
connection through http.client(), whose event hook re-runs the same resolve-then-connect
check on each redirect.

Why it is worth fixing even though it is fiddly

Nothing else in the chain is a second line of defence. The whole reason capture refuses
private addresses is that the URL came from a stranger's page — and a stranger who controls
a hostname controls its time to live.

Whether this is reachable in practice depends on the resolver in front of Postulo: a
caching resolver that honours a one-second time to live makes it straightforward, and one
that enforces a floor makes it a race. Neither is a reason to leave it.

Shape

Resolve once, and connect to what was checked:

Resolve the hostname; keep the addresses that passed.
Connect to one of those addresses, with the Host header set to the hostname and,
for HTTPS, the TLS server name set to it as well, so certificate checking still works.
On a redirect, do the same again for the new host.

httpx supports this through a custom transport, or through resolving into the connection
pool. Whichever way it goes, the check and the connection must not be two separate
lookups.

An AddressPinnedTransport in plugins/http.py would fix every caller at once, which is
the argument for putting it there rather than in each one.

Classification

Bug, security. Not breaking.

Depends on

Nothing, but it is worth doing after the link checker is moved onto the guarded client, so
there is one path to harden rather than two.

## What happens `validate_public_url` in `plugins/fetching.py` resolves a hostname, checks every address it answers with, and returns the URL as a string: ```python addresses = _addresses_for(parts.hostname) if not all(address.is_global for address in addresses): raise UnsafeURL(...) return urlunparse(parts) ``` The caller then hands that string to httpx, which **resolves the name again** before connecting. Between the two lookups the answer is free to change, and a record with a one-second time to live is free to give a public address to the first query and `127.0.0.1` to the second. The check passes and the connection goes somewhere else. This is the standard way a resolve-then-connect check is defeated, and it applies to every caller: capture (`fetch_page`), company logos, portfolio links, and every plugin connection through `http.client()`, whose event hook re-runs the same resolve-then-connect check on each redirect. ## Why it is worth fixing even though it is fiddly Nothing else in the chain is a second line of defence. The whole reason capture refuses private addresses is that the URL came from a stranger's page — and a stranger who controls a hostname controls its time to live. Whether this is reachable in practice depends on the resolver in front of Postulo: a caching resolver that honours a one-second time to live makes it straightforward, and one that enforces a floor makes it a race. Neither is a reason to leave it. ## Shape Resolve once, and connect to what was checked: 1. Resolve the hostname; keep the addresses that passed. 2. Connect to one of **those addresses**, with the `Host` header set to the hostname and, for HTTPS, the TLS server name set to it as well, so certificate checking still works. 3. On a redirect, do the same again for the new host. httpx supports this through a custom transport, or through resolving into the connection pool. Whichever way it goes, the check and the connection must not be two separate lookups. An `AddressPinnedTransport` in `plugins/http.py` would fix every caller at once, which is the argument for putting it there rather than in each one. ## Classification Bug, security. Not breaking. ## Depends on Nothing, but it is worth doing after the link checker is moved onto the guarded client, so there is one path to harden rather than two.

tiagoagueda added this to the 0.2.0 milestone

2026-09-06 16:05:53 +00:00

tiagoagueda added the

bug

security

tier/4

labels

2026-09-06 16:05:53 +00:00

tiagoagueda referenced this issue from a commit

2026-09-06 16:35:53 +00:00

Connect to the address that was approved, not to a second lookup

tiagoagueda closed this issue