Restore: the key check counts only connections and says nothing is lost when the mail secrets are unreadable #481

New issue

Closed

opened 2026-10-01 10:47:00 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:47:00 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the restore report against every column encrypted with the field key.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The code is now at core/site.py:302 (the read of SiteSettings.email_password, which #248 moved up); core/backup.py, the restore command and the cited lines of core/models.py have not moved.

The archive carries a fingerprint of the field key so that a restore can say whether the secrets it put back can still be read. The count behind that message looks only at plugin connections. SiteSettings keeps four more stores under the same key: the SMTP password, the mail OAuth client secret and tokens, the mail transport's secrets and the text transport's secrets. When the key differs and no connection holds a secret, restore prints that nothing is lost, while the instance's own outgoing mail can no longer authenticate.

Where:
- core/backup.py:131-134, _connections_with_secrets: Connection.objects.exclude(secrets_encrypted="").count() and nothing else.
- core/backup.py:567-570, where restore_backup fills the report.
- core/management/commands/restore.py:61-77, the two messages; the second reads "no connection holds a secret, so nothing is lost by it".
- The stores it misses: email_password_encrypted, email_oauth_secrets_encrypted, transport_secrets_encrypted and text_secrets_encrypted on SiteSettings (core/models.py:964, :998, :1010, :1021).
- The wiki's Backups and your data, "Afterwards", repeats the connections-only wording.
What happens: an operator restores onto a rebuilt host with a new POSTULO_SECRET_KEY and no POSTULO_FIELD_KEY. The instance sends mail with an SMTP password or Microsoft 365 consent stored in Server settings. restore reports the key mismatch as harmless. From then on SiteSettings.email_password (read by core/site.py:340) and email_oauth_secrets raise SecretsUnreadable, so password resets, sign-in codes and notifications that go by mail or text fail.
Why it matters: this is exactly the silent failure, noticed weeks later, that the fingerprint was added to prevent (the command's own comment says so), and here the message actively reassures the operator.
Fix: count every encrypted store: the connections plus each non-empty *_encrypted column on SiteSettings (secrets.encrypt({}) writes "", so non-empty means a secret is there). Better, try to decrypt each and report which could not be read. Name the mail settings in the message, for example "re-enter the SMTP password or consent again on Server settings → Email", and extend the wiki's "Afterwards" to match.

Done when

A test in tests/test_backup.py stores an SMTP password in SiteSettings, takes a backup, restores under a different key, and asserts that the report counts it and the command does not print "nothing is lost"; it fails today.

Related to #234 (closed), which introduced the fingerprint check this one is incomplete against.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the restore report against every column encrypted with the field key. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The code is now at `core/site.py:302` (the read of `SiteSettings.email_password`, which #248 moved up); `core/backup.py`, the `restore` command and the cited lines of `core/models.py` have not moved. The archive carries a fingerprint of the field key so that a restore can say whether the secrets it put back can still be read. The count behind that message looks only at plugin connections. `SiteSettings` keeps four more stores under the same key: the SMTP password, the mail OAuth client secret and tokens, the mail transport's secrets and the text transport's secrets. When the key differs and no connection holds a secret, `restore` prints that nothing is lost, while the instance's own outgoing mail can no longer authenticate. - **Where:** - `core/backup.py:131-134`, `_connections_with_secrets`: `Connection.objects.exclude(secrets_encrypted="").count()` and nothing else. - `core/backup.py:567-570`, where `restore_backup` fills the report. - `core/management/commands/restore.py:61-77`, the two messages; the second reads "no connection holds a secret, so nothing is lost by it". - The stores it misses: `email_password_encrypted`, `email_oauth_secrets_encrypted`, `transport_secrets_encrypted` and `text_secrets_encrypted` on `SiteSettings` (`core/models.py:964`, `:998`, `:1010`, `:1021`). - The wiki's *Backups and your data*, "Afterwards", repeats the connections-only wording. - **What happens:** an operator restores onto a rebuilt host with a new `POSTULO_SECRET_KEY` and no `POSTULO_FIELD_KEY`. The instance sends mail with an SMTP password or Microsoft 365 consent stored in Server settings. `restore` reports the key mismatch as harmless. From then on `SiteSettings.email_password` (read by `core/site.py:340`) and `email_oauth_secrets` raise `SecretsUnreadable`, so password resets, sign-in codes and notifications that go by mail or text fail. - **Why it matters:** this is exactly the silent failure, noticed weeks later, that the fingerprint was added to prevent (the command's own comment says so), and here the message actively reassures the operator. - **Fix:** count every encrypted store: the connections plus each non-empty `*_encrypted` column on `SiteSettings` (`secrets.encrypt({})` writes `""`, so non-empty means a secret is there). Better, try to decrypt each and report which could not be read. Name the mail settings in the message, for example "re-enter the SMTP password or consent again on Server settings → Email", and extend the wiki's "Afterwards" to match. ## Done when - A test in `tests/test_backup.py` stores an SMTP password in `SiteSettings`, takes a backup, restores under a different key, and asserts that the report counts it and the command does not print "nothing is lost"; it fails today. Related to #234 (closed), which introduced the fingerprint check this one is incomplete against.

tiagoagueda added this to the 0.5.0 milestone

2026-10-01 10:47:00 +00:00

tiagoagueda added the

bug

label

2026-10-01 10:47:00 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 10:04:59 +00:00

Count every secret under the field key when a restore finds another key

tiagoagueda closed this issue

2026-10-04 10:04:59 +00:00