Sync connections: a run that fails is tried again every interval for ever, and a sync cannot retire its connection #1162

New issue

Closed

opened 2026-10-31 03:04:26 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-31 03:04:26 +00:00

Owner

What exists today

Core, src/postulo/plugins/syncing.py:

is_due (lines 70-75) looks only at synced_at and the interval (15, 60, 240 or 1440 minutes, default 60, lines 35-41). It does not read last_error or last_ok_at.
_run_connection (lines 117-146) records report.error in last_error, stamps synced_at, and the connection is due again one interval later. Nothing counts consecutive failures.
It catches SecretsUnreadable and any Exception, but not ConnectionUnusable (plugins/base.py:772-792). That exception, which switches a connection off with a sentence for the person, is honoured for stores (documents/archiving.py:177) and notifiers (notifications/service.py:94), not for syncs: a sync plugin that raised it would be logged with a traceback as a crash and dialled again next interval.

postulo-imap, src/postulo_imap/mailbox.py:582-589 (connect): a refused LOGIN becomes MailboxError("Sign-in refused…"), which sync.py:253-255 returns as SyncReport(error=…). So after a password change, every interval opens DNS, TCP, TLS and a failed LOGIN against the person's mail server, until the person notices the error on the connection. The same holds for postulo-dav and postulo-francetravail, which go through the same scheduler.

The lockout risk the scout named is real but smaller than stated: a lockout policy that counts failures in a short window (fail2ban's default 5 in 10 minutes, Active Directory's usual 30-minute observation window) is not reached by one attempt every 15 minutes or more. What does happen is a steady stream of failed sign-ins on the account: providers that count them per day, or send a "blocked sign-in attempt" mail for each, react to it, and the person's other clients can be caught in a lockout the stream contributes to.

The scout's other point, "two queries per message" in matching.py:169-185 (match): it runs only for a message the rules recognised (sync.py:377-380), which is a handful per pass; the thread and contact queries are indexed lookups. Not worth an issue; dropped.

What would change

In the core:

Back off while failing. In is_due, when last_error is set, wait the longer of the interval and the time the connection has been failing (synced_at - last_ok_at, or since created_at when it never worked), capped at one day. That is an exponential back-off with no new column: 1 h, 2 h, 4 h … 24 h, and the next success puts it back on its interval. Sync now is unaffected, since it calls sync_connection directly. The connection card should say when the next attempt is, so the slowdown is not mistaken for a hang (a server-rendered sentence beside the existing last-error text).
Honour ConnectionUnusable for syncs. In _run_connection, catch it as the store path does and call connection.retire(str(dead), keep_secrets=dead.keep_secrets).

In postulo-imap, only if the user chooses to retire (Decisions): raise ConnectionUnusable(…, keep_secrets=True) when the LOGIN answer carries the RFC 5530 [AUTHENTICATIONFAILED] response code, and keep the ordinary error for [UNAVAILABLE] and for servers that send no code. That would be a separate, small issue on postulo-imap.

No library: the back-off is three lines of arithmetic on fields the row already has.

What it saves or protects

medium (reliability): a connection whose password was changed stops dialling the mail, DAV or France Travail server 24 times a day (96 at the 15-minute interval) with a credential known to fail

Accessibility

none

Decided by the maintainer (2026-10-31)

back off every failing sync, capped at one day; never retire a connection on its own

Decisions

Retire, or only slow down, on a refused sign-in? Recommended: slow down in the core for every failing sync (one issue on postulo), and leave retiring to a later postulo-imap issue; a server that briefly refuses sign-ins during maintenance would otherwise switch the connection off.
Cap of the back-off. Recommended: one day, the longest interval the form offers.
Split into: the core issue only (back-off plus ConnectionUnusable for syncs, one PR); the matching queries are not filed.

Found by the efficiency and reliability audit of 2026-10-30 (finding PR3; index postulo/postulo#1108). Checked against main at daee30b15 (2026-10-30).

## What exists today Core, `src/postulo/plugins/syncing.py`: - `is_due` (lines 70-75) looks only at `synced_at` and the interval (15, 60, 240 or 1440 minutes, default 60, lines 35-41). It does not read `last_error` or `last_ok_at`. - `_run_connection` (lines 117-146) records `report.error` in `last_error`, stamps `synced_at`, and the connection is due again one interval later. Nothing counts consecutive failures. - It catches `SecretsUnreadable` and any `Exception`, but not `ConnectionUnusable` (`plugins/base.py:772-792`). That exception, which switches a connection off with a sentence for the person, is honoured for stores (`documents/archiving.py:177`) and notifiers (`notifications/service.py:94`), not for syncs: a sync plugin that raised it would be logged with a traceback as a crash and dialled again next interval. postulo-imap, `src/postulo_imap/mailbox.py:582-589` (`connect`): a refused `LOGIN` becomes `MailboxError("Sign-in refused…")`, which `sync.py:253-255` returns as `SyncReport(error=…)`. So after a password change, every interval opens DNS, TCP, TLS and a failed `LOGIN` against the person's mail server, until the person notices the error on the connection. The same holds for postulo-dav and postulo-francetravail, which go through the same scheduler. The lockout risk the scout named is real but smaller than stated: a lockout policy that counts failures in a short window (fail2ban's default 5 in 10 minutes, Active Directory's usual 30-minute observation window) is not reached by one attempt every 15 minutes or more. What does happen is a steady stream of failed sign-ins on the account: providers that count them per day, or send a "blocked sign-in attempt" mail for each, react to it, and the person's other clients can be caught in a lockout the stream contributes to. The scout's other point, "two queries per message" in `matching.py:169-185` (`match`): it runs only for a message the rules recognised (`sync.py:377-380`), which is a handful per pass; the thread and contact queries are indexed lookups. Not worth an issue; dropped. ## What would change In the core: 1. **Back off while failing.** In `is_due`, when `last_error` is set, wait the longer of the interval and the time the connection has been failing (`synced_at - last_ok_at`, or since `created_at` when it never worked), capped at one day. That is an exponential back-off with no new column: 1 h, 2 h, 4 h … 24 h, and the next success puts it back on its interval. *Sync now* is unaffected, since it calls `sync_connection` directly. The connection card should say when the next attempt is, so the slowdown is not mistaken for a hang (a server-rendered sentence beside the existing last-error text). 2. **Honour `ConnectionUnusable` for syncs.** In `_run_connection`, catch it as the store path does and call `connection.retire(str(dead), keep_secrets=dead.keep_secrets)`. In postulo-imap, only if the user chooses to retire (Decisions): raise `ConnectionUnusable(…, keep_secrets=True)` when the `LOGIN` answer carries the RFC 5530 `[AUTHENTICATIONFAILED]` response code, and keep the ordinary error for `[UNAVAILABLE]` and for servers that send no code. That would be a separate, small issue on postulo-imap. No library: the back-off is three lines of arithmetic on fields the row already has. ## What it saves or protects medium (reliability): a connection whose password was changed stops dialling the mail, DAV or France Travail server 24 times a day (96 at the 15-minute interval) with a credential known to fail ## Accessibility none ## Decided by the maintainer (2026-10-31) back off every failing sync, capped at one day; never retire a connection on its own ## Decisions - **Retire, or only slow down, on a refused sign-in?** Recommended: slow down in the core for every failing sync (one issue on postulo), and leave retiring to a later postulo-imap issue; a server that briefly refuses sign-ins during maintenance would otherwise switch the connection off. - **Cap of the back-off.** Recommended: one day, the longest interval the form offers. - **Split into:** the core issue only (back-off plus `ConnectionUnusable` for syncs, one PR); the matching queries are not filed. --- Found by the efficiency and reliability audit of 2026-10-30 (finding PR3; index postulo/postulo#1108). Checked against `main` at `daee30b15` (2026-10-30).

tiagoagueda added this to the 0.6.0 milestone

2026-10-31 03:04:26 +00:00

tiagoagueda added the

performance

bug

labels

2026-10-31 03:04:26 +00:00

tiagoagueda referenced this issue

2026-10-31 03:04:41 +00:00

Efficiency and reliability audit of 2026-10-30: index #1108

tiagoagueda referenced this issue from a commit

2026-11-10 13:37:55 +00:00