Store: a revoked or wrong token is retried six times per document instead of retiring the connection #13

New issue

Closed

opened 2026-10-01 10:48:19 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-10-01 10:48:19 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo-paperless at 973969bb7, reading how the store reports each answer Paperless can give and comparing it with what the core does with a store's exceptions.

The core lets a store say that the other side has finished with a connection: it raises ConnectionUnusable, and send_copy switches the connection off with the reason and stops dialling. This plugin never raises it. A 401 from Paperless is turned into an ordinary PaperlessError, so a dead token is tried again for every document, six times each, and nothing tells the person until they notice the failed badges.

Where: src/postulo_paperless/store.py:116-126 (Archive._get, Archive._post), :149-157 (Archive.post_document) and :207-213 (Archive.set_custom_field) raise PaperlessError(_explain(response)) for every status that is not a success, 401 included. The import list at :36-44 has no ConnectionUnusable. In the core, documents/archiving.py:140-150 catches ConnectionUnusable and calls connection.retire(...); documents/archiving.py:151-153 records anything else as a failure to retry; plugins/base.py:551-567 names "a token the provider revoked" as the case the exception is for.
What happens: a person regenerates their Paperless API token, or the Paperless account is removed. The first request of every put (archive.ensure("document_types", ...), store.py:319) answers 401. Each document's copy is recorded as failed: PaperlessError: HTTP 401: Invalid token. and retried after 5, 10, 20, 40 and 80 minutes, then gives up at MAX_ATTEMPTS. The connection stays switched on, shows no error of its own, and is dialled again for every new document. Once the token is corrected, the copies that used their six attempts are not picked up again: pending_copies filters on attempts__lt=MAX_ATTEMPTS, and Send everything leaves existing copies as they are, so each document has to be re-sent by hand with Send to stores now.
Why it matters: the mechanism postulo/postulo#243 added exists so that one dead credential is one message on the connection rather than a failure on every document. With it, the connection is switched off once with a sentence the person can read, and the other pending copies keep their attempts and resume when it is switched back on (documents/archiving.py:179-184).
Fix: in Archive, raise ConnectionUnusable with a translated sentence when Paperless answers 401, from one helper used by _get, _post, post_document and set_custom_field. Leave 403 as a PaperlessError: it means the token's account lacks one permission (creating a document type, say), which is fixed in Paperless and worth retrying. Decide keep_secrets: the default forgets the token, which suits a revoked one. PaperlessStore.test() must go on answering with a TestResult, so add the new exception to what it catches at store.py:304. Add ConnectionUnusable to the names tests/test_surface.py checks on the surface.

Related to postulo/postulo#243 (closed), which made the core honour ConnectionUnusable for stores and records that #2 "stopped short of raising it" because the core did not act on it then. The core this repository pins (1d6cdd7e0) already does; the plugin's half was not done.

Done when

A test in tests/test_paperless.py shows that PaperlessStore().put(...) with a wrong token raises ConnectionUnusable, and that test() with the same token still returns a TestResult that is not ok.
A test in tests/test_paperless.py, through send_pending(), shows that a 401 leaves the connection switched off with the reason in last_error, and that a second pending copy on the same connection is not attempted.

Found on 2026-09-29 in a read-only review of postulo-paperless at 973969bb7, reading how the store reports each answer Paperless can give and comparing it with what the core does with a store's exceptions. The core lets a store say that the other side has finished with a connection: it raises `ConnectionUnusable`, and `send_copy` switches the connection off with the reason and stops dialling. This plugin never raises it. A 401 from Paperless is turned into an ordinary `PaperlessError`, so a dead token is tried again for every document, six times each, and nothing tells the person until they notice the *failed* badges. - **Where:** `src/postulo_paperless/store.py:116-126` (`Archive._get`, `Archive._post`), `:149-157` (`Archive.post_document`) and `:207-213` (`Archive.set_custom_field`) raise `PaperlessError(_explain(response))` for every status that is not a success, 401 included. The import list at `:36-44` has no `ConnectionUnusable`. In the core, `documents/archiving.py:140-150` catches `ConnectionUnusable` and calls `connection.retire(...)`; `documents/archiving.py:151-153` records anything else as a failure to retry; `plugins/base.py:551-567` names "a token the provider revoked" as the case the exception is for. - **What happens:** a person regenerates their Paperless API token, or the Paperless account is removed. The first request of every `put` (`archive.ensure("document_types", ...)`, `store.py:319`) answers 401. Each document's copy is recorded as *failed: PaperlessError: HTTP 401: Invalid token.* and retried after 5, 10, 20, 40 and 80 minutes, then gives up at `MAX_ATTEMPTS`. The connection stays switched on, shows no error of its own, and is dialled again for every new document. Once the token is corrected, the copies that used their six attempts are not picked up again: `pending_copies` filters on `attempts__lt=MAX_ATTEMPTS`, and *Send everything* leaves existing copies as they are, so each document has to be re-sent by hand with *Send to stores now*. - **Why it matters:** the mechanism postulo/postulo#243 added exists so that one dead credential is one message on the connection rather than a failure on every document. With it, the connection is switched off once with a sentence the person can read, and the other pending copies keep their attempts and resume when it is switched back on (`documents/archiving.py:179-184`). - **Fix:** in `Archive`, raise `ConnectionUnusable` with a translated sentence when Paperless answers 401, from one helper used by `_get`, `_post`, `post_document` and `set_custom_field`. Leave 403 as a `PaperlessError`: it means the token's account lacks one permission (creating a document type, say), which is fixed in Paperless and worth retrying. Decide `keep_secrets`: the default forgets the token, which suits a revoked one. `PaperlessStore.test()` must go on answering with a `TestResult`, so add the new exception to what it catches at `store.py:304`. Add `ConnectionUnusable` to the names `tests/test_surface.py` checks on the surface. Related to postulo/postulo#243 (closed), which made the core honour `ConnectionUnusable` for stores and records that #2 "stopped short of raising it" because the core did not act on it then. The core this repository pins (`1d6cdd7e0`) already does; the plugin's half was not done. ## Done when - A test in `tests/test_paperless.py` shows that `PaperlessStore().put(...)` with a wrong token raises `ConnectionUnusable`, and that `test()` with the same token still returns a `TestResult` that is not ok. - A test in `tests/test_paperless.py`, through `send_pending()`, shows that a 401 leaves the connection switched off with the reason in `last_error`, and that a second pending copy on the same connection is not attempted.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:48:19 +00:00

tiagoagueda added the

bug

label

2026-10-01 10:48:19 +00:00

tiagoagueda referenced this issue

2026-10-17 10:39:53 +00:00

SEC2-061: Store reads every Paperless answer whole with no deadline, so a server the account holder names can hold the shared scheduler #16

tiagoagueda referenced this issue from a commit

2026-10-19 07:05:31 +00:00

Read Paperless's answers within a deadline and a cap, retire a dead token, decline unconsumable files, and stop mis-reading the duplicate's id