Document stores: local media built in, an interface for keeping copies elsewhere #13

New issue

Closed

opened 2026-09-05 12:29:08 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-09-05 12:29:08 +00:00

Owner

Why this exists

postulo-paperless (#15) wants every document — rendered CVs, rendered letters, the files a person uploaded — in Paperless. The right shape, per the rule in CONTRIBUTING ("prefer an interface with a built-in plugin over a hard-coded implementation"), is a store interface with today's behaviour as the built-in store, not a Paperless-shaped hook in the documents app.

What exists today

Two kinds of file: UploadedDocument.file and RenderedDocument.file (documents/models.py), under the private MEDIA_ROOT, served through serve_private_file (core/files.py). Renders are produced by snapshot_cv and snapshot_letter when a person records what they sent.
Export packs media into the archive (core/export.py, MEDIA_PREFIX).
Nothing observes a file being created except the code that creates it.

Shape

Entry-point group postulo.stores. Contract: put(document, file, metadata) -> ExternalRef, open(ref) or url(ref), optional delete(ref), optional browse(query) for a later read-back stage, and test(). metadata is a small dataclass — kind (CV render, letter render, upload and its DocumentKind), title, company, application, dates, language, tags — enough for any archive to file it sensibly.
Built-in store: local media, which is exactly what happens today expressed through the same contract, so the core has one code path and a plugin is not a special case.
Policy: local stays the source of truth. External stores receive copies. Rendering, serving, export and the review of what was sent keep working with no network at all. A job search must not stop because an archive server is down. An "external only" mode is imaginable and deliberately not proposed.
When. On creation of a RenderedDocument or UploadedDocument, through the tasks framework — network calls in a request are what #4 already warned about — with retries and a per-document status the person can see ("archived to Paperless", "failed: …", "not sent").
Where. A per-person choice on the connection (#11): which kinds go to which store. A Send now action to backfill documents that existed before the store was configured.
ExternalRef stored beside the document (store, id, URL) and carried in the export as metadata, so a restored instance still knows where its copies went.

Classification

Enhancement. Not breaking: with no external store configured, nothing changes.

Open questions

Should the built-in local store be selectable at all, or always on? Always on; the interface exists so that others can be added, not so that local can be removed.
Read-back (attach a document that already lives in a store to an application, e.g. a rejection letter that arrived on paper) is a second stage; the contract leaves browse() optional for it.

## Why this exists postulo-paperless (#15) wants every document — rendered CVs, rendered letters, the files a person uploaded — in Paperless. The right shape, per the rule in CONTRIBUTING ("prefer an interface with a built-in plugin over a hard-coded implementation"), is a store interface with today's behaviour as the built-in store, not a Paperless-shaped hook in the documents app. ## What exists today - Two kinds of file: `UploadedDocument.file` and `RenderedDocument.file` (`documents/models.py`), under the private `MEDIA_ROOT`, served through `serve_private_file` (`core/files.py`). Renders are produced by `snapshot_cv` and `snapshot_letter` when a person records what they sent. - Export packs media into the archive (`core/export.py`, `MEDIA_PREFIX`). - Nothing observes a file being created except the code that creates it. ## Shape 1. **Entry-point group `postulo.stores`.** Contract: `put(document, file, metadata) -> ExternalRef`, `open(ref)` or `url(ref)`, optional `delete(ref)`, optional `browse(query)` for a later read-back stage, and `test()`. `metadata` is a small dataclass — kind (CV render, letter render, upload and its `DocumentKind`), title, company, application, dates, language, tags — enough for any archive to file it sensibly. 2. **Built-in store: local media**, which is exactly what happens today expressed through the same contract, so the core has one code path and a plugin is not a special case. 3. **Policy: local stays the source of truth.** External stores receive *copies*. Rendering, serving, export and the review of what was sent keep working with no network at all. A job search must not stop because an archive server is down. An "external only" mode is imaginable and deliberately not proposed. 4. **When.** On creation of a `RenderedDocument` or `UploadedDocument`, through the tasks framework — network calls in a request are what #4 already warned about — with retries and a per-document status the person can see ("archived to Paperless", "failed: …", "not sent"). 5. **Where.** A per-person choice on the connection (#11): which kinds go to which store. A *Send now* action to backfill documents that existed before the store was configured. 6. **`ExternalRef`** stored beside the document (store, id, URL) and carried in the export as metadata, so a restored instance still knows where its copies went. ## Classification Enhancement. Not breaking: with no external store configured, nothing changes. ## Open questions 1. Should the built-in local store be selectable at all, or always on? Always on; the interface exists so that others can be *added*, not so that local can be removed. 2. Read-back (attach a document that already lives in a store to an application, e.g. a rejection letter that arrived on paper) is a second stage; the contract leaves `browse()` optional for it.

tiagoagueda added this to the 0.2.0 milestone

2026-09-05 12:29:08 +00:00

tiagoagueda added the

enhancement

label

2026-09-05 12:29:08 +00:00

tiagoagueda referenced this issue

2026-09-05 12:29:12 +00:00

postulo-paperless: keep every document in Paperless-ngx #15

tiagoagueda referenced this issue

2026-09-05 12:29:20 +00:00

Plugin connections: per-person configuration and secrets for plugins that talk to another service #11