Copy handed-over documents into a chosen WebDAV folder, as a store in this package #5

New issue

Open

opened 2026-09-30 20:30:01 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-09-30 20:30:01 +00:00

Owner

The core has what a store needs: StorePlugin.put(document, file, metadata, config, user) returns an ExternalRef or None, per person, retried by the scheduler (src/postulo/documents/archiving.py in the core), for what was handed over and never for a PDF exported on its own (postulo/postulo#236). A switch per document kind comes free. delete() and browse() are never called (src/postulo/plugins/base.py:781-793 in the core).

Reading: "storing documents" means keeping a copy of each in the person's WebDAV folder; the alternative is under Decisions.

What the plugin should do

A store of its own under postulo.stores, with its own name (say webdav): plugin names are unique across the instance (find_any, src/postulo/plugins/registry.py:314-325 in the core). Its own connection (a connection has one plugin and one kind): folder address, user name, password. The address is the folder's full URL, since RFC 6764 defines well-known URIs for CardDAV and CalDAV only; on Nextcloud, /remote.php/dav/files/<user>/<folder>/. validate() keeps the sync's https:// rule.
The client learns files (src/postulo_dav/dav.py), work shared with #6:
- bytes, streamed: get() returns response.text and put() encodes a str (dav.py:223-242);
- a plain MKCOL: create_collection makes only address books and calendars (dav.py:180-200);
- plain folders in listings, which collections() drops (dav.py:162-167) and etags() skips (dav.py:216-218), at Depth 1 (RFC 4918 §9.1 lets a server refuse infinity).
Where a file goes comes from DocumentMetadata (kind_label, title, filename, company, role, sent_on, created_at), each part made safe for a path. Missing folders are made one level at a time: MKCOL answers 409 while a parent is missing (RFC 4918 §9.3.1).
Create, never overwrite. PUT with If-None-Match: *, as for new cards (dav.py:232). On 412 the file there is read back: the same size and SHA-256 (metadata.checksum) is a retry after a lost reply and counts as sent; anything else gets a suffixed name.
Answers a person can act on. 507 (RFC 4918 §11.5) says the folder is full and is retried. 401 raises ConnectionUnusable, so the connection is switched off with the reason (and its password forgotten, unless kept) instead of failing for every document (postulo/postulo#243).
The reference is the file's href, not Nextcloud's oc:fileid, which is not documented to survive a move.
Test reads the folder and DAV:quota-available-bytes (asked by name, RFC 4331 §3) and writes nothing, as postulo/postulo-dav#2 settled for the sync.
Size. Uploads through the form are capped at 20 MB (src/postulo/documents/forms.py:26 in the core) and renders are PDFs: one plain PUT is enough.
Scope. Every request stays under the chosen folder, and an href the server returns is checked against it first; #13 owns that rule for the whole client, and the store has it from the start. Most servers cannot narrow a credential to a folder (a Nextcloud app password can only lose file access; Fastmail's are per protocol), so the README recommends what does: WebDAV access to a Nextcloud public share that allows uploads (the share token as user name; it needs an administrator's sharing setting), or a Hetzner Storage Box sub-account, which reaches only its own sub-directory. Neither was tried against a real server.

New strings go through uv run postulo-messages extract, fr-fr and pt-pt filled on the commit (the test gates them), the 23 EU catalogues before a tag (uv run pytest -m release). The names needed (StorePlugin, DocumentMetadata, ExternalRef, ConnectionUnusable) are on the surface at the pinned 1d6cdd7e0, so no repin; tests/test_surface.py gains them.

Decisions

Copies or primary storage. Recommended: copies. Primary storage would break the core's rule that local media is the source of truth (src/postulo/documents/stores.py), a core change not proposed here.
Layout. By company (<company>/<role>/<date> <filename>, dated by sent_on, else created_at; <kind_label>/ without an application) or flat with descriptive names. Recommended: by company by default, flat as a choice. Separate folders per kind already work as two connections.
Deleting and reading back. A copy stays when its document is deleted here. Removing copies or browsing the folder needs the core to call delete() and browse(); storing does not, so not filed.

Needs from the core

Nothing. A folder picker instead of a typed address comes with #13.

Done when

A CV sent with an application lands in the folder under the chosen layout and shows as archived.
A clashing name gets a suffix, a retry after a lost reply leaves one file, a full folder is reported and retried, and a refused password switches the connection off.
The tests' in-memory DAV server gains plain folders and bytes; no request leaves the folder, even when a listing names an href elsewhere, and Test sends no MKCOL or PUT.
The README and package descriptions cover the store, the Nextcloud address and confined credentials.

From the list of DAV plugin improvements of 2026-09-30.

The list of DAV improvements asks for "the storage component (WebDAV), that for now is not implemented, but will be use for storing documents (on the user side)". This package registers one entry point today, `postulo.syncs` (`pyproject.toml:30-31`). postulo/postulo#241 (item 2) proposed a separate `postulo-webdav-store` repository reusing `postulo_dav.dav`; the list makes it a component of this plugin instead: a second entry point, `postulo.stores`, in this package, which takes up #241's item 2. The core has what a store needs: `StorePlugin.put(document, file, metadata, config, user)` returns an `ExternalRef` or `None`, per person, retried by the scheduler (`src/postulo/documents/archiving.py` in the core), for what was handed over and never for a PDF exported on its own (postulo/postulo#236). A switch per document kind comes free. `delete()` and `browse()` are never called (`src/postulo/plugins/base.py:781-793` in the core). **Reading:** "storing documents" means keeping a *copy* of each in the person's WebDAV folder; the alternative is under Decisions. ## What the plugin should do - **A store of its own** under `postulo.stores`, with its own name (say `webdav`): plugin names are unique across the instance (`find_any`, `src/postulo/plugins/registry.py:314-325` in the core). Its own connection (a connection has one plugin and one kind): folder address, user name, password. The address is the folder's full URL, since RFC 6764 defines well-known URIs for CardDAV and CalDAV only; on Nextcloud, `/remote.php/dav/files/<user>/<folder>/`. `validate()` keeps the sync's `https://` rule. - **The client learns files** (`src/postulo_dav/dav.py`), work shared with #6: - bytes, streamed: `get()` returns `response.text` and `put()` encodes a `str` (`dav.py:223-242`); - a plain `MKCOL`: `create_collection` makes only address books and calendars (`dav.py:180-200`); - plain folders in listings, which `collections()` drops (`dav.py:162-167`) and `etags()` skips (`dav.py:216-218`), at Depth 1 (RFC 4918 §9.1 lets a server refuse infinity). - **Where a file goes** comes from `DocumentMetadata` (`kind_label`, `title`, `filename`, `company`, `role`, `sent_on`, `created_at`), each part made safe for a path. Missing folders are made one level at a time: `MKCOL` answers 409 while a parent is missing (RFC 4918 §9.3.1). - **Create, never overwrite.** `PUT` with `If-None-Match: *`, as for new cards (`dav.py:232`). On 412 the file there is read back: the same size and SHA-256 (`metadata.checksum`) is a retry after a lost reply and counts as sent; anything else gets a suffixed name. - **Answers a person can act on.** 507 (RFC 4918 §11.5) says the folder is full and is retried. 401 raises `ConnectionUnusable`, so the connection is switched off with the reason (and its password forgotten, unless kept) instead of failing for every document (postulo/postulo#243). - **The reference** is the file's href, not Nextcloud's `oc:fileid`, which is not documented to survive a move. - **Test** reads the folder and `DAV:quota-available-bytes` (asked by name, RFC 4331 §3) and writes nothing, as postulo/postulo-dav#2 settled for the sync. - **Size.** Uploads through the form are capped at 20 MB (`src/postulo/documents/forms.py:26` in the core) and renders are PDFs: one plain `PUT` is enough. - **Scope.** Every request stays under the chosen folder, and an href the server returns is checked against it first; #13 owns that rule for the whole client, and the store has it from the start. Most servers cannot narrow a credential to a folder (a Nextcloud app password can only lose file access; Fastmail's are per protocol), so the README recommends what does: WebDAV access to a Nextcloud public share that allows uploads (the share token as user name; it needs an administrator's sharing setting), or a Hetzner Storage Box sub-account, which reaches only its own sub-directory. Neither was tried against a real server. New strings go through `uv run postulo-messages extract`, fr-fr and pt-pt filled on the commit (the test gates them), the 23 EU catalogues before a tag (`uv run pytest -m release`). The names needed (`StorePlugin`, `DocumentMetadata`, `ExternalRef`, `ConnectionUnusable`) are on the surface at the pinned 1d6cdd7e0, so no repin; `tests/test_surface.py` gains them. ## Decisions - **Copies or primary storage.** Recommended: copies. Primary storage would break the core's rule that local media is the source of truth (`src/postulo/documents/stores.py`), a core change not proposed here. - **Layout.** By company (`<company>/<role>/<date> <filename>`, dated by `sent_on`, else `created_at`; `<kind_label>/` without an application) or flat with descriptive names. Recommended: by company by default, flat as a choice. Separate folders per kind already work as two connections. - **Deleting and reading back.** A copy stays when its document is deleted here. Removing copies or browsing the folder needs the core to call `delete()` and `browse()`; storing does not, so not filed. ## Needs from the core Nothing. A folder picker instead of a typed address comes with #13. ## Done when - A CV sent with an application lands in the folder under the chosen layout and shows as archived. - A clashing name gets a suffix, a retry after a lost reply leaves one file, a full folder is reported and retried, and a refused password switches the connection off. - The tests' in-memory DAV server gains plain folders and bytes; no request leaves the folder, even when a listing names an href elsewhere, and *Test* sends no `MKCOL` or `PUT`. - The README and package descriptions cover the store, the Nextcloud address and confined credentials. From the list of DAV plugin improvements of 2026-09-30.

tiagoagueda added the

enhancement

label

2026-09-30 20:30:01 +00:00

tiagoagueda referenced this issue

2026-09-30 20:30:02 +00:00

Send each verified instance backup to a WebDAV folder, as a backup destination in this package #6

tiagoagueda referenced this issue

2026-09-30 20:30:02 +00:00

Pair by collection and UID with the address as a cache, never rewrite a remote UID, and re-pair a re-added connection instead of duplicating it #7

tiagoagueda referenced this issue

2026-09-30 20:30:04 +00:00