Events as iCalendar: export every dated thing, import an .ics file, and replace the hand-written reader and writer with a library #661

New issue

Closed

opened 2026-10-03 01:58:41 +00:00 by tiagoagueda · 1 comment

tiagoagueda commented

2026-10-03 01:58:41 +00:00

Owner

What exists today

A hand-written writer. applications/ical.py (270 lines): escape (:63), parameter (:74), fold (:92), event_lines for an interview (:137), DayEntry and day_lines for a whole day (:202, :220), calendar (:248, METHOD:PUBLISH). Its docstring says a dependency “would bring the rest of the standard along for nothing” (:1-6). Content-line injection was closed in #218; the interview UID is still written unescaped and an unknown outcome raises (#450, open).
What leaves. InterviewCalendarView (applications/views.py:1023) serves interviews.ics: upcoming interviews, and, for the next 183 days (FEED_DAYS, :1020), the application deadlines, the listing closing dates and the offer answer-by dates as whole days (dated_days, :1058, read through agenda.events_between, applications/agenda.py:197); and one interview. The API serves the interviews only (api/routers/interviews.py:72-77, #451); the page's feed needs a browser session (#451). Reminders are on the calendar page (agenda.py:75) and in no file.
Identity. An interview has a stored uid, unique per owner (applications/models.py:692-694, :734, :762). A whole day's UID is a UUID over its absolute address (ical.py:202-218), and the view builds that address from the request (views.py:1076, build_absolute_uri): an instance reached under two allowed host names would give one deadline two UIDs. Not reproduced; tests/test_deadlines.py:270 fetches twice from one host. A reminder has no UID (applications/models.py:600).
No import. Nothing in the core reads .ics. An interview cannot exist without an application (Interview.application is required), and schedule_interview (applications/services.py:378) is how one is made, with its timeline entry and reminder.
The plugin's reader. postulo-dav/src/postulo_dav/text.py:234-303: text_to_events does not track nested components, so a VALARM's text replaces an interview's notes (postulo/postulo-dav#11); parse_moment reads an unknown zone, a floating time or a date as UTC and moves the interview (postulo/postulo-dav#24). Its open issues design the rest: pairing an event by collection and UID (#7, #11), reminders and milestones as VTODO (#12). Core's surface offers event_lines, calendar_status, reschedule_interview and settle_interview (plugins/api.py:199-226); escape is deliberately not on it, so the plugin keeps its own copy.

Checked against main at f09b6010b.

Proposal

Commit one: the dependency and the reader. Add icalendar with a floor of 7.1.3 (see Prior art); a module that reads text into plain values (uid, start, end or day, summary, location, description with an alarm's text kept apart, status, whether it recurs) and refuses what it cannot place. Its bounds are checked before the parser runs: size, number of components, nesting depth, line length.
Import an .ics through the review page #659 factors out: each VEVENT or VTODO listed with what it would become, nothing made until the person chooses. A timed event becomes an interview on an application they pick, through schedule_interview (the kind chosen; the timeline records the file); an event or a task becomes a reminder; or it is skipped. A UID equal to an interview's is shown as already there, so the same file twice adds nothing. Not read: ATTACH, URL and CONFERENCE are never fetched, ATTENDEE is not turned into contacts (a stranger's names), VALARM is ignored, RRULE and RDATE are refused with a note, as postulo-dav#11 leaves recurrence out.
Time is never guessed. A TZID is resolved through zoneinfo, then the library's Windows-name table, then the file's own VTIMEZONE; one that does not resolve, a floating time and a date alone are shown on the review page as needing a choice, and never silently become UTC (postulo-dav#24).
Commit two: the writer on the library. The interview and whole-day output moves to it, with golden tests that parse the old and the new output with the library and compare components. #450 is fixed by the move, not beside it.
Commit three: export the rest. Reminders as VTODO (SUMMARY, DUE, STATUS, COMPLETED, RELATED-TO the application), in an address of their own: the interviews feed is what existing subscribers read, and its meaning is not changed under them. A Download as iCalendar on the calendar page for the kinds its legend shows, from agenda.events_between, the one place that decides what a deadline is.
On the plugin surface, the reader's plain values and the VTODO mapping, under names to be chosen, with their lines in Writing a plugin and a repin of postulo-dav, which then drops its parser. The library itself is not on the surface; a plugin that wants it declares its own dependency.

Decisions

Library, or hand-written. Recommended: icalendar, for reading and writing. The new reader is new code, which is what the rule asks to check; the writer is touched to add VTODO and to fix #450, which is when #653 says to look again. Two defects of the plugin's parser, a zone read as UTC and an alarm's text taken for the notes, are what a component tree and a zone table are for; that argues the same way.
The day-entry UID. Options: the address (today, host-dependent); POSTULO_PUBLIC_URL-derived (config/settings/base.py:538, optional); a stored column. Recommended: POSTULO_PUBLIC_URL where set, else the address as today; and a stored uid on Reminder, minted as an interview's is, because a pairing by UID (postulo-dav#7) needs one that survives an export and a move. The column is a migration and an archive bump.
Reminders in the interviews feed. Recommended: no; their own address.
What an all-day event becomes on import. Options: a reminder on that day; an application's deadline (one per application); skip. Recommended: a reminder, because a deadline is one date per application and an import should not overwrite it.
METHOD:REQUEST invitations. Recommended: read as events; no reply is ever sent.

Prior art

Looked for under #653, 2026-10-02:

icalendar 7.3.0 (2026-08-19), BSD-2-Clause, Plone Foundation (collective/icalendar, pushed 2026-10-02, 169 open issues and pull requests); needs python-dateutil and tzdata, and typing-extensions below Python 3.13, of which tzdata and typing-extensions are in uv.lock already and python-dateutil and six would be new; it ships a Windows-to-IANA table and a fuzzing corpus in the repository. OSV: one advisory (listed twice, as GHSA and PYSEC), GHSA-cv84-9p8j-fj68 / CVE-2026-55099, High (CVSS 7.5): Component.__eq__ takes exponential time in the nesting depth, and the parser has no depth limit; introduced in 7.1.0, fixed in 7.1.3 (2026-06-15). Patched, so not the rule's exception; the floor is >=7.1.3 and the importer bounds depth and never compares parsed components. Adopt.
ical 14.2.0 (2026-09-11), Apache-2.0, allenporter/ical, used by Home Assistant; needs pydantic (locked, 2.13.5), python-dateutil and tzdata; Python 3.11 or later; no advisory. Strict validation and built-in recurrence. Not preferred: fourteen major versions and a coupling to pydantic's upgrades. A fair second choice if icalendar disappoints.
ics 0.7.3 (2026-04-15), Apache-2.0; needs arrow, tatsu below 5.16, attrs, six, python-dateutil; a 0.8 development line since the same day; tatsu's latest release lists BSD-4-Clause (advertising clause), not verified for the pinned range. Reject.
recurring-ical-events 3.8.2, LGPL-3.0-or-later: only to expand recurrences, which this issue leaves out.
django-scheduler (0.12.0, BSD-3-Clause, no advisory) is event models with recurrence rules, not an iCalendar reader; it would not replace the calendar page, which draws other records (agenda.py:15-21). Reject.
Hand-written (ical.py, text.py): the precedent, and the cost recorded above.
For the DAV plugin's transport, a CalDAV client library (caldav 3.3.1, GPL-3.0-or-later or Apache-2.0) is a question for that repository.

Builds on / Needed by

Builds on #14, #204, #238, #218, #450, #451, #330, #331, postulo/postulo-dav#7, #11, #12, #24 and #653, #659. Needed by postulo-dav#11 and #12.

Done when

tests/test_ical_import.py (new): a timed event, a task and an all-day event each read as the table says; a TZID in a Windows name resolves; a floating time and a date alone are never UTC unseen; an alarm's text never reaches the notes; a recurring event is refused with a note; a UID already held is shown as present; a file nested thousands of components deep, one with an enormous number of components and one past the size bound are refused before parsing.
tests/test_interviews.py, tests/test_deadlines.py and tests/security/test_exports.py hold the new writer to what they hold today, including that no control character, lone carriage return or unescaped UID can add a property; tests/security/test_feeds_and_pictures.py covers the new download and the reminders address.
tests/test_security_audit.py gains a test that the lock holds icalendar 7.1.3 or later.
tests/e2e/test_ical_import.py (new): upload, choose an application, confirm; the interview exists with its timeline entry. Pages join tests/test_page_coverage.py and tests/security/test_isolation_sweep.py; new surface names get their line in Writing a plugin (tests/test_wiki_surface.py) and a test in tests/test_plugin_surface.py.
If Reminder gains a uid: a migration, the archive format (29 today) moved to one more than whatever it is when the commit lands, the importer and the API's reminder shape carrying it, the candidate format (4) untouched. Strings in en-GB, fr-FR and pt-PT (drafts) and Weblate (#349); the wiki's calendar page.

From the list of improvements of 2026-10-02.

From the maintainer's list: “Preferable export/import format of events details is iCalendar. A step closer to official DAV plugin.” Postulo writes iCalendar for interviews and for three kinds of whole day, by hand, and reads none. The DAV plugin reads and writes it with a second hand-written parser, and the two have already disagreed about escaping, time zones and alarms. This issue completes the export, adds the import, and settles the library question under the rule of #653; it is three commits, in the order given. ## What exists today - **A hand-written writer.** `applications/ical.py` (270 lines): `escape` (`:63`), `parameter` (`:74`), `fold` (`:92`), `event_lines` for an interview (`:137`), `DayEntry` and `day_lines` for a whole day (`:202`, `:220`), `calendar` (`:248`, `METHOD:PUBLISH`). Its docstring says a dependency “would bring the rest of the standard along for nothing” (`:1-6`). Content-line injection was closed in #218; the interview `UID` is still written unescaped and an unknown outcome raises (#450, open). - **What leaves.** `InterviewCalendarView` (`applications/views.py:1023`) serves `interviews.ics`: upcoming interviews, and, for the next 183 days (`FEED_DAYS`, `:1020`), the application deadlines, the listing closing dates and the offer answer-by dates as whole days (`dated_days`, `:1058`, read through `agenda.events_between`, `applications/agenda.py:197`); and one interview. The API serves the interviews only (`api/routers/interviews.py:72-77`, #451); the page's feed needs a browser session (#451). **Reminders are on the calendar page (`agenda.py:75`) and in no file.** - **Identity.** An interview has a stored `uid`, unique per owner (`applications/models.py:692-694`, `:734`, `:762`). A whole day's UID is a UUID over its absolute address (`ical.py:202-218`), and the view builds that address from the request (`views.py:1076`, `build_absolute_uri`): an instance reached under two allowed host names would give one deadline two UIDs. Not reproduced; `tests/test_deadlines.py:270` fetches twice from one host. A reminder has no UID (`applications/models.py:600`). - **No import.** Nothing in the core reads `.ics`. An interview cannot exist without an application (`Interview.application` is required), and `schedule_interview` (`applications/services.py:378`) is how one is made, with its timeline entry and reminder. - **The plugin's reader.** `postulo-dav/src/postulo_dav/text.py:234-303`: `text_to_events` does not track nested components, so a `VALARM`'s text replaces an interview's notes (postulo/postulo-dav#11); `parse_moment` reads an unknown zone, a floating time or a date as UTC and moves the interview (postulo/postulo-dav#24). Its open issues design the rest: pairing an event by collection and `UID` (#7, #11), reminders and milestones as `VTODO` (#12). Core's surface offers `event_lines`, `calendar_status`, `reschedule_interview` and `settle_interview` (`plugins/api.py:199-226`); `escape` is deliberately not on it, so the plugin keeps its own copy. Checked against `main` at `f09b6010b`. ## Proposal 1. **Commit one: the dependency and the reader.** Add `icalendar` with a floor of 7.1.3 (see *Prior art*); a module that reads text into plain values (uid, start, end or day, summary, location, description with an alarm's text kept apart, status, whether it recurs) and refuses what it cannot place. Its bounds are checked before the parser runs: size, number of components, nesting depth, line length. 2. **Import** an `.ics` through the review page #659 factors out: each `VEVENT` or `VTODO` listed with what it would become, **nothing made until the person chooses**. A timed event becomes an interview on an application they pick, through `schedule_interview` (the kind chosen; the timeline records the file); an event or a task becomes a reminder; or it is skipped. A `UID` equal to an interview's is shown as already there, so the same file twice adds nothing. Not read: `ATTACH`, `URL` and `CONFERENCE` are never fetched, `ATTENDEE` is not turned into contacts (a stranger's names), `VALARM` is ignored, `RRULE` and `RDATE` are refused with a note, as postulo-dav#11 leaves recurrence out. 3. **Time is never guessed.** A `TZID` is resolved through `zoneinfo`, then the library's Windows-name table, then the file's own `VTIMEZONE`; one that does not resolve, a floating time and a date alone are shown on the review page as needing a choice, and never silently become UTC (postulo-dav#24). 4. **Commit two: the writer on the library.** The interview and whole-day output moves to it, with golden tests that parse the old and the new output with the library and compare components. #450 is fixed by the move, not beside it. 5. **Commit three: export the rest.** Reminders as `VTODO` (`SUMMARY`, `DUE`, `STATUS`, `COMPLETED`, `RELATED-TO` the application), in an address of their own: the interviews feed is what existing subscribers read, and its meaning is not changed under them. A *Download as iCalendar* on the calendar page for the kinds its legend shows, from `agenda.events_between`, the one place that decides what a deadline is. 6. **On the plugin surface,** the reader's plain values and the VTODO mapping, under names to be chosen, with their lines in *Writing a plugin* and a repin of postulo-dav, which then drops its parser. The library itself is not on the surface; a plugin that wants it declares its own dependency. ## Decisions - **Library, or hand-written.** **Recommended:** `icalendar`, for reading and writing. The new reader is new code, which is what the rule asks to check; the writer is touched to add `VTODO` and to fix #450, which is when #653 says to look again. Two defects of the plugin's parser, a zone read as UTC and an alarm's text taken for the notes, are what a component tree and a zone table are for; that argues the same way. - **The day-entry UID.** Options: the address (today, host-dependent); `POSTULO_PUBLIC_URL`-derived (`config/settings/base.py:538`, optional); a stored column. **Recommended:** `POSTULO_PUBLIC_URL` where set, else the address as today; and a stored `uid` on `Reminder`, minted as an interview's is, because a pairing by UID (postulo-dav#7) needs one that survives an export and a move. The column is a migration and an archive bump. - **Reminders in the interviews feed.** **Recommended:** no; their own address. - **What an all-day event becomes on import.** Options: a reminder on that day; an application's deadline (one per application); skip. **Recommended:** a reminder, because a deadline is one date per application and an import should not overwrite it. - **`METHOD:REQUEST` invitations.** **Recommended:** read as events; no reply is ever sent. ## Prior art Looked for under #653, 2026-10-02: - **icalendar** 7.3.0 (2026-08-19), BSD-2-Clause, Plone Foundation (`collective/icalendar`, pushed 2026-10-02, 169 open issues and pull requests); needs `python-dateutil` and `tzdata`, and `typing-extensions` below Python 3.13, of which `tzdata` and `typing-extensions` are in `uv.lock` already and `python-dateutil` and `six` would be new; it ships a Windows-to-IANA table and a fuzzing corpus in the repository. OSV: one advisory (listed twice, as GHSA and PYSEC), **GHSA-cv84-9p8j-fj68 / CVE-2026-55099**, High (CVSS 7.5): `Component.__eq__` takes exponential time in the nesting depth, and the parser has no depth limit; introduced in 7.1.0, **fixed in 7.1.3** (2026-06-15). Patched, so not the rule's exception; the floor is `>=7.1.3` and the importer bounds depth and never compares parsed components. **Adopt.** - **ical** 14.2.0 (2026-09-11), Apache-2.0, `allenporter/ical`, used by Home Assistant; needs `pydantic` (locked, 2.13.5), `python-dateutil` and `tzdata`; Python 3.11 or later; no advisory. Strict validation and built-in recurrence. **Not preferred:** fourteen major versions and a coupling to pydantic's upgrades. A fair second choice if `icalendar` disappoints. - **ics** 0.7.3 (2026-04-15), Apache-2.0; needs `arrow`, `tatsu` below 5.16, `attrs`, `six`, `python-dateutil`; a 0.8 development line since the same day; `tatsu`'s latest release lists BSD-4-Clause (advertising clause), not verified for the pinned range. **Reject.** - **recurring-ical-events** 3.8.2, LGPL-3.0-or-later: only to expand recurrences, which this issue leaves out. - **`django-scheduler`** (0.12.0, BSD-3-Clause, no advisory) is event *models* with recurrence rules, not an iCalendar reader; it would not replace the calendar page, which draws other records (`agenda.py:15-21`). **Reject.** - **Hand-written** (`ical.py`, `text.py`): the precedent, and the cost recorded above. - For the DAV plugin's transport, a CalDAV client library (`caldav` 3.3.1, GPL-3.0-or-later or Apache-2.0) is a question for that repository. ## Builds on / Needed by Builds on #14, #204, #238, #218, #450, #451, #330, #331, postulo/postulo-dav#7, #11, #12, #24 and #653, #659. Needed by postulo-dav#11 and #12. ## Done when - `tests/test_ical_import.py` (new): a timed event, a task and an all-day event each read as the table says; a `TZID` in a Windows name resolves; a floating time and a date alone are never UTC unseen; an alarm's text never reaches the notes; a recurring event is refused with a note; a `UID` already held is shown as present; a file nested thousands of components deep, one with an enormous number of components and one past the size bound are refused before parsing. - `tests/test_interviews.py`, `tests/test_deadlines.py` and `tests/security/test_exports.py` hold the new writer to what they hold today, including that no control character, lone carriage return or unescaped `UID` can add a property; `tests/security/test_feeds_and_pictures.py` covers the new download and the reminders address. - `tests/test_security_audit.py` gains a test that the lock holds `icalendar` 7.1.3 or later. - `tests/e2e/test_ical_import.py` (new): upload, choose an application, confirm; the interview exists with its timeline entry. Pages join `tests/test_page_coverage.py` and `tests/security/test_isolation_sweep.py`; new surface names get their line in *Writing a plugin* (`tests/test_wiki_surface.py`) and a test in `tests/test_plugin_surface.py`. - If `Reminder` gains a `uid`: a migration, the archive format (29 today) moved to one more than whatever it is when the commit lands, the importer and the API's reminder shape carrying it, the candidate format (4) untouched. Strings in en-GB, fr-FR and pt-PT (drafts) and Weblate (#349); the wiki's calendar page. From the list of improvements of 2026-10-02.

tiagoagueda added this to the 0.6.0 milestone

2026-10-03 01:58:41 +00:00

tiagoagueda added the

enhancement

label

2026-10-03 01:58:41 +00:00

tiagoagueda referenced this issue

2026-10-03 01:59:06 +00:00

Your career: presentations, conferences and seminars, one section with a role and a type #694

tiagoagueda referenced this issue from Postulo/postulo-dav

2026-10-03 01:59:29 +00:00

Pair an event in a chosen calendar with an interview, make an interview from one, and stop an alarm's text replacing the notes #11