Data protection: a contact's document omits the applications and interviews that name them; erasure and the dry run miss some #370

New issue

Closed

opened 2026-10-01 10:46:12 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:46:12 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, by comparing what each of the data-protection functions and the contact merge treat as "pointing at a contact".

Checked again on 2026-10-01 against main at 0a5cad587: still present.

#297 built a data-subject export of "everything the instance holds on one other person" and an erasure that says what it removed. Four functions each decide for themselves what refers to a contact, and they disagree. The merge moves four kinds of reference; the subject's document carries only one of them, and the erasure report and the retention dry run each leave out a different one.

Where:
- core/gdpr.py:150-175 (contact_document): the contact row, numbers, addresses, links, listing_events and plugin rows. Nothing for the applications where the person is the main contact (Application.contact), the ones they referred (Application.referred_by), or the interviews they were at (Interview.contacts, applications/models.py:722-724).
- core/gdpr.py:279-309 (erase_contact): unlinked counts applications, referrals and listing entries, not interview seats; the M2M rows go with the delete and the report never says so.
- core/gdpr.py:331-363 (retention_dry_run): would_remove counts numbers, addresses, links and applications (as contact or referrer), not listing entries or interviews.
- jobs/merging.py:548-621 (plan_contacts) and :645-701 (merge_contacts): the one place that handles all four, and backs itself with a collector check (_refuse_what_is_left).
- core/gdpr.py:81-123: the subject document's number, address and link rows restate the archive's field lists by hand (core/export.py:149-167) and already differ from them: "primary" where the archive writes is_primary.
What happens: one contact is the main contact on one application, the referrer on another, sits on one interview and is the source of one listing-history entry.
- The document from Export this person's data (jobs/views.py:826) has contact, phone_numbers, postal_addresses, web_links, listing_events and plugins; neither application and not the interview appear.
- The retention dry run reports applications_unlinked: 2 and nothing for the listing entry or the interview.
- Erasing the contact reports one application, one referral and one listing entry unlinked; the interview loses the person and the sentence shown afterwards (jobs/views.py:800) does not mention it.
Why it matters: an operator answering an access request under Article 15 hands over a document that leaves out that the person referred the account holder and took part in an interview, while the module docstring promises the document is complete "or has to say it is not". The erasure and the dry run understate what they touch, which is what they exist to state.
Fix:
- One function, for example core.gdpr.references(contact), that returns every row pointing at a contact: applications as contact and as referrer, interview seats, listing entries.
- Use it in contact_document (new sections applications, referrals and interviews, named by role and company the way _listing_event_rows names a listing, bumping DOCUMENT_VERSION), in erase_contact (an interviews count in unlinked and a sentence for it), in retention_dry_run, and in plan_contacts.
- Build the subject document's rows with export._fields and the archive's PHONE_NUMBER_FIELDS, POSTAL_ADDRESS_FIELDS and WEB_LINK_FIELDS, so the two cannot drift.

Related to #297 (closed), which built the export, the erasure and the dry run.

Done when

tests/test_gdpr.py references one contact in all four ways and asserts that the document, the erasure report and the dry run each account for every one.
A test in tests/test_gdpr.py walks Contact._meta.related_objects (as merging._refuse_what_is_left does with the collector) and fails when a relation to Contact exists that references() does not cover.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, by comparing what each of the data-protection functions and the contact merge treat as "pointing at a contact". Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. #297 built a data-subject export of "everything the instance holds on one other person" and an erasure that says what it removed. Four functions each decide for themselves what refers to a contact, and they disagree. The merge moves four kinds of reference; the subject's document carries only one of them, and the erasure report and the retention dry run each leave out a different one. - **Where:** - `core/gdpr.py:150-175` (`contact_document`): the contact row, numbers, addresses, links, `listing_events` and plugin rows. Nothing for the applications where the person is the main contact (`Application.contact`), the ones they referred (`Application.referred_by`), or the interviews they were at (`Interview.contacts`, `applications/models.py:722-724`). - `core/gdpr.py:279-309` (`erase_contact`): `unlinked` counts applications, referrals and listing entries, not interview seats; the M2M rows go with the delete and the report never says so. - `core/gdpr.py:331-363` (`retention_dry_run`): `would_remove` counts numbers, addresses, links and applications (as contact or referrer), not listing entries or interviews. - `jobs/merging.py:548-621` (`plan_contacts`) and `:645-701` (`merge_contacts`): the one place that handles all four, and backs itself with a collector check (`_refuse_what_is_left`). - `core/gdpr.py:81-123`: the subject document's number, address and link rows restate the archive's field lists by hand (`core/export.py:149-167`) and already differ from them: `"primary"` where the archive writes `is_primary`. - **What happens:** one contact is the main contact on one application, the referrer on another, sits on one interview and is the source of one listing-history entry. - The document from *Export this person's data* (`jobs/views.py:826`) has `contact`, `phone_numbers`, `postal_addresses`, `web_links`, `listing_events` and `plugins`; neither application and not the interview appear. - The retention dry run reports `applications_unlinked: 2` and nothing for the listing entry or the interview. - Erasing the contact reports one application, one referral and one listing entry unlinked; the interview loses the person and the sentence shown afterwards (`jobs/views.py:800`) does not mention it. - **Why it matters:** an operator answering an access request under Article 15 hands over a document that leaves out that the person referred the account holder and took part in an interview, while the module docstring promises the document is complete "or has to say it is not". The erasure and the dry run understate what they touch, which is what they exist to state. - **Fix:** - One function, for example `core.gdpr.references(contact)`, that returns every row pointing at a contact: applications as contact and as referrer, interview seats, listing entries. - Use it in `contact_document` (new sections `applications`, `referrals` and `interviews`, named by role and company the way `_listing_event_rows` names a listing, bumping `DOCUMENT_VERSION`), in `erase_contact` (an `interviews` count in `unlinked` and a sentence for it), in `retention_dry_run`, and in `plan_contacts`. - Build the subject document's rows with `export._fields` and the archive's `PHONE_NUMBER_FIELDS`, `POSTAL_ADDRESS_FIELDS` and `WEB_LINK_FIELDS`, so the two cannot drift. Related to #297 (closed), which built the export, the erasure and the dry run. ## Done when - `tests/test_gdpr.py` references one contact in all four ways and asserts that the document, the erasure report and the dry run each account for every one. - A test in `tests/test_gdpr.py` walks `Contact._meta.related_objects` (as `merging._refuse_what_is_left` does with the collector) and fails when a relation to `Contact` exists that `references()` does not cover.

tiagoagueda added this to the 0.5.0 milestone

2026-10-01 10:46:12 +00:00

tiagoagueda added the

bug

label

2026-10-01 10:46:12 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 10:05:00 +00:00

Account for every reference to a contact in the document, erasure and dry run

tiagoagueda closed this issue

2026-10-04 10:05:00 +00:00