import_data: fails with IntegrityError when the account's own details already hold a number, address, link or identifier #375

New issue

Closed

opened 2026-10-01 10:46:14 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:46:14 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the archive importer and loading small archives into an account whose profile was already filled in.

Checked again on 2026-10-01 against main at 0a5cad587: still present.

The archive importer restores the profile's telephone numbers, postal addresses, web links and person identifiers as if the profile were empty. It chooses the primary row only among the archive's own rows and never looks at what the holder already has, so the one-primary constraints refuse the insert; a value the profile already holds is refused by the per-holder uniqueness rules in the same way. account_is_empty does not count any of these, so an account whose owner has only filled in Your details passes the guard and the whole import then rolls back with a raw traceback.

Where:
- core/importer.py:198-225, _restore_web_links; :228-260, _restore_postal_addresses; :263-297, _restore_phone_numbers: primary_taken starts empty or False and seen holds only the archive's rows.
- core/importer.py:380-388: PersonIdentifier.objects.get_or_create(profile=..., scheme=..., value=...) clashes with the one-per-scheme constraint when the profile holds the same scheme with another value.
- core/importer.py:130-138, account_is_empty: looks at applications, companies, CVs and experience only.
- core/management/commands/import_data.py:35: catches ArchiveError and BadZipFile only.
What happens (run against the snapshot, account_is_empty true in every case):
- a primary website on the profile, and a different primary website in the archive: UNIQUE constraint failed: core_weblink.content_type_id, core_weblink.object_id, core_weblink.kind;
- the same website: ... core_weblink.url;
- a primary address, and a different primary address in the archive: ... core_postaladdress.content_type_id, core_postaladdress.object_id; the same address: ... core_postaladdress.owner_id, core_postaladdress.comparable;
- a primary telephone number, and a different primary number in the archive: ... core_phonenumber.content_type_id, core_phonenumber.object_id;
- an ORCID on the profile and a different ORCID in the archive: ... accounts_personidentifier.profile_id, accounts_personidentifier.scheme.
Why it matters: the natural way to move to a new instance is to sign up, fill in one's details, and ask the operator to import the archive. That import fails every time, with a traceback rather than a sentence, until the operator deletes the person's rows by hand. The candidate-file import already handles this case (resume/candidate.py:1315, _hang: "The primary the account has stays the primary"); the archive importer does not.
Fix:
1. Seed each restorer's primary_taken and seen from the holder's existing rows (by kind for links, by comparable_form() for addresses, by URL for links).
2. For identifiers, keep the profile's value for a scheme it already holds and report the archive's as skipped.
3. Either count the profile's rows in account_is_empty or say in its docstring and the command's help that they do not count.
4. Report whatever is skipped through ImportReport.skipped, and turn any remaining IntegrityError into an ArchiveError with a readable message.

Done when

tests/test_backup.py (or a new tests/test_import_into_filled_profile.py): for each of web link, postal address, telephone number and person identifier, importing an archive into an account whose profile already has a primary row of that kind succeeds, keeps the existing primary, and lists anything skipped in the report.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the archive importer and loading small archives into an account whose profile was already filled in. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The archive importer restores the profile's telephone numbers, postal addresses, web links and person identifiers as if the profile were empty. It chooses the primary row only among the archive's own rows and never looks at what the holder already has, so the one-primary constraints refuse the insert; a value the profile already holds is refused by the per-holder uniqueness rules in the same way. `account_is_empty` does not count any of these, so an account whose owner has only filled in *Your details* passes the guard and the whole import then rolls back with a raw traceback. - **Where:** - `core/importer.py:198-225`, `_restore_web_links`; `:228-260`, `_restore_postal_addresses`; `:263-297`, `_restore_phone_numbers`: `primary_taken` starts empty or `False` and `seen` holds only the archive's rows. - `core/importer.py:380-388`: `PersonIdentifier.objects.get_or_create(profile=..., scheme=..., value=...)` clashes with the one-per-scheme constraint when the profile holds the same scheme with another value. - `core/importer.py:130-138`, `account_is_empty`: looks at applications, companies, CVs and experience only. - `core/management/commands/import_data.py:35`: catches `ArchiveError` and `BadZipFile` only. - **What happens** (run against the snapshot, `account_is_empty` true in every case): - a primary website on the profile, and a different primary website in the archive: `UNIQUE constraint failed: core_weblink.content_type_id, core_weblink.object_id, core_weblink.kind`; - the same website: `... core_weblink.url`; - a primary address, and a different primary address in the archive: `... core_postaladdress.content_type_id, core_postaladdress.object_id`; the same address: `... core_postaladdress.owner_id, core_postaladdress.comparable`; - a primary telephone number, and a different primary number in the archive: `... core_phonenumber.content_type_id, core_phonenumber.object_id`; - an ORCID on the profile and a different ORCID in the archive: `... accounts_personidentifier.profile_id, accounts_personidentifier.scheme`. - **Why it matters:** the natural way to move to a new instance is to sign up, fill in one's details, and ask the operator to import the archive. That import fails every time, with a traceback rather than a sentence, until the operator deletes the person's rows by hand. The candidate-file import already handles this case (`resume/candidate.py:1315`, `_hang`: "The primary the account has stays the primary"); the archive importer does not. - **Fix:** 1. Seed each restorer's `primary_taken` and `seen` from the holder's existing rows (by kind for links, by `comparable_form()` for addresses, by URL for links). 2. For identifiers, keep the profile's value for a scheme it already holds and report the archive's as skipped. 3. Either count the profile's rows in `account_is_empty` or say in its docstring and the command's help that they do not count. 4. Report whatever is skipped through `ImportReport.skipped`, and turn any remaining `IntegrityError` into an `ArchiveError` with a readable message. ## Done when - `tests/test_backup.py` (or a new `tests/test_import_into_filled_profile.py`): for each of web link, postal address, telephone number and person identifier, importing an archive into an account whose profile already has a primary row of that kind succeeds, keeps the existing primary, and lists anything skipped in the report.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:46:14 +00:00

tiagoagueda added the

tier

bug

labels

2026-10-01 10:46:14 +00:00

tiagoagueda referenced this issue

2026-10-03 01:58:39 +00:00

Export and import one kind of record as JSON: companies, contacts, listings and applications, each as a file of its own #659

~~tiagoagueda referenced this issue 2026-10-03 01:59:01 +00:00~~

Your career: publications, typed and shaped like BibTeX entries #687