Write every language code as a BCP 47 tag in its canonical form: pt-BR, not pt-br or pt_BR #337
Labels
No labels
accessibility
authentication
breaking change
bug
documentation
enhancement
interface
internationalisation
observability
performance
security
tier
1
tier
2
tier
3
tier/4
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
Postulo/postulo#337
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The requirement: all language codes must follow BCP 47. A BCP 47 tag (RFC 5646) is made of subtags from the IANA registry joined by hyphens, and §2.1.1 gives the form to write one in: the language in lower case, a script in title case, a region in upper case (
pt-BR,en-GB,sr-Cyrl). Tags compare without regard to case, sopt-bris a valid tag written the wrong way, andpt_BRis not a tag at all: it is a gettext locale name.Postulo writes every code "as Django writes it" (
src/postulo/core/languages.py:136): all lower case. That is what is stored, compared, exported and sent out, except in a few places that already disagree with it.What exists today
NATIVE_NAMESholds 69 codes, the regional ones asen-gb,fr-fr,pt-ptandpt-br. The same spellings keyFLAG_COUNTRIES,PLURAL_FORMS,SOURCE,FROM_LANGUAGEincore/phones.pyandlocale/status.json;LANGUAGE_CODEis"en-gb".CharField(max_length=10)(or a list) with no check:Profile.languageandrecord_language;SiteSettings.default_languageandoffered_languages;languageonCV,CoverLetter,UploadedDocumentandRenderedDocument;resume.Translation.language. The pages writept-br; nothing normalises on save.<html lang>of the interface (templates/base.html;500.htmlhard-codesen-gb) and of every rendered document, which is where a PDF takes its language;lang=on options and spans;languageon CVs, letters and documents in the API, a plain string in the OpenAPI description; the personal archive;Notification.languageand the webhook payload;DocumentMetadata.language, handed to store plugins.language_tag()indocuments/docx.pywritespt-PTandzh-Hans, "BCP 47 is case-insensitive and Word is not always". It is the only mention of BCP 47 in the repository;"en-GB"indocuments/models.py:108andnotifications/service.py:30;"language": "en-GB"and Writing a plugin says "as a Postulo code (pt-PT)", where the code sendsen-gbandpt-pt.^[a-z]{2,3}(?:-[a-z0-9]{2,8}){0,2}$after lower-casing and says a code is "like en-gb or pt-pt" (resume/candidate.py); Europass and captured pages lower-case what they read; the archive import takes what the file holds.translatable.normalise,patterns.tag_ofandeuropass._localelower-case;docx.language_tagwrites the canonical form.What a framework forces, and stays
Language:header are gettext locale names (fr_FR,pt_BR), made by Django'sto_locale. They are not language tags, and are derived from one inlanguages.locale_dir_nameandmessages_tool.get_language()returns, so{{ LANGUAGE_CODE }}and theContent-Languageheader are lower case. It does not force the case ofLANGUAGE_CODEor of theLANGUAGESkeys:get_languages()lower-cases the keys andto_localeits input (read in Django 6.1.1's source, not run).Proposal
core/languages.pygains the canonical writer (whatdocx.language_tagdoes; underscores accepted in, never written out) and a comparison that ignores case and falls back to the primary subtag (RFC 4647 §3.4). They replace the three lower-casers and the exact-match lookups (flag_country,PLURAL_FORMS.get,site.offers,status.get,NATIVE_NAMES.get,{% language_flag %}).en-GB,fr-FR,pt-PT,pt-BR, instatus.jsontoo.ca-ES-valencia(RFC 5646 §4.4.1 advises room for 35).<html lang>re-cased from Django's value. The API'slanguageis described as a BCP 47 tag.FORMAT_VERSIONgoes up, with its note.nb; the ESCO data is keyedno(jobs/management/commands/fetch_esco.py:96), soesco._reading("nb")falls back to English and a Norwegian reader gets English occupation and skill names. Europass mapsnortono, and a page sayinglang="no"matches nothing. The comparison needs to knownoandnbtogether.sr; the catalogue is Cyrillic.srhas no Suppress-Script in the registry, sosr-Cyrlis the tag that says so.SCRIPTSnames scripts in English ("Cyrillic","Ethiopic") where BCP 47 has subtags (Cyrl,Ethi), andis_rtlreads the primary subtag only, soku-Latnwould be drawn right to left.tests/test_smoke.py,test_theme.py,test_translations.py,test_language_variants.pyand others);docs/TRANSLATING.mdmixespt-brandpt-BRin one file.Decisions
pt-brfollows BCP 47 to the letter, but Postulo writes the canonical form wherever it writes a tag. It is the form the RFC gives, the one the wiki already promises and Word needs, and one form ends the mixture.LANGUAGE_CODEand theLANGUAGESkeys. Decided with the above: canonical. Django accepts either, and the list is then written once.Content-Languageheader. Decided with the above: canonical. Django writes it in lower case, so the middleware re-cases it.pt-AOsaves. No library is added.srorsr-Cyrl. Decided on 2026-10-01:sr-Cyrl, now. The list sayssr-Cyrl, the catalogue directories becomesr_Cyrl, and a storedsris migrated.A breaking change for whatever reads these codes: API clients, webhook receivers, store plugins and the archive see
pt-PTwhere they sawpt-pt. It goes in the changelog as one.The other repositories
postulo/postulo-chromium#12, postulo/postulo-dav#14, postulo/postulo-imap#3, postulo/postulo-apprise#5, postulo/postulo-paperless#5 and postulo/postulo-helloworld#4 follow this one. postulo-firefox holds no language code and follows through its pin; postulo-templates holds none yet; postulo-mcp only relays the API's
languageand takes its OpenAPI snapshot again.Done when
<html lang>, and finds canonical tags.pt_brandPT-brtyped or imported are stored aspt-BR.docs/TRANSLATING.mdand the wiki state the rule once: a code is a BCP 47 tag in its canonical form, and a locale directory's name is gettext's, derived from it.Requested on 2026-10-01: all language codes must follow BCP 47.
Decided on 2026-10-01: always go canonical.
Postulo writes a language code in its canonical BCP 47 form wherever it writes one. The body's decisions are updated to say so:
LANGUAGE_CODEand theLANGUAGESkeys in canonical form;Content-Languageheader re-cased, since "always" leaves no place for Django's lower-case form where Postulo can change it.Locale directory names and PO headers are not language tags and stay gettext's.
Still open: whether tags are checked against the registry or only for shape, and
srorsr-Cyrl. Canonical form does not settle the second:sris already canonical, and the question is whether to state the script.Final decision on translations (maintainer, 2026-10-01)
What this changes here
Only the spelling of what the decision names, which is this issue's subject:
fr_CA,en_CA).LANGUAGESentry and a catalogue of its own, seeded from en-GB.zh-Hansandzh-Hant, which bears on the open point here about script subtags (srversussr-Cyrl).Decided on 2026-10-01, which closes the two points left open here:
pt-AOstill saves. No library is added.sr-Cyrl, now. The list sayssr-Cyrl, the catalogue directories becomesr_Cyrlin every catalogue set, and a storedsris migrated to it. A browser asking forsr,sr-RSorsr-Latnis still given this catalogue, as it is today.Checked against the registry dated 2026-09-17: all 69 codes of the list are registered, none is deprecated,
srhas no suppressed script, andnbbelongs to the macrolanguageno, which is the relation the Norwegian ESCO names need.The work is on the branch
337-bcp47-language-codesand is merged intomainonly on the maintainer's word.What follows for the other repositories: each plugin's own
locale/srbecomeslocale/sr_Cyrlwhen it moves its pin, before its firstextract.Landed on
mainasa6363a397, with the wiki inpostulo.wiki@9fff446.What the repositories that follow this one need to know, since some of it differs from what the body proposed:
en-GB,fr-FR,pt-PT,pt-BRandsr-Cyrl.postulo-messagesrefuses a code spelt otherwise than the list spells it (po_path("fr-fr")raises), so a test that still names("fr-fr", "pt-pt")fails loudly once the pin moves, instead of holding fewer catalogues than it thinks.sr_Cyrl. Renamelocale/srtolocale/sr_Cyrlbefore the firstextractunder the new pin, or the tool writes an emptysr_Cyrlbeside the old directory.postulo.plugins.api):language_tag,is_language_tag,language_matchandlanguage_matches. They are Postulo's own writer, shape check and comparison; a plugin that compares or stores a code uses these and writes no normaliser of its own.languageis described as a BCP 47 tag and a letter's is filled now, sopostulo-mcpretakes its OpenAPI snapshot.Notification.languageandDocumentMetadata.languageare canonical tags. A receiver compares without regard to case.What it did not do: Armenian and Georgian are still missing from
languages.SCRIPTS, which is #455, andlocale/status.jsonwas re-keyed and not regenerated, which is #495.