Translations: the catalogue check passes a stray '%' that makes the page formatting it fail #497

New issue

Closed

opened 2026-10-01 10:47:06 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:47:06 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, by running the snapshot's problems_in against hand-made catalogue entries.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The code is now at core/templatetags/postulo.py:510-520 for the percent filter (#248 moved the flag lookup out of that file); core/messages_tool.py is unchanged.

scripts/messages.py check is Postulo's only placeholder gate: the tool replaces msgfmt, and with it msgfmt --check-format. It compares the sets of matches of one pattern, and that leaves two blind spots:

Any other % sequence in a translation is invisible to it, so it passes text that makes Python's % operator raise.
Because it compares sets, it lets a singular message drop a positional placeholder.

docs/TRANSLATING.md:140 promises that "a translation that would raise at render time, does not get in".

Where: core/messages_tool.py:200 (PLACEHOLDER = re.compile(r"%\((\w+)\)[sdifr]|%[sdifr%]|\{(\w*)\}")); core/messages_tool.py:699-746 (placeholders, problems_in, where not got <= expected is the only test for a message with no named placeholder); core/templatetags/postulo.py:528-538 (the percent filter, gettext("%(share)s%%") % {"share": value}, with nothing catching an error).

What happens: run against problems_in in this snapshot, each of these passes the check, and the runtime then fails when it formats the text:

msgid	translation	at run time
`%(share)s%%`	`%(share)s %`	`ValueError: incomplete format`
`%(share)s%%`	`% %(share)s`	`ValueError: unsupported format character '%'`
`%(n)s done`	`%(n)s % d`	`TypeError: not enough arguments for format string`
`Delete %s?`	`Supprimer ?`	`TypeError: not all arguments converted`

The first two are the shapes the percent filter's own docstring invites: a space before the sign in French, and the sign in front in Turkish. A translator who writes the sign as one % instead of %% gets through the check, the catalogue compiles, and every page that uses |percent fails for everybody reading that language. Those pages are the funnel, industries, sources and insight-counter widgets on the dashboard, and the translation bars on Defaults.

Why it matters: nothing fails today. All 14,561 translated python-format forms in the snapshot's 18 catalogue sets format cleanly, and no msgid in them uses a positional placeholder yet. But the gate does not do what the documentation says it does. Translations arrive from Weblate and from pull requests, and this check is what CI relies on, so the first such translation would surface as a 500 on the dashboard.
Fix: in problems_in, for each translated message flagged python-format, format every form the way the runtime will. Use form % {name: 1 for name in named} for named placeholders, or a tuple of the right length for positional ones, and report any exception. For brace messages, check that string.Formatter().parse(form) parses and yields only the msgid's field names. For singular messages, compare positional placeholders as a list rather than a set. Together that reproduces msgfmt --check-format.

Done when

tests/test_messages_tool.py has a case for each row of the table above. Each asserts that problems_in reports the entry. All four pass the check today.

Found on 2026-09-29 in a read-only review of postulo at 95348908d, by running the snapshot's `problems_in` against hand-made catalogue entries. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The code is now at `core/templatetags/postulo.py:510-520` for the `percent` filter (#248 moved the flag lookup out of that file); `core/messages_tool.py` is unchanged. `scripts/messages.py check` is Postulo's only placeholder gate: the tool replaces `msgfmt`, and with it `msgfmt --check-format`. It compares the sets of matches of one pattern, and that leaves two blind spots: - Any other `%` sequence in a translation is invisible to it, so it passes text that makes Python's `%` operator raise. - Because it compares sets, it lets a singular message drop a positional placeholder. `docs/TRANSLATING.md:140` promises that "a translation that would raise at render time, does not get in". - **Where:** `core/messages_tool.py:200` (`PLACEHOLDER = re.compile(r"%\((\w+)\)[sdifr]|%[sdifr%]|\{(\w*)\}")`); `core/messages_tool.py:699-746` (`placeholders`, `problems_in`, where `not got <= expected` is the only test for a message with no named placeholder); `core/templatetags/postulo.py:528-538` (the `percent` filter, `gettext("%(share)s%%") % {"share": value}`, with nothing catching an error). - **What happens:** run against `problems_in` in this snapshot, each of these passes the check, and the runtime then fails when it formats the text: | msgid | translation | at run time | | --- | --- | --- | | `%(share)s%%` | `%(share)s %` | `ValueError: incomplete format` | | `%(share)s%%` | `% %(share)s` | `ValueError: unsupported format character '%'` | | `%(n)s done` | `%(n)s % d` | `TypeError: not enough arguments for format string` | | `Delete %s?` | `Supprimer ?` | `TypeError: not all arguments converted` | The first two are the shapes the `percent` filter's own docstring invites: a space before the sign in French, and the sign in front in Turkish. A translator who writes the sign as one `%` instead of `%%` gets through the check, the catalogue compiles, and every page that uses `|percent` fails for everybody reading that language. Those pages are the funnel, industries, sources and insight-counter widgets on the dashboard, and the translation bars on *Defaults*. - **Why it matters:** nothing fails today. All 14,561 translated python-format forms in the snapshot's 18 catalogue sets format cleanly, and no msgid in them uses a positional placeholder yet. But the gate does not do what the documentation says it does. Translations arrive from Weblate and from pull requests, and this check is what CI relies on, so the first such translation would surface as a 500 on the dashboard. - **Fix:** in `problems_in`, for each translated message flagged `python-format`, format every form the way the runtime will. Use `form % {name: 1 for name in named}` for named placeholders, or a tuple of the right length for positional ones, and report any exception. For brace messages, check that `string.Formatter().parse(form)` parses and yields only the msgid's field names. For singular messages, compare positional placeholders as a list rather than a set. Together that reproduces `msgfmt --check-format`. ## Done when - `tests/test_messages_tool.py` has a case for each row of the table above. Each asserts that `problems_in` reports the entry. All four pass the check today.

tiagoagueda added this to the 0.6.0 milestone

2026-10-01 10:47:06 +00:00

tiagoagueda added the

internationalisation

tier

bug

labels

2026-10-01 10:47:06 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 04:15:10 +00:00

Check translations by formatting them, not only by comparing placeholders

tiagoagueda closed this issue