Spreadsheet import: \"45,000 - 55k\" becomes 55,000 to 45 million, and \"R$\" is read as US dollars #471

New issue

Closed

opened 2026-10-01 10:46:56 +00:00 by tiagoagueda · 0 comments

tiagoagueda commented

2026-10-01 10:46:56 +00:00

Owner

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the salary parsing in the CSV importer and running its functions.

Checked again on 2026-10-01 against main at 0a5cad587: still present. The code is now at core/csv_import.py:749-750 (parse_salary_range), :622-638 (CURRENCY_SYMBOLS, with "$" on line 624 and "R$" on line 633) and :715-717 (parse_currency).

The salary parsing that #224 repaired still gets two common forms wrong. When the low figure is written in full and the high figure has a k, the low one is multiplied by a thousand anyway, so the range is stored a thousandfold wrong. And a Brazilian real amount written with its symbol is stored as US dollars, although the symbol table maps R$ to BRL on purpose. Related to #224 (closed), whose fix is incomplete here.

Where: core/csv_import.py:754-755 in parse_salary_range; core/csv_import.py:618-634 (CURRENCY_SYMBOLS) and :720-722 in parse_currency.
What happens:
1. parse_salary_range scales the low figure whenever the high side carries a k, the low side does not, and low < high. That test is true for "45,000 - 55k" (45,000 is less than 55,000), so the low figure becomes 45,000,000 and the swap stores (55000, 45000000). The same happens to "50000-60k" → (60000, 50000000), "€40,000 to 50k" → (50000, 40000000) and "900 - 1.2k" → (1200, 900000).
2. parse_currency walks CURRENCY_SYMBOLS in insertion order and tests symbol in raw. "$" (line 620) comes before "R$" (line 629), so "R$ 5000" and "R$8.000" both return USD. plugins/builtin/patterns.py:103-115, which reads salaries from captured pages, already lists R$ and leaves out the bare $; the importer's table has drifted from it.
Why it matters: the wrong figure is written into the listing or application with nothing on the mapping preview or the result page to show it was misread. It distorts the salary column's sorting and anything that compares salaries, and a Brazilian's whole sheet arrives in the wrong currency.
Fix: carry the k across only when the scaled low figure stays at or below the high one (low * 1000 <= high), so a low figure already written in full is left alone. In parse_currency, try symbols longest first (sorted(CURRENCY_SYMBOLS.items(), key=lambda item: -len(item[0]))), so R$ is matched before $.

Done when

tests/test_counting.py asserts parse_salary_range("45,000 - 55k") == (45000, 55000), parse_salary_range("900 - 1.2k") == (900, 1200) and, as today, parse_salary_range("50-60k") == (50000, 60000).
tests/test_counting.py::test_the_currency_is_read_from_the_cell gains ("R$ 8.000", "BRL").

Found on 2026-09-29 in a read-only review of postulo at 95348908d, reading the salary parsing in the CSV importer and running its functions. Checked again on 2026-10-01 against `main` at `0a5cad587`: still present. The code is now at `core/csv_import.py:749-750` (`parse_salary_range`), `:622-638` (`CURRENCY_SYMBOLS`, with `"$"` on line 624 and `"R$"` on line 633) and `:715-717` (`parse_currency`). The salary parsing that #224 repaired still gets two common forms wrong. When the low figure is written in full and the high figure has a `k`, the low one is multiplied by a thousand anyway, so the range is stored a thousandfold wrong. And a Brazilian real amount written with its symbol is stored as US dollars, although the symbol table maps `R$` to BRL on purpose. Related to #224 (closed), whose fix is incomplete here. - **Where:** `core/csv_import.py:754-755` in `parse_salary_range`; `core/csv_import.py:618-634` (`CURRENCY_SYMBOLS`) and `:720-722` in `parse_currency`. - **What happens:** 1. `parse_salary_range` scales the low figure whenever the high side carries a `k`, the low side does not, and `low < high`. That test is true for `"45,000 - 55k"` (45,000 is less than 55,000), so the low figure becomes 45,000,000 and the swap stores `(55000, 45000000)`. The same happens to `"50000-60k"` → `(60000, 50000000)`, `"€40,000 to 50k"` → `(50000, 40000000)` and `"900 - 1.2k"` → `(1200, 900000)`. 2. `parse_currency` walks `CURRENCY_SYMBOLS` in insertion order and tests `symbol in raw`. `"$"` (line 620) comes before `"R$"` (line 629), so `"R$ 5000"` and `"R$8.000"` both return `USD`. `plugins/builtin/patterns.py:103-115`, which reads salaries from captured pages, already lists `R$` and leaves out the bare `$`; the importer's table has drifted from it. - **Why it matters:** the wrong figure is written into the listing or application with nothing on the mapping preview or the result page to show it was misread. It distorts the salary column's sorting and anything that compares salaries, and a Brazilian's whole sheet arrives in the wrong currency. - **Fix:** carry the `k` across only when the scaled low figure stays at or below the high one (`low * 1000 <= high`), so a low figure already written in full is left alone. In `parse_currency`, try symbols longest first (`sorted(CURRENCY_SYMBOLS.items(), key=lambda item: -len(item[0]))`), so `R$` is matched before `$`. ## Done when - `tests/test_counting.py` asserts `parse_salary_range("45,000 - 55k") == (45000, 55000)`, `parse_salary_range("900 - 1.2k") == (900, 1200)` and, as today, `parse_salary_range("50-60k") == (50000, 60000)`. - `tests/test_counting.py::test_the_currency_is_read_from_the_cell` gains `("R$ 8.000", "BRL")`.

tiagoagueda added this to the 0.5.0 milestone

2026-10-01 10:46:56 +00:00

tiagoagueda added the

tier

bug

labels

2026-10-01 10:46:56 +00:00

tiagoagueda referenced this issue from a commit

2026-10-04 10:04:57 +00:00

Read "45,000 - 55k" and "R$" correctly in imported salaries

tiagoagueda closed this issue

2026-10-04 10:04:57 +00:00