[oliver-kriska]

Faster fuzzy merging without changing match selection

Related to #440. The expensive part of fuzzy merging is scoring every old message for each unmatched message, not constructing the matcher closure. This PR skips candidates whose maximum possible Jaro similarity is below the threshold or the best score already found. Surviving candidates still use String.jaro_distance/2; this does not substitute an approximate matcher or change the fuzzy threshold.

Lengths and grapheme frequencies are prepared once per PO merge, lazily when a nonempty, valid fuzzy query needs them. The bounds assume no transpositions and at most as many matching graphemes as the strings have in common. They use graphemes rather than bytes or codepoints, count repeated graphemes, and preserve the scorer's floating-point addition order. Candidate traversal order and the first winner on equal scores are retained. Exact matches and --no-fuzzy do not build the index. All old messages remain eligible, including ones already used by another match.

Empty queries use the original scan rather than preparing the catalog. Malformed UTF-8 on either side also uses the original full scan, including after candidate preparation: pruning or stopping at a perfect match must not suppress platform Jaro exceptions. Equality and empty-string fast paths remain the scorer's responsibility.

Measurements

Baseline: upstream b2c62ff. Apple M4 Pro, Elixir 1.20.4 / OTP 29.0.5, MIX_ENV=prod, four schedulers. These are medians of seven alternating baseline/candidate pairs after an untimed warmup, using fresh input files for every invocation. Timings include the complete merge task: parsing, preparing candidates, merging, pruning references, composing and writing PO files. They exclude VM startup, compilation and copying the fixtures.

Workload	Before	After
Real application change: 16 locales, 176 PO files, 11 unmatched source keys	7.41 s	1.94 s
25,900 messages, one locale, 100 edits	25.81 s	2.98 s
Real application change, --no-fuzzy control	1.30 s	1.33 s
25,900 messages, all exact matches	1.36 s	1.32 s

On Elixir 1.19.5 / OTP 28.4.1 (the same major versions as the issue reporter), with the same methodology:

Workload	Before	After
10,000 messages, one locale, 10 edits	0.950 s	0.675 s
10,000 messages, one locale, 100 edits	14.07 s	0.964 s

The large fixtures expand a private application's real catalog using unique msgid suffixes, retaining its translations, plurals, contexts and references. The real-change row replays an actual before/after catalog snapshot without generating strings. This is not the reporter's catalog, and the locale count and number of changed messages in that report are unknown. The machine had other work running, so these are indicative paired measurements, not guaranteed timings or a reproduction of the reported six-minute merge. The control results do not establish a speedup.

Separately traced Jaro-call counts fell from 779,328 to 6,096 for the real change, and from 2,590,000 to 3,128 for the 25,900-message/100-edit case. Allocated heap words inside merge/5 for the latter fell from 6.63 billion to 98.7 million; these are cumulative allocations, not peak memory. Profiling the 25,900-message/10-edit search shows the remaining work is mostly preparing grapheme counts and checking bounds rather than running Jaro comparisons.

Equivalence and limits

• Every measured task invocation checks merge statistics and SHA-256 hashes of every output PO against the baseline.
• Differential validation compares complete message structs, statistics and composed bytes for 2,022 seeded multi-message cases and 4,800 exhaustive short-string threshold-boundary cases on each of OTP 27, 28 and 29. Another 120 malformed/empty-input cases compare full results or exception type and message. Cases cover Unicode combining characters/emoji, repeated graphemes, contexts, singular/plural conversion, exact/fuzzy matches, obsolete messages and previous-message metadata.
• The Elixir 1.16 scoring implementation was also exercised on the installed runtime; this is not a replacement for running CI on Elixir 1.16 / OTP 24.
• The isolated branch passes mix test --warnings-as-errors (197 passed: 5 doctests, 192 tests), formatting and whitespace checks. Kept tests target inclusive thresholds, repeated/Unicode graphemes, best-match/tie behavior, zero thresholds and platform handling of malformed UTF-8.

This trades temporary per-catalog indexing memory for fewer expensive comparisons. Histograms remain live until the merge finishes; peak memory was not measured. Worst-case scanning remains quadratic in old messages × unmatched new messages. Candidates with identical grapheme histograms can all survive the bounds, making the extra checks slower than the baseline. It also does not change extraction's timeout or compilation behavior, and does not claim to resolve every cause of the timeout in issue 440. Keeping this as a draft for feedback on the complexity/performance tradeoff.