Skip to content

fix: a written number is read across punctuation and next to another one - #43

Merged
jqueguiner merged 2 commits into
mainfrom
fix/written-numbers-words2num2
Nov 11, 2026
Merged

jqueguiner merged 2 commits into
mainfrom
fix/written-numbers-words2num2

Conversation

@jqueguiner

@jqueguiner jqueguiner commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Reads a written number across punctuation and beside another written number, using words2num2's sentence mode after the existing English expansion.

Type of change

  • Edit existing language (fix a replacement, tweak config, …)
  • Bug fix (other)

Edit existing language

  • New/changed word substitutions go in replacements.py, not inline in operators.py — no substitution added here
  • If you changed a config field that can be None: the step reading it still handles None gracefully — no config field changed
  • Unit tests updated or added
  • E2e CSV updated if the expected output changed

Why

Two rows of the pipecat STT benchmark were scored as engine errors only because the expansion stopped early:

load_pipeline('gladia-3', 'en').normalize("That's fifty.")
# before: 'that is fifty'      after: 'that is 50'

load_pipeline('gladia-3', 'en').normalize('where people hit sixty, seventy.')
# before: 'where people hit 60 seventy'   after: 'where people hit 6070' (the digit collapsing the
#                                                       same input already gets when it is written 60, 70)

EnglishNumberNormalizer walks words one by one and stops at the first punctuation mark, and it consumes a single number out of two side by side. words2num2's words2num_sentence is built for transcripts and reads both, so it runs on the class's output.

It runs after, not instead, because the class reads forms words2num2 does not and those keep their value:

"20 thirteen"   -> 2013     (words2num_sentence alone: "20 13")
"25 hundred"    -> 2500     (words2num_sentence alone: "25 hundred")
"1 thousand five hundred" -> 1500   (words2num_sentence alone: "1 1500")
"eleventh 2000 and twelve" -> 112012

Those are the mixed digit-and-word forms ASR writes, and the two packages' repositories are archived, so they will not grow them: the class stays the place that reads them, and words2num_sentence is only the second pass. The order cannot be swapped — running it first leaves "20 thirteen" as "20 13", which the class then no longer joins.

It only runs when a written number is left in the text. On digits and dots alone, words2num_sentence reads a version or an address as a decimal, so the guard keeps them:

"version 2.5.1"  -> 'version 2 dot 5 dot 1'   (without the guard: 'version 2 dot 50 point 1')
"192.168.1.1"    -> '192 dot 168 dot 1 dot 1'

A lone one stays: it is a pronoun as often as a number, and the class already keeps it.

num2words2 is the inverse of words2num2 and is what the new test writes the numbers with.

How was this tested?

uv run pytest            # 1422 passed
uv run ruff check .
uv run ruff format --check .
uv run ty check

The new tests assert, through the published preset:

  • the two rows above, and I paid fifty dollars.
  • version 2.5.1, 192.168.1.1, 1.1.1.1 keep their dots
  • every number num2words2 writes from 0 to 120, plus 999, 1234 and 10000, comes back as its digits

E2e rows added to tests/e2e/files/gladia-3/en.csv:

"That's fifty.",that is 50
"where people hit sixty, seventy.",where people hit 6070

Fixes #42

The English expansion stopped at the first punctuation mark and read only one of two
adjacent written numbers, so "that is fifty." kept "fifty" and "sixty, seventy" became
"60 seventy". words2num2's sentence mode reads both, and it reads transcripts: "no one"
stays, a phone number stays a phone number.

It runs on this class's output, not instead of it, because the class reads forms
words2num2 does not ("20 thirteen" -> 2013, "25 hundred" -> 2500) and those keep their
value. It only runs when a written number is left, so a version or an address is not read
as a decimal ("2 dot 5 dot 1" stays, instead of "2 dot 50 point 1"). A lone "one" is left
alone too: it is a pronoun as often as a number.

num2words2 comes with words2num2 as its inverse and is what the new test writes the
numbers with: every number from 0 to 120, plus 999, 1234 and 10000, read back as its
digits through the published preset.

Fixes #42
Conflicts: tests/e2e/files/gladia-3/en.csv — both sides append rows at the end
of the file (this branch: a written number across punctuation; main: nonfiction
/ non-fiction from #41). Kept both.
@jqueguiner
jqueguiner merged commit 1cba554 into main Nov 11, 2026
10 checks passed
@jqueguiner
jqueguiner deleted the fix/written-numbers-words2num2 branch November 11, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gladia-3 leaves a written number unexpanded at the end of a sentence, and skips the second of two adjacent numbers

2 participants