Skip to content

time: fix parse_float with ten or more fractional digits - #5575

Merged
Watson1978 merged 2 commits into
fluent:masterfrom
NUAAwqy:fix/numeric-time-parser-float-seconds
Oct 27, 2026
Merged

Watson1978 merged 2 commits into
fluent:masterfrom
NUAAwqy:fix/numeric-time-parser-float-seconds

Conversation

@NUAAwqy

@NUAAwqy NUAAwqy commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Which issue(s) this PR fixes:

None — self-found while auditing NumericTimeParser#parse_float.

What this PR does / why we need it:

NumericTimeParser#parse_float (lib/fluent/time.rb) kept ten characters of the fractional part (nsec_s[0..9]), so a timestamp with ten or more decimal digits stored an nsec of 1_000_000_000 or more. EventTime assumes nsec fits inside a second: to_f/to_r report the wrong instant, to_time normalises into the next second, %N prints ten digits, and to_msgpack_ext forwards the oversized field as-is, so other fluentd nodes inherit it. Of fraction lengths 1..20, eleven produce an out-of-range nsec:

Fluent::NumericTimeParser.new(:float).parse("1700000000.1234567890").to_f
# => 1700000001.2345679   (should be 1700000000.123456789)

The fix: keep the first nine fraction digits (nsec_s[0, 9], sub-nanosecond digits dropped, which is what Time#nsec does). Padding and caching behaviour are untouched, so the hot path keeps its shape.

Per the review, the negative-timestamp adjustment introduced earlier in this PR was dropped again: it changed how negative values interact with existing formatting and MessagePack serialization, and EventTime is also defined in the Forward protocol, so changing its encoding needs a separate discussion. Negative values now parse exactly as before this PR.

Validation (Ruby 3.3.6, TZ=UTC):

check result
new test (test/test_time_parser.rb) 31 tests, 0 failures, 0 errors
regression slice: test_time_formatter 77, test_event_time 22, test_formatter 36, test_mixin 21, plugin/test_parser 23, plugin/test_parser_json 66, plugin/test_parser_apache 3, plugin_helper/test_compat_parameters 10, plugin/test_out_exec_filter 22 all 0 failures, 0 errors
rubocop 1.92.1 + rubocop-performance 1.27.0 no offenses

plugin/test_in_exec reports 2 failures (can handle non-ascii characters) that reproduce identically at f984d65 with the fix reverted — the flaky encoding tests pinned in #5562/#5572, unrelated to time parsing.

Docs Changes:

None needed.

Release Note:

time: fix parse_float with ten or more fractional digits

  • With time_type float, a time string with ten or more fractional digits was parsed into a wrong time. Digits beyond nanoseconds are now dropped.

NumericTimeParser#parse_float sliced 10 characters out of the fraction
(nsec_s[0..9]), so a value with 10 or more decimal digits stored a nanosecond
field of 1_000_000_000 or more. EventTime assumes nsec fits in a second, so
such a value reports the wrong second everywhere: to_f and to_r are off,
to_time normalises into the next second, %N prints ten digits, and the pair is
forwarded as-is through to_msgpack_ext, so a downstream fluentd inherits it.

It also truncated the whole part towards zero. EventTime keeps nsec
non-negative, so a negative timestamp has to floor to the second below and
carry the remainder, exactly like Time#to_i and Time#nsec. Instead "-1234.5"
became (-1234, 500000000) -- one second late, and "-0.5" became (0, 500000000),
i.e. +0.5. Every pre-1970 timestamp parsed with time_type float was wrong, and
the parser disagreed with the string based parser for the same instant.

Both are now checked against the decomposition Time.at(Rational(value)) gives:
a randomised corpus of 200_000 positive and negative values with 1 to 9
fraction digits had 98_614 wrong decompositions before this change and 0 after,
and nsec stays in [0, 1_000_000_000) for fractions of up to 20 digits.

Signed-off-by: Qingyuan Wang <162230229@nuaa.edu.cn>
@Watson1978 Watson1978 added the bug Something isn't working label Oct 27, 2026

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding this. I reviewed 606851a on Ruby 4.0.9 (Linux).

Keeping nine digits of the fraction looks good. It fixes the out-of-range nsec.

You are right that master parses -0.5 as +0.5. However, the negative-timestamp adjustment changes behavior in ways that interact badly with existing formatting and MessagePack serialization. Details are in the inline comments. Let's keep this PR focused on the nine-digit limit and address negative timestamps separately.

Negative timestamps require a separate discussion of formatting and serialization compatibility. EventTime is also defined in the Forward Protocol, so changing its encoding affects other implementations too.

  • MUST: Drop the negative-timestamp adjustment and its test. Both inline comments have suggestions for this.
  • MUST: Update the PR description and the release note to reflect the narrower scope.

With the two suggestions applied, test/test_time_parser.rb (31 tests) and test/test_time_formatter.rb (77 tests) pass.

Comment thread lib/fluent/time.rb Resolved
Comment thread test/test_time_parser.rb Resolved
Per review: dropping the negative-timestamp adjustment again. It changed
how negative values interact with out_file formatting and MessagePack
serialization, and EventTime is also defined in the Forward protocol, so
changing its encoding needs a separate discussion. This keeps the PR
focused on the nine-digit fraction limit.

Signed-off-by: NUAAwqy <NUAAwqy@users.noreply.github.com>
@NUAAwqy

NUAAwqy commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review — both MUSTs are applied in 5173c74:

  • Dropped the negative-timestamp adjustment and its test; negative values parse exactly as before. Agreed the formatting/serialization compatibility (and the Forward protocol EventTime encoding) needs a separate discussion.
  • Rebased the PR description and the release note onto the narrower nine-digit-fraction scope.

test/test_time_parser.rb (31 tests) and test/test_time_formatter.rb (77 tests) pass with the change.

@Watson1978 Watson1978 changed the title time: fix parse_float overlong nanoseconds and negative second flooring time: fix parse_float with ten or more fractional digits Oct 27, 2026
@Watson1978 Watson1978 added this to the v1.21.0 milestone Oct 27, 2026

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update.
LGTM

@Watson1978 Watson1978 added the backport to v1.19 We will backport this fix to the LTS branch label Oct 27, 2026
@Watson1978
Watson1978 merged commit 846fc17 into fluent:master Oct 27, 2026
20 checks passed
Watson1978 pushed a commit that referenced this pull request Nov 1, 2026
…gits (#5575) (#5580)

**Which issue(s) this PR fixes**:
Backport #5575

None — self-found while auditing `NumericTimeParser#parse_float`.

**What this PR does / why we need it**:

`NumericTimeParser#parse_float` (`lib/fluent/time.rb`) kept **ten**
characters of the fractional part (`nsec_s[0..9]`), so a timestamp with
ten or more decimal digits stored an `nsec` of `1_000_000_000` or more.
`EventTime` assumes `nsec` fits inside a second: `to_f`/`to_r` report
the wrong instant, `to_time` normalises into the next second, `%N`
prints ten digits, and `to_msgpack_ext` forwards the oversized field
as-is, so other fluentd nodes inherit it. Of fraction lengths 1..20,
eleven produce an out-of-range `nsec`:

```
Fluent::NumericTimeParser.new(:float).parse("1700000000.1234567890").to_f
# => 1700000001.2345679   (should be 1700000000.123456789)
```

**The fix**: keep the first nine fraction digits (`nsec_s[0, 9]`,
sub-nanosecond digits dropped, which is what `Time#nsec` does). Padding
and caching behaviour are untouched, so the hot path keeps its shape.

Per the review, the negative-timestamp adjustment introduced earlier in
this PR was dropped again: it changed how negative values interact with
existing formatting and MessagePack serialization, and `EventTime` is
also defined in the [Forward
protocol](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1#eventtime-ext-format),
so changing its encoding needs a separate discussion. Negative values
now parse exactly as before this PR.

**Validation** (Ruby 3.3.6, `TZ=UTC`):

| check | result |
| --- | --- |
| new test (`test/test_time_parser.rb`) | 31 tests, **0 failures, 0
errors** |
| regression slice: test_time_formatter 77, test_event_time 22,
test_formatter 36, test_mixin 21, plugin/test_parser 23,
plugin/test_parser_json 66, plugin/test_parser_apache 3,
plugin_helper/test_compat_parameters 10, plugin/test_out_exec_filter 22
| all **0 failures, 0 errors** |
| rubocop 1.92.1 + rubocop-performance 1.27.0 | no offenses |

`plugin/test_in_exec` reports 2 failures (`can handle non-ascii
characters`) that reproduce identically at `f984d65` with the fix
reverted — the flaky encoding tests pinned in #5562/#5572, unrelated to
time parsing.

**Docs Changes**:

None needed.

**Release Note**:

time: fix parse_float with ten or more fractional digits
* With `time_type float`, a time string with ten or more fractional
digits was parsed into a wrong time. Digits beyond nanoseconds are now
dropped.

Signed-off-by: Qingyuan Wang <162230229@nuaa.edu.cn>
Signed-off-by: NUAAwqy <NUAAwqy@users.noreply.github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Qingyuan Wang <146524024+NUAAwqy@users.noreply.github.com>
Co-authored-by: NUAAwqy <NUAAwqy@users.noreply.github.com>
@github-actions github-actions Bot added the backported "backport to LTS" is done label Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport to v1.19 We will backport this fix to the LTS branch backported "backport to LTS" is done bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants