Skip to content

Make RedisValue numeric comparison consistent across storage types - #3253

Open
banlor wants to merge 1 commit into
StackExchange:mainfrom
banlor:fix/redisvalue-numeric-normalization
Open

banlor wants to merge 1 commit into
StackExchange:mainfrom
banlor:fix/redisvalue-numeric-normalization

Conversation

@banlor

@banlor banlor commented Sep 23, 2026 •

Copy link
Copy Markdown

Fixes #3233.

Today default equality depends on how a RedisValue is stored. Strings are parsed with double.TryParse(..., NumberStyles.Any, ...), but bytes (what comes back from the server) go through the byte parser, which rejects those forms. So "1,000" equals "1000", but not its own UTF-8 bytes after a round trip.

This PR makes ==/Equals, GetHashCode and CompareTo use the byte parser for strings, byte[]/memory and ReadOnlySequence<byte> alike. Explicit conversions ((long), (int), (double), ...) are left alone.

Before and after

bytes "x" is Encoding.UTF8.GetBytes("x"). Hash equality matches == in every row.

x y x == y before after Math.Sign(x.CompareTo(y)) before after
"1,000" "1000" or 1000L true false 0 -1
"1,000" bytes "1,000" false true 1 0
"(5)" "-5" or -5L true false 0 -1
"(5)" bytes "(5)" false true 1 0
" 5 " "5" or 5L true false 0 -1
" 5 " bytes " 5 " false true 1 0
"1,000" "999" false false 1 -1
two-segment sequence of "000...001" (41 chars) "1" false true -1 0
two-segment sequence of "1,000" "1,000" false true -1 0

So new HashSet<RedisValue> { "1,000", "1000" } has 1 item today and 2 with this PR, while { "1,000", bytes "1,000" } goes from 2 to 1. Hash codes of the affected strings change.

Unchanged

  • "1000", bytes "1000" and 1000L are all equal, and "5." == "5", "+5" == "5", "1e2" == "100" still hold.
  • "000...001" (41 chars, string or byte[]) equals "1" today and still does. That's why inputs longer than Format.MaxDoubleTextLen are still parsed, and why Simplify rents a buffer for them. Strings and sequences are only copied and parsed if every char is one of 0-9 + - . e E.
  • "inf", "+inf" and "nan" don't equal double.PositiveInfinity or double.NaN. The ordering fallback is unchanged too, so "+inf".CompareTo(double.PositiveInfinity) is still 0 while == is false.
  • Explicit conversions: (long)"1,000" is 1000, (int)" 5 " is 5, (long)"(5)" is -5, (double)"inf" is infinity, and (long) of bytes "1,000" still throws InvalidCastException.
  • RedisValue.EqualityComparer.Binary.

This is a behaviour change to default comparison. If you'd rather keep the default and ship this as an opt-in comparer next to Binary, I can rework the PR that way.

I got the table by running the same small program against main (0a92ae4) and this branch (2f7af8e) on net10.0. dotnet test tests/StackExchange.Redis.Tests -c Release -f net10.0 --filter "FullyQualifiedName~RedisValue" passes on this branch (261 tests, including the new RedisValueNumericNormalizationTests). I haven't run other target frameworks or the integration suite against a real server.

Checklist

  • I fully and freely contribute this code in accordance with the project license (and am legally able to do so)
  • I take responsibility for this contribution's quality and correctness, including any portions produced with AI assistance (see CONTRIBUTING.md).

Use byte-parser normalization for comparison and hashing while retaining
permissive explicit conversions. Reject impossible numeric characters
before renting scratch buffers for strings and fragmented sequences.

Add storage, conversion, length and allocation regression coverage.
@mgravell

mgravell commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

In the PR body (so: here) can we please be very clear and explicit exactly what scenario(s) this is addressing? Specifically, any change is a potential break to someone, so it would be good to be very clear to see what we're looking to change here. In particular, I wonder whether what we actually want here is an equality comparer instance, like Binary, but with some other, preferred and well-defined value-interpreting semantics.

@mgravell

Copy link
Copy Markdown
Collaborator

If the point is to defer any lossy conversions until compare time: that's great, I just think we should be super clear about what actually changes in the PR body.

}
if (forComparison || seq.Length <= Format.MaxDoubleTextLen)
{
int len = checked((int)seq.Length);

@mgravell mgravell Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to be mindful of what scenarios we need to allow/disallow here, and what means for the inputs; for example, if we can't simplify something that is over-long, it seems unlikely that we'd ever need the lease here - we can presumably discount such values immediately. So: what forComparison scenario forces this lease? Something with pathological numbers of leading zeros... probably isn't a useful case. I also wonder about [+-]inf etc.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lease preserves numeric comparison for overlong inputs, including leading zeroes; it isn't needed for inf, +inf, -inf or nan, which remain text for equality even though double casts can parse them. I tested removing it with a uniform 40-byte cutoff, but that also breaks existing long byte-array comparisons, so I haven't pushed that change. The body now gives the round-trip mismatch and breaking changes explicitly: would you prefer the bounded default behavior or an opt-in comparer?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, it is hard for me to opine because the PR body doesn't really make it clear what it intends to change, behaviour-wise, in concrete terms / examples.

@mgravell

Copy link
Copy Markdown
Collaborator

I wonder if we can simplify the intent here. For example, what if the non-comparing simplify only handled strict integers (no commas etc) and floating points that parse and round-trip back to the exact same bytes/chars, so it is never lossy, and the compare parse does what it has always done. Does that solve the underlying issue more simply? Is that what this PR is trying to do?

@banlor

banlor commented Sep 26, 2026

Copy link
Copy Markdown
Author

leaving comparison parsing unchanged would still leave #3233: "1,000" equals "1000" as strings, but not its own UTF-8 bytes after a Redis round trip. This PR targets that comparison mismatch; a lossless non-comparing Simplify change is narrower and would also need separate fallbacks to preserve existing numeric casts. Would you prefer that narrower change here and a separate comparison fix?

@mgravell

Copy link
Copy Markdown
Collaborator

What I'd prefer is for the PR body to be very clear, with concrete examples, what it proposed to change, to what.

@banlor

banlor commented Sep 26, 2026

Copy link
Copy Markdown
Author

Updated the description with concrete before/after examples, including why over-long inputs still need the rented buffer.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RedisValue equality treats "1,000", "(5)" and " 5 " as numbers, via NumberStyles.Any

2 participants