Skip to content

Fix pairwise Summator merges and self-aliasing - #495

Open
jmh530 wants to merge 3 commits into
libmir:masterfrom
jmh530:codex/fix-pairwise-summator-merges
Open

jmh530 wants to merge 3 commits into
libmir:masterfrom
jmh530:codex/fix-pairwise-summator-merges

Conversation

@jmh530

@jmh530 jmh530 commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Fixes #494.

Repeatedly adding or subtracting a two-value pairwise Summator overruns its partial-sum buffer on the 65th merge. The merge inserts source partials through put, then incorrectly adjusts the destination counter to count the source’s original observations. This leaves the counter inconsistent with the destination’s binary tree.

Keep the counter maintained by put, treating each inserted partial as a leaf in the destination tree.

Also preserve the source during self-addition and self-subtraction. Inserting partials can otherwise overwrite values that the same merge has yet to read: an accumulator containing 1, 2, 3 produces 18 rather than 12 after s += s. Copy the source only when it aliases the destination; ordinary merges retain the existing traversal without a copy.

Public signatures and other summation algorithms are unchanged.

Regression coverage

Add @safe pure nothrow @nogc tests for float, double, and real, covering:

  • Repeated accumulator addition and subtraction beyond the original failure threshold.
  • Empty and varied-size batches, with scalar insertions between merges.
  • Exact expected sums, source preservation, and the binary-tree state invariant.
  • Self-addition and self-subtraction followed by further insertion.

Validation

  • Confirmed the buffer overrun and incorrect self-addition against unchanged source.
  • Focused optimized regressions pass on Windows DMD 2.113 and LDC 1.43, including DIP1000.
  • Windows DMD unoptimized module harness: 27 modules passed.
  • Windows LDC optimized module harness with DIP1000: 27 modules passed.
  • Focused optimized regressions pass on Linux LDC 1.43 and 1.28.1.
  • Diff whitespace checks pass.

Two broader optimized harness failures also reproduce with unchanged source: Windows DMD fails the reduce test at iteration.d:914, and Linux LDC fails the existing precise-summation residual assertion. Neither is changed here.

No full repository suite or performance benchmark was run. Correcting the tree can change floating-point grouping after merges; bit-for-bit agreement with the previous invalid merge state is not promised.

AI-Assisted-By: Codex (GPT-6 Astra Medium)

Preserve the counter maintained by put when merging source partial
sums. Adjusting it afterward to count the source's original inputs
breaks the destination tree invariant and causes a buffer overrun
on the 65th repeated merge of a two-value accumulator.

Copy the source for self-addition and self-subtraction so inserting
partials cannot overwrite values still needed by the merge.
Ordinary merges retain the existing source traversal without a copy.

Add float, double, and real regressions for repeated addition and
subtraction, varied and empty batches, interleaved scalar insertion,
source preservation, tree invariants, and self-merges.

Fixes libmir#494.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Add a regression using a seven-element source merged twice into a
distinct accumulator. The original bookkeeping causes a partial-buffer
index underflow on the second merge.

Cover addition and subtraction for float, double, and real, checking
both accumulated results and preservation of the source sum.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Insert a source's sole partial directly for addition and subtraction,
bypassing the alias check and self-merge snapshot. The argument is read
before put modifies the destination, so this is safe for self-merges.

Retain destination counter bookkeeping and snapshot protection for
multi-partial self-merges, without changing the arithmetic order.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@jmh530

jmh530 commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Added two follow-up commits:

  • b60588e adds the seven-element-source regression: merging it twice into a distinct accumulator previously underflowed the partial-buffer index. Coverage includes addition and subtraction for float, double, and real.
  • 90238ee adds a fast path for sources with one stored partial. The value is read before insertion, so even self-merging needs no snapshot. Multi-partial self-merges retain snapshot protection.

Local Windows validation passed with DMD and LDC, including the 27-module test harness and additional checks for compile-time evaluation, cancellation, infinities, NaNs, and self-merges against explicit snapshots.

Local optimized LDC microbenchmarks showed that the fast path substantially reduces the self-merge overhead. In the tested double-addition cases, repeated ordinary merges were about 2% slower than the original implementation, while one-partial self-merges were slightly faster. Results vary by compiler and workload; these are not application-level performance guarantees.

A final branch review found no further actionable issues. Smaller snapshots were also investigated, but their mixed performance did not justify additional changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pairwise Summator addition and subtraction fail on the 65th repeated merge

1 participant