runcontainer: count or test before allocating in run-bitmap intersections

[tamirms]

Description

Several run-container intersections build an 8 KiB bitmap before they know whether the result is empty, a small array or a bitmap. This counts or tests first, then allocates only what is returned.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Performance improvement
• Code refactoring
• Documentation update
• Test improvements
• Build/CI changes

Changes Made

What was changed?

• runContainer16.andBitmapContainer (run AND bitmap) counts the result first, then builds an empty array, an exact-size array or a bitmap.
• bitmapContainer.iandRun16 (in-place bitmap AND run) counts the same way. A small result becomes an array; otherwise the gaps between the run's intervals are cleared in place.
• An in-place And with a full range returns the container unchanged. A bitmap used to be copied, and a run intersected.
• runContainer16.intersects returns at the first shared value instead of building a container to answer a boolean.
• resetBitmapRange clears whole words with clear instead of one at a time. The in-place path above uses it for every gap.
• Added the helpers clearBitmapGaps and containerFromWords, and runbitmap_and_test.go with tests and BenchmarkRunAndBitmap.

Why was it changed?

• An empty or array-sized result paid for an 8 KiB bitmap it never returned. An empty result is the common case when a run meets a sparse bitmap.
• Intersects allocated a container to answer a boolean. Every mixed Intersects call that touches a run goes through it.

How was it changed?

• The result is sized from its cardinality before it is built, the way bitmapContainer.andBitmap already works.
• A run with more than 64 intervals goes through a stack scratch bitmap instead of interval by interval, but only where that is faster: when the run covers more than 4096 values, or when the CPU has the AVX-512 VBMI2 fill that makes reading the scratch back cheap. Measured on an M1 and a Xeon 8375C.
• Array results of 1024 values or more use that vector fill where it is available, as bitmapContainer.andBitmap does.
• intersects scans masked words for bitmaps, merges like andArray for arrays (or searches per value when the intervals far outnumber the values), and tests interval overlap for runs.
• No API change.

Testing

• runbitmap_and_test.go checks each path against its old definition, newBitmapContainerFromRun(rc).andBitmap(bc) and !rc.and(c).isEmpty(), over random container pairs.
• Edge cases: empty and full bitmaps, intervals on word boundaries, results exactly at the array and bitmap threshold on both the interval and the scratch path, runs of one to hundreds of intervals, and a few values against many intervals.
• Allocation tests: the empty result, the boolean path and an in-place And with a full range allocate no container.

go test ./...
go test -tags appengine ./...
GOOS=linux GOARCH=386 go build ./...    # also arm, arm64, ppc64

All pass on linux/amd64 (Xeon 8375C, with AVX-512 VBMI2), darwin/arm64, and amd64 without AVX-512.

Formatting

gofmt -s -l . and go tool unconvert ./... report nothing. go vet ./... reports only the existing ReadFrom signature warning from master.

Fuzzing

The FuzzSmat target in the template does not exist on master (smat.go refers to a missing smat_fuzz_test.go), so -fuzz=FuzzSmat matches nothing. The smat corpus runs and passes, but none of its sequences reaches these functions:

go test -tags=gofuzz -run 'TestGenerateSmatCorpus|TestSmatHits'
ok  github.com/RoaringBitmap/roaring/v2

The randomized comparison under Testing is what exercises the change.

Performance Impact

BenchmarkRunAndBitmap (added) intersects a run-optimized bitmap with a dense one, over one key. The in-place cases use a fresh copy each time, made outside the timer; the "copy +" cases include the copy. Xeon 8375C:

	master	this PR	speedup
And, empty result	2.8 µs, 8328 B, 4 allocs	597 ns, 104 B, 2 allocs	4.7x
And, array result	3.9 µs, 14504 B, 8 allocs	2.2 µs, 6280 B, 6 allocs	1.77x
And, bitmap result	3.7 µs, 16560 B, 8 allocs	2.0 µs, 8336 B, 6 allocs	1.85x
And, 1024-interval run, array result	9.8 µs, 12456 B, 8 allocs	7.9 µs, 4232 B, 6 allocs	1.25x
And, 1024-interval run, bitmap result	9.1 µs, 16560 B, 8 allocs	6.9 µs, 8336 B, 6 allocs	1.31x
in-place And, bitmap result	2.2 µs, 8224 B, 2 allocs	472 ns, 0 B, 0 allocs	4.6x
in-place And, 1024-interval run	8.0 µs, 8224 B, 2 allocs	5.1 µs, 0 B, 0 allocs	1.56x
copy + in-place And, bitmap result	3.7 µs, 16628 B, 9 allocs	2.1 µs, 8404 B, 7 allocs	1.76x
copy + in-place And, 1024-interval run	9.0 µs, 16628 B, 9 allocs	6.7 µs, 8404 B, 7 allocs	1.35x
Intersects, disjoint	2.8 µs, 8248 B, 3 allocs	118 ns, 0 B, 0 allocs	23x
Intersects, overlapping	3.8 µs, 14392 B, 4 allocs	9 ns, 0 B, 0 allocs	449x
Intersects, 3 values vs 1024 intervals	1.2 µs, 32 B, 2 allocs	61 ns, 0 B, 0 allocs	20x

Performance Analysis

• Every row is faster and allocates less. The common empty result goes from 2.8 µs and 8 KiB to 597 ns and 104 B.
• An in-place And no longer allocates: 2.2 µs to 472 ns with a bitmap result.
• On an M1, which has no vector fill, the gains are similar: 5.6 µs to 3.2 µs for the array result and 1.8 µs to 409 ns for the empty one.
• BenchmarkIntersection*, BenchmarkUnion* and BenchmarkAndAny/* are unchanged.

Breaking Changes

None.

Related Issues

None.

Additional Notes

bitmapContainer.iandArray converts the array to a bitmap the same way; that is left for a separate change.

🤖 Generated with Claude Code