Skip to content

Clean up CodeQL alerts in library and test code - #792

Merged
leerho merged 3 commits into
mainfrom
codeql-cleanup
Nov 2, 2026
Merged

leerho merged 3 commits into
mainfrom
codeql-cleanup

Conversation

@leerho

@leerho leerho commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Cleans up most of the open CodeQL alerts. The library-code changes don't change behavior; one test fix corrects a real test bug.

Library code

Alert Where Change
index out of bounds (error) Util.convertToLongArray Walk each 8-byte group forward with an explicit exclusive end, so the bounds are visible. The output is identical (UtilTest.checkConvertToLongArray still passes).
value may be null (2) ReservoirItemsUnion / ReservoirLongsUnion toByteArray() Derive empty from gadgetBytes == null, so the guard covers the dereference directly.
value may be null (2) KllPreambleUtil.toString() (debug helper) Each items branch takes its serDe from Objects.requireNonNull(serDe, ...), so the guard sits at the dereference. The method already required it at the top, but CodeQL could not connect that check to the later else //ITEMS_SKETCH branches.

Util.convertToLongArray also gets its flag renamed from littleEndian to bigEndian. true always produced Big-Endian longs (bytes {1..8} become 0x0102030405060708), so the name now matches the behavior and the code is unchanged. It is package-private and only UtilTest calls it. It also gets a javadoc: a final group of fewer than 8 bytes is flush right in its long in either byte order. The test now also checks full groups against ByteBuffer in both byte orders.

Tests

  • Real bug: UtilTest.checkCeilingMultiple2expK used (1 << 36) - 1L. That's an int shift, which Java masks to 1 << 4, so the test only ever checked n = 15. It now uses 1L << 36.
  • PositionalSegmentTest: compute offsets as (long) i * X.BYTES (21 integer-multiplication alerts).
  • Remove locals that were assigned but never read. Where the call is what's being tested, the call stays as a statement. Two tests got stronger along the way: CpcWrapperTest now asserts the empty-sketch estimate and bounds it used to discard, and ArrayOfDoublesCompactSketchTest asserts the type its unused cast was checking.

Deliberately not changed

These are better dismissed in the code-scanning UI:

  • The println(String s) { //System.out.println(s); } debug helpers (unused-parameter alerts): the commented-out body is the intended toggle.
  • Locals read only by commented-out debug printlns (IsomorphicTest), and the deliberate bad cast in HeapIntersectionTest.checkFamilyID.
  • deprecated-call in tests, and uncaught-number-format-exception in FrequentLongsSketch.getInstance(String), which other open work already touches. A NumberFormatException is an IllegalArgumentException in any case.

Testing

mvn verify: 2320 tests, 0 failures. The CodeQL run on this PR will show which alerts close.

🤖 Generated with Claude Code

leerho and others added 2 commits November 1, 2026 13:20
Library code, no behavior change:
- Util.convertToLongArray: walk each 8-byte group forward with an
  explicit exclusive end, so the index bounds are visible. The output
  is unchanged, and UtilTest.checkConvertToLongArray still passes.
- ReservoirItemsUnion and ReservoirLongsUnion toByteArray(): derive
  "empty" from gadgetBytes itself, so the null check guards the
  dereference directly.

Tests:
- UtilTest.checkCeilingMultiple2expK used (1 << 36) - 1L, an int shift
  that evaluates to 15, so the test only ever checked n = 15. It now
  uses 1L << 36 as intended.
- PositionalSegmentTest: compute byte offsets in long arithmetic.
- Remove locals that were assigned but never read. Where the call is
  what is tested, the call stays as a statement. CpcWrapperTest now
  asserts the empty-sketch estimate and bounds it previously discarded,
  and ArrayOfDoublesCompactSketchTest asserts the type its cast checked.

Left as they are: the println(String s) debug helpers whose bodies are
commented out (the unused-parameter alerts), locals used by commented-out
debug println lines, and the deliberate bad cast in HeapIntersectionTest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The flag was named littleEndian, but true produced Big-Endian longs:
bytes {1..8} became 0x0102030405060708. Renaming it to bigEndian makes
the name match the behavior, so the code itself is unchanged. The
method is package-private and only UtilTest calls it.

The javadoc documents the byte order, and that a final group of fewer
than 8 bytes is flush right in its long in either byte order.

UtilTest.checkConvertToLongArray keeps its expectations and adds checks
of full groups against ByteBuffer in both byte orders, using bytes with
the high bit set, plus an empty array.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leerho
leerho requested review from proost and tisonkun November 1, 2026 21:45
The debug toString(MemorySegment, SketchType, boolean, serDe) already
required a non-null serDe for an items sketch, but the check was at the
top of the method, and CodeQL could not connect it to the later
"else //ITEMS_SKETCH" branches. Each items branch now takes its serDe
from Objects.requireNonNull(serDe, ITEMS_SERDE_MSG), so the guard is at
the dereference. No behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leerho
leerho requested a review from freakyzoidberg November 2, 2026 20:30
@leerho
leerho merged commit bff7769 into main Nov 2, 2026
6 checks passed
@leerho
leerho deleted the codeql-cleanup branch November 2, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants