Skip to content

Upgrade zstd-jni to 1.5.7-26 - #2388

Open
mkurz wants to merge 1 commit into
AsyncHttpClient:mainfrom
mkurz:zstd-jni-1.5.7-26
Open

mkurz wants to merge 1 commit into
AsyncHttpClient:mainfrom
mkurz:zstd-jni-1.5.7-26

Conversation

@mkurz

@mkurz mkurz commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Problem

Since #2376, a compressed HTTP/1.1 body that ends inside its compressed stream fails the request. One zstd case was documented as not noticed: a body of several frames completed when it was cut inside a frame that started in the same chunk as the end of the previous one.

zstd-jni set a flag when a frame finished and cleared it only on its next read from the source. A frame that started in input zstd-jni had already read was decoded with the flag still set, so the end of the input looked like a clean end at a frame boundary. AHC hands zstd-jni one chunk per read, so such a body completed and the rest of it was lost without an error.

Change

  • zstd-jni.version goes from 1.5.7-25 to 1.5.7-26.
  • Two new tests in Http1ContentDecompressorTest, with heap and direct buffers:
    • zstdBodyCutInAFrameThatStartedInTheChunkThatEndedThePreviousOneFails sends a first frame and half of a second. It uses three splits, each with the end of the first frame and the start of the second in the same chunk: all in one last chunk, all in one chunk followed by an empty last chunk, and a split inside the first frame. The Content-Length matches the bytes sent, so the HTTP body is complete and only the zstd stream is cut. The test expects a DecompressionException.
    • zstdFramesThatShareAChunkDecodeInFull sends both frames complete, with the same splits, and expects the whole body.
  • README and Javadoc: the note now applies only to zstd-jni before 1.5.7-26. zstd-jni is an optional dependency, so the application's own version is the one used. The README therefore recommends 1.5.7-26 or later, and its example dependency shows that version (it showed 1.5.7-7).

The other gaps the README describes are unchanged:

  • a snappy body cut between two of its chunks;
  • HTTP/2 bodies;
  • transfer codings;
  • bodies decompressed as soon as they are read.

Dependabot's monthly update would bump the version as well; this PR adds the tests and the documentation change on top.

AI disclosure

Claude Code on behalf of Matthias Kurz. The commit includes Co-Authored-By per AGENTS.md.

Test plan

  • Fail with zstd-jni 1.5.7-25 (-Dzstd-jni.version=1.5.7-25), pass with this PR:
    • zstdBodyCutInAFrameThatStartedInTheChunkThatEndedThePreviousOneFails, on JDK 11 (JNI) and JDK 25 (FFM). With 1.5.7-25, all 6 cases (3 splits, heap and direct) end normally without an error; with 1.5.7-26, all 6 fail with DecompressionException.
  • Pass with both versions:
    • zstdFramesThatShareAChunkDecodeInFull, on JDK 11 and JDK 25.
  • On JDK 25 the tests ran zstd-jni's FFM implementation: the test JVM loaded com.github.luben.zstd.ZstdBinding, which only exists in that implementation (checked with -Xlog:class+load).
  • ./mvnw -B -ntp -Dgpg.skip=true clean verify on JDK 11: 1,996 tests, 0 failures, 0 errors, 11 skipped; Revapi passed. No test-skipping flags were used.

zstd-jni before 1.5.7-26 took the end of the input for a clean end at
a frame boundary when the stream was cut inside a frame that started
in the same read as the end of the previous frame
(luben/zstd-jni#435). AHC hands zstd-jni one chunk per read, so a zstd
body of several frames that was cut inside a frame starting in the
same chunk as the end of the previous one completed without an error.
1.5.7-26 fails such a body.

Add a test for that cut, with the end of the first frame and the start
of the second in one chunk, and a test that two complete frames in one
chunk decode in full. The first fails with 1.5.7-25.

zstd-jni is an optional dependency, so the application decides which
version is used. Limit the note in the README and the Javadoc of
Http1ContentDecompressor to versions before 1.5.7-26, recommend
1.5.7-26 or later, and show that version in the README.

Claude Code on behalf of Matthias Kurz

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant