Repository navigation
fix(security): close bitstream stream leak on HEAD/304/412/416/S3-redirect - #1559
Conversation
…responses The /content download opened the storage InputStream eagerly while building response headers (getChecksum()/contentLength() called fetchDocument() which cached an open stream). The stream was only closed when Spring wrote the response body, so HEAD, 304, 412 and the S3 redirect (302) returned without a body and leaked the stream (an S3 connection for S3 assetstores). fetchDocument() now caches only the etag and length (and materializes the cover page into a byte[]); getInputStream() opens the stream lazily when the body is actually written, via a fresh short-lived Context.
The existing testS3DirectDownloadRedirect is gated behind @IfProfileValue and is skipped in a standard CI run, leaving the highest-severity leak path (the S3 302 redirect) without a CI-enforced guard. This adds an ungated test that drives the redirect branch with a mocked presigned-URL service and runtime config, asserts a 302 and verifies the storage stream is never opened.
There was a problem hiding this comment.
🟡 Changes recommended
Citation-page materialization and fallback stream handling can still leak on bodyless paths, and the test must restore the original S3 configuration value.
1 open issue
What changed in this PR
This pull request updates bitstream downloads to defer storage stream creation until the response body is written, with regression coverage for bodyless responses, redirects, and stream closure.
Changes:
- Caches bitstream metadata and lazily retrieves normal content streams.
- Adds coverage for
HEAD, conditional responses, S3 redirects, and successful downloads. - Preserves fresh-stream behavior for repeated or range-based reads.
| File | Summary |
|---|---|
dspace-server-webapp/src/test/java/org/dspace/app/rest/BitstreamRestControllerIT.java |
Adds regression coverage for bodyless responses, redirects, and stream usage. |
dspace-server-webapp/src/main/java/org/dspace/app/rest/utils/BitstreamResource.java |
Implements lazy bitstream retrieval and cached document metadata. |
Review effort: Lite
Why the file stayed openThe server opened the file while it built the headers. Spring closed it only after it wrote the body. With no body, nothing closed it. This PR opens the file only when Spring writes the body. flowchart LR
R["GET or HEAD /content"] --> H["build headers"]
H -->|"before: opens the file"| B1{"body?"}
B1 -->|"200"| C1["closed after the body"]
B1 -->|"HEAD, 304, 412, 416, 302"| L["stays open until GC"]
H -->|"after: does not open it"| B2{"body?"}
B2 -->|"200"| C2["opened, sent, closed"]
B2 -->|"HEAD, 304, 412, 416, 302"| N["nothing to close"]
style L stroke:red,stroke-width:2px
412 and 416 work like HEAD, but are rare. On S3, opening a file first downloads up to 5 MB to a temp file. So on S3 each such request costs a download and a temp file. It does not keep an S3 connection open. |
A non-matching If-Match ends in 416 without a body, like HEAD, 304 and 412, so the storage stream must not be opened there either. The file is now opened only when Spring writes the body. A test checks that a file missing in the assetstore still ends in 500, not in an empty 200. The tests added in this PR used var. The rest of the repo uses explicit types, so they now declare BitstreamStorageService. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Port of upstream DSpace PR 12249 by Bram Maegerman, taken from its dspace-8_x port, DSpace PR 12534 (merge commit 6c8b05f). BitstreamResource.java is now the dspace-8_x file with one change: the cover-page fallback still closes the stream of the original file. RequestItemEmailNotifier.java gets the hunks from 12534 and closes the attachment streams after the mail is sent. The four tests from 12534 are copied as they are. Our own tests stay only for cases upstream does not test: 304, 412, 416, the S3 redirect, the cover-page fallback and a missing file. Our HEAD test repeated the upstream one and is removed. The gated S3 test is back to the dtq-dev version, the ungated S3 test covers the same case. Co-authored-by: Bram Maegerman <bram.maegerman@telenet.be> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The upstream HEAD test only checks that a stream is closed if it was opened. This test checks that HEAD on /content does not call retrieve on the storage at all, like the 304, 412 and 416 tests do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

References
Description
Port of the upstream fix for bitstream streams that were never closed.
/api/core/bitstreams/{uuid}/contentnow opens the stored file only when it writes the body (a citation cover page is still built up front), so HEAD, 304, 412, 416 and the S3 redirect (302) no longer leave it open, and request-a-copy closes the files it attaches.Instructions for Reviewers
List of changes in this PR:
BitstreamResource.javais the vanilladspace-8_xfile with one difference. In the cover-page fallback ingetCoverpageByteArray(), vanilla leaves the stream of the original file open (also onmain). Here it is closed with try-with-resources.RequestItemEmailNotifier.javagets the same hunks as DSpace PR 12534: the attachment streams are closed infinally. On dtq-dev the single-file branch sits inside the check from dtq-dev PR 1520, so that hunk is one level deeper.BitstreamRestControllerIT: the four tests from DSpace PR 12534, unchanged. Our own tests cover what upstream does not check: HEAD, 304, 412, 416 and the S3 redirect never open the file, a failed cover page closes its streams, and a GET for a file missing in the assetstore ends in 500. The upstream HEAD test only checks that HEAD closes what it opens.How to test:
On a running instance with a local assetstore, send
curl -I .../server/api/core/bitstreams/<uuid>/contenta few times and runlsof -p <tomcat pid> | grep assetstore. Before, every HEAD left one more file open until garbage collection. After, none.On S3, each response without a body also downloaded the first chunk of the file (5 MB by default) to a temp file. Now it downloads no file content, unless a citation cover page has to be built.
Changes a client can see when a file is missing in the assetstore and no citation cover page is built:
Checklist
mainbranch of code (unless it is a backport or is fixing an issue specific to an older branch).pom.xml), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.🤖 Generated with Claude Code