[Kasinhou]

Security — exclude restricted-bitstream full text from the Discovery index

Part of dataquest-dev/dspace-customers#1104, see #1083.

Problem

The extracted full text of every bitstream in an item's TEXT bundle was concatenated into the item's single Solr fulltext field with no per-bitstream access check, while Discovery access control is applied only at the item level. So for a publicly-readable item that contains an access-restricted file, an anonymous keyword search matched words from the restricted file — a content oracle (probe word-by-word), and the highlighter could even return the restricted snippet.

Fix

In FullTextContentStreams.buildFullTextList(), a TEXT-bundle bitstream is indexed only when it is readable by the Anonymous group — checking both the derivative and its matching source ORIGINAL bitstream (media-filter copies policies, but that copy can drift, so both are verified), with date-valid policies and fail-closed on error. Restricted text therefore never enters fulltext, which closes both the search oracle and the highlight/snippet channel at the source (all search_text / spellcheck / fulltext_hl fields derive from fulltext). The Anonymous group is resolved once per item for reindex performance.

Operational note — reindex required

This affects new indexing only. Existing Solr documents keep previously-indexed restricted text until a full reindex is run on deploy: dspace index-discovery -b.

Intentional trade-off

The gate is anonymous-based on a shared field, so a file readable only by an authenticated group is no longer full-text searchable even for authorized users. Closing the anonymous oracle this way is the minimal, safe fix; per-user full-text search of restricted content would require a larger per-bitstream-field redesign and is out of scope.

Tests

• Unit (FullTextContentStreamsTest, 8 passing): a restricted derivative is excluded; a readable one is included; and a readable derivative whose source ORIGINAL is restricted is excluded (the policy-drift case).
• Integration (DiscoveryRestControllerIT): public item with a restricted file whose unique token is searched — anonymous gets 0 hits, a token in a public file still returns the item (no regression), admin also gets 0 for the restricted token.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

A critical fail-open path can still index restricted content when the matching source bitstream cannot be resolved.

1 open issue

• Fail closed when derivative source cannot be resolved · New

What changed in this PR

This pull request prevents restricted bitstream text from being indexed in DSpace Discovery’s shared Solr fulltext field by checking Anonymous access to text derivatives and their matching source files.

Changes:

• Adds Anonymous, date-valid read-policy checks for TEXT derivatives and corresponding ORIGINAL bitstreams.
• Adds unit and integration coverage for restricted derivatives, restricted sources, and publicly readable content.
• Leaves an unresolved access-control gap when a matching source bitstream cannot be resolved.

File	Description
dspace-server-webapp/​src/​test/​java/​org/​dspace/​app/​rest/​DiscoveryRestControllerIT.java	Verifies Discovery search behavior for restricted and publicly readable file content.
dspace-api/​src/​test/​java/​org/​dspace/​discovery/​FullTextContentStreamsTest.java	Tests exclusion of restricted derivatives and source bitstreams.
dspace-api/​src/​main/​java/​org/​dspace/​discovery/​FullTextContentStreams.java	Filters full-text streams based on Anonymous access policies.

_{Review effort: Lite}