From 3d64a90741a5abaff957b9c01f16dae2af7aa9a7 Mon Sep 17 00:00:00 2001 From: Kasinhou <129340513+Kasinhou@users.noreply.github.com> Date: Tue, 20 Oct 2026 13:56:27 +0200 Subject: [PATCH] mendelu: disallow heavy Discovery facet/pagination browse URLs Cherry-pick of upstream DSpace/dspace-angular 9b03e82a14. On 8.x+ the community landing page renders the search section on the bare /communities/ route (default child route), so its pagination is emitted as /communities/?spc.page=N - not covered by the existing /communities/*/search or facet rules. Also blocks /collections/*?f. Not applied to 7.x branches: there the community page has no such route, so ?spc.page is never generated. Refs dataquest-dev/dspace-customers#1085 Co-Authored-By: Claude Opus 4.8 --- src/robots.txt.ejs | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/robots.txt.ejs b/src/robots.txt.ejs index 64d93747026..a54a06490a8 100644 --- a/src/robots.txt.ejs +++ b/src/robots.txt.ejs @@ -31,6 +31,9 @@ Disallow: /communities/*/search # one (&f.), and robots.txt has no way to express "either". Disallow: /*?f. Disallow: /*&f. +# Heavy Discovery facet queries on browse pages +Disallow: /collections/*?f +Disallow: /communities/*?spc.page # Optionally uncomment the following line ONLY if sitemaps are working # and you have verified that your site is being indexed correctly.