Skip to content

fix: harden CLARIN shared surface + dtq owningCollection reauth - #24

Open
jr-rk wants to merge 2 commits into
test/clarin-usage-coveragefrom
fix/clarin-shared-surface-hardening
Open

jr-rk wants to merge 2 commits into
test/clarin-usage-coveragefrom
fix/clarin-shared-surface-hardening

Conversation

@jr-rk

@jr-rk jr-rk commented Aug 20, 2026

Copy link
Copy Markdown

Purpose

The dspace_rest_client/ production fixes that were previously bundled with the
test/CI PR, split out per review so the test PR stays scoped. Stacked on
test/clarin-usage-coverage (base of this PR) — it supplies the _helpers
builders, pytest.ini markers and the differential-contract job these tests
run under. Merge that PR first.

Targets dtq. The failure-mode assertions are dtq_only and deselected on the
main leg; main carries the same bugs, but propagating these fixes to main is
deliberately out of scope here.

Fixes (each characterization-tested: test written first, then the method)

Method Was Now
get_bundle_by_name / get_items_from_collection / get_resource_policy NoneType subscript crash on non-200 clean None / []
get_resource_policy IndexError on empty policy list; no error/empty distinction None, records last_err
get_user_by_email truthy uuid-less User on 404 (slips past if user:) None, records last_err, no noisy 404 log
create_submit_group Group(None) crash on a 201 with empty body None
add_member posts a bare /epersons/{uuid} href → DSpace 422/404 canonical /eperson/epersons/{uuid}
Group(None) / User(None) TypeError empty object
get_owningCollection (dtq-only) empty truthy Collection on non-200 → ISstag _audit.py 401-reauth was dead code None + last_err

Verified locally

Leg A (full suite, dtq impl with these fixes): 137 pass.

jr-rk and others added 2 commits October 7, 2026 09:20
Characterization-tested fixes (test written to the intended behaviour first,
then the method corrected) to methods the main-lineage consumers call:

  get_bundle_by_name / get_items_from_collection / get_resource_policy
      non-200 or empty body -> clean None/[] instead of a NoneType subscript
      crash; get_resource_policy also records last_err so a caller can tell an
      HTTP error from a genuine empty result.
  get_user_by_email        404/any non-200 -> None (+ last_err), not a truthy
      uuid-less User that slips past a consumer `if user:` guard.
  create_submit_group      201 with an empty body -> None, not Group(None).
  add_member               post the canonical /eperson/epersons/{uuid} href;
      the bare /epersons/ path 404s and DSpace rejects the uri-list.
  Group(None) / User(None) empty object, not a TypeError.
  get_owningCollection     (dtq-only) non-200 -> None + last_err, so ISstag's
      src/repo/_audit.py `owning_col is None and last_err.status_code == 401`
      reauth retry stops being dead code.

Stacked on the test/CI PR, which supplies the _helpers builders, pytest.ini
markers and the differential-contract job these tests run under. Merge that
PR first. The failure-mode assertions are marked dtq_only (deselected on the
main leg): main carries the same bugs, but propagating these fixes to main is
deliberately out of scope here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- get_items_from_collection: a failed request returned [], the consumers'
  end-of-collection signal, silently truncating their paging loops. It now
  goes through fetch_resource and returns None (last_err set on a non-200).
- get_user_by_email: DSpace answers an unknown email with 204 No Content.
  204 is the clean miss (None, no log, no last_err); 404 and any other
  non-user answer return None but log ERROR and keep last_err; the raw
  email is never logged. Transport errors propagate instead of looking
  like a miss.
- create_group / create_user / create_submit_group: return None when the
  parsed body has no uuid, since Group(None)/User(None) are now accepted
  and would be truthy, uuid-less objects.
- get_owningCollection: an unparseable 200 records last_err; a transport
  error returns None (logged) so one timeout does not abort a prefetch loop.
- add_member: correct the comment - DSpace resolves the posted eperson by
  its trailing UUID, the bare /epersons/ href was not broken.
@vidiecan
vidiecan force-pushed the fix/clarin-shared-surface-hardening branch from 2a95b6f to 4eaf688 Compare October 7, 2026 07:26
@vidiecan
vidiecan force-pushed the test/clarin-usage-coverage branch from 0c7411b to 13ba16c Compare October 7, 2026 07:26
@vidiecan

vidiecan commented Oct 7, 2026

Copy link
Copy Markdown

Rebased onto the new #23 head (dtq a62a63e) + review fixes

New head: 4eaf688 (rebased commit 88c4c56 + one review commit), stacked on #23 13ba16c.

Conflict resolution

Review fixes (each test written first and seen failing)

Finding Change Test
get_items_from_collection turned any failed request into [] with no last_err, so the consumers' paging loop (which stops on None/empty) silently truncated Goes through fetch_resource; returns None on failure (last_err set on a non-200). I chose None over raising because it keeps this PR's goal (a failed call stops cleanly, no crash) and the consumer loop (dspace-import get_all_items) already stops on is None, while None + last_err now tells an error from the real end ([]) test_non_200_returns_none_and_records_last_err
get_user_by_email: DSpace answers an unknown email with 204, not 404 204 = clean miss: None, no log, no last_err. 404 (misconfiguration), 401/403/5xx, an unparseable or uuid-less body: None, logged at ERROR, last_err kept. The raw email is never logged. The blanket except Exception is gone, so a timeout propagates instead of looking like a clean miss test_204_no_content_is_a_clean_miss, test_error_status_returns_none_logs_error_and_records_last_err (404/401/403/500, email not in logs), test_transport_error_is_not_a_clean_miss; contract test now uses 204
Group(None)/User(None) accepted, so a failed create returned a truthy uuid-less object create_group/create_user return None when the parsed body has no uuid; create_submit_group checks j.get('uuid') TestCreateGroupAndUser (201 typed, shared; failure None, dtq_only), test_201_without_uuid_returns_none
get_owningCollection: unparseable 200 did not record last_err; a timeout aborted the caller's prefetch loop Unparseable 200 sets last_err; requests.RequestException is caught, logged, None returned test_unparseable_200_returns_none_and_sets_last_err, test_transport_error_returns_none_without_raising
The add_member comment / dtq_only claimed the bare /epersons/{uuid} href "does not resolve" Comment corrected (DSpace matches only the trailing UUID); canonical /eperson/epersons/{uuid} kept; the dtq_only test is replaced by #23's shared one shared add_member body test

Note on transport errors: last_err holds a Response, and the ISstag _audit.py predicate dereferences last_err.status_code, so it is not set to an exception object. In get_owningCollection a None result is always a failure, so returning None with a logged error is unambiguous. In get_user_by_email, None with no last_err means "no such user", so a timeout has to propagate instead.

Verified locally

  • Full suite + --cov-fail-under=70: 163 passed, 73.9% coverage.
  • Differential contract: dtq leg 66 passed; main leg 46 passed, 20 dtq_only deselected.
  • Python 3.8.20: 163 passed.
  • Offline consumer check (dspace-rest-test, read-only): every DSpaceClient call it makes (26 call shapes, incl. create_clarinlruallowances, get_clarinlruallowances_by_bitstream_and_user, get_user_by_email, add_member, create_submit_group) binds to the rebased signatures, and the return contracts it relies on hold (User/Group with uuid, None on miss, True from add_member, allowance list with _links). Also checked get_resource_policy(...)["id"] and get_bundle_by_name.

Deferred (not changed here)

  • One fetch helper for all the CLARIN getters (several still open-code api_get + parse_json).
  • get_resource_policy returns policies[0] of a possibly multi-policy list.
  • create_dso still crashes on a 201 with an empty body (new_dso["type"] on None).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants