Skip to content

test(two-node-sim): model F25 (takeover waits for the append lock on a frozen peer's JM) - #1507

Merged
schmidt-scaled merged 1 commit into
mainfrom
test/two-node-sim-f25
Oct 19, 2026
Merged

schmidt-scaled merged 1 commit into
mainfrom
test/two-node-sim-f25

Conversation

@schmidt-scaled

Copy link
Copy Markdown
Contributor

Bed run c12 (2026-10-19, node image = ultra main + #217 + #219 incl. F24, CP main-ceff9f53): 02c (the vol1 primary sn2 frozen 8 s) no longer aborts the survivor (F24 fixed), but stalls 6/7/8 s with 0 IO errors.

Mechanism (proven on the bed logs): at the takeover the new leader re-locks the appender on every JM and helper_ensure_append_locks_ready waits for all lock requests. The one on the frozen peer's JM (a READ on remote_jm_…) neither completes nor fails until the peer aborts on wake or the 8 s NVMe IO timeout fires ("locke failed _res_lock -3"), and the lvstore update's used-blobids write, with all client IO, waits until then.

This PR models it:

  • CAP_APPENDER_LOCK_SKIPS_PEER_JM (the fix: ultra Adding lvol id to lvol events #219, commit b4d92c71)
  • with F23+F24 but not F25, a solo takeover of a frozen peer's LVS lasts until the peer's frozen_until → violation F25
  • new scenario F25-frozen-primary-takeover-append-lock, judged on the client stall. The F24 scenario had stall_limit_s=None, which is why the simulator did not catch this before the bed did.
  • F25's capability is added to every pending-fix set that includes F24's
  • test_f25_needs_its_own_fix

Local run against FDB: 64 passed, 2 skipped, 8 xfailed.

🤖 Generated with Claude Code

…ck on a frozen peer's JM

Bed c12/02c (image with ultra #219 incl. F24): no survivor abort any more, but
the new leader's first write waited for its append-lock request on the frozen
peer's JM until the peer aborted on wake (6-8 s client stall, 0 IO errors).

- simnode: CAP_APPENDER_LOCK_SKIPS_PEER_JM (ultra #219, F25 commit)
- actors: with the F23+F24 capabilities but not F25's, a solo takeover of a
  frozen peer's LVS lasts until the peer's frozen_until, violation "F25"
- scenarios: F25-frozen-primary-takeover-append-lock, judged on the client
  stall (the F24 scenario has no stall limit, which is why it missed this);
  F25's capability added to every pending-fix set that includes F24's
- test_f25_needs_its_own_fix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the target/main Targets the development branch label Oct 19, 2026
@schmidt-scaled
schmidt-scaled merged commit 4aa5fc0 into main Oct 19, 2026
21 checks passed
@schmidt-scaled
schmidt-scaled deleted the test/two-node-sim-f25 branch October 19, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

target/main Targets the development branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants