Skip to content

B3: fix the image-browser segfault and the 99% hang after a worker crash - #1184

Merged
trissim merged 2 commits into
mainfrom
fix/B3-image-browser-segfault
Oct 10, 2026
Merged

trissim merged 2 commits into
mainfrom
fix/B3-image-browser-segfault

Conversation

@trissim

@trissim trissim commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Symptom

pytest tests/unit tests/core -n 2 --dist loadfile segfaulted an xdist worker at about 96%, in test_image_browser_streaming_config.py::test_streaming_enableable_is_title_only_and_targets_its_config_group. The session then hung at 99% for hours.

Cause 1: the segfault (pyqt-reactive flash coordinator)

_GlobalFlashCoordinator keeps a flash registration "pending" while its widget is outside a QMainWindow/QDialog. Nothing removed these registrations except a successful attach, so they piled up for the whole process. In this reproducer, 300+ were pending by the image-browser file. Every local flash called widget.isVisible() and widget.window() on each one and caught RuntimeError to skip deleted widgets.

test_step_code_widget_roundtrip.py leaves a registration whose widget is a ReflowingVerticalScrollArea viewport. Qt creates that viewport, and the code reaches it through parentWidget(). Its owner is deleted later, during the image-browser test's first QTest.qWait. I attached a watcher to the viewport: its destroyed signal fired, but sip.isdeleted(viewport) stayed False. The next ParameterFormManager flush therefore called window() on freed memory, and the crash stack goes through _process_pending_registrations → get_for_window.

Fix (OpenHCSDev/PyQT-reactive#19, pinned here as 0.3.30 / 10ba5b4): a pending registration now ends when Qt destroys its widget or any widget its element closes over. It uses the _DestructionCallback mechanism the mixin already uses for its own records. The RuntimeError probe is gone.

Cause 2: the 99% hang (pytest-xdist loadfile scheduler)

When a worker crashes, LoadScopeScheduling.remove_node puts every file assigned to the dead worker back in the work queue, including files it had already finished. The replacement worker is handed one queued file per test it completes, and an already-finished file contains no tests. With ~470 finished files queued, the workers run out of tests while files are still queued. tests_finished then never becomes true, and the controller waits forever without sending shutdown. The OSError: cannot send (already closed?) lines at the end of the logs come from the workers' pytest_sessionfinish running only when the session was killed. Our conftest is not involved. A 41-file synthetic suite whose last file segfaults hangs the same way: it hit timeout 120 with xdist 3.8.0.

Fix: pytest.ini adds --max-worker-restart=0, so a crashed worker ends the session with the crashed test reported. The same synthetic suite now exits 1 in under a second (worker 'gw0' crashed while running ...).

Reproduction

Without the fix, tests/unit/test_napari_reset_transitions.py tests/unit/pyqt_gui/test_step_code_widget_roundtrip.py tests/unit/pyqt_gui/test_image_browser_streaming_config.py segfaults every time at the same frame. Without the napari file it does not crash. That three-file set came from bisecting the crashed worker's 48 GUI and viewer files, and the full 48-file sequence also segfaulted on both of its unfixed runs.

Verification (QT_QPA_PLATFORM=offscreen, DISPLAY unset, worktree basetemp)

  • Three-file reproducer: 5/5 passed (63 passed each run).
  • The crashed worker's 48-file sequence: no crash. Every failure is also in the main full-suite failure list except test_async_failure_finalizes_before_pipeline_editor_bridge_presentation (5 s poller timeout under load). That test passed 3/3 alone and passed in the tests/unit/pyqt_gui run.
  • test_image_browser_streaming_config.py: 20/20 serially, 20/20 with -n 2.
  • Image browser and napari tests (13 files, -n 2): 309 passed. 11 failed and 7 errored, all in the existing main failure list.
  • tests/unit/pyqt_gui -n 2 --dist loadfile: no crash. 953 passed, 10 failed, all in the existing main failure list.
  • pyqt-reactive tests -n 2: 610 passed.

🤖 Generated with Claude Code

…t worker ends the session

Repin pyqt-reactive 0.3.30 (10ba5b4). Its flash coordinator kept
registrations for widgets outside a flash window forever and called
window() on a scroll-area viewport after Qt had freed it. sip had not
marked that wrapper deleted, so the call segfaulted in
test_image_browser_streaming_config.py.

pytest.ini: --max-worker-restart=0. After a crash, xdist's loadfile
scheduler requeues every file the dead worker had already finished. It
hands the replacement worker one of those files per completed test, so
the workers run out of tests with files still queued, and the controller
waits forever at 99%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 10, 2026 19:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@trissim
trissim merged commit 188ea19 into main Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants