[kixelated]

Problem

On NVIDIA under Linux (nixos: RTX 3070 Ti, driver 580.178.04), 7 compositor GPU tests passed and then crashed when the process exited, with SIGSEGV or SIGABRT (free(): invalid pointer): a_move_glides_an_overlay, an_overlay_* (4), a_stinger_mask_runs_on_under_a_preview, and scaling_bleeds_no_transparent_color. Found in moq-dev/studio#74.

Reproduced at main (2bbdd5b): 41/48 compositor::tests pass, the same 7 abort; a_move_glides_an_overlay run alone crashed 6 of 6 times.

Root cause, from the core dump: the overlay thread is detached. When the compositor drops, the thread wakes, frees Vello's pipelines (Painter drop, vkDestroyDescriptorPool in libnvidia-eglcore), while the main thread has already returned and is in exit(), running libEGL_nvidia's exit handlers that tear down the driver. The overlay thread then crashes inside the driver. Every failing test opens an overlay; Mesa's lavapipe has no such exit-time teardown, so it never crashed there. Not a wgpu, driver, or moq bug: freeing GPU objects after main returns is the studio's.

Thread 1 "overlays": ?? <- libnvidia-eglcore <- ash::Device::destroy_descriptor_pool
  <- wgpu_hal::vulkan::Device::destroy_bind_group_layout <- ... <- drop_glue::<moq_scene::overlay::Painter> <- moq_scene::overlay::run
Thread 3 (main): __pthread_clockjoin_ex <- libnvidia-eglcore <- libEGL_nvidia <- __run_exit_handlers <- exit

Approach

• Overlays keeps its thread's JoinHandle. Dropping it sends a new Request::Stop and joins the thread, so the documents and the painter are freed before the compositor finishes dropping. Stop is needed because each Overlay holds a sender clone, so the channel would not close on its own.
• Matte has the same pattern: its detached thread holds a device clone and the readback buffer, and frees them after the matte drops. It now drops its job sender and joins the thread on drop. No test hit this one (the matte tests wait for the thread before dropping), but the race is the same.
• This matches moq-video's Completion and Worker, which join their GPU threads on drop.

Impact

• The compositor GPU tests exit cleanly on NVIDIA.
• Dropping a compositor that has overlays now waits for the overlay thread: the whole compositor drop took 5.5 to 6.2 ms in a_move_glides_an_overlay on the RTX 3070 Ti (debug and release). This happens only at shutdown, or when the overlay thread has already failed.
• Dropping a matte while it is cutting out a picture waits for that one inference: 4.5 ms in release (8.7 ms in debug), or up to 57 ms for the first one in the process, which also loads the model. A matte drops when a source stops removing its background.
• Public API: none (both types are pub(crate)). Wire: none. README unchanged: no flag, panel, scene file, or output change.

Alternatives

• Skip the tests or set the thread aside at exit: ruled out by the quest.
• Close the overlay thread when every sender is gone, with no Stop: depends on every Overlay dropping before Overlays, and a stray one would hang the join forever.
• Keep the matte thread free of GPU objects by reading back on the compositor's thread and sending only bytes to the model thread: a bigger change to fix a race this small. Joining costs one inference at most.

Choices made without asking

• Fix the matte thread too, since it is the same bug (a detached thread freeing GPU objects after its owner drops), though no run crashed on it.
• Leave the browser reader thread alone: it also holds the device, but Browser::stop already joins it, and moq-dev/studio#76 is working in that module.

Validation

On nixos (RTX 3070 Ti, NVIDIA 580.178.04), at 85536dd:

Run	main (2bbdd5b)	this PR
compositor::tests (--run-ignored only -j 4)	41/48, 7 abort at exit	48/48, 10 runs in a row (480/480)
a_move_glides_an_overlay alone, the test binary directly	6 of 6 crash (4 SIGSEGV, 2 SIGABRT)	passes
compositor::tests + browser::layers::tests, NVIDIA		49/49
compositor::tests + browser::layers::tests, lavapipe (VK_DRIVER_FILES)		49/49
engine::tests::the_program_*, setup::tests::times_every_canvas, NVIDIA		4/4
just ci		pass (278 tests)

• The backtrace above is from coredumpctl on a crash at main; under gdb the test passed, since gdb's slower teardown hides the race.
• -j 4 because another process holds 5.6 of the GPU's 8 GB; moq-dev/studio#74 hit RequestDeviceError(OutOfMemory) at default parallelism. No out-of-memory errors at -j 4.
• Drop and inference times were measured with temporary eprintln! probes, not committed.
• Stressing the two matte tests 10 times without the matte join did not crash, as expected since they wait for the thread first.
• macOS not run: the change is platform-independent thread teardown, and the crash is Linux NVIDIA's.

Follow-ups

None.

Review

• Reviewer: Codex (quest review), gpt-6.1-sol, medium effort.
• Round 1 at 85536dd: no findings.
• Reviewed head: 85536dd.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

[pullfrog]

  ^{｜ Review this ➔}

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 28 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 7032bcf4-3ead-4ae2-9a4f-2740a7b6e1a7

Commits

Reviewing files that changed from the base of the PR and between 3ff7dc9 and fcaa0b2.

Files selected for processing: 4

• quest/m0/README.md
• quest/m0/gpu-tests-exit.md
• rs/moq-scene/src/matte.rs
• rs/moq-scene/src/overlay.rs

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: 85536dd
Base: 2bbdd5b

No actionable findings.

Direction: joining the GPU-owning threads fixes their lifetime rather than hiding the exit race. The explicit overlay Stop message avoids depending on every Overlay sender clone being dropped first; the join also covers a worker already unwinding after a panic. Matte closes its sole job sender before joining, keeps the device/readback resources alive through that join, and its worker polls its own submitted GPU work. Its unbounded result channel does not require the dropping compositor to consume a result before the worker exits. The existing one-picture-at-a-time ownership bounds the work drained during matte shutdown.

Verification: GitHub-only static review of the changes, worker request/result loops, GPU resource ownership, compositor cleanup/failure paths and quest closure. No builds, tests, core-dump inspection, NVIDIA/Metal runs or timing measurements performed. The reported repeated GPU-suite passes and shutdown latency were not independently reproduced. This is a COMMENT, not approval.

(Written by review (OpenAI))