Skip to content

Decouple emulator wake pipe from window subsystem - #143

Open
Cuda-Chen wants to merge 1 commit into
sysprog21:masterfrom
Cuda-Chen:decouple-wake-pipe
Open

Cuda-Chen wants to merge 1 commit into
sysprog21:masterfrom
Cuda-Chen:decouple-wake-pipe

Conversation

@Cuda-Chen

@Cuda-Chen Cuda-Chen commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

The self-pipe trick (wake_fd) was previously coupled to the window backend: wake_fd was only
declared under VIRTIOINPUT || VIRTIOGPU, pipe creation required g_window.window_main_loop, and poll/drain/signal-wakeup handling shared the window guards.

When building without SDL2 (ENABLE_SDL=0), the Makefile disables VIRTIOGPU and VIRTIOINPUT while other background threads such as VIRTIOSND and signal handlers still need to unblock poll(-1) in the SMP scheduler. In addition, virtio-snd never notified wake_fd on TX worker completions, and unmasked helper threads spawned by virtio_snd_init() / PortAudio could handle SIGINT/SIGTERM after teardown closed the pipe.

Changes

  • Decouple wake-pipe lifecycle: Make wake_fd[2], pipe setup under vm->n_hart > 1 (both ends non-blocking), signal_wake_fd, semu_close_wake_pipe(), semu_cleanup_backends(), and semu_run() poll/drain paths unconditional while keeping g_window calls under VIRTIOINPUT || VIRTIOGPU.
  • Drain and exit ordering: Drain all queued wake bytes in semu_run(), run emu_tick_peripherals() immediately, and break cleanly on emu->stopped || signal_received.
  • Plain teardown: Close wake_fd with plain close() and reset fields to -1 in semu_close_wake_pipe(), relying on callers running on the emulator thread or after pthread_join().
  • VirtIO-Sound wakeup & signal masking: Expose virtio_snd_set_wake_fd(), write one wake byte from the virtio-snd TX worker when completions or virtio_snd_set_fail() raise InterruptStatus, and block SIGINT/SIGTERM around pthread_create(), Pa_Initialize(), Pa_OpenStream(), and Pa_StartStream().
  • Unit test & CI: Add tests/test_vsnd_wake.c, PA_LDFLAGS / unit-test targets in Makefile, .ci/test-unit.sh, Linux/macOS CI steps in .github/workflows/main.yml, and .gitignore entries for test artifacts.

Summary by cubic

Decouples the emulator wake pipe from the window subsystem so VIRTIOSND and other async devices can wake the SMP scheduler's poll(-1) in headless or ENABLE_SDL=0 builds.

Previously the wake pipe was only created when a window backend was present (g_window.window_main_loop and VIRTIOINPUT/VIRTIOGPU), so builds without SDL2 had no way to unblock the emulator's idle wait from background threads.

  • Creates the wake pipe whenever the SMP scheduler runs (vm->n_hart > 1), marks both ends non-blocking, and drains all queued wake bytes before ticking peripherals.
  • Wires the VIRTIOSND TX worker to write a wake byte on completions and device-fail through virtio_snd_set_wake_fd().
  • Blocks SIGINT/SIGTERM around helper-thread creation (window init, virtio-snd TX worker, PortAudio init/open/start) so spawned threads inherit the mask; signals stay unblocked only while the emulator or window loop runs, and are re-blocked before teardown.
  • Keeps all g_window calls under VIRTIOINPUT || VIRTIOGPU, with teardown through semu_close_wake_pipe() and semu_cleanup_backends().
  • Adds a unit-test Makefile target with a test_vsnd_wake harness, run in Linux and macOS CI.

Written for commit 4b38fae. Summary will update on new commits.

View guided diff

@Cuda-Chen
Cuda-Chen marked this pull request as ready for review November 14, 2026 14:55
@Cuda-Chen
Cuda-Chen requested review from Mes0903 and jserv November 14, 2026 14:55
cubic-dev-ai[bot]

This comment was marked as resolved.

Comment thread device.h Resolved
Comment thread main.c Resolved
Comment thread main.c
#if SEMU_HAS(VIRTIOINPUT) || SEMU_HAS(VIRTIOGPU) || SEMU_HAS(VIRTIOSND)
static void semu_close_wake_pipe(emu_state_t *emu)
{
signal_wake_fd = -1;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearing signal_wake_fd does not stop a handler that has already loaded the old value. With virtio-snd, the TX thread and the audio backend thread can receive SIGINT, then write() after the close() below, onto a closed or reused fd. This race already existed in the window path, but this change extends it to headless builds. Follow-up: block SIGINT/SIGTERM with pthread_sigmask() around pthread_create() in virtio_snd_init(), so signals always reach the emulator thread and poll() returns EINTR on its own.

Comment thread main.c Resolved
Comment thread main.c
ssize_t bytes_drained =
read(emu->wake_fd[0], &wake_byte, sizeof(wake_byte));
(void) bytes_drained;
emu->peripheral_update_ctr = 0;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reset itself is fine: it makes the next emu_tick_peripherals() run at once. But this counter also drives boot_complete (peripheral_update_ctr > BOOT_SETTLE_ITERATIONS), and since emu_tick_peripherals() caps it at 64, that test is never true. So the 1ms timer fd is always polled and poll(-1) wakes every millisecond whether or not this pipe fires, which will hide any wake-latency problem. Follow-up: give the boot-settle check its own monotonic counter.

@Cuda-Chen
Cuda-Chen requested a review from jserv November 15, 2026 07:17
Comment thread Makefile Outdated
scripts/build-artifacts.sh $(ARTIFACTS)

clean:
$(Q)$(RM) -r $(UNIT_TEST_BINS) $(addsuffix .dSYM,$(UNIT_TEST_BINS))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This clean rule only removes $(UNIT_TEST_BINS), which is empty when VIRTIOSND is off. After switching configs, make clean ENABLE_VIRTIOSND=0 leaves a stale tests/test_vsnd_wake and its .dSYM behind. Name the test artifacts unconditionally here, as .gitignore already does.

Suggested change
$(Q)$(RM) -r $(UNIT_TEST_BINS) $(addsuffix .dSYM,$(UNIT_TEST_BINS))
$(Q)$(RM) -r tests/test_vsnd_wake tests/test_vsnd_wake.dSYM

Comment thread main.c Outdated
* cause a deadlock where harts remain stuck waiting even after
* events arrive.
*/
if (emu->stopped || signal_received)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This early exit now sits between the "Resume all hart coroutines ... we must always resume harts" comment and the loop that comment describes, so the comment reads as if it were about the break. Move the check above that comment, next to the drain block it depends on.

Comment thread main.c
}

#if SEMU_HAS(VIRTIOINPUT) || SEMU_HAS(VIRTIOGPU)
/* Callers run on the emulator thread or after 'pthread_join()', so no other

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment says signal_handler() only runs on threads joined before teardown, but only the virtio-snd helper threads are masked. Threads created by SDL_Init() in window_init(), or by any later library, still accept SIGINT/SIGTERM and can load the old signal_wake_fd just before the close(). The race predates this PR, but the comment now promises something the code does not enforce. Either weaken it to best effort, or as a follow-up block SIGINT/SIGTERM once in main() before semu_init() and unblock them only on the thread that runs semu_run() and the window loop. That one mask would also replace the per-call masking in virtio_snd_init().

Comment thread tests/test_vsnd_wake.c Outdated
assert(vsnd.priv != NULL);

int pfd[2];
assert(pipe(pfd) == 0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pipe() call, the fcntl() calls and the read() calls that drain the pipe all sit inside assert(). Built with -DNDEBUG (for example via an overridden CFLAGS), the test never creates the pipe yet still prints PASS. Do the call first and assert on the saved result.

Suggested change
assert(pipe(pfd) == 0);
int rc = pipe(pfd);
assert(rc == 0);

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 4 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="main.c">

<violation number="1" location="main.c:1985">
P2: `SIGINT` and `SIGTERM` stay blocked throughout `semu_init()`, but their handlers are installed only afterward. If a backend initializer stalls, termination signals remain pending and cannot stop the process; provide an init-time cancellation path or narrow the blocking interval.</violation>
</file>

Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack

View guided diff | Re-trigger cubic

Comment thread main.c Outdated
semu_run_debug(emu);
else
semu_run(emu);
semu_set_term_signals_blocked(true);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: SIGINT and SIGTERM stay blocked throughout semu_init(), but their handlers are installed only afterward. If a backend initializer stalls, termination signals remain pending and cannot stop the process; provide an init-time cancellation path or narrow the blocking interval.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At main.c, line 1985:

<comment>`SIGINT` and `SIGTERM` stay blocked throughout `semu_init()`, but their handlers are installed only afterward. If a backend initializer stalls, termination signals remain pending and cannot stop the process; provide an init-time cancellation path or narrow the blocking interval.</comment>

<file context>
@@ -1961,10 +1977,12 @@ static void *emu_thread_func(void *arg)
         semu_run_debug(emu);
     else
         semu_run(emu);
+    semu_set_term_signals_blocked(true);
 
     /* Unblock 'window_main_loop()' on the main thread so it can return. */
</file context>

Comment thread .github/workflows/main.yml Outdated
shell: bash
timeout-minutes: 5
- name: unit test
run: .ci/test-unit.sh

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.ci/test-unit.sh only changes to the repository root and runs make unit-test. A workflow step already starts there, and the test prints its own PASS line. Call make directly in both jobs (the macOS step too) and drop the script, the same way the build step runs plain make.

Suggested change
run: .ci/test-unit.sh
run: make unit-test

Comment thread main.c Outdated
{
int ret;
emu_state_t emu;
semu_set_term_signals_blocked(true);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking SIGINT/SIGTERM for the whole of semu_init() changes what Ctrl-C does at startup. Before this, the default action killed the process. Now the signal stays pending until init returns and the handler is installed. A slow or stuck init (loading a large initrd or disk image, setting up the network backend) can then only be stopped with SIGKILL. If that is intended, say so in the commit message. Otherwise, mask only around the init calls that spawn helper threads (window_init(), virtio_snd_init()), saving and restoring the mask the way virtio-snd.c already does around Pa_OpenStream().

Comment thread tests/test_vsnd_wake.c Outdated
Comment on lines +234 to +238
wait_used_idx(ram, ADDR_TX_USED, tx_idx);
poll_read.revents = 0;
nev = poll(&poll_read, 1, 50);
assert(nev == 0);
assert(rreg(&vsnd, VIRTIO_InterruptStatus) & VIRTIO_INT__USED_RING);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait_used_idx() returns as soon as the used index moves. The TX worker stores that index before it sets VIRTIO_INT__USED_RING in InterruptStatus. If the worker is preempted for longer than the 50 ms poll() window, which can happen on a loaded CI runner, this assertion fails even though the code is correct. Wait for the interrupt bit instead: loop on rreg(&vsnd, VIRTIO_InterruptStatus) with the same bound as wait_used_idx(), then check that the pipe stayed empty.

Comment thread virtio-snd.c Outdated
* around calls on unblocked threads (e.g., the emulator thread after init)
* that dynamically spawn helper threads, which inherit the blocked mask.
*/
static void vsnd_block_term_signals(sigset_t *old_mask)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This builds the same SIGINT/SIGTERM set as semu_set_term_signals_blocked() in main.c. Keep one save/restore pair in a shared header (for example semu_block_term_signals(sigset_t *old) in utils.h, restored with pthread_sigmask(SIG_SETMASK, old, NULL)) and call it from both files, so the two copies of the signal set cannot drift apart. The description also says virtio_snd_init() blocks these signals around pthread_create() and Pa_Initialize(), but it does not. That is safe only because main() blocks them first, and tests/test_vsnd_wake calls virtio_snd_init() with nothing blocked. Either wrap those two calls with this helper or fix the description.

The self-pipe trick ('wake_fd') was previously coupled to the window
backend: 'wake_fd' was only declared under 'VIRTIOINPUT || VIRTIOGPU',
pipe creation required 'g_window.window_main_loop', and poll/drain/
signal-wakeup handling shared the window guards.

When building without SDL2 ('ENABLE_SDL=0'), the Makefile disables
'VIRTIOGPU' and 'VIRTIOINPUT' while other background threads such as
'VIRTIOSND' and signal handlers still need to unblock 'poll(-1)' in
the SMP scheduler. In addition, 'virtio-snd' never notified 'wake_fd'
on TX worker completions, and unmasked helper threads spawned during
window/audio initialization or stream setup could handle
'SIGINT'/'SIGTERM' after teardown closed the pipe.

Decouple wake-pipe lifecycle management and wire 'virtio-snd' + CI
coverage:
- Make 'wake_fd[2]', pipe setup under 'vm->n_hart > 1' (both ends
  non-blocking), 'signal_wake_fd', 'semu_close_wake_pipe()',
  'semu_cleanup_backends()', and 'semu_run()' poll/drain paths
  unconditional while keeping 'g_window' calls under
  'VIRTIOINPUT || VIRTIOGPU'.
- Drain all queued wake bytes in 'semu_run()', tick peripherals
  immediately, and check 'emu->stopped || signal_received' before
  resuming hart coroutines.
- Close 'wake_fd' with plain 'close()' and reset fields to -1 in
  'semu_close_wake_pipe()', relying on callers running on the emulator
  thread or after 'pthread_join()'.
- Add 'semu_block_term_signals()' in 'utils.h' so 'main.c' and
  'virtio-snd.c' share a single 'SIGINT'/'SIGTERM' mask definition.
  Keep non-thread-spawning 'semu_init()' steps interruptible, mask
  only around 'g_window.window_init()' and inside 'virtio_snd_init()'
  ('pthread_create()' / 'Pa_Initialize()'), 'Pa_OpenStream()', and
  'Pa_StartStream()', restore the caller's mask only while the emulator
  and window loops run, and re-block before closing the wake pipe.
- Expose 'virtio_snd_set_wake_fd()' and write one wake byte from the
  virtio-snd TX worker when completions or 'virtio_snd_set_fail()'
  raise 'InterruptStatus'.
- Add 'tests/test_vsnd_wake.c' (safe under '-DNDEBUG' and waiting on
  'InterruptStatus' before polling for a quiet wake pipe), 'PA_LDFLAGS'
  and 'unit-test' targets in 'Makefile', unconditional 'clean' removal
  of unit-test artifacts, direct 'make unit-test' steps in Linux/macOS
  CI workflows, and '.gitignore' entries for test artifacts.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 8 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/main.yml">

<violation number="1" location=".github/workflows/main.yml:165">
P3: These CI steps exercise only the SDL-enabled build, and `test_vsnd_wake` uses its own pipe rather than the emulator poll loop. Add a no-SDL regression check that exercises `semu_run()` so the configuration motivating this change is covered.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

View guided diff | Re-trigger cubic

shell: bash
timeout-minutes: 5
- name: unit test
run: make unit-test

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: These CI steps exercise only the SDL-enabled build, and test_vsnd_wake uses its own pipe rather than the emulator poll loop. Add a no-SDL regression check that exercises semu_run() so the configuration motivating this change is covered.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/main.yml, line 165:

<comment>These CI steps exercise only the SDL-enabled build, and `test_vsnd_wake` uses its own pipe rather than the emulator poll loop. Add a no-SDL regression check that exercises `semu_run()` so the configuration motivating this change is covered.</comment>

<file context>
@@ -161,6 +161,10 @@ jobs:
       shell: bash
       timeout-minutes: 5
+    - name: unit test
+      run: make unit-test
+      shell: bash
+      timeout-minutes: 5
</file context>

@Cuda-Chen
Cuda-Chen requested a review from jserv November 17, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants