Skip to content

perf(ffi): look up the running loop only while something on it waits - #581

Merged
kixelated merged 3 commits into
mainfrom
quest/ffi-python-loop-lookup
Nov 4, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/ffi-python-loop-lookup

Conversation

@kixelated

@kixelated kixelated commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

#564 resumes a same-thread wake without the socket pair by calling asyncio._get_running_loop() at the start of every call made on a thread whose loop has polled. Most of those calls wake nothing, yet each pays the lookup: a handle call inside a loop costs about 15 ns more than one outside it. moq's Python calls run inside a loop.

Approach

  • Each loop's Queue counts what may wake on it: its futures while parked, and its Wakers while alive.
  • The thread-local that said "a loop polled or took here" now holds that count, for the loop that last polled or took on the thread. A call looks up the running loop only while the count is nonzero, and the thread stops looking once a lookup finds no loop running, as before.
  • A future leaves the count when its wake, a new poll, or its close moves it out of PARKED. A close now marks the node woken, so a wake Rust makes after the close queues nothing.

Skipping the lookup is always safe: with no running loop noted, a wake takes the socket path, as a wake from another thread does. The lookup itself is unchanged, so it never trusts a stale loop, the hazard #564's PR body describes.

#564's tests still pass: no lost or double wake, same-thread vs cross-thread, a wake during cancel, a wake before the first await, and 64 concurrent receives woken from both threads. tests/api.py now also checks that a Waker's wake on the loop's thread writes no byte when nothing else waits. Before this, the check after the settle could not fail, since the dispatch drains the socket. Without the Waker count, the new check fails.

moq/bench/export.py now times a handle call inside the loop the woken reads ran on, once nothing waits on it. This is the state a moq app's sync calls run in.

Measurements

The Python leg of just ffi bench (moq/bench/export.py), on main and on this branch, interleaved with BoltFFI 0.31.1's Python harness from moq@0f986d9 in the same run, rotating order. Shared host with load average 4 to 5 on 32 threads, CPython 3.14.8. Medians of 9 rounds, min to max in parentheses:

main this PR BoltFFI
handle call, inside a loop 96.0 ns (92.7 to 97.6) 80.8 ns (79.4 to 83.1) sequence 142 ns (140 to 147)
handle call, outside a loop 81.6 ns (78.8 to 83.4) 79.1 ns (78.5 to 85.1)
frame written and read, never suspending 951 ns (924 to 973) 869 ns (849 to 912)
frame written and read after a wake 15,130 ns (14,502 to 15,707) 14,806 ns (14,415 to 15,852) yield_once 27,477 ns (26,876 to 28,770)

An in-loop call now costs within 2 ns of one outside a loop. The woken read keeps #564's direct path. An earlier 9-round run at load 7 to 11 measured before the in-loop timing was moved after a loop had polled. In that position main skipped the lookup too, so it showed no difference there. Its other rows matched these.

Impact

  • Public API: none.

Alternatives

  • Remembering the loop's thread instead of looking it up. Rejected in perf(ffi): resume a Python wake on the loop's own thread without the socket pair #564: it goes stale once a loop stops and later runs on another thread, and can deadlock a blocking call.
  • A process-wide count instead of one per loop. It is simpler, but any loop's parked futures, or a future leaked on a stopped loop, would make every in-loop call on every thread pay the lookup.
  • Calling _get_running_loop's C function directly. That would shave the vectorcall dispatch, but every in-loop call would still pay part of the cost, and it relies on CPython internals.

Follow-ups

None.

Review

  • Reviewer: quest review, Codex gpt-6.1-sol at medium effort, round 1 at head 26dbec8. No findings.

Checks run locally on 26dbec8: just ci (ffi lint and the 21 fixture tests driving C, C++, Kotlin, Python, Swift, Go, and Dart), and just ffi moq on every language (run on 416487f; later commits touch only the bench script and the quest tree). Not run: just ffi android, which needs a device.

🤖 Generated with Claude Code

kixelated and others added 3 commits November 4, 2026 02:16
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 4, 2026 10:34
@pullfrog

pullfrog Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

This run was cancelled before it finished.

Pullfrog  | View workflow run | via Pullfrog

@coderabbitai

coderabbitai Bot commented Nov 4, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 24 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 96f84071-6889-4a3a-a5b8-87659da527a5
Commits

Reviewing files that changed from the base of the PR and between 5ad8ee2 and 26dbec8.

Files selected for processing: 5
  • ffi/moq/bench/export.py
  • ffi/runtime/python.rs
  • ffi/tests/api.py
  • quest/m3/README.md
  • quest/m3/ffi-python-loop-lookup.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated
kixelated added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 7f976d5 Nov 4, 2026
7 of 8 checks passed
@kixelated
kixelated deleted the quest/ffi-python-loop-lookup branch November 4, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant