Repository navigation
perf: borrow the executing function from its module instance - #74
matthargett wants to merge 4 commits into
Conversation
3e11936 to
8370a09
Compare
8370a09 to
2bdebab
Compare
|
Interesting change! In my preliminary tests I don't see reliable gains / some regressions so I'm a bit hesitant to merge this as-is due to the large amount of changes / complication of the execution machinery. |
|
This was meant to stack on some other PRs in my fork, but I can pull it in this one piece so the individual PR is more of a slam-dunk win on both of our sets of benchmarks. Let me know what you think, and if future PRs should be a little less incremental so the ultimate win is more obvious. |
Entering a function reserved and zero-filled all three value-stack lanes even when the function never touches the 64- or 128-bit lane, which is the common case. Skip a lane whose locals and operand-stack depth are both zero. A single-result return now moves the result down and truncates once, instead of popping, truncating and pushing it back.
Every call and return between two functions cloned the callee's `Shared<WasmFunction>` and dropped the previous one, two refcount updates each way, and looked the function up in the store. A module instance now keeps its own functions, which the store allocates contiguously. The interpreter holds the instance for the whole run, so the executor borrows the executing function and its module from it: a call or return within the instance switches a reference, and a direct call to one of the module's own functions skips the address table, the host check and the owner check. Execution that continues in another instance's function (a call through an import, table or reference, a return, or an exception unwinding into it) ends the run, and `InterpreterRuntime` resumes that frame with an executor for its instance. Fuel and time budgets carry over, so a run suspends at the same points as before.
The match loop spilled the executor's function and reloaded it, then the instruction slice's pointer and length, on every step. The executing function is borrowed from the module instance, not from the executor, so the loop can hold its instructions in a local and reload them only when a step switches functions.
The instance kept its own copy of the module's function list for the executor to borrow from, which cost an allocation and a reference count per function on every instantiation, and as many again on drop. Hold the module instead, which costs one reference count per instance.
948e9ea to
97fc951
Compare
|
@explodingcamera I think this is ready for another review/benchmark pass, if you're up for it. I pulled in a few of the incremental changes so it's a clearer win across CPU architectures without using any unstable or bleeding-edge Rust features. I'd like to get some of this core performance stuff squared away before I start prototyping the out-of-tree WASIp3 features like async/modules |
Stack, in merge order: #74, this PR → #81, which is #79's six commits on top of #74. #79 is the same six commits on
nextalone. All three share one table layout for side-by-side comparison.Every call or return between two functions cloned the callee's
Shared<WasmFunction>and dropped the previous one (two atomic refcount updates each way) and looked the function up in the store.A module instance now keeps a reference to its module, whose functions the store allocates contiguously (helping with locality for cache and prefetch).
InterpreterRuntimeholds the instance for the whole run and the executor borrows the executing function from it, so a call or return inside the instance switches a reference (I think this is what makes cache evict more often than I would expect). A direct call to one of the module's own functions also skips the address table, the host check and the owner check. Execution that moves into another instance's function (through an import, a table, a function reference, a return or an unwinding exception) ends the run and resumes with an executor for that instance. Fuel and time budgets carry over, so budgeted runs should suspend at the same points as before.Entering a function also skips the value-stack lanes it never uses (most functions only touch the 32-bit lane), and a single-result return moves its result down once instead of popping and pushing it.
Because the executing function is borrowed from the instance rather than owned by the executor, the default (non-tail-call) dispatch loop keeps the function's instructions in a local and reloads them only when a call or return switches functions, instead of reloading the function and the slice's pointer and length on every step.
tests/cross_instance_calls.rscovers calls, tail calls, table calls, callbacks and exceptions across an instance boundary in both directions, and fuel- and time-budgeted runs that cross it.A loop calling a one-line function drops from 410 to 319 instructions per iteration with the default dispatch, and from 441 to 349 with
nightly-tail-calls.Results
CoreMark is −4.9% averaged over the five chips. Call-heavy code gains most (the EH parser). With the default dispatch, CoreMark is −12.7% on the A13's and −7.0% on the A14's performance cores.
Change in cycles per call against
nextwithnightly-tail-calls, median of five interleaved runs per build. i7-8650U: a Surface Book 2 (Kaby Lake, 2017) on Windows x64, pinned to one core, core cycles counted with VTune. A10X: Apple TV 4K (1st gen). A12, A13: iPhone XS Max and iPhone SE, performance cores. M4: performance cores. argon2id, compress and decompress arecargo bench-suite's fixtures, the EH parser and GC binary trees are WasmBench rows, and CoreMark runs with a clock that advances a fixed step per read, so every build runs the same iterations.