Skip to content

Rust 1.100.0, GPUI Kit 0.9.0, faster Q8 decode at long contexts - #34

Merged
voidwest merged 4 commits into
mainfrom
chore/rust-1.100-gpui-kit-0.9
Nov 16, 2026
Merged

voidwest merged 4 commits into
mainfrom
chore/rust-1.100-gpui-kit-0.9

Conversation

@voidwest

Copy link
Copy Markdown
Owner

Summary

This branch moves Ember to Rust 1.100.0 and GPUI Kit 0.9.0, makes Q8 decode faster at long contexts, and removes a duplicate test run from the gate. Each part is a separate commit, so you can review or revert it alone.

Commit Change
e4c381e build Pins Rust 1.100.0 for GUI and development builds, and moves GPUI Kit from 0.6.6 to 0.9.0. Fixes nine new clippy findings. Makes the Python bindings inherit the workspace lints.
dd3966f perf Makes the Q8_0 fast decode path on aarch64 use the register-tiled attention kernel that prefill and planned decode already use.
3f41a92 ci Makes the audio step build the tests with --no-run, so that it no longer runs the 675 library tests a second time.

The headless MSRV stays at Rust 1.92. The Ember version stays at 1.0.0.

Toolchain and GPUI Kit (e4c381e)

  • No source changes needed: GPUI Kit 0.9.0 compiles with no source change. One code comment now names the 0.9.0 source, because its claim about Tab::ix was checked again in that version.
  • Scroll bounce: GPUI Kit 0.8 turned on scroll bounce by default on macOS. Lists, tables and selectors now bounce at the scroll limit.
  • New clippy findings: Clippy 1.100 finds more redundant_clone and needless_range_loop cases, and the workspace denies them. This commit fixes nine.
  • tests/k_parity.rs: A new assert keeps the test as strict as before, so a short reference run still fails.
  • Lockfile: Cargo.lock changes a lot, because GPUI Kit moves its own dependency tree. The headless MSRV build still passes.

Q8 decode speedup (dd3966f)

The tiled kernel does the same arithmetic in the same order for each element, so the output is bit-identical. Single-row reference decode keeps the generic body, so it stays an independent check of the kernel.

Paired decode A/B, Llama-3.2-1B Q8_0, M1 Pro, 8 alternating rounds:

Context (tokens) 4 threads 8 threads
32 +1.8% +0.7%
512 +7.2% +5.0%
1024 +12.5% +11.3%
2048 +21.9% +15.5%

Warning

Known cost, cause not found: Q6_K prefill is 6–8% lower, and Q4_K prefill is about 2% lower. This change is not on that route, and the Q6 kernel's instructions do not change. Loop alignment, fat LTO, PGO, allocator settings, and prompt size do not remove the drop. Q4_K and Q6_K decode do not change.

Tests (3f41a92)

  • Before: The full gate reported 1,764 test results, but only 1,067 distinct tests exist. The audio step reran all 675 library tests, and no unit test uses the audio code.
  • After: The step still compiles and links the audio code. No test was removed.

Verification

Check Result
scripts/ci_local.sh full on Rust 1.100.0, e4c381e tree Pass, including the Rust 1.92 MSRV check (cargo audit was skipped, because it is not installed locally)
scripts/ci_local.sh quick on the final tree Pass
Headless tests on the final tree 1,012 passed, 0 failed
Captures, outputs, and tokenization, Rust 1.99.1 against 1.100.0, three execution modes Byte-identical
Q8 decode and prefill speed, Rust 1.99.1 against 1.100.0 Within noise
Real-model GUI worker test, golden morphology check, GUI render (80 images) Pass, exact-semantic
Q8 speedup bit check (per head, real model data) and a 96-token real-text run Identical outputs
Q4_K/Q6_K parity (validate_k_parity.sh) 7/7 each on Rust 1.99.1 with this change; not rerun on Rust 1.100.0

Not verified

  • The Q4_K/Q6_K parity gate did not run on Rust 1.100.0, because the model drive was not mounted.
  • The speedup was measured on one machine, an M1 Pro, and only on Llama-3.2-1B. Qwen2.5-1.5B Q8_0 showed no decode gain in one 8-thread check at 1024 tokens of context, and the reason is not known.
  • No independent performance review has run.

🤖 Generated with Claude Code

voidwest and others added 4 commits November 16, 2026 08:56
Pin the GUI and development toolchain to Rust 1.100.0 and GPUI Kit to
0.9.0. The headless MSRV stays at Rust 1.92. The GPUI Kit update needs
no source change.

Clippy 1.100 finds more redundant clones and index loops, which the
workspace denies. Remove nine of them. The k_parity loop keeps its
strictness: a short reference run still fails the test.

The Python bindings now inherit the workspace lints, as Cargo 1.100
asks. The crate passes them with no finding.

Against Rust 1.99.1, captures, outputs and tokenization are
byte-identical in the reference, planned and planned-fused modes, and
Q8 decode and prefill speed is within noise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On aarch64, the Q8_0 fast decode path now uses the register-tiled
attention kernel that prefill and planned decode already use. The
kernel computes each element with the same arithmetic in the same
order, so the output is bit-identical. The reference path keeps the
generic body for single-row decode, as an independent check.

Paired decode A/B, Llama-3.2-1B Q8_0, M1 Pro, 4 and 8 threads:
+1.8% / +0.7% at 32 tokens of context, +7.2% / +5.0% at 512,
+12.5% / +11.3% at 1024, +21.9% / +15.5% at 2048. A per-head bit check
on real model data, a 96-token real-text run and the golden morphology
check all match.

Known cost: Q6_K prefill is 6-8% lower and Q4_K prefill about 2% lower,
although this branch is not on that route and the Q6 kernel's
instructions do not change. Loop alignment, fat LTO, PGO, allocator and
prompt-size changes do not remove it. The cause is not known.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The audio step ran all 675 library tests a second time, but no unit
test uses the audio code. Build and link the tests with the feature
instead. The step keeps its purpose: it still compiles and links the
audio code, which clippy alone does not link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cargo audit fails on quinn-udp 0.5.15. In that version, a remote
sender can make the receive loop spin on targets that read one datagram
per recvmsg call, Apple platforms among them (denial of service). The
fix is in 0.5.16.

The crate comes in through GPUI Kit's HTTP client (gpui-kit-assets ->
gpui-pre-reqwest -> quinn). main has the same version; the advisory
was published after main's last audit run. The update also changes
which windows-sys version five crates resolve to, for Windows only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@voidwest
voidwest merged commit 2d1b745 into main Nov 16, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant