Skip to content

Avx2 775 trim - #782

Merged
TomWambsgans merged 7 commits into
mainfrom
avx2-775-trim
Nov 18, 2026
Merged

TomWambsgans merged 7 commits into
mainfrom
avx2-775-trim

Conversation

@TomWambsgans

@TomWambsgans TomWambsgans commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

experimental: AVX2 speedups from #775, trimmed

7 of the 18 commits of #775 by @kevaundray, each simplified, with tests added and re-measured: +1,172/-551 in 14 files, against #775's +2,943/-875 in 31. Rebased on main. The proof bytes are unchanged, the verifier is untouched, and the stable prover changes only through the shared grind, which returns the same nonces.

commit what
keccak256: search grinding nonces with first_nonce the grind permutes its candidates in lanes and reads the one lane it tests, rather than transposing whole states through keccak_f_many; same smallest nonce
primitives: AVX2 arms for bit_transpose_64bytes where AVX-512 VBMI with GFNI is missing: byte gather by unpacks, then one GFNI affine map, or three masked-swap rounds without GFNI
keccak_gkr: table rows by byte maps on AVX2 GFNI affine or vpshufb nibble maps on every AVX2 target; the x86 fold_groups arms it leaves unused go
lean_vm_core: bus GKR vertical kernels on AVX2+VPCLMULQDQ the eight-lane kernels as two x4 halves; x8 and f128x8 share their method names
keccak_gkr: GFNI product sums on values at six points sums on values at six points of GF(2^8), turned back into coefficients once a layer
keccak_gkr: whole-position parts on AVX2 a digit-bucket part is a whole block position; it runs only on AVX2 without GFNI, and is a simplification elsewhere
keccak_gkr: fold digit bucket sums by nibbles 32 sums per digit on the stack instead of 256 halved eight times

Left out of #775 (11 commits)

  • 9d7e41f76 (rows padded to four lanes, +175/-118) is slower at 8 threads on Zen 4 and Arrow Lake: rows that aren't a multiple of 8 lanes lose the encode's streaming stores. 08209e93a (L0 leaves hashed from their data lanes, +91/-51) gains nothing at 8 or 4 threads without it.
  • 8a3fe5188 (level-0 lane fold on AVX2, +536/-12) gains at most 0.4% on Arrow Lake at 8 and 4 threads.
  • 8f7705d64 (BitFold row transpose, +202/-126): on GFNI CPUs it only touches the ring-switch lift, and its own A/B shows under 1% single-threaded. Not measured here.
  • Seven commits for CPUs without GFNI or VPCLMULQDQ (Haswell-era, Zen 3), +860/-187:
    • the PCLMULQDQ-only bus GKR, level-0 and NTT paths (c52813010, 836ec09f0, 918df52d1);
    • skip_clmul (c1fc8efc3, 491743d4a);
    • the row products (ae7b9f5d5, ff71b8aa1).
  • first_nonce is a 15-line search over the unchanged permute, rather than experimental: AVX2 single-thread speedups (taken from riscv branch) #775's permute refactor. The two are equal end to end on Arrow Lake; on Zen 4, experimental: AVX2 single-thread speedups (taken from riscv branch) #775's version is about 1.2% faster (1400 leanXMSS at 8 and 4 threads, interleaved).

Results

Proving time against main, --repeat 3, two interleaved rounds:

  • aggregate: aggregate --xmss 1400 --log-inv-rate 2
  • recursion: recursion --n 4 --xmss-per-leaf 1400 --leaf-log-inv-rate 2 --log-inv-rate 1
aggregate, 8 threads aggregate, 4 threads recursion, 8 threads recursion, 4 threads
Arrow Lake (AVX2, GFNI, VPCLMULQDQ) -4.8% -5.7% -4.3% -5.4%
Zen 4, AVX-512 disabled -8.0% -8.7% -5.3% -7.8%
Zen 4 (AVX-512) -2.4% -3.5% -2.9% -1.9%
Zen 5 (AVX-512) -5.0% -4.7% -3.2% -4.3%
Apple M4 Max* -2.5% -2.1% not run not run

* On the previous revision of this branch. The only change since on Apple's paths is one memset fewer per digit in the bucket sums.

Nothing measured is slower. Not measured: the sphincs and blob aggregations, and CPUs without GFNI (Haswell, Zen 3), whose paths commits 2, 3, 6 and 7 change.

Checks

  • identical_proofs at 1 and 3 threads on 9 x86 builds: native Zen 5, znver4, znver3, alderlake, skylake-avx512, haswell, x86-64-v3+gfni, x86-64-v4+vpclmulqdq, and znver4 without AVX-512.
  • Every commit passes clippy -D warnings and its tests on its own, on native, haswell, znver3 and znver4 without AVX-512.
  • At the head, clippy is clean on aarch64 (neoverse-n2, apple-m1), and fmt and rustdoc are clean.
  • Each new kernel is tested against a portable or scalar reference.
    • CI runs tests on haswell only, so the GFNI, VPCLMULQDQ and AVX-512 arms are exercised only by the native and per-target runs above.
  • Eight adversarial reviews found no soundness, proof-byte or memory-safety issue.
    • Two rounds of mutation testing (90 mutants) found 6 test gaps, all now closed.
    • The grind returns main's nonce in over 8,500 differential checks on four builds.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Benchmarks: base against this PR, on one runner

1902ed0 merged into its base, against the base 9d37339: each benchmark run in turns on one runner, base and PR 5 times each. A size is shown when it changed; a time or a peak memory when it moved the same way in every round and its median by at least 1% (the change is the median ratio, with the rounds' range).

Threads shows the named configuration; default means no thread count is encoded in the benchmark ID.

Keccak GKR

benchmark threads runner measure base this PR change
keccak-hash-batch-262144 default arm64 Proving time 2.71 s 2.68 s -1.41% (-2.02% to -0.85%)
keccak-hash-batch-262144 default x86-64 Proving time 3.83 s 3.57 s -6.29% (-7.48% to -5.77%)
Every result

Keccak GKR

benchmark threads runner CPU measure base this PR change
keccak-hash-batch-262144 default arm64 Neoverse-N2 Proving time 2.71 s 2.68 s -1.41% (-2.02% to -0.85%)
keccak-hash-batch-262144 default x86-64 AMD EPYC 9V45 96-Core Processor Proving time 3.83 s 3.57 s -6.29% (-7.48% to -5.77%)

TomWambsgans and others added 7 commits November 18, 2026 07:54
From #775: the grind permutes its candidates in lanes and reads the one lane it tests, rather than transposing whole states through keccak_f_many; same smallest nonce. 1400 leanXMSS at 8 / 4 threads: Zen 4 -1.3% / -1.2%, Arrow Lake -0.7% / -1.1%.

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From #775: where AVX-512 VBMI with GFNI is missing, gather the byte columns with unpacks and a shuffle, then bit-transpose them by one GFNI affine map, or by three masked-swap rounds without GFNI. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.7% / -1.2%.

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From #775: the table pass builds its rows by byte maps, GFNI affine or vpshufb nibble lookups, on every AVX2 target rather than only on AVX-512 with GFNI; the x86 fold_groups arms it leaves unused go. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.8% / -1.4%.

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From #775: products_into, the round message and the fused fold run eight lanes at a time with VPCLMULQDQ on AVX2 too, as two x4 halves, with x8 and f128x8 sharing their method names. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.5% / -2.0%.

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From #775: the product sums are taken on each product's values at six points of GF(2^8), by byte maps and GF2P8MULB, rather than on its carry-less coefficients, which the inverse evaluation map recovers once a layer. 1400 leanXMSS at 8 / 4 threads: Zen 4 -2.4% / -3.2%, Arrow Lake -2.1% / -3.8%.

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From #775: a digit-bucket part is a whole block position on AVX2 too, six registers, so each bucket's loop runs once per digit rather than twice and the half-position arm goes. Only AVX2 without GFNI (Zen 3, Haswell) runs the buckets; elsewhere it is a simplification.

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e made

From #775: each bucket's sum adds into the sums of its value's two nibbles as it is made, rather than into 256 sums halved eight times, so a digit's scratch is 32 parts on the stack. It pays where the buckets are the product path; 1400 leanXMSS on an Apple M4 Max at 8 / 4 threads: -0.8% / -0.2%.

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TomWambsgans
TomWambsgans merged commit f37c5e5 into main Nov 18, 2026
23 of 24 checks passed
@TomWambsgans
TomWambsgans deleted the avx2-775-trim branch November 18, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant