Repository navigation
experimental: AVX2 single-thread speedups (taken from riscv branch) - #775
Closed
kevaundray wants to merge 18 commits into
Closed
kevaundray wants to merge 18 commits into
kevaundray wants to merge 18 commits into
Conversation
kevaundray
force-pushed
the
kw/avx2-single-thread
branch
from
November 17, 2026 22:19
ddf5401 to
a3361b4
Compare
Benchmarks: base against this PR, on one runner08209e9 merged into its base, against the base 6a828ec: each benchmark run in turns on one runner, base and PR 5 times each. A size is shown when it changed; a time or a peak memory when it moved the same way in every round and its median by at least 1% (the change is the median ratio, with the rounds' range). Threads shows the named configuration; Keccak GKR
Every resultKeccak GKR
|
Without AVX-512 VBMI, gather each byte column with two unpack rounds, a qword permute and a byte shuffle, then transpose the bits of four columns per YMM: with GFNI one affine map (rows reversed by the gather), without it the three masked-swap rounds. The consumers are the PCS level-0 projection and the Keccak openings. Throwaway microbench, one 64-byte block, single thread on an EPYC 9454P (Zen 4) core: portable 7.43 ns, AVX2 (haswell) 4.48 ns, AVX2+GFNI 1.33 ns.
On AVX2 without the AVX-512 GFNI kernel, the row-major fold transposes its
rows into byte columns (16x16 in-lane byte transposes, rows of 8 bytes paired
across the two halves) and runs the column kernel, instead of one 256-entry
subset-sum lookup per byte, so the byte tables are no longer built there.
The nibble kernel (AVX2 without GFNI) stores each 16-entry table once and
broadcasts it to both lanes, which halves its table footprint, and writes its
output through the same 16x16 byte transpose as the GFNI-256 kernel instead of
a word buffer. Rows of 128 bytes, whose tables outgrow L1, take four output
bytes of both 32-row halves at a time so each table is read once per block.
Throwaway microbench, ns per 64-row block, single thread, EPYC 9454P core 40,
min of runs:
fold_columns, nibble (haswell) C=16 256 -> 205, C=32 634 -> 386,
C=64 1502 -> 899, C=128 2775 -> 2024
fold_block, nibble (haswell) C=16 349 -> 228, C=32 1229 -> 437
fold_block, GFNI-256 C=16 327 -> 130, C=32 2619 -> 245
aggregate --xmss 100, LEANVM_NUM_THREADS=1, core 40, both commits against
upstream main, min over six alternating runs, ms (base -> new):
vp: Prove 979 -> 971, Project 59.4 -> 54.4, Openings 7.6 -> 5.4,
Lift 1.5 -> 1.2
gfni: Prove 931 -> 923, Project 52.1 -> 49.0, Openings 5.6 -> 3.9,
Lift 1.5 -> 1.0
hsw: Prove 1330 -> 1330, Project 82.1 -> 77.0, Openings 7.5 -> 5.5,
Lift 2.0 -> 1.7
The table pass's rows come from 32 x 16 byte maps per register of positions on every AVX2 target, not only AVX-512 with GFNI: one 256-bit affine product per map with GFNI, two vpshufb nibble lookups without. The 6 * BITS scratch and its reduction pass remain only off x86. The module is renamed table_gfni -> table_maps; the AVX-512 arm is unchanged but for sharing the unpack rounds. Single thread, aggregate --xmss 100, last pass, Tables span, same core (16), base then candidate: - gfni (x86-64-v3 + gfni): 49.1 -> 22.6, and 49.1 -> 23.3 on a repeat - vp (x86-64-v3 + vpclmulqdq): 49.1 -> 33.7 - hsw (haswell): 81.9 -> 43.9 - native (Zen 4, AVX-512 arm): 19.5 -> 19.6 Rows alone, microbenchmark per 1600-position block: 11.3 us with GFNI, 22.3 us with nibble lookups.
The proof-of-work grind permuted 64 whole states through keccak_f_many, transposing each into lanes and back, and kept 25 output lanes to read one. `first_nonce` builds the states in lanes from the template, takes the nonce lane as a constant so LLVM hoists the first round's work on the untouched lanes out of the loop, and computes only the output lane in the last round. It returns the same smallest passing nonce, so proofs are unchanged; both provers grind through it. Single thread, AMD EPYC 9454P core 12, same core, back to back. Grind kernel, ns per permutation, keccak_f_many on 64 states before, first_nonce after: hsw 97.9 / 98.1 -> 85.6 / 81.4. aggregate --xmss 100, last pass, ms, base -> this, two runs each: - hsw: prove 1344 / 1355 -> 1320 / 1323; Bus grind 34.1 -> 27.5; PCS open 286 -> 272; Final level 12.9 -> 10.8 - vp: prove 983 / 992 -> 958 / 960; Bus grind 34.0 -> 27.5; PCS open 238 -> 223; Final level 13.1 -> 10.8 - gfni: prove 942 / 939 -> 915 / 915 - native: prove 551 / 552 -> 522 / 529; Bus grind 22.5 -> 15.7; PCS open 151 -> 135; Final level 8.6 -> 5.9 The prove totals include the next commit's AVX2 absorb, which moves only Commit.
kevaundray
force-pushed
the
kw/avx2-single-thread
branch
from
November 17, 2026 22:31
a3361b4 to
4cd49cc
Compare
The transposed eight-lane kernels of the bus GKR (`products_into`, the round message and the fused fold) ran only with AVX-512. They now also run with AVX2 and VPCLMULQDQ, on `f128x8`'s AVX2 `F128x8`: two `x4` halves of four lanes. The message runs one half after the other on `x4::F128x4`, so its live set fits sixteen YMM registers; interleaving the halves, or looping over them, measured slower. `x4` gains `Sum4` (a lane sum of unreduced products, Karatsuba's terms summed before one fold), `F128x4::mul_sum` and `Mul`; `f128x8`'s AVX2 arm gains `load_runs`, `store_interleaved` and `halves`. The AVX-512 path is unchanged. Single thread, core 32 of an EPYC 9454P, `-C target-cpu=x86-64-v3 +pclmulqdq,+vpclmulqdq` (vp), upstream main vs this, back to back: - a 2^18-row layer of three trees (min of 8): first message 10.36 -> 6.20 ms, first fused fold 11.03 -> 7.18, second 5.24 -> 4.84, the rest 5.31 -> 3.80; `products_into` over 2^20 elements 1.75 -> 1.10 ms. - `aggregate --xmss 100`, three pairs: Bus GKR 61.3 / 61.7 / 61.1 -> 40.4 / 40.0 / 41.4 ms; Bus leaves 29.9 -> 27.8 ms (the four- and sixteen-tuple products).
The radix-8 and radix-4 register kernels now also build where AVX2 has only 128-bit PCLMULQDQ (the haswell tier, and AVX-512 without VPCLMULQDQ), with one four-lane vector per row in flight instead of two. Before, that tier ran each butterfly through memory, a row pair at a time. The product is shared with the row-pair kernel as `mul_xor_ymm`; the VPCLMULQDQ and AVX-512 arms are unchanged, and the transposed butterfly keeps its old arms. Single thread, `aggregate --xmss 100`, AMD EPYC 9454P core 28, same-core A/B against upstream main, three alternating passes each, ms (base / new): hsw (-C target-cpu=haswell): proving time 1346-1349 / 1318-1319 Commit / NTT 208 / 195 Commit L1 57.3 / 53.8 Commit L2 25.6 / 24.5 Commit L3 11.2 / 10.9 Open L0 18.3 / 14.4-14.8 Open L1 6.5 / 5.3 NTT alone (no hashing, jemalloc), hsw: base encode (2^18 x 48, 44 lanes) 86 -> 73, extension encodes 2^17/2^16/2^15 x 32 words 21.0/7.5/2.15 -> 17.5/6.3/1.87; kernel 2.86-3.0 -> 2.44-2.49 ns per four-lane butterfly, against 2.43 for the same butterflies kept in registers. vp and gfni compile to the same kernels: proving time 982/982-984 and 937-938/937, Commit / NTT 173/173-174 and 174/174.
Without GFNI the skip pass embedded and extended every position, then took per slot and position a product of two embeddings and two products of it with the weight's words: fifteen carry-less products a slot. Those products bound it on these tiers. `skip_clmul` takes a row's sum as `e[2] A + e[4] B + e[0] C`, the cofactors `A = w[0] e[1] + w[1] e[3]`, `B = w[2] e[3] + w[3] e[0]` and `C = w[4] e[1]` multiplied out on the 32 packs (ten products a pack) and extended like them. A cofactor has three words, so a slot takes nine products, each of an embedding and a word: 1490 a row instead of 1950. The extension's binary points of new vectors are stored, and its low mixed points summed as the products read them. With VPCLMULQDQ a register's two lanes are the two groups at the same output; with PCLMULQDQ alone the groups take a pass each. NEON and the portable form keep the row kernel. Measured on one core of an EPYC 9454P (Zen 4), single thread, `aggregate --xmss 100`, last of three passes, ms, base and new run back to back twice on the same core: vp tier (x86-64-v3 + VPCLMULQDQ): span base new base2 new2 Prove 990 941 977 940 Keccak reduction 397 348 384 348 Keccak / Layer / Skip 174.4 139.1 174.2 138.8 hsw tier (haswell): span base new base2 new2 Prove 1330 1280 1330 1270 Keccak reduction 611 553 612 551 Keccak / Layer / Skip 318.5 258.1 318.9 258.0 `SkipBlock::run` on one random block (microbenchmark, best of 4 runs): vp 194 us before, 152 us after; hsw 356 us before, 292 us after. Tried and rejected on the vp tier (us per block, same core): - The extension stored to a slot array, field by field, as 128-bit lanes (the products then reading two slots a register): 262 against 194. - The extension stored to an output array a register at a time: 216 against 194. Storing only the new vectors and summing the low points as the products read them is the kept 152. - Residues without GFNI (nibble lookups for the affine maps, shift and add for the byte products): not built; a byte product costs about sixty AVX2 operations against one, and the products dominate.
The GFNI outer products staged each lane byte's products from CLMUL, a word per pack, then transposed them to column bytes. A product of two embeddings has degree below 48, so its values at six points of GF(2^8) with distinct minimal polynomials fix it: the staging now takes the embeddings' values by affine byte maps of the witness bytes, the products' values by GF2P8MULB, and one 8x32 byte transpose per point. The sums run over the value bits, and the inverse of the evaluation map sends them back to coefficients once a layer. Single thread, `aggregate --xmss 100`, Zen 4 core 24, same core back to back, last pass, ms: span gfni base gfni native base native Product 89.0 49.6 50.7 39.3 Keccak 399.0 352.0 160.0 144.0 Per block (pack_products microbench, n = 4096, gfni): 87.4 -> 51.2 us; staging 49.7 -> 13.2 us, accumulation unchanged at about 30 us.
The digit buckets summed each block position's products in two parts of
three AVX2 registers, so each bucket's loop ran twice per digit. A part
of the whole position, six registers, runs it once.
Single thread, `aggregate --xmss 100`, Zen 4 core 24, two runs each,
same core back to back, Product span, ms:
vp: base 115.7 / 115.8 this 110.3 / 109.7
hsw: with the next commit's PCLMULQDQ row products only 135.6 / 135.5,
with this commit as well 127.7 / 127.9
The digit buckets' products are `ones * (a + c) + b * c`, two carry-less
products a pack. `ones * a` is a linear map of the pack, so with
PCLMULQDQ alone (a product a lane) a row now takes it once a position by
nibble lookups, an 8x32 byte transpose to a word a pack, and the products
of the row XOR two of them. That halves the carry-less products. With
VPCLMULQDQ the lookups cost more than they save (row products per block,
microbenchmark: 62.3 us with them, 54.4 us without), so that tier keeps
the products. `kernels::row_products` hands the five products of a row to
a closure, so the other arms keep their shape.
Single thread, `aggregate --xmss 100`, Zen 4 core 24, two runs each,
same core back to back, Product span, ms:
hsw: base 148.9 / 148.9 lookups 135.6 / 135.5
with the previous commit as well 127.7 / 127.9
vp: unchanged code path
The digit buckets' payloads hold a position's products 48 bits each. On AVX2 every product went through memory as a word, then four at a time through scalar shifts into three words, then into the payload. Now a lane's two products come out of the carry-less products already packed (the second shifted up six bytes), and each lane's twelve bytes are stored straight into the payload, twelve bytes apart, the last lane's without the four bytes past it (`kernels::row_packed`). With PCLMULQDQ alone the nibble lookups of `ones * a` produce that packed layout too. A part is a whole position on x86, so nothing else is copied; the other arms keep products and `Part::pack`. Single thread, `aggregate --xmss 100`, Zen 4 core 24, same core back to back, Product span, ms: vp: before 109.9 / 110.0 after 101.8 / 101.5 hsw: before 129.5 / 128.4 / 135.4 after 119.7 / 119.3 / 119.5 pack_products microbench (n = 4096) per block: vp 122.7 -> 114.2 us, hsw 143.3 -> 130.5 us; the row kernel alone per block: vp 49.7 -> 40.5 us, hsw 82.0 -> 63.8 us.
…e made Each digit's 256 bucket sums were stored, then halved eight times into the digit's rows, which with whole-position parts on AVX2 spills past L1. Now each bucket's sum, made in registers, adds into the sum of its value's high nibble (kept in registers across the 16 values that share it) and into that of its low nibble (16 parts), and the two sets of 16 make the rows of the digit's high and low bits. Shared by every arm that uses the digit buckets. Single thread, `aggregate --xmss 100`, Zen 4 core 24, two runs each, same core back to back, Product span, ms: vp: before 101.2 / 103.1 after 93.2 / 93.5 hsw: before 119.3 / 119.5 after 108.8 / 109.1 pack_products microbench (n = 4096) per block: vp 113.8 -> 104.8 us, hsw 131.4 / 134.2 -> 121.9 / 121.4 us.
… AVX2
With VPCLMULQDQ and no AVX-512, the level-0 rounds wrote each folded lane
(`P = G + alpha F + beta X F`) and multiplied it by `X` through the plane form
of `F128x8`, which transposes eight elements to two coefficient planes on load
and back on store. `primitives::field::x4` gains `F128x2` and `F64x2`: two
elements a register in memory order, with Karatsuba (`Prod2`) and mixed
(`Mixed2`) products that reduce straight back to that order. `Folded::block`
and `mul_x_block` run on them, and an `E` block's `X` product multiplies only
its even columns, the odd columns' constant being one; a `K` block's gathers
its four even sums into one register.
Single thread, `aggregate --xmss 100`, last pass, same core, back to back, ms:
vp: base / new
PCS open 237 238 / 230
Fold round L0 / Project 59.3 59.8 / 51.9
gfni:
PCS open 233 / 222 224
Fold round L0 / Project 52.8 / 44.2 44.2
hsw and AVX-512 compile the unchanged arms.
…CLMULQDQ
`primitives::field::x4` now builds with AVX2 and PCLMULQDQ, a 256-bit
carry-less product being two 128-bit ones where VPCLMULQDQ is missing
(`x4::clmul`). The level-0 lane fold (`Folded::block`) and the `X` products
(`mul_x_block`) take their `F128x2` arms on the haswell tier too, in
place of the portable eight-lane path. The leaf builder and `f128x8` keep
their VPCLMULQDQ gates.
Single thread, `aggregate --xmss 100`, last pass, same core, back to back,
ms (base is upstream main):
hsw: base / new
PCS open 289 288 / 279 282
Fold round L0 / Project 82.7 82.2 / 74.0 75.1 (78.5 78.7 before)
…minimal polynomial Only a block's skip sums at `s` matter: the prover evaluates them there. So `skip_clmul` reduces each cofactor modulo the minimal polynomial of `s` (Barrett's reduction from `PowerField`'s modulus, three carry-less products) from three words to two, and a slot takes six products instead of nine: per row 32 * 19 + 130 * 6 = 1388 products instead of 1490. `SkipBlock::new` takes the `PowerField`; the other arms keep exact sums, and the kernel test compares sums modulo the minimal polynomial. `SkipBlock::run` on one random block (microbenchmark, best of 6 rounds, one core of an EPYC 9454P, against kw/avx2-single-thread at e557ff7): vp 154.9 us before, 145.0 us after; hsw 319.4 us before, 301.9 us after. On an earlier tip of the branch: vp 152.3 to 142.8, hsw 291.9 to 275.7. `aggregate --xmss 100`, single thread, last pass, Skip span in ms, ref and new back to back twice on core 8 (host load average about 50): hsw: 262.7, 267.9 before; 255.1, 250.5 after. vp: unusable, the reference alone swung from 142 to 272.
…LMULQDQ With `x4` building on AVX2 and PCLMULQDQ alone (a 256-bit carry-less product as two 128-bit ones), `f128x8`'s AVX2 arm, the bus GKR's vertical kernels (`products_into`, the round message, the fused fold) and the lane-wise leaf fill now run on the haswell tier too, in place of the portable eight-lane types and the scalar row loops. They are gated on AVX2 and PCLMULQDQ, with VPCLMULQDQ where AVX-512 is present (the `x8` path), so an AVX-512 machine without VPCLMULQDQ keeps the portable path. Single thread, core 32 of an EPYC 9454P, `aggregate --xmss 100`, `target-cpu=haswell`, the previous commit vs this, two quiet pairs back to back, ms: Bus leaves 49.5 49.3 -> 35.5 34.5 Bus GKR 72.8 72.4 -> 57.9 57.7 Prove bus 156 156 -> 127 126 Prove constraints 22.0 21.8 -> 19.8 20.3 proving time 1138 1134 -> 1135 (noisy) 1103 The other stages are unchanged. vp, gfni and native compile to the same code as before.
kevaundray
force-pushed
the
kw/avx2-single-thread
branch
from
November 17, 2026 22:46
4cd49cc to
c528130
Compare
The base commit encoded n_lanes data lanes as next_multiple_of(8), so the NTT only shrank when the committed cells crossed an eighth of the stack. Rows are now padded to `ntt::ROW_LANES`: four on x86 with AVX2 (AVX-512 included), eight on NEON and the portable build, where the kernels' tail goes word by word. - The lane-major first gathered pass and the opening's rebuild of it move whole octets of lanes and then a quad where the row ends in one (`stage::lane_groups`), with a four-line stage tile. - The AVX2 register kernels take the count of vectors that hold lanes, so a last chunk of one vector no longer multiplies a vector of zeros with VPCLMULQDQ (two vectors in flight). - Leaf images are unchanged: the padding is the image's leading zeros, and only the data lanes ride the proof. A narrower row also absorbs more of the leading zeros as the shared zero-prefix state, so a leaf can take one Keccak permutation fewer. - The base encode's span reports the data lanes beside the row width. Single thread, EPYC 9454P core 44, `aggregate --xmss N`, three passes, A/B back to back against origin/kw/avx2-single-thread (ms): | tier | N | data / row lanes | Commit/NTT | Open L0 | Commit L1..L3 NTT | total | | gfni | 200 | 42: 48 -> 44 | 356 -> 323 | 13.5 -> 13.0 | 168 -> 169 | 1276 -> 1244 | | gfni | 1300 | 43: 48 -> 44 | 1470 -> 1340 | 16.5 -> 15.7 | 693 -> 695 | 5811 -> 5666 | | gfni | 1440 | 38: 40 -> 40 | 2480 -> 2490 | 370 -> 374 | 1412 -> 1424 | 8744 -> 8761 | | hsw | 200 | 42: 48 -> 44 | 401 -> 364 | 18.7 -> 17.6 | 186 -> 186 | 2115 -> 2000 | | hsw | 1300 | 43: 48 -> 44 | 1670 -> 1510 | 23.7 -> 21.7 | 775 -> 775 | 10321 -> 10179 | | hsw | 1440 | 38: 40 -> 40 | 2840 -> 2830 | 512 -> 510 | 1588 -> 1592 | 14297 -> 14370 | | native (AVX-512) | 200 | 42: 48 -> 44 | 227 -> 220 | 9.6 -> 9.6 | 102 -> 103 | 932 -> 928 | | native (AVX-512) | 1300 | 43: 48 -> 44 | 941 -> 913 | 11.9 -> 12.1 | 425 -> 424 | 4422 -> 4382 | | native (AVX-512) | 1440 | 38: 40 -> 40 | 1640 -> 1640 | 301 -> 301 | 866 -> 867 | 6555 -> 6568 | At 1440 the row is 40 lanes either way, the no-regression shape; its differences are within the runs' spread (hsw ref 200 had 11% spread). On AVX-512 the same tree with eight-lane rows measured 225 ms at 200 and 942 ms at 1300, so four lanes is kept there too. The extension commits encode 16 lanes of data each and need no padding.
A base codeword row leads with zero lanes up to the NTT's lane quantum, and the leaf hasher counted only the leaf's own padding as the shared zero-prefix state. It now takes the data lanes (`MerkleBuilder::new` and `CutTree::open` gain `data_words`), so the whole zero prefix, the row's padding included, is one sponge state, and a leaf takes one Keccak permutation fewer whenever the padding crosses a rate block. The hashed bytes are unchanged; the extension commits pass their full row. Single thread, EPYC 9454P core 44, `aggregate --xmss 150` (46 data lanes in rows of 48, leaf images 512 -> 376 bytes), two A/B rounds back to back against the previous commit (Commit/NTT is the last pass's span; total the three-pass mean): | tier | Commit/NTT ms | total s | | gfni | 176 -> 152 | 0.849, 0.835 -> 0.813, 0.801 | | hsw | 196 -> 174 | 1.376, 1.371 -> 1.360, 1.352 | | native (AVX-512) | 112 -> 98.4 | 0.609, 0.611 -> 0.596, 0.594 | At `--xmss 200` (42 data lanes in rows of 44) both hash the same image; the totals matched within spread (gfni 1.253 both, hsw 2.091 -> 2.026 at 6% spread, native 0.935 -> 0.942 at 6% spread).
Collaborator
|
Merged
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
From #775: the grind permutes its candidates in lanes and reads the one lane it tests, rather than transposing whole states through keccak_f_many; same smallest nonce. 1400 leanXMSS at 8 / 4 threads: Zen 4 -1.3% / -1.2%, Arrow Lake -0.7% / -1.1%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
From #775: where AVX-512 VBMI with GFNI is missing, gather the byte columns with unpacks and a shuffle, then bit-transpose them by one GFNI affine map, or by three masked-swap rounds without GFNI. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.7% / -1.2%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
From #775: the table pass builds its rows by byte maps, GFNI affine or vpshufb nibble lookups, on every AVX2 target rather than only on AVX-512 with GFNI; the x86 fold_groups arms it leaves unused go. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.8% / -1.4%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
From #775: products_into, the round message and the fused fold run eight lanes at a time with VPCLMULQDQ on AVX2 too, as two x4 halves, with x8 and f128x8 sharing their method names. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.5% / -2.0%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
From #775: the product sums are taken on each product's values at six points of GF(2^8), by byte maps and GF2P8MULB, rather than on its carry-less coefficients, which the inverse evaluation map recovers once a layer. 1400 leanXMSS at 8 / 4 threads: Zen 4 -2.4% / -3.2%, Arrow Lake -2.1% / -3.8%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
From #775: a digit-bucket part is a whole block position on AVX2 too, six registers, so each bucket's loop runs once per digit rather than twice and the half-position arm goes. Only AVX2 without GFNI (Zen 3, Haswell) runs the buckets; elsewhere it is a simplification. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
…e made From #775: each bucket's sum adds into the sums of its value's two nibbles as it is made, rather than into 256 sums halved eight times, so a digit's scratch is 32 parts on the stack. It pays where the buckets are the product path; 1400 leanXMSS on an Apple M4 Max at 8 / 4 threads: -0.8% / -0.2%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TomWambsgans
added a commit
that referenced
this pull request
Nov 18, 2026
* keccak256: search grinding nonces with first_nonce From #775: the grind permutes its candidates in lanes and reads the one lane it tests, rather than transposing whole states through keccak_f_many; same smallest nonce. 1400 leanXMSS at 8 / 4 threads: Zen 4 -1.3% / -1.2%, Arrow Lake -0.7% / -1.1%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * experimental primitives: AVX2 arms for bit_transpose_64bytes From #775: where AVX-512 VBMI with GFNI is missing, gather the byte columns with unpacks and a shuffle, then bit-transpose them by one GFNI affine map, or by three masked-swap rounds without GFNI. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.7% / -1.2%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * experimental keccak_gkr: table rows by byte maps on AVX2 From #775: the table pass builds its rows by byte maps, GFNI affine or vpshufb nibble lookups, on every AVX2 target rather than only on AVX-512 with GFNI; the x86 fold_groups arms it leaves unused go. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.8% / -1.4%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * experimental lean_vm_core: bus GKR vertical kernels on AVX2+VPCLMULQDQ From #775: products_into, the round message and the fused fold run eight lanes at a time with VPCLMULQDQ on AVX2 too, as two x4 halves, with x8 and f128x8 sharing their method names. 1400 leanXMSS at 8 / 4 threads: Arrow Lake -0.5% / -2.0%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * experimental keccak_gkr: GFNI product sums on values at six points From #775: the product sums are taken on each product's values at six points of GF(2^8), by byte maps and GF2P8MULB, rather than on its carry-less coefficients, which the inverse evaluation map recovers once a layer. 1400 leanXMSS at 8 / 4 threads: Zen 4 -2.4% / -3.2%, Arrow Lake -2.1% / -3.8%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * experimental keccak_gkr: whole-position parts on AVX2 From #775: a digit-bucket part is a whole block position on AVX2 too, six registers, so each bucket's loop runs once per digit rather than twice and the half-position arm goes. Only AVX2 without GFNI (Zen 3, Haswell) runs the buckets; elsewhere it is a simplification. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * experimental keccak_gkr: fold digit bucket sums by nibbles as they are made From #775: each bucket's sum adds into the sums of its value's two nibbles as it is made, rather than into 256 sums halved eight times, so a digit's scratch is 32 parts on the stack. It pays where the buckets are the product path; 1400 leanXMSS on an Apple M4 Max at 8 / 4 threads: -0.8% / -0.2%. Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Tom Wambsgans <TomWambsgans@users.noreply.github.com> Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
|
merged here #782 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Single-thread speedups for the experimental prover on x86 with AVX2 and no AVX-512, from the ideas in #769, which records what came of each one. Proof bytes are unchanged. AVX-512 and NEON arms keep their behaviour.
The branch keeps only the changes with a large gain. Eleven smaller commits, each worth a few ms at most, are on
kevaundray:kw/avx2-single-thread-fullif any are wanted back.Results
Single thread (
LEANVM_NUM_THREADS=1),aggregate --xmss 100, Ryzen 9 7950X3D pinned to one core, minimum pass over two runs of three, upstream main28dcbbcbfagainst this PR, s:The three builds ran back to back on the same core, on a busier machine than the earlier runs, so compare within a row only. Each commit message has its own same-core A/B, taken on an EPYC 9454P (also Zen 4).
What changed, by stage
ones * aby lookup.vpshufbnibbles. The scratch pass is gone, and the maps run inside Exp/keccak tail fold #771'sRound::pass.s. With GFNI, upstream's1e674918akernel is kept.Xproducts run two coefficients a register, Haswell included.bit_transpose_64bytes.BitFold::fold_blockgoes through the column kernels.keccak256): searches withfirst_nonce, which gives the same smallest nonce.--xmss 200with 42 data lanes, where rows go from 48 to 44.--xmss 150Commit / NTT 176 → 152 ms on GFNI.Tried and rejected, with numbers in #769:
Checks
keccak256,fiat_shamir_prover, the experimental crates,stable_lean_vm_core, andleanvm-cli, which includesidentical_proofs.clippy --workspace --all-targets -D warningsis clean on CI's x86 targets (x86-64, alderlake, znver3, x86-64-v4, znver4) and on haswell.cargo fmtis clean.