Repository navigation
Conversation
Builds the CLI at the PR's merge commit and at its first parent on one runner, runs `cargo leanvm recursion --n 2 --xmss-per-leaf 900 --log-inv-rate 2 --repeat 3` on each, base first, and writes both reports to the job summary. The cycle, row and committed counts and the proof size are deterministic; the times are the runner's. It gates nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… its WHIR query rows with it
Once a child proof's transcript is fixed, what the recursion guest does with an opened WHIR row is affine in the row: its dot product with eq weights the guest derives from the child's fold challenges. That cost a MUL and an XOR per term, plus a JUMP, a MUL and an XOR per lane pair to prove the hinted lanes in K and pack them into the cells KECCAK hashes.
DOT [o_x, o_p, o_w, o_acc, o_out] asserts mem[fp·o_out] = mem[fp·o_acc] + Σ_{j<4} w_j·x_j, the x_j the four K-lanes of two canonical frame cells (lane 2k + h is limb h of cell k), the w_j E words read through a pointer, p·o_w·g^j, as DEREF reads its target. The table commits the lanes as K columns and reads each lane cell as (lo, hi, 0), so the bus proves the cells canonical and their lanes in K; the result rides the bus as the degree-2 coordinate acc_l + Σ_j w_{j,l}·x_j per limb, as MUL's product does, so the table has no identity. Its relation is batched with every other memory access by the bus fingerprint and the one table sumcheck: no new challenge, the error bound unchanged in form (Proposition prop:pad, 4·2^μ/|E|), the table sumcheck's degree unchanged.
The guest hints a query's leaf as the canonical cells KECCAK hashes and chains DOTs against a run of lane weights it builds once a level from opening_row_weights' two tensor factors (level 0's lanes are its words; a deeper level's lane 3i + j is limb j of word i, weighed y^j times the word's weight). The host packs the leaf hint's lanes in pairs.
Every column and bus block of a table costs a recursive verifier a selector per child over the child's position bits above the table's height, and a leaf child has no DOT rows. Four lanes (37 columns, 11 flush pairs) and a least log height of 13 balance that against the rows saved: eight lanes and no floor saved 83.3 k rows in the queries and spent 44.6 k on the selectors; four lanes and the floor spend 19.5 k.
The table is in all three verifiers, the compiler builtin is dot(x, w, acc), and the ISA, the table and its floor are in doc/leanvm (§02, §06, §07, §08, §09) and zkDSL.md. Tests: DOT's value and proof, a non-canonical lane cell refused, a forged DOT result leaving the bus unbalanced, the compiler builtin, and the shared verifier fixture (Rust and Python, a tampered DOT column evaluation rejected by both).
2→1 recursion over leaves of 900 leanXMSS signatures, --log-inv-rate 2 (deterministic):
cycles 542,071 → 481,347 (−11.2 %): MUL 201,734 → 175,094, XOR 148,802 → 118,244, DEREF 117,238 → 119,056, SET 38,029 → 38,441, JUMP 23,605 → 11,595, KECCAK 12,663 → 12,693, DOT 6,224
TOTAL_COMMITTED 2^24.02 → 2^23.849, the stacked witness 2^25 → 2^24 words
proof 217.9 → 233.2 KiB: in a stack of 2^24 the level-0 rows hold almost no zero lanes, which the wire form leaves out; forced to a 2^25 stack the same node's proof is 222.2 KiB
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A WHIR query position is linear in the squeezed word's bits, Σ_b bit_b·x^b, and so is the reconstruction that ties the bits back to the word. Each bit cost a MUL by its constant and an XOR; DOT now sums a run two bit cells a row (a bit cell is the lane pair (b, 0)) against a table of the powers x^b that exponent_tables builds once, an odd last bit by a product. Booleanity stays one MUL a bit, and a group straddling a limb boundary still splits into the two runs that stay inside a limb. 2→1 recursion over leaves of 900 leanXMSS signatures, --log-inv-rate 2 (deterministic), after the previous commit: cycles 481,347 → 465,265: decode_query_bits 37,902 → 21,624; MUL 175,094 → 165,015, XOR 118,244 → 108,104, DOT 6,224 → 11,620 TOTAL_COMMITTED 2^23.849 → 2^23.878 (the DOT table passes 2^13 rows) proof 233.2 → 233.8 KiB Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
exponent_tables returns the bit-weight table too since the previous commit; the test's own main bound two values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e a nibble at a time, through a hinted digit A recursive verifier opens each column and bus block of its child through a selector: eq over the column's position bits above its height, read off per-nibble tables, with g^offset rebuilt from the same bits and pinned to the certified offset. The bits were hinted one by one, each pinned boolean (a MUL) and folded into the nibble's g-power index (a MUL, an XOR and a MUL): four rows a bit. hint_decompose_nibbles_exponent(digits, x, shift, n, rev) hints the base-16 digits of (dlog x) >> (dlog shift) as the g-powers g^d, low digit first, each digit's bits reversed when rev = 1 (the column tables index a nibble top bit first). The guest range-checks each digit below its nibble's table size (assert log(d) < 2^width, three rows) and uses it as the index directly. Soundness is the old argument with digits for bits: each digit is in range, so the product of the shares g^(d_j·16^j·2^κ) is g^E with E below 2^(4·nibbles) < 2^64 − 1, the order of g, and pinning it to g^offset makes E = offset, whose base-16 digits are unique; so every digit is the offset's, the offset's low κ bits are zero, and each eq factor read is the one for the offset's own bits. The range check is load-bearing: without it 0x21 = 1 + 16·2 = 17 + 16·1 would let a digit read past its table (test). 2→1 recursion over leaves of 900 leanXMSS signatures, --log-inv-rate 2: cycles 465,265 → 446,274 (−4.1 %); TOTAL_COMMITTED 2^23.878, the 2^24 stack and the padded table heights unchanged; proof 233.8 → 234.0 KiB. Tests: the hint's digits (plain, reversed, shifted), and an out-of-range digit refused by the range check though the product accepts it; the guest's column selectors still match the native eq (guest_column_selectors_match_native_eq). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uest walks a WHIR query's path one row a level A Merkle path's level cost the recursion guest six rows: the query bit read into the frame, a MUL and an XOR for the address of the child pair it selects, two DEREF stores of the running node into that pair, and the KECCAK over the hinted pair. KECCAK_M (opcode g^7) is KECCAK with odd_in = 0 and two changes: the low limb of the cell b8 is a bit s instead of lane 16, which is Keccak-256's last padding byte for a one-block message, and the two m pairs are read in the order s selects (m0, m1 then m2, m3 when s = 0, the other way round when s = 1). A level is now the bit's read and one KECCAK_M over the previous level's output cells and the hinted sibling. The KECCAK table proves both opcodes. Five columns (mk, b8, s, a0, a1) and eight degree-2 identities: mk(mk + 1) = 0, so the bytecode read's opcode coordinate g^5 + mk·(g^5 + g^7) is KECCAK's or KECCAK_M's; mk·odd_in = 0; b8 + u8 + mk·(b8 + in_16) = 0 (on a KECCAK row b8 is the lane its layout puts in that limb, u8 = in_16 + odd_in·(in_16 + in_15)); mk·(in_16 + 0x80·2^56) = 0; s + mk·b8 = 0 and s(s + 1) = 0; a0 + o_m0 + s·(o_m0 + o_m2) = 0 and a1 + o_m1 + s·(o_m1 + o_m3) = 0. The first two m cells are read at fp·a0, fp·a1, the other two at fp·(a0 + o_m0 + o_m2), fp·(a1 + o_m1 + o_m3), all degree-2 coordinates. On a KECCAK row every flush is the old one; on a KECCAK_M row the cells feeding the Keccak reduction's input lanes are the swapped ones. No flush is added (a row still reads 27 times), no challenge is drawn, the identities take their own range of the table sumcheck's ξ-powers, and its degree is unchanged; the zero padding row satisfies them. In all three verifiers (Rust crates/verifier, python-verifier, the guest's table-sumcheck terminal) and the spec (§02, §07), zkDSL.md (keccak(a, b, out, swap=bit)). The host hints a level's sibling only. The bit is the query bit decode_query_bits pins boolean and reconstructs against the squeezed word, as before, so the path the guest hashes is the query position's. 2→1 recursion over leaves of 900 leanXMSS signatures, --log-inv-rate 2: cycles 446,274 → 417,750 (−6.4 %; −22.9 % against the base 542,071); TOTAL_COMMITTED 2^23.878 → 2^23.886, the 2^24 stack and the padded heights unchanged; proof 234.0 → 235.6 KiB. Tests: KECCAK_M hashes in its bit's order and proves (both bits); a bit of 2 faults; a state of the wrong order is refused; the guest's Merkle path reaches the root and misses it with a wrong sibling, a flipped bit or another leaf; the shared Rust/Python verifier fixture runs KECCAK_M rows (each asserted against a plain KECCAK) and a tampered a1 evaluation fails both at TableSumcheckTerminal; aggregate_two_to_one and aggregate_three_levels (a parent verifying a child with KECCAK_M rows). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…CAK, whose Debug form now names its swap flag The digest hashes each Op's Debug text, and Op::Keccak gained the field swap (KECCAK_M, e82ff81); the instructions these programs compile to are the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. This proposes the "affine verifier" recursion step for leanVM's 2→1 recursion: a
DOTinstruction, aKECCAK_MMerkle step, and the recursion guest rewritten to use them. Oncargo leanvm recursion --n 2 --xmss-per-leaf 900 --log-inv-rate 2it cuts cycles by 22.9 % and CI proving time to ×0.770, at +8 % proof size (figures below).It was developed and measured in a fork, scaraven#7, and is rebased here onto
mainat3cbac047without conflicts. The first commit (9e703bf6) adds aRecursion benchworkflow that runs the 2→1 benchmark at the PR and at its base on one runner. It gates nothing and can be dropped before merge.The idea
Once a child proof's transcript is fixed (all its challenges), almost everything the child's verifier does with the proof's data is affine in that data, with coefficients that are functions of the challenges alone. Only the challenge-only quantities (eq tables, Lagrange and novel-basis weights) and a few data × data finals are genuinely nonlinear. In leanVM the guest already computes the challenge-only part natively, one
MULperEproduct. The data-affine part, however, is paid as VM rows, aMULand anXORper term:JUMP/MUL/XORper lane pair to prove the hinted lanes inKand pack them for Keccak;Σ_b x^b·bit_b.That is about half of a level-0 query's 425 rows.
The change
DOT [o_x, o_p, o_w, o_acc, o_out], a seventh instruction and table:mem[fp·o_out] = mem[fp·o_acc] + Σ_{j<4} w_j·x_j.x_jare the fourK-lanes of two canonical frame cells (lane2k + his limbhof cellk).w_jareEwords read through a pointer, atp·o_w·g^j, the wayDEREFreads its target.Kcolumns and reads each lane cell as(lo, hi, 0), so the bus alone proves the cells canonical and their lanes inK.acc_l + Σ_j w_{j,l}·x_j, asMUL's product does, so the table has no identity. It has 37 columns and 11 flush pairs.dot(x, w, acc)),doc/leanvmandzkDSL.md.DOTs against a run of lane weights built once a level fromopening_row_weights' tensor factors.DOT, two bit cells a row, against a shared table of the powersx^b. Booleanity stays oneMULa bit..github/workflows/recursion-bench.yml, runs the 2→1 benchmark at this PR and at its base.543c2df1, guest only). A recursive verifier opens each column and bus block of its child through a selector: eq over the column's position bits above its height, read off per-nibble tables, withg^offsetrebuilt from the same bits and pinned to the certified offset. The bits used to be hinted one at a time, each pinned boolean and folded into the nibble's index: four rows a bit.hint_decompose_nibbles_exponent(digits, x, shift, n, rev), writes the base-16 digits of(dlog x) >> (dlog shift)as the g-powersg^d.assert log(d) < 2^width, three rows) and uses it as the table index directly.KECCAK_M, a Merkle step (68a7bae6). A Merkle path level cost six rows: read the query bit, aMULand anXORto address the child pair it selects, twoDEREFstores of the running node into that pair, and theKECCAK.KECCAK_M(opcodeg^7) isKECCAKin the even layout with two changes. The low limb of the cellb8is a bits, in place of lane 16, which is Keccak-256's last padding byte for a one-block message. The twompairs are read in the ordersselects.KECCAK_Mover the previous output cells and the hinted sibling.KECCAKtable proves both opcodes, with five more columns (mk,b8,s,a0,a1) and eight identities of degree 2.keccak(a, b, out, swap=bit)),doc/leanvm§02 and §07, andzkDSL.md. The host now hints a level's sibling only.Why four lanes and a floor of 13. A recursive verifier pays a fixed cost for each column and each bus block of every table of the child proof: a selector over the child's position bits above the table's height. A leaf child has no
DOTrows, so itsDOTtable has height 0, the worst case.The floor costs 2^13 padding rows in every proof.
Soundness
What is batched. Every
DOTrow's relation is one more memory interaction on leanVM's single bus. The pull/push pair on the result cell carries the derived affine form, and the other reads carry its inputs. All interactions of all tables are checked by the one multiset equalityΠ_push(β − π_α(t)) = Π_pull(β − π_α(t)).Which random combination. The fingerprint
π_α(t) = Σ_{i<16} eq(α, i)·t_iwithα ∈ E^4andβ ∈ E, proven by the one batched GKR. The table's leaves are settled by the one table sumcheck at the GKR pointζ, batched by powers ofξ ∈ E. The column claims go to the one WHIR opening. There is no new challenge and no new phase: the "one batched linear check" of the affine idea is the bus fingerprint, applied to rows whose coordinate carries the affine form.Error over GF(2^192). Unchanged in form: the bus contributes
4·2^μ/|E|(docprop:pad), plus the GKR and table-sumcheck terms, plus the PCS.DOTadds about 2^16.5 tuples a side against the node's 2^23, soμdoes not move: about2^-166.MUL's, so the sumcheck's degree is unchanged.27·7·2^32 < 2^40(§06).Transcript ordering (the child's challenges fixed before the parent's random combination). The child's challenges and opened data are cells of the parent's memory, computed or hinted by the guest. They are committed in the parent's one commitment, whose root the parent's transcript absorbs before drawing
α, β, ξ, ζor the opening'sλ. In the same order:DOTinput is bound:KECCAKhashes up the Merkle path;addr, and the bus forcesp·o_w·g^jinto range;Completeness.
Σ_i W_i·word_i = Σ_{i,j} W_i·y^j·limb_{i,j}, and the positions are the same sums.DOTrow satisfies its relation, and its padding row is the zero row.Selector digits (
543c2df1)The digit hint is advice, so the guest's checks carry the soundness. Each digit
g^{d_j}is range-checked,d_j < 2^{w_j}, wherew_jis 4, or the partial nibble's width atκ. So each nibble's table lookup reads an entry of that nibble's table. The product of the shares isg^EwithE = Σ_j d_j·16^j·2^κ(bit-reversed within a nibble for the column tables, whose index runs top bit first).Eis below2^{4·nibbles} < 2^64 − 1, the order ofg, so pinning the product to the certifiedg^offsetgivesE = offsetas integers. Base-16 digits are unique, so:κare zero;This is the old argument with digits for bits. The range check is load-bearing:
0x21 = 1 + 16·2 = 17 + 16·1, and only the check refuses the second split (test). The guest computes the same selector values, so nothing about the recursive statement changes.KECCAK_M(68a7bae6)What the table proves. The bytecode read carries the opcode coordinate
g^5 + mk·(g^5 + g^7). The identities are, in characteristic 2:mk·(mk + 1) = 0;mk·odd_in = 0;b8 + u8 + mk·(b8 + in_16) = 0, whereu8 = in_16 + odd_in·(in_16 + in_15)is the lane its layout puts inb8's low limb;mk·(in_16 + 0x80·2^56) = 0;s + mk·b8 = 0;s·(s + 1) = 0;a0 + o_m0 + s·(o_m0 + o_m2) = 0;a1 + o_m1 + s·(o_m1 + o_m3) = 0.The first two
mcells are read atfp·a0andfp·a1, the other two atfp·(a0 + o_m0 + o_m2)andfp·(a1 + o_m1 + o_m3). Read cell 8's low limb is the columnb8.On a
KECCAKrow (mk = 0), identities 3, 5, 7 and 8 forceb8to be the layout lane,s = 0,a0 = o_m0anda1 = o_m1. Every flush is then the old one, soKECCAK's relation is unchanged.On a
KECCAK_Mrow, identity 1 makes the opcode exactlyKECCAK_M's: a non-bitmkwould give some otherg^k, and the bus pairs each row with the program entry at itspc. Then:s = b8is the low limb of the cell the bus reads atfp·o_b8, and it is a bit;fp·o_m{0,1,2,3}in the ordersnames.That is the relation in §02.
The error bound is unchanged.
μstand.ξ-powers, at degree 2, so the sumcheck's degree is unchanged.q_keccak's spare-slot columns (the table's first committed ones) are the same.The guest. A level's bit is the query bit that
decode_query_bitspins boolean and reconstructs against the squeezed word, as before. The node cells are the previous step's output cells, read in place; the sibling is advice, as the child pair was. So the path the guest hashes to the cap is the query position's.Completeness. The host hints the true siblings.
KECCAK_Mwith bitbcomputeshash_pairin the order the index parity gives, so an honest path reaches the same cap.Tests.
DOT's value and proof.DOTresult leaves the bus unbalanced (dot_result_is_bound).DOTrows, and a tamperedDOTcolumn evaluation fails the same check (TableSumcheckTerminal) in both verifiers.KECCAK_Mhashes in its bit's order, and proves and verifies for both bits.keccak_m_order_is_bound).KECCAK_Mrows, each asserted against a plainKECCAK. A tampereda1evaluation fails both verifiers atTableSumcheckTerminal.aggregate_two_to_onepasses, and so doesaggregate_three_levels, where a parent verifies a child that hasKECCAK_Mrows.Rustworkflow (cargo testall, clippy, rustdoc, rustfmt) andDocumentation PDFpass on the head.Measurements
Measured in the fork, before the rebase, against base
1e9d8cff. The fork's commits map one-to-one onto this branch's:f69ee6be→9e703bf6,bdde3a52→156a6317,3836659f→2877dcda,7f773cbd→f4d227e1,f777af77→73dc2b48,806a5cc0→543c2df1,e82ff814→68a7bae6,7137b72a→f8c5c428. TheRecursion benchjob reruns them againstmainon this PR.cargo leanvm recursion --n 2 --xmss-per-leaf 900 --log-inv-rate 2(cycles, rows, committed size and proof size are deterministic):1e9d8cffDOTrows (bdde3a52)DOTpositions (3836659f)806a5cc0)KECCAK_M(e82ff814)KECCAK_M)(The two new columns' table counts are from the CLI's
2^xfigures, so they are rounded; the cycles are exact.) Neither new commit moves a table across a power of two:MULis still above 2^17 and pads to 2^18. So their effect on proving time is at most the GKR's and the table sumcheck's work on unpadded rows, which CI does not resolve.CI (
Recursion bench, both sides on one GitHub runner, base first,--repeat 3,-C target-cpu=haswell):f777af77(DOT)e82ff814(+ digits,KECCAK_M)7137b72a(head, same code)The cycle, committed and proof-size counts in CI are identical to the table above. The two new commits cut cycles by a further 10.2 % but leave proving time where
DOTput it (0.770 on like runners). No table crosses a power of two, so the padded witness the prover commits and sums over is the same.Local (x86-64, AVX-512 but no PCLMULQDQ/GFNI, so absolute times are not comparable with the README's M4 Max). Each side is pinned to 8 cores with
LEANVM_NUM_THREADS=8, the same for every side, and run with--repeat 3. The host is shared, at load 4 to 30.DOTrowsDOTpositionsKECCAK_M(e82ff814)The last column ran at a lighter load (4 to 12) than the others, so its 4 % edge over
DOTpositions is not a measured gain; CI, run on the same runner, shows none.The proof gets larger, not smaller. The base's committed data, 2^24.02 words, sits just over a power of two, so its 2^25 stack is almost half zero. The wire form leaves the zero lanes out of every level-0 row it opens. In a 2^24 stack the rows are nearly full. Forced to a 2^25 stack, the same
DOTnode's proof is 222.2 KiB.Not done here
fp-dependent addresses, i.e. the parent's witness, so the outer verifier would need them exported and would pay for them again.DOToverEdata, for the sumcheck finals and claim batching: a few thousand rows.KECCAKreads); the level-0 query loop (≈ 30 k); the residual novel-basis eq per query (≈ 43 k, challenge-only);decode_query_bits(≈ 22 k, aMULper bit for all 192 bits of each squeezed word).MULneeds to lose ≈ 14 k more rows to fall to 2^17.🤖 Generated with Claude Code