Skip to content

leanvm sidecar process and client connections - #768

Closed
vladimir-ea wants to merge 7 commits into
leanEthereum:mainfrom
gattaca-com:ve/sidecar
Closed

vladimir-ea wants to merge 7 commits into
leanEthereum:mainfrom
gattaca-com:ve/sidecar

Conversation

@vladimir-ea

Copy link
Copy Markdown
Contributor

No description provided.

TomWambsgans added a commit that referenced this pull request Nov 18, 2026
`bins/leanvm-sidecar/clients/go` (package `leanvmsidecar`, 152 lines): `Dial` checks the hello, then `Aggregate` and `Verify` take and return claims, proofs, signatures and blobs as the README's byte forms, so the client only frames requests and splits responses. Refusals are a `*Rejection` with the status and message; responses are capped. Its test proves a blob-only leaf and a node over it, verifies both ways and checks that a refusal leaves the connection usable; CI's Test job runs it, with gofmt and go vet, against the sidecar `cargo testall` builds.

#768's Go client speaks another protocol (shared-memory rings), so this one is new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TomWambsgans TomWambsgans mentioned this pull request Nov 18, 2026
TomWambsgans added a commit that referenced this pull request Nov 18, 2026
* bench: the benchmark CLI moves to bench/ as leanvm-bench, its harness crate is bench_harness

bins/leanvm (package leanvm-cli, binary leanvm) becomes bench/ (package and binary leanvm-bench, `cargo leanvm-bench`), with bench.sh beside it, so that bins/ holds only what users run. The harness library crates/bench becomes crates/bench_harness, so that "bench" names the folder alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* thread count: set in code, LEANVM_NUM_THREADS removed

`lean_vm::setup(threads)` and leanvm-bench's `--threads` fix the prover's pool size before it starts (0: one per core), through `isa::Prover::set_num_threads`, which each tree's pool implements and which panics if the pool already runs with another count. The pools read no environment variable any more; bench.sh passes `--threads`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: leanvm-sidecar proves `aggregate` for local clients over a Unix socket

`crates/sidecar` holds the protocol (length-prefixed frames, a hello with the verifier tag, the program's Fiat-Shamir IV and the prover), the server, generic over the prover, and the Rust client, re-exported as `lean_vm::sidecar`. `bins/leanvm-sidecar` runs it (`--prover`, `--socket`, `--max-request-mib`, `--threads`); its README specifies the protocol. `AggregateClaims` gains `to_bytes`/`from_bytes`, `isa::Prover` a `NAME`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: a verify operation, for clients without the library, taken from #768

Verify (operation 2) answers whether a proof proves claims, on the connection's thread rather than behind the proofs being made.

#768

Co-Authored-By: vladimir-ea <85992906+vladimir-ea@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: warm up before serving, taken from #768

Two discarded proofs before the socket appears, a one-blob leaf and a 16-child recursion over it at the highest rate, so the first requests no longer pay for first-touched memory. On an M-series Mac, experimental prover: the first 900-signature leaf at rate 1 went from 12-30% slower than the next ones to 4%, the first 16-child recursion from 7-70% to 2%, rates 1 and 2 both covered; the warm-up costs about 1.8 s at startup.

#768

Co-Authored-By: vladimir-ea <85992906+vladimir-ea@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* review fixes: bench.sh runs from the repo root again, and doc touch-ups

bench/bench.sh moved up a level with the rename but still went up two, leaving the repository. The rest is
documentation: setup's first sentence ran into the next, AggregateClaims::to_bytes had no doc, Client::verify's
doc spoke of clients without the library, and the sidecar test's header and two doc lines were rewrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* AGENTS.md: the verifier's Keccak under Zen 4 tuning

Built for znver4 (target-cpu=native on Zen 4), LLVM's SLP vectorizer packs the verifier's Keccak-f
lanes 4 or 8 wide and spends the gain on cross-lane shuffles. On h1 (Ryzen 7 PRO 8700GE), one
permutation takes 437 ns, against 196 ns with SLP off and 197 ns with -slp-max-vf=2; generic
x86-64 (232 ns), x86-64-v3 (205 ns) and x86-64-v4 (203 ns) are unaffected, as is an M4 Max (136 ns).
Decoding a 1000-leanXMSS leaf proof, which rebuilds its pruned Merkle paths, and verifying it take
2.96 and 8.12 ms there, against 1.60 and 6.37 ms built without AVX-512.

Tried in the source, none kept: a black_box barrier per round (Zen 4 down to 256 ns at best, other
targets 15 to 60% slower), row and column loops the compiler cannot unroll (2 to 4 times slower
everywhere), and fully unrolled locals or the shared crate's row schedule (vectorized the same).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: a client that builds no prover, proving and verifying through the sidecar

lean_vm gets a default `prover` feature. Without it, it builds no prover tree, no aggregation, lean_da
or zkDSL compiler: it offers the sidecar's client, the claim and signature types and the rate bounds,
and proves and verifies through the sidecar alone, so not while the sidecar is down. With it, the
library proves and verifies in the process as before; experimental_prover implies it.

- claims (new): AggregateClaims and its byte form, OutputClaims and the two claim aliases, moved out of
  aggregation, which re-exports them, so that both the library and the client name them. It depends on
  lean_xmss and lean_sphincs alone.
- sidecar: the protocol and the client by default; the server, and the protocol's server half, behind
  a `server` feature, which bins/leanvm-sidecar turns on. The client takes and returns proofs as
  bytes, so that its API is the same with or without the prover, and no longer checks the program IV
  or the claims a response carries: it trusts the sidecar it verifies through. aggregation's
  published_claims, which only that check used, goes, and node.rs is main's again.
- formal's guest export, which compiles aggregation's sources, depends on claims and imports the two
  key types aggregation no longer brings.
- CI runs clippy on lean_vm without default features.

Verifying through the sidecar costs about what verifying in-process does once the proof is decoded,
which a client receiving bytes pays either way: for a 1000-leanXMSS leaf at rate 1 (277 KiB), 6.7 ms
through the sidecar against 6.5 ms decode and verify in-process on an M4 Max, and 11.1 against 11.2 ms
on a Ryzen 7 PRO 8700GE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: warm up at rate 1/4, the README's hello as the client checks it, the site as main has it

The warm-up proved at the highest rate, whose memory the process then keeps: on the stable prover its peak RSS was 11.9 GB, now 4.8 GB at rate 1/4 (M4 Max), ready in about 2 s either way. A higher rate pays for its extra memory on its first proof.

The client checks the hello's magic, version and verifier tag; the program identifies the build, which only a client that also verifies with the library checks.

The site's reproduce command describes the benchmarked commit it pins, from before the rename, so it stays as on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* review fixes: the no-prover lint over every target, the thread count on experimental, the leaves' file, the socket's directory

- lean_vm's `api` test needs the `prover` feature, so CI lints the build without it with `--all-targets`.
- `--threads` and `setup` say that on experimental they count the performance cores' workers.
- `write_leaves` describes the file as it is: the claims carry their length too.
- The sidecar's README: the socket goes in a directory only its user can write, and a connection may hold about twice the max request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* final review fixes: the child count refused before decoding, AGENTS.md names the sidecar, and cleanups

- The server refuses more than MAX_RECURSIONS children right after the count, as `aggregate` would, so a request's decoded children never outgrow its bytes: a request of empty children otherwise held about 4x the max request.
- The README's hello: the program identifies the build, and no longer promises a check the Rust client does not make.
- AGENTS.md: the sidecar takes `--threads` and `--tracing` and installs jemalloc too.
- Cleanups: the library's log lines lose their `leanvm-sidecar:` prefix, the client's limits are named constants, `server.rs` imports the claims from `claims`, experimental's `set_num_threads` counts the efficiency cores only when it sizes the pool, and the end-to-end test checks a refusal keeps the connection usable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* thread count: the help texts lose the experimental caveat, now that both provers count every worker

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: the max request is a constant, 64 MiB, and --max-request-mib goes

A request holds at most 32 blobs (4 MiB) and 16 children (a few MiB), and 64 MiB leaves room for about 25,000 leanXMSS or 8,000 leanSPHINCS signatures besides, so the limit needs no knob; at 64 MiB rather than 1 GiB, a connection holds at most about 128 MiB. The hello still announces it, so a later build can change it without a protocol version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: no --tracing, which the benchmark CLI covers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* allocator: jemalloc and its configuration in a crate of their own, so the sidecar no longer links bench_harness

bench_harness re-exports it, so the benchmark CLI and the benches are unchanged. jemalloc still reads the configuration in both binaries (`_RJEM_MALLOC_CONF=stats_print:true` shows `opt.dirty_decay_ms: -1`), though nothing calls into the crate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar README

* sidecar: at most 16 connections at once, so the memory requests hold is bounded

Each connection holds at most one request, queued or proving, of about twice the max request, but nothing bounded the connections, so a client opening them in a loop could exhaust memory. The accept loop now takes one of 16 slots before accepting, and a connection's thread frees it when it ends, even by panicking: about 2 GiB at most. A connection beyond the cap waits for its hello rather than being refused, so the protocol is unchanged; checked live: the 17th waits, and gets its hello once one closes.

The README says again how access is controlled: the socket takes the umask, and connecting needs write permission on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar README: the connection cap without its memory, and no supervisor advice

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lean_vm: two exclusive modes, `prover` (the default) or `sidecar`

With `prover`, the library proves and verifies in the process and has no sidecar client; with `sidecar` (and `default-features = false`), it builds no prover and proves and verifies only through the sidecar. Both at once is a compile error, so forgetting `default-features = false` does not silently build the prover. The sidecar is a plain optional dependency, re-exported as `lean_vm::sidecar`; the end-to-end test uses the `sidecar` crate's client directly, and CI lints the sidecar mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* add todo

* lean_vm: stable by default, `experimental_prover` or `sidecar`; the sidecar picks its prover at build time

A user names the mode in one word: nothing for the stable prover in this process, `experimental_prover` for the experimental one, `sidecar` to prove and verify through the sidecar. `sidecar` with `experimental_prover` is a compile error that says where the choice belongs: the sidecar binary, which now builds the stable prover, or the experimental one with its own `experimental_prover` feature, instead of taking `--prover`, so it links one prover.

In sidecar mode the library still compiles the stable prover (about 5 s on a clean build, 38 crates against 10) but hides and never runs it, so the workspace allows Cargo's unused-dependencies lint; CI's cargo machete still finds unused dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: `serve` takes the socket and the thread count, no Config duplicating the CLI's

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar client: `connect` takes only the socket, waiting as long as the sidecar takes

No fixed timeout fits every request, and a sidecar that dies closes the connection, so a call fails at once rather than hanging; `set_timeout` stays for a client that wants a bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar test: verify only through the sidecar, as a client does

The leaf and the node are checked with the client's `verify`, and one proof against other claims is refused, so a `verify` that accepts everything still fails the test. It imports `lean_vm` in its default mode: in its `sidecar` mode, as a dev-dependency, `cargo test` would build `lean_vm` so for the whole workspace and its `api` test would run nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* doc

* lean_vm: prover.rs is in_process.rs, the counterpart of the sidecar mode

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* final review fixes: the experimental pool test tests again, no program in the hello, one claims spec

- The experimental pool test fixed the pool's performance count, which is no longer its size wherever efficiency cores exist, so it panicked on its first line and never reached the one under test; it now uses `num_threads()`, as stable's does.
- The hello's program IV was sent and decoded, but nothing checked or exposed it: dropped from protocol version 1, unreleased.
- The claims' byte form is specified once, in the sidecar's README; `crates/claims` points there.
- The warm-up's doc had the rate's direction backwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* simplify

* guest-export: machete ignores `claims`, used by the aggregation sources it copies

CI's `cargo machete --with-metadata` (0.9.2) failed on it; it passes now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: the lock file is opened without following a symlink or waiting on a FIFO

A FIFO at `<socket>.lock` hung startup forever, silently, and a dangling symlink there made the sidecar create an empty file at its target; both now fail at once (ENXIO, ELOOP). Either needs a directory others can write, which the README rules out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar client: one 16 MiB room for every response, beyond the request's length

It replaces 64 KiB for a hello or a verdict and 256 MiB for a proof: proofs take a few hundred KiB, and the claims a response carries are no longer than the request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: AGENTS.md's x86 check and allocator row hold again, the sidecar's errors and `serve` say what they do

- `bench_harness` no longer pulls in jemalloc, so the x86 check no longer leaves it out.
- `keccak_gkr` (both trees) takes `allocator` as a dev-dependency, as `pcs` does: only its bench uses it, so no library links it, as AGENTS.md says.
- `SidecarError::Incompatible` also covers a peer that is no sidecar, `Rejected` a request the client refuses itself, and `serve` returns on any error before serving.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lean_vm: MAX_RECURSIONS in every mode, so a sidecar client building a tree knows the fan-out

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sidecar: a Go client, tested in CI against the sidecar

`bins/leanvm-sidecar/clients/go` (package `leanvmsidecar`, 152 lines): `Dial` checks the hello, then `Aggregate` and `Verify` take and return claims, proofs, signatures and blobs as the README's byte forms, so the client only frames requests and splits responses. Refusals are a `*Rejection` with the status and message; responses are capped. Its test proves a blob-only leaf and a node over it, verifies both ways and checks that a refusal leaves the connection usable; CI's Test job runs it, with gofmt and go vet, against the sidecar `cargo testall` builds.

#768's Go client speaks another protocol (shared-memory rings), so this one is new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Tom Wambsgans <TomWambsgans@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vladimir-ea <85992906+vladimir-ea@users.noreply.github.com>
@TomWambsgans

Copy link
Copy Markdown
Collaborator

adapted here: #784, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants