[LouisHaftmann]

📝 Summary

The cache server ran on Nitro/Node.js: a ~250 MB image that idles at ~280 MB of memory and spends most of its CPU time in the runtime rather than moving bytes. This rewrites the whole server as a single statically linked Rust binary with a deliberately strict toolchain, while keeping every external contract so existing deployments upgrade by swapping the image.

Kept unchanged: the Twirp CacheService (JSON and protobuf), Azure-style block uploads, downloads with Range, Results Passthrough, the REST management API with its OpenAPI docs at /management-api/_docs, /health, /metrics metric names, every environment variable, the cleanup schedules, and the storage layout. The database schema and Kysely's bookkeeping tables are reused as-is, including migrations $7/$8 from #283 and #286. Migrations take the same advisory locks, so a database written by either implementation stays readable by the other. ADR-0010 records the decision.

src/
├── main.rs        # config → db (migrate) → storage → axum router, graceful shutdown
├── config.rs      # typed env parsing (same names, defaults, exclusivity rules)
├── db.rs, db/     # sqlx::Any + sea-query for Postgres/MySQL/SQLite, Kysely-compatible migrations
├── storage.rs     # uploads, matching, merge-on-read tee, leases, Capacity-based Eviction
├── storage/       # filesystem adapter, one object_store adapter for S3/GCS/Azure
├── routes/        # Twirp, block upload/download, passthrough, management API
├── cleanup.rs     # the six cleanup tasks on wall-clock schedules
└── tests/         # per-test isolated servers over every DB × storage backend

Strictness: the crate is unsafe-free and denies warnings, clippy all/pedantic/nursery/cargo plus ~70 restriction lints, including:

• unwrap_used, expect_used and panic
• indexing_slicing
• as_conversions and arithmetic_side_effects
• allow_attributes: exceptions must be #[expect(…, reason = "…")]

It also denies missing_docs, unreachable_pub and unused_crate_dependencies. Formatting uses nightly rustfmt with error_on_line_overflow, grouped and granular imports, and comment wrapping.

Performance, beyond the rewrite itself (details and numbers in the benchmark comment):

• SQLite:
  • WAL with synchronous=normal, checkpointing every 10000 pages.
  • Read-only connections beside the single writer.
  • Built without memory statistics (.cargo/config.toml).
  • Prefix matching with substr; at most two queries per lookup.
• Filesystem: merges are a kernel copy (copy_file_range), and a first download reads the Parts directly.
• Single-Part entries: served from their Part, with Range, and never merged.
• Object storage:
  • Parts and merges of known length stream to a presigned URL in one PUT.
  • Objects seen to exist are trusted for a minute (ADR-0005 amended).
  • Merges that can't be presigned upload four multipart parts at a time.
• Allocator: mimalloc, tuned in the Dockerfile with MIMALLOC_PURGE_DELAY=0 MIMALLOC_ALLOW_THP=0.

Against Node 9.9.0 on SQLite + filesystem: uploads are 6.7× faster, first downloads 22×, lookups 17×, and small save+restore cycles go from 4 to 1143 per second.

Fixed along the way:

• DELETE /management-api/cache-entries ignored the repoId filter.
• An empty MANAGEMENT_API_KEY enabled the API without auth; it now disables it, as before.
• Deleting a storage location removed its data before the database commit (ADR-0001), and it ignored active readers (ADR-0002).
• The Helm chart's float-formatted integers (1.073741824e+11) are accepted.
• A negative CACHE_CLEANUP_OLDER_THAN_DAYS stops startup; the TypeScript server deleted every entry daily with it.
• Restarting inside a repeated DST hour no longer runs cleanup every 30 seconds (a Croner bug); runs wait for the next boundary.

✅ Verification

• Test suite: 68 Rust tests: those ported from the vitest suite, the fix(storage): stop concurrent CreateCacheEntry calls from reserving the same entry twice #283/fix(upload): fail a block with 404 instead of silently dropping it #285/fix(storage): match cache keys exactly on every database, like GitHub does #286/fix(download): answer Range on an empty cache entry the same way on every storage driver #291 tests, and a test pinning the match order. They pass locally on these backend combinations:

  • sqlite + filesystem / gcs / azblob
  • postgres + s3 / gcs
  • mysql + azblob

  CI runs the full 3 × 4 matrix through scripts/test-services.sh.

• Real client: @actions/cache 6 save and restore against the release binary, for 1 B, 2 MiB, 64 MiB and 300 MiB, including the restore-key fallback, with both lazy and eager merge: all byte-identical.

• In-place upgrade: entries saved by the 9.8.0 TypeScript server (one merged, one unmerged) restore byte-identically through the Rust server on the same SQLite database and storage, with no migration rerun. A test also upgrades a $6 database dumped from the TypeScript server through $8.

• Docker: the image builds for linux/amd64 and linux/arm64 by cross-compiling (no QEMU). The final image is about 29 MB, versus 243 MB.

• Checks: cargo +nightly fmt --check, cargo clippy --all-targets --locked (0 diagnostics) and node --test scripts/*.test.mjs all pass.

A benchmark comparing Node 9.9.0, this branch before the performance commits, and the current head (startup, memory, CPU, transfer speed, lookup latency) is in this comment.

🔍 Reviews

Review	Found	Fixed
bug-review, round 1	17	12
bug-review, round 2	4	2
bug-review, round 3	0	0
ponytail-review, round 1	34	34
ponytail-review, round 2	9	9
ponytail-review, round 3	2	2
code-review (standards + spec), round 1	21	14
code-review (standards + spec), round 2	11	8
code-review (standards + spec), round 3	4	3

After merging dev (#291), a second review pass ran with GPT-6.1 Sol:

Review	Found	Fixed
bug-review, round 1	8	8
bug-review, round 2	4	4
bug-review, round 3	2	2
bug-review, round 4	0	0
ponytail-review, round 1	5	5
ponytail-review, round 2	1	1
ponytail-review, round 3	0	0
code-review (standards + spec), round 1	12	11
code-review (standards + spec), round 2	5	5
code-review (standards + spec), round 3	3	2
code-review (standards + spec), round 4	1	0

The unfixed ones are the two DST and retention differences above; one round-3 spec finding was already fixed.

Unfixed by decision:

• A wrongly typed token claim now answers 401 "Invalid token" instead of a claim-specific 401 message.
• migrate returns a plain bool.
• OIDC discovery runs serially when it fails; the fallback JWKS lives on the same host.
• nodejs_* metrics are gone, and the process_* set differs slightly.

Out of scope because the TypeScript server behaves the same (see Needs attention):

• concurrent finalize of one upload
• incomplete multipart uploads after a failed completion
• stray top-level objects
• a 404 on /management-api/cache-entries/ with a trailing slash

Not adopted, as judgement calls: newtype ids, a merge-state enum, splitting the cleanup summary, and moving one upload test.

💥 Merge danger

Door: two-way

The schema, migration bookkeeping and storage layout are shared with the TypeScript server, so rolling back to the previous image works on the same database and volume.

Blast Radius: everything

Every deployment runs the new binary. Removed:

• /management-api/_rpc and the TypeScript SDK
• the -caged image tags and Helm's image.variant
• BENCHMARK, NITRO_PORT and NITRO_HOST
• server-side merge composition on S3 and GCS: Eager Merge now streams the Parts through the server

DELETE /management-api/storage-locations/{id} answers 409 while the location is being downloaded.

Behaviour that changed:

• SQLite: runs in WAL mode with synchronous=normal. A power loss or OS crash can drop the last commits, but never corrupts the database. The mode is stored in the file, and the TypeScript server opens it as is.
• Single-Part entries: never get a merged object; the TypeScript server would merge them on first read after a rollback.
• Object storage:
  • Parts are uploaded through presigned URLs, which needs credentials that can sign. Without them, uploads fall back to multipart.
  • Data deleted behind the server's back (another replica, a bucket lifecycle rule) can still be promised for up to a minute.

⚠️ Needs attention

• The docs site (gha-cache-server.falcondev.io) still describes the SDK, the -caged variant and the Node runtime.
• The release workflow now bumps Cargo.toml through scripts/release-metadata.mjs (tested with node --test). It runs for real only at the next release.
• Memory: peak memory rose from 24 to 88 MiB with mimalloc. musl's allocator peaks at about 46 MiB but needs about 75% more CPU.
• GCS: the GCS path is only tested with small objects. No emulator supports the XML multipart uploads object_store uses above 5 MiB.
• Gaps carried over from the TypeScript server, worth issues:
  • Two concurrent finalizes of one upload create two storage locations for one folder, which cleanup can later delete under the live entry.
  • A multipart upload whose completion fails is not aborted; an S3 lifecycle rule for incomplete uploads covers it.
  • delete_folder misses stray objects at the top level.

🤖 Generated with Claude Code

[LouisHaftmann]

Benchmark: Node 9.9.0 vs this PR

To reproduce: cargo run --release --example benchmark -- node=ghcr.io/falcondev-oss/github-actions-cache-server:9.9.0 rust=<image built from this branch>

Setup: each image runs as a container with SQLite and filesystem storage on a fresh Docker volume (btrfs on NVMe). Cleanup jobs are off, and token validation is skipped so the numbers measure the server, not GitHub's JWKS. A protocol client sends the same requests as @actions/cache (32 MiB blocks). CPU time is cpu.stat and memory is anonymous memory (memory.stat anon, excluding page cache), both read from each container's cgroup and sampled every 10 ms. Host: Ryzen 9 5950X (32 threads), 64 GB RAM. rust-before is this branch before the performance commits (11ddb3a8); rust is the current head.

Metric	node	rust-before	rust
Image size	260.1 MiB	28.6 MiB	28.9 MiB
Startup to healthy	3178.6 ms	511.7 ms	507.8 ms
Idle memory	332.9 MiB	2.4 MiB	3.0 MiB
Peak memory	436.0 MiB	24.0 MiB	88.4 MiB
Total CPU time	58.56 s	11.19 s	4.66 s
Upload (8 × 128 MiB, parallel)	356 MiB/s	497 MiB/s	2371 MiB/s
Upload CPU time	2.82 s	0.88 s	1.07 s
First download (merge on read)	368 MiB/s	794 MiB/s	8060 MiB/s
First download CPU time	6.12 s	2.77 s	1.18 s
Later download (merged)	829 MiB/s	2338 MiB/s	2312 MiB/s
Later download CPU time	2.34 s	1.99 s	0.50 s
Lookups per second (64 concurrent)	1953	3577	33998
Lookup p50	27.7 ms	16.4 ms	1.7 ms
Lookup p99	87.7 ms	122.1 ms	7.0 ms
Small save+restore per second (256 KiB, 32 concurrent)	4	4	1143
Small save+restore CPU time	44.12 s	3.77 s	0.63 s

Against Node

• Footprint: the image is 9× smaller, the server is healthy 6× sooner, and it idles at 3 MiB instead of 333 MiB.
• Speed: uploads are 6.7× faster and first downloads 22× faster. Lookups are 17× faster, with a p99 of 7 ms instead of 88 ms. Small save+restore cycles run at 1143 per second instead of 4.
• CPU: the whole run takes 12.6× less CPU.

What the performance commits changed (rust-before → rust)

• SQLite:
  • WAL with synchronous=normal: commits no longer fsync, which took small saves from 4 to over 400 per second.
  • Checkpoints every 10000 pages instead of 1000.
  • A pool of read-only connections next to the single writer.
  • Built without memory statistics, whose global mutex serialized the readers.
  • Prefix matching with substr instead of a bound LIKE, which SQLite recompiled on every run.
  • At most two queries per lookup.
• Filesystem merges: done with a kernel copy (copy_file_range, a reflink on btrfs/xfs). A first download reads the Parts directly instead of waiting on the merged copy being written.
• Single-Part entries (every save up to 128 MiB) are served from their Part, with Range, and never merged.
• Object storage:
  • Parts and merges of known length stream to a presigned URL in one PUT instead of buffered 5 MiB multipart parts.
  • Objects seen to exist are trusted for a minute, so lookups skip a HEAD request.
  • On S3 that took uploads from 323 to 1172 MiB/s, first downloads from 325 to 970 MiB/s, lookups from 3500 to 29000 per second, and small saves from 3.5 to 500 per second.
• mimalloc replaces musl's allocator.

Where the current build is not ahead

• Peak memory went from 24 to 88 MiB. mimalloc keeps freed memory per thread. musl's allocator peaks at about 46 MiB on the same build, but needs about 75% more CPU and halves first-download speed.
• Later downloads stay at disk speed on btrfs. The merged file is a reflink of the Parts, so it starts outside the page cache and is read from the NVMe.
• Node's numbers may be slightly low. The machine was finishing image builds when its run started (1-minute load 3.4).