Skip to content

perf(onchain): reuse a synced wallet across composes - #188

Merged
jvsena42 merged 2 commits into
masterfrom
perf/reuse-synced-compose-wallet
Nov 5, 2026
Merged

jvsena42 merged 2 commits into
masterfrom
perf/reuse-synced-compose-wallet

Conversation

@jvsena42

@jvsena42 jvsena42 commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Related to synonymdev/bitkit-ios#1187. App-side parts: synonymdev/bitkit-ios#1221 and synonymdev/bitkit-android#1785.

onchain_compose_transaction built a fresh in-memory BDK wallet and ran a full Electrum sync on every call. This keeps the synced wallet so later composes for the same account only run an incremental sync.

Description

  • Add onchain/wallet_cache.rs: a process-wide cache of synced watch-only wallets keyed by descriptors (extended key and script type, without key origin), network and Electrum URL. At most 8 wallets are kept; the least recently used one is dropped.
  • Every compose still syncs before building PSBTs. BDK's incremental sync reuses stored transactions, removes transactions the server no longer reports along with their UTXOs, and recomputes spent flags, so UTXOs are as current as with a cold sync.
  • PSBTs are built on a copy of the synced database, using the descriptors with the key origin when a fingerprint is given. Building a PSBT advances the change index; the copy keeps that out of the cached wallet, so change indexes stay the same as before.
  • Fee-estimate composes (no fingerprint) and signing composes (fingerprint) share one cached wallet.
  • Calls for the same wallet are serialized on a per-wallet mutex. A failed sync drops the cached wallet so the next call starts cold; a poisoned lock is reset. A new Electrum connection is still opened per compose.
  • Public UniFFI API unchanged. No binding changes.

Measured breakdown on the cold path (Trezor emulator all seed, m/84'/1'/0', regtest):

Server Txs Last used ext/int Connect Sync PSBT build
Staging electrs ssl://electrs.bitkit.stag0.blocktank.to:9999 (iOS regtest default) 106 60 / 75 0.24–0.27 s 7.1–13.1 s 6–12 ms
Local Docker electrs tcp://127.0.0.1:60001 4 53 / 1 <1 ms 0.82–0.94 s 11–19 ms

The 7.3–7.5 s in the issue matches the staging electrs sync, not the local Docker stack.

Review round 1

  • Compose connections now have a 30 s socket timeout, so a server that stops answering fails the compose and releases the per-wallet lock instead of blocking later composes for the account (1ea7ad2). Test: stalled_server_fails_within_timeout_and_releases_wallet, which hangs without the timeout and passes with it.

Preview

End-to-end compose_transaction, release build, two runs each:

Server Before (cold) After (warm, no fingerprint) After (warm, fingerprint)
Staging electrs 7.43 s, 7.37 s 3.36 s, 3.41 s 3.38 s, 3.39 s
Local Docker electrs 0.55 s, 0.53 s 0.39 s, 0.39 s 0.39 s, 0.39 s

Fees were identical across the three composes in each run.

QA Notes

  • cargo test --lib modules::onchain: 83 passed, 26 ignored (at 1ea7ad2).

  • New tests in onchain/tests.rs (compose_wallet_cache): database copy keeps scripts, UTXOs, txs, indexes and sync time; a PSBT built on the copy carries the fingerprint and m/84'/1'/0'/... paths, change goes to the next internal index and the source index is untouched; cache key ignores the fingerprint but not server or script type; LRU eviction.

  • Ignored live test: COMPOSE_CACHE_ELECTRUM_URL=<url> cargo test --lib compose_reuses_synced_wallet -- --ignored --nocapture (defaults to staging electrs). Run against staging and local Docker as above.

  • Full cargo test: 735 passed, 11 failed. All 11 are Blocktank tests that call api.stag.blocktank.to, which was unreachable from this machine.

  • cargo clippy and cargo fmt: no new warnings.

  • In the iOS app: this commit cherry-picked onto v0.8.0-rc11 (the version bitkit-ios pins; the Swift bindings are unchanged), built with build_ios.sh and linked as a local package into bitkit-ios at fix: compose a hardware-funded request once bitkit-ios#1221 (Debug E2E_BUILD, staging regtest, iPhone 17e simulator iOS 26.5, Trezor emulator over Bridge). Payment requests from tools/paykit-fixture-sender (synonymdev/bitkit-e2e-tests), 1,000,000 sats each, paid from the Trezor:

    • Request 1: fee preview compose 7.45 s (cold); signing compose with the fingerprint 3.46 s (warm); the device signed and the app showed Bitcoin Sent.
    • Request 2, a few minutes later: fee preview compose 3.89 s; signing compose 3.58 s; broadcast succeeded (txid 965774ae6fde8a53d19174f1fe5c6ca5dd964138ca18235d95e9826f19d7854c), so the cached wallet did not reuse the inputs spent by request 1.

Gaps:

  • Not run in the Android app.
  • About 3.4 s per warm compose against staging remains. It is the per-script history requests BDK sends in batches of 21 (about 9 round trips for this account) plus a new TLS connection. Reusing the Electrum connection or skipping the sync when script statuses are unchanged could cut it further; not done here.
  • get_account_info still does a full sync per call and does not use this cache.
  • Within one compose with several fee rates, change indexes still advance per rate as before.

🤖 Generated with Claude Code

compose_transaction built a fresh in-memory BDK wallet and ran a full
Electrum sync on every call. Against the staging regtest Electrum with the
Trezor emulator account (106 txs) that is about 7.4 s per compose, almost
all of it in the sync.

Keep the synced wallet per extended key, script type, network and Electrum
URL. Later composes run an incremental sync that reuses the stored
transactions, then build PSBTs on a copy of the synced database so the
cached wallet's change index is never advanced. The cache key leaves out the
fingerprint, so fee-estimate composes and signing composes share a wallet.
A failed sync drops the cached wallet; at most 8 wallets are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jvsena42 jvsena42 self-assigned this Nov 5, 2026
@jvsena42
jvsena42 marked this pull request as ready for review November 5, 2026 13:28
@jvsena42
jvsena42 requested review from a team, ben-kaufman and pwltr and removed request for a team November 5, 2026 13:28
Comment thread src/modules/onchain/wallet_cache.rs Resolved

@talosmachina talosmachina left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 finding (1 P1). Keeps the synced watch-only BDK wallet per descriptor, network and Electrum URL, so later composes run an incremental sync and build PSBTs on a copy of its database. Reviewed aa32fa4, full tier.

Build skipped: talos-pr-run could not check the author's permission on this repo with the token it holds, and the check fails closed. So this is a static review: the tests were not run here.

What I checked, and 6 candidates I ruled out

Read in full: wallet_cache.rs, the compose.rs changes, the new tests in tests.rs, and in implementation.rs the functions resolve_wallet_setup, create_wallet, sync_wallet, connect_electrum and run_account_info_blocking. In bdk 0.30.2, Wallet::sync and blockchain/script_sync.rs (into_db_update).
CI: green (Android and iOS bindgen, manifests, usdt-tests, wake-tests). No CI job runs modules::onchain.

Ruled out

  • Stale UTXOs on a warm sync: into_db_update deletes txs the server no longer reports, along with their UTXOs, and recomputes is_spent from every observed tx, so a spent input is not reused.
  • Change index leaking into the cache: PSBTs are built on copy_database's copy, and sync keeps max(db_index, sync_index). The test asserts the source index stays at 3.
  • Descriptor checksum mismatch on the copy: checksums are deliberately not copied, and the cached wallet is always built from the origin-less descriptors, so the two Wallet::new calls never conflict.
  • Fingerprint left out of the key: a key origin changes only the PSBT metadata, not the derived scripts, so both setups watch the same scripts.
  • Eviction while a slot is in use: the caller's Arc keeps the slot alive. The worst case is one extra cold sync, never shared state.
  • Poisoned slot after a panic: catch_unwind inside spawn_blocking poisons the mutex, and synced_database_snapshot resets the slot to None, so the next compose starts cold.

Merge confidence: 3/5, the P1 on the shared lock during an untimed sync.

Comment thread src/modules/onchain/wallet_cache.rs Resolved
@jvsena42
jvsena42 marked this pull request as draft November 5, 2026 14:11
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jvsena42
jvsena42 marked this pull request as ready for review November 5, 2026 14:51
@jvsena42
jvsena42 requested review from a team and ben-kaufman November 5, 2026 14:51

@pwltr pwltr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit 1ea7ad2. No supported correctness findings.

Compose connections now use a 30-second socket timeout, addressing the earlier stalled-server concern. The added timeout and wallet-lock release regression passed (1 test). This is a per-operation timeout; retries and multiple requests can extend total compose time.

Review details

General and targeted validation: GPT-6.1 Sol / High.
Prior security: Kimi K3 / High.

@jvsena42
jvsena42 merged commit 4affda1 into master Nov 5, 2026
6 checks passed
@jvsena42
jvsena42 deleted the perf/reuse-synced-compose-wallet branch November 5, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants