Repository navigation
Conversation
An own approval stays at its nonce until it is mined. When the account's mined nonce stays below it and the node holds nothing in between (a mined approval whose block was reorged away, or a skipped foreign transaction that later left the pool), nothing will ever fill the empty nonce, and every later approval queues behind it until the process restarts. The follow-up now detects that case and moves the approval down to the empty nonce, or, when the migration no longer needs it, leaves the nonce to the next approval. The attempt left behind merely reverts if it is ever mined.
A checkpoint records the attestor address and its vault attestor generation together. One without either still loads as written before the fields existed and is bound to the current key on startup, but one with only one of them was edited, and binding it would skip the rewind the identity check asks for. Refuse such a checkpoint with a message pointing at the runbook.
The "own approval not mined" page went out once per transaction. While an attestor waits on such a transaction it processes nothing else, so no other alert follows, and a page whose delivery failed left it silent. Page again 10 minutes after the first page, then at doubling intervals up to every 4 hours, for as long as the transaction stays unmined. Repeat pages say "still not mined (page N)".
A page whose webhook delivery failed (relay down, rate limited, timed out) was logged and dropped. Try a failed delivery again after 2 s and after 10 s more. A 4xx answer other than 408 and 429 is not retried: the relay refused the request itself, which repeating cannot change.
PENDULUM_GENESIS_HASH lets rehearsals run against a local test chain. Nothing tied the override to a test Base chain, so an environment carried over from a rehearsal could pin a test chain's genesis while pointing at the mainnet vault. Refuse any genesis other than Pendulum's when BASE_CHAIN_ID is 8453.
… running Membership and attestor generation were read once, at startup. A key removed and re-added while the daemon ran kept making progress although its earlier approvals no longer counted, and only the next restart refused the checkpoint. Read both again on every finalized head (two calls) and exit when either differs from startup; the next start then asks for the checkpoint to be rewound. The startup check uses the same reads.
The ReleasePending scan split its blocks into ranges of MAX_BLOCK_RANGE (default 10,000) and failed the whole cycle when the endpoint refused that size. Base Sepolia's public endpoint caps eth_getLogs at 200 blocks, so every cycle failed and paged until the setting was lowered by hand. A range the endpoint refuses for its size is now read again at half the size, down to a single block. After ten full-size reads in a row succeed the size doubles again, never above MAX_BLOCK_RANGE, which stays the maximum. Other failures propagate as before. MAX_BLOCK_RANGE must now be a whole number of at least 1, and each cycle's log line shows the current size.
Cover the rule that only a consumption read at the finality boundary drops an entry: a release consumed only at the latest state, or a NonceAlreadyConsumed from a latest-state simulation, keeps the entry; a boundary read drops it. Also cover the interval throttle of blocked pages and the vault binding of the state file.
An unexpected release failure paged on every cycle for every affected entry, so one persistent condition produced a critical page per entry per minute on the webhook the attestors share. Throttle it per release like a blocked one: the same page about a release goes out again only after BLOCKED_ALERT_INTERVAL_MS, and a different page about it at once.
A release transaction not mined within viem's 180 s receipt wait paged "unexpected release failure", and the next cycle signed another release at the next nonce, queued behind the stuck one, until fees fell again. Sign each release at the pending nonce with the capped fee estimate and wait FEE_BUMP_AFTER_SECONDS for it. One not mined by then stays in flight: no other release is sent behind it, and from cycle to cycle it is re-sent as the identical call at the same nonce with both fees raised, at most FEE_BUMP_MAX times and never above MAX_FEE_PER_GAS_GWEI, then it pages "release transaction not mined", throttled like the other release pages. Once its nonce is used by another transaction, or nothing below it can be mined, the release is simulated and sent anew. The settings and defaults match the attestor's.
POLL_INTERVAL_MS=0 turned the cycle loop into a busy loop against the RPC, and READ_BATCH_SIZE=0 failed every cycle that had anything pending. Require a poll interval of at least one second and a whole read batch size of at least 1 at startup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
Attestor
PENDULUM_GENESIS_HASHother than Pendulum's is refused whenBASE_CHAIN_IDis 8453.Attestor and releaser
Releaser
ReleasePendinglog range the endpoint refuses as too large is read again at half the size, down to one block. It doubles again after ten full-size reads, never aboveMAX_BLOCK_RANGE.FEE_BUMP_AFTER_SECONDSstays in flight. It is re-sent at the same nonce with both fees raised byFEE_BUMP_PERCENT, at mostFEE_BUMP_MAXtimes and never aboveMAX_FEE_PER_GAS_GWEI, and then pages "release transaction not mined". The settings and defaults match the attestor's and are documented inreleaser/README.md.BLOCKED_ALERT_INTERVAL_MS, like blocked releases.POLL_INTERVAL_MSbelow one second and aREAD_BATCH_SIZEbelow 1.Testing
npm testinattestor/(94 pass) andreleaser/(34 pass).eth_getLogslimit, the releaser shrank its range from 10,000 by itself and completed every cycle.