Skip to content

Resend B2B webhook updates when a deposit or account returns to an earlier state - #1437

Merged
ebma merged 6 commits into
stagingfrom
fix/monerium-webhook-repeated-state-event-id
Oct 21, 2026
Merged

ebma merged 6 commits into
stagingfrom
fix/monerium-webhook-repeated-state-event-id

Conversation

@ebma

@ebma ebma commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Why

DEPOSIT_UPDATED and ACCOUNT_UPDATED named each event after a hash of the snapshot alone, and the snapshots carry no timestamp. When a deposit or account came back to a snapshot it had shown before, the new event got the earlier event's id. The outbox dropped it as a duplicate (ignoreDuplicates on (webhook_id, event_id)), and our docs tell partners to skip an eventId they already processed, so the partner was left on the stale state.

Both cases showed up in the live sandbox tests:

  • Deposit: one quote reverted for a single keeper cycle, so the waiting reason went below_floor → no_route → below_floor. The keeper keeps waiting.since across reasons, so the last snapshot was byte-identical to the first, and the partner's last update said no_route for ten minutes.
  • Account: suspend, reactivate, suspend, reactivate. Only the first two ACCOUNT_UPDATED events were delivered, so a partner could treat a live client as suspended, or a suspended one as live.

What changes

The id suffix is now the first 16 hex of SHA-256 over the previous snapshot hash sent, the new snapshot hash and the row's updated_at. A write to the row moves updated_at, so a repeated state gets a fresh id. A change that touches only another table leaves updated_at alone (a failed swap drops out of conversions[]), but then the snapshot it replaces differs, so two such returns still get different ids. A re-emit after a crash between the enqueue and the marker write sees the same marker and row and keeps its id, so the outbox still absorbs it. Change detection is untouched (the stored lifecycle_event_hash is still the plain snapshot hash), so deploying this re-sends nothing.

The same pass also stamped its marker (lifecycle_event_at) with the clock at the write. A write that landed between the pass's read and that marker, such as the webhook inbox moving a deposit to returned, ended up older than the marker, so the settled deposit was never re-evaluated and the partner never got its last update. The marker now records the row version that was evaluated, so such a write re-queues the deposit on the next pass.

Compatibility

The payloads and the documented id shape (deposit-updated:<depositId>:<16 hex>, account-updated:<accountId>:<16 hex>) stay the same; only the ids of new events differ. The partner docs (page 07), the shared JSDoc and security spec invariant 16 now say the id is unique per change rather than per snapshot. Wire-contract snapshot unchanged.

Tests

  • A deposit whose waiting reason flips back sends three updates with three distinct ids.
  • A converting deposit whose two retry swaps are each seen pending and then fail sends four updates with four distinct ids, with no write to the deposit row.
  • An account suspended and reactivated twice sends all five updates.
  • An update re-emitted after a lost marker write is still absorbed: the test restores the earlier marker and expects no new delivery.
  • A pending deposit that the provider returns while a pass is evaluating it is still reported as returned on the next pass.

The id tests fail without the fix, the crash test fails if the id is not stable across the re-emit, and the settle test fails with the clock-stamped marker (all checked by mutation). manager-events.test.ts passes (17), and lint, verify, typecheck and wire-contract are clean.

CI: migrator.test.ts now resets the test database first. The 066 migration test leaves KYC cases with verification state behind, which made reverting 066 fail whenever bun ran that file first (also seen on other open PRs).

ebma added 2 commits October 20, 2026 10:25
DEPOSIT_UPDATED and ACCOUNT_UPDATED named each event after the snapshot
hash alone. A deposit or account that came back to an earlier snapshot
(a waiting reason flipping back after one failed quote, a second suspend
and reactivate) reused that snapshot's event id, so the outbox dropped
it as a duplicate and the partner was left on the stale state. Both
cases showed up on the sandbox. The id now also covers the row's
updated_at; a re-emit after a crash before the marker write keeps its id.
@netlify

netlify Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for vortex-sandbox ready!

Name Link
🔨 Latest commit 7bb223c
🔍 Latest deploy log https://app.netlify.com/projects/vortex-sandbox/deploys/6ad89c67e9ab5a6a3c6a0226
😎 Deploy Preview https://deploy-preview-1437--vortex-sandbox.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify

netlify Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for vrtx-dashboard canceled.

Name Link
🔨 Latest commit 7bb223c
🔍 Latest deploy log https://app.netlify.com/projects/vrtx-dashboard/deploys/6ad89c671701c3e6ddb96e6c

@netlify

netlify Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for vortexfi canceled.

Name Link
🔨 Latest commit 7bb223c
🔍 Latest deploy log https://app.netlify.com/projects/vortexfi/deploys/6ad89c67706812307df05414

ebma added 4 commits October 21, 2026 12:45
A failed swap writes only its execution row and drops out of
conversions[], so a deposit already converting could return to an
earlier snapshot at the same updated_at and reuse that event id. The
previous marker tells those returns apart, and a re-emit after a crash
before the marker write still sees the same marker and keeps its id.
The crash test now restores a real earlier marker; its instance update
was a no-op, so it passed with any id.
The DEPOSIT_UPDATED marker was stamped with the clock at its write. A
write that landed between the pass's read and that marker, such as the
webhook inbox moving a deposit to returned, ended up older than the
marker, so the settled deposit was never re-evaluated and the partner
stayed on the stale snapshot. Stamping the row version that was
evaluated re-queues it on the next pass.
Reverting migration 066 refuses while any KYC case holds verification
state, and the 066 test leaves such cases behind. Whenever bun ran it
before the migrator suite, the revert test failed. Resetting first
makes the suite independent of file order.
@ebma
ebma merged commit 4b4cb2d into staging Oct 21, 2026
6 checks passed
@ebma
ebma deleted the fix/monerium-webhook-repeated-state-event-id branch October 21, 2026 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant