From e75c7aa1ce48c3aad1625407706e2c40bb49f9b5 Mon Sep 17 00:00:00 2001 From: Jaromir Hamala Date: Thu, 5 Nov 2026 14:36:26 +0100 Subject: [PATCH 1/2] Document partial commits behind a QWP NACK The replay invariant in design/qwp-nack-policy-v2.md said a NACK means the server applied nothing of the rejected batch. The server's commit batch now spans every table of a receive pass and commits the tables one at a time (questdb#8195). It rejects the whole batch for every refusal it can check before committing: a read-only node, an ACL denial, or a dropped, renamed or suspended table. A storage fault on one table's commit, or a DROP, RENAME or SUSPEND WAL landing while the batch commits, can still fail a table after others committed. Their rows from frames at or after the NACKed frame are then applied, and the client's RETRIABLE replay sends them again. The invariant now states what a NACK names, when the replay duplicates nothing, and the exception with its bound: one server receive pass (http.recv.buffer.size, 2 MiB by default). DEDUP tables absorb the duplicates. The client's policy does not change: it replays rather than risk loss. --- design/qwp-nack-policy-v2.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/design/qwp-nack-policy-v2.md b/design/qwp-nack-policy-v2.md index 2a9ff891c..782e8f63a 100644 --- a/design/qwp-nack-policy-v2.md +++ b/design/qwp-nack-policy-v2.md @@ -134,8 +134,18 @@ until a primary is reachable. Consequently: ## Invariants to hold the line on -- **Replay idempotency:** NACK ⇒ batch atomically not applied server-side. - RETRIABLE replay depends on it; violating it trades loss for duplication. +- **Replay safety:** a NACK names the first frame the server did not commit + in full, and no ack covers it or any later frame. Every refusal the server + can check before committing (read-only node, ACL denial, a dropped, renamed + or suspended table) rejects its whole commit batch with nothing applied, + so the replay duplicates nothing. The one exception: the server commits + each table of a batch separately (questdb#8195), so when one table's commit + fails after others committed, on a storage fault or a DROP, RENAME or + SUSPEND WAL landing mid-commit, those tables' rows from frames at or after + the NACKed frame are already applied, and the replay sends them again. A + batch never outlives one server receive pass (`http.recv.buffer.size`, + 2 MiB by default), which bounds those duplicates; DEDUP tables absorb + them. RETRIABLE replay accepts that duplication rather than risk loss. - **NACK-before-close:** a server rejecting bytes must NACK before closing; a close without a terminal NACK is treated as transport. - **No watermark advance on any rejection**, ever (fuzz-tested: From 592fba0054089df2ad2079c07ed9a442c0ec0262 Mon Sep 17 00:00:00 2001 From: Jaromir Hamala Date: Mon, 9 Nov 2026 13:19:22 +0100 Subject: [PATCH 2/2] Rewrite the replay invariant in plain terms The invariant packed the ack rule, the pre-commit refusals and the partial-commit exception into one long sentence chain, and it no longer matched the server: questdb#8195 now checks a table's refusals as the table joins the commit batch, and a DROP, RENAME or SUSPEND WAL can fail a table at any time while it waits for its commit, not only mid-commit. The bullet now states the ack rule, what a RETRIABLE replay normally re-sends, when it re-sends rows the server already stored, and what bounds them: one server receive pass for plain frames, or the whole group for a FLAG_DEFER_COMMIT group, which the old text left out. --- design/qwp-nack-policy-v2.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/design/qwp-nack-policy-v2.md b/design/qwp-nack-policy-v2.md index 782e8f63a..da89e0df9 100644 --- a/design/qwp-nack-policy-v2.md +++ b/design/qwp-nack-policy-v2.md @@ -134,18 +134,19 @@ until a primary is reachable. Consequently: ## Invariants to hold the line on -- **Replay safety:** a NACK names the first frame the server did not commit - in full, and no ack covers it or any later frame. Every refusal the server - can check before committing (read-only node, ACL denial, a dropped, renamed - or suspended table) rejects its whole commit batch with nothing applied, - so the replay duplicates nothing. The one exception: the server commits - each table of a batch separately (questdb#8195), so when one table's commit - fails after others committed, on a storage fault or a DROP, RENAME or - SUSPEND WAL landing mid-commit, those tables' rows from frames at or after - the NACKed frame are already applied, and the replay sends them again. A - batch never outlives one server receive pass (`http.recv.buffer.size`, - 2 MiB by default), which bounds those duplicates; DEDUP tables absorb - them. RETRIABLE replay accepts that duplication rather than risk loss. +- **No ack past a NACK:** the server acks neither a NACKed frame nor any + frame after it, and after a RETRIABLE NACK the client replays them all. + Normally the server has stored none of their rows. The exception: the + server commits each table on its own. If one table's commit fails after + other tables committed, the rows those tables got from the replayed + frames are already stored, and the replay stores them a second time. + This takes a storage fault, or a DROP, RENAME or SUSPEND WAL that lands + while the table waits for its commit. The server commits plain frames at + least once per receive pass (`http.recv.buffer.size`, 2 MiB by default; + questdb#8195), which bounds what a replay re-sends; a `FLAG_DEFER_COMMIT` + group can re-send the whole group. Tables with DEDUP absorb the + duplicates. The client accepts them: without the replay, the failed + table's rows would be lost. - **NACK-before-close:** a server rejecting bytes must NACK before closing; a close without a terminal NACK is treated as transport. - **No watermark advance on any rejection**, ever (fuzz-tested: