Repository navigation
Conversation
Posted reviews state the review level but never the model that wrote them, so a reader cannot tell which model to credit for a review or compare the reviews that different models write. The review-pr skill now asks for the model on the same opening line as the level, and Step 4 makes that line the first line of the report. The console output, fix-pr's report.md and the review that approve-pr and reject-pr post therefore all carry it. The skill reads the model from the harness, never from memory, since models misstate their own name and version: - pi: $PI_PROVIDER/$PI_MODEL and $PI_REASONING_LEVEL. pi sets them per session, so a review that fix-pr runs in a subagent names its own model. - Claude Code: the exact model ID its system prompt states. - Codex: the model and effort of the session's latest turn, from the rollout file that $CODEX_THREAD_ID names; the command needs jq. When the harness reports no model, the line says "model unknown" instead of a guess. The pi and Codex commands ran against a live pi session and a real Codex rollout file, with and without the variables set.
Reviews kept reporting missing tests for scenarios few users reach, such as ATTACH and DETACH PARTITION edge cases or non-WAL tables that ILP over TCP writes, even when the reviewer's own probe showed the code worked. The risk rating that lets rare bugs ship covers only Critical findings, so these gaps still arrived as Moderate findings, or as never-run lines that the coverage map listed. review-pr now splits the ways to reach a change. The main path is the behavior the PR adds or fixes, reached the way its users routinely reach it. It includes the trigger of a fixed bug and the main path's own error and cleanup paths, so a missing test there is still a gap. Every other path is exotic: a non-default setting or table type, a rarely used operation or its edge cases, a less common route into the change, or an unusual combination of features. The reviewer probes an exotic path where the change may behave differently. A failing probe is a functional finding, admitted and rated like any other bug. Otherwise the row is EXOTIC: no finding, and the report asks for no test. The rule reaches every place that turned a missing test into a finding: the Step 2.6 classification and probe rule, Agent 5's cross-context check and mutation spot-check, the execution-mode and cross-context checklists, the Step 3b admission of coverage gaps and the Moderate definition. The coverage map still lists never-run lines, as a record that asks for no test. The Claude Code and Codex copies of the skill get the same rule.
PR bodies often pinned a tandem PR's head or a submodule pointer by commit hash, or counted the tests a change added. Both go stale as soon as the branch moves, and reviews then reported the stale body; the Claude Code and Codex copies of review-pr listed it as a Moderate finding. CLAUDE.md also banned design narrative, so a body said what users gain but not how the change gets there. CLAUDE.md now asks for effect bullets, then the approach: one to three sentences per fix or feature on the root cause or design and how the change addresses it. Bodies link tandem PRs by number and leave out commit hashes and counts of tests, files, lines or commits. review-pr checks that format and asks for hashes and counts to go, without checking them against the code. Its report opens with the revisions it reviewed, a tandem PR's head included. The Claude Code and Codex copies get pi's Step 2, its Title and description section, which keeps body problems out of the findings, and its Agent 7, which leaves the body to Step 2. fix-pr's analysis reports each item's approach next to its effect bullet, and the PR body that fix-pr prints carries both.
A person can push back on a review item in a PR comment, for example to explain why they won't fix it. review-pr ignored such replies: Step 2.5f re-tested every earlier finding and re-raised each one that still reproduced, fresh discovery agents could find the same issue again, and the Claude Code and Codex copies had no rule for earlier reviews at all. A follow-up review on #7263 re-raised as Critical an item its author had already declined. Replies in review threads on the diff never reached the review either, since gh pr view --comments leaves them out. An item a person pushed back on is now settled: a reply declines it, or disputes it with a reason. The review never re-tests a settled item and never raises it again in any section of the report, even when it still reproduces. Admission omits a candidate that restates it. Only a defect with another symptom, or one that a later commit makes worse than the reply answered, counts as new. Step 1 also reads review-thread comments. Discovery agents get the settled items with the surface map and must not return them, Step 3b gains an OMITTED settled outcome, Step 2 leaves out declined title and body problems, and the out-of-scope list names settled items. The Claude Code and Codex copies get a 2.5f that runs at every level.
A QWP connection's commit batch now holds plain frames of any tables from one receive pass, interleaved in any order: each table commits once when the pass drains, instead of a table switch committing the batch. Frames with several table blocks join too. The batch commits its tables one by one, in the order of their first frames, and stops at the first that fails. The ack covers the frames before the failed table's first frame, which hold rows of committed tables only; the NACK names that frame. Rows that later frames added to committed tables are in the WAL and arrive again on the client's replay: a partial failure duplicates, it never loses. A frame that appended rows next to the batch's and then fails, or then meets a block that cannot join, leaves rows no writer can drop alone. The processor then rolls the batch back and redoes its frames one at a time from the receive buffer, each committed on its own, and answers the frame as it would be answered without a batch. This replaces the salvage branches: a frame that fails before touching any writer is a boundary, and the batch commits before its answer. A frame that redefines symbol dictionary ids can only open a batch: redoing the frames before it would read the new strings. Each table now keeps its own schema parse (QwpTableBlockCursor, keyed by table name, bounded by the per-connection table limit) and its own Phase 1 column mapping (on its WalTableUpdateDetails). The joinability check is a cache hit, so the single-entry caches made every table switch a batch boundary and every frame allocate. QwpIngressMultiTableBenchmark measures one connection's ingestion across 1-100 interleaved tables, with a raw pre-encoded client that measures the server and with the Java client. Tests that pinned single-table batching now pin the new contract; the ack contract fuzz writes three tables, frames of two tables and frames whose second block fails, and accepts the partial-failure duplicates only where they are allowed.
The ack contract fuzz writes frames of two tables, so it now reaches a group-closing frame of two tables whose own commitAll commits one table and fails on the other, as on master. Its I2 invariant accepted such duplicates only when the NACK named the frame after the ack; a rowless deferred frame between them holds the ack lower. The invariant now asks for any NACK above the ack, or a CLOSE, and takes the first frame with rows after the ack as the one whose failed table committed nothing. QwpIngressOracleFuzzTest gains a multi-table variant: producers cycle their rows through three DEDUP tables while the server bounces, so the real client's replays meet batches that span tables, and every table must end up with exactly its rows. A new test drives the live read-only gate's check before a frame opens a batch, which only the fuzz reached so far. The redo flag is set as the last statement that prepares the redo, so the catch-all needs not reset it, and the block cursor counts every cached schema section as retained, as an entry always holds its bytes.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
|
Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run. |
Since #7917, level-3 reviews take about four times as long, and they raise findings that collapse as soon as a developer challenges them. On this machine, 63 level-3 reviews in October took a median of 45 minutes; five on Nov 2 took 113 to 194. The main model's own time did not change: the rest was waiting on agents. #7917 gave discovery agents a shell, max thinking and two hours, so they proved what they suggested by building, starting servers and running stress loops, and falsifiers then proved it again. Discovery goes back to the builtin reviewer, which reads code at high thinking within the 30-minute default. Only falsifiers run code, with a one-hour cap and a 30-minute budget in their task. Earlier findings and slowdowns the PR body states go to falsifiers as candidates, since discovery no longer runs anything. The risk round, meant to let unlikely defects ship, escalated instead: the assessor took "the worst consequence the defect can reach", and the parent could "never make a rating rarer or milder". A proved 160 MB allocation during restore became an out-of-memory failure at startup and a Critical on #8166, until a challenge showed that peak memory did not change. The assessor now rates the consequence the evidence shows, and first tests the finding against the author's strongest reply, such as a cost the server already pays in normal operation; a reply that holds removes the finding. The parent corrects ratings in both directions, and a Blocking finding shows the reply it answers. fix-pr drops a bug whose reply holds and lists it for the user.
Review (level 1)Verdict: request changes, for one blocking finding. The raw-replay speedups reproduce: multi-table one-row frames ingest 5.7-8.4x faster, and no listed case got slower. With 100_000 alternating one-row frames, each table gets 14 WAL transactions instead of 50_000. Title and description
BlockingB1. The per-table schema cache keeps parsed schemas after disconnect, up to 32,768 per decoder
ModerateM1. The new oracle fuzz test times out for some seeds
M2. The same test never exercises a failed batch commit, despite its Javadoc
Minor
Coverage
Summary
|
Review finding B1: the per-table schema cache in QwpTableBlockCursor outlived its tables. Utf8SequenceObjHashMap.clear() leaves the values array populated, so neither the reset at capacity nor the release after a parse error freed anything, a disconnect left the cache alone on the pooled connection state, and the UDP receiver's cursor lives as long as the server. Up to 32,768 parses could stay reachable per decoder. A table's last schema parse now lives on its WalTableUpdateDetails, like its Phase 1 column mapping. QwpTudCache implements a new QwpSchemaCache for the block cursor, so a parse is kept only while the connection holds its table open, is bounded by the per-connection table limit like the table's writer, and goes when the entry is freed: on disconnect, on eviction, or when the cache distresses. Every spelling of a table name shares one parse. A closed entry drops its QWP caches, since the table map's clear() keeps freed entries reachable too. The block cursor keeps no parse itself; without a cache it parses every block. A table's first block on a connection parses before its entry exists. The appender then leaves that parse in the new entry, so the table's next block finds the object its column mapping was resolved for and keeps joining the commit batch. The schema lookup and the commit batch lookup ask the table cache for the same table one after the other, so the cache checks the entry it found last before hashing; the batch's own last-joined shortcut goes. QwpParserCacheRetentionTest pinned that the block cursor retains the schema section; it now pins that the table's entry does and the cursor does not. Also from the review: - M1: the multi-table oracle fuzz runs without fsync, which made a seed outlast the client's drain timeout on disk. - M2: its Javadoc no longer claims to cover a batch commit that fails partway; nothing in it fails a commit. - A misplaced Javadoc in QwpIngressCommitBatchTest moves back to its method. - A new test covers tables whose batch position is stale from an earlier pass, or that a rowless block opened.
An OK ack names every table its frames committed, and a durable ack every table whose frontier moved. With a lowered http.send.buffer.size either frame could outgrow the send buffer, and trySendAck and trySendDurableAckFrame then dropped the connection after the rows had committed. The client replayed them on reconnect, the next pass failed the same way, and tables without DEDUP kept gaining copies. The commit batch made this easy to reach: one ack now covers every table a receive pass wrote, not the tables of at most 64 frames. Base already failed the same way for a single frame carrying enough tables. getFrameAddress now writes such a frame to staging memory on the connection state, and sendStagedFrame hands it to the send buffer one buffer at a time. A blocked send parks the send state as before; the send buffer keeps the unsent part of its piece and the rest stays staged. resumeSend and drainPendingResponse finish the staged frame before their per-state continuation, so no other frame can interleave with it. The frame bytes on the wire are unchanged, so clients see the same single ack. The staging memory grows to the largest frame and is freed when the connection ends; if it cannot be allocated, the connection drops as before. QwpIngressUpgradeProcessorResumeRecvTest#testAckBufferTooSmall asserted the old disconnect for an ack larger than a 10-byte buffer. It now asserts that the ack goes out in pieces of 10 and 3 bytes, which is the behavior this change introduces. QwpIngressOversizeAckTest drives the pinned client against a live server with a 1 KiB send buffer, for OK acks of one frame and of many frames per pass, and for local durable acks. QwpIngressStagedAckFrameTest pins piece sizes around the buffer size, resumption after blocked pieces, an error queued behind a staged ack, teardown, a peer lost mid-frame, multi-byte table names and allocation failure. The multi-table oracle fuzz now uses up to 26 tables and lowers the send buffer in half of its runs.
Rename getFrameAddress to reserveFrame: it allocates staging memory, and sendFrame relies on that, which its javadoc now says. Drop a reset in stageFrame that its precondition already guarantees, split its assert so the message matches the condition, and stop restating the staging memory's lifetime in three places. In the tests, Connection takes its frames up front and lets PhasedNetworkFacade deliver them instead of overriding it, stageAck picks name lengths in one expression, the frame-size test checks reuse of the staging memory for every frame no larger than one already staged instead of two hard-coded sizes, and RestartableQwpServer no longer exposes a -1 sentinel named like a buffer size.
Master's #7936 returns a quiet connection's idle WAL writers to the pool on a keepalive PING: QwpTudCache.releaseIdleWriters() removes and frees every cache entry without uncommitted rows. This branch added lastCachedTud, the entry getCachedTableUpdateDetails() checks before the hash lookup, and treats an entry without a writer as freed. The textual conflicts kept both sides: the distressed clear() and reset() forget lastCachedTud and the released tables; the processor imports both Vect and MicrosecondClock; and QwpIngressCommitBatchTest drops its own readLeShort and okAckSeqTxn, which master moved into QwpIngressWireHarness, and keeps this branch's symbolFrame. One conflict had no markers. releaseIdleWriters() swallows a failing writer close, such as the close of a table dropped while the connection idled, and that failure leaves the writer set on the released entry. lastCachedTud then kept serving it: a second frame for the re-created table found no batch entry for the stale TUD, took the table as outside the batch while its new writer held the first frame's rows, and the frame was refused. releaseIdleWriters() now forgets lastCachedTud when it releases that entry. QwpIngressIdleWriterReleaseTest#testIdleReleaseFailureDoesNotServeReleasedEntry covers it and fails without the reset; the failing-msync facade it shares with testIdleReleaseSurvivesWriterCloseFailure moves into a helper.
getCachedTableUpdateDetails() checks lastCachedTud before the hash lookup and took an entry without a writer as freed. A free that fails leaves the writer set: evictStaleTud() swallows that failure, and the failed-lookup path in getTableUpdateDetails() only rethrows it without distressing the cache. The shortcut then kept serving the removed entry for its name. When a stale entry's writer failed to close, the next two frames of the re-created table in one pass found no batch entry for the stale TUD; the second took the table as outside the batch while its new writer held the first frame's rows, and the frame was refused. Every site that removes an entry now calls onEntryRemoved(), which updates the table count and forgets lastCachedTud, so lastCachedTud is always null or a mapped entry and the writer check goes. QwpIngressCommitBatchTest#testRecreatedTableAfterFailedEvictionSharesOneBatch reproduces the refusal through a failing close of the evicted writer. QwpIngressProcessorStateTest#testRemovedEntryIsNotServedAgain drives each of the seven removal paths with the removed entry found last and with another entry found last; reverting any single call site to the old count update fails it.
A review names its level and its model, but not the version of the review-pr skill that wrote it. The skill changes often, so a reader cannot tell which rules produced a posted review, or compare the reviews that different versions of the skill write. Each copy of review-pr (pi, Claude Code and Codex) now states its version under its title, starting at 4.0, and asks for it on the opening line, after the level and the model. Step 4 makes that line the first line of the report, so the console output, fix-pr's report.md and the reviews that approve-pr and reject-pr post all carry it. The version sits in the body, not in the frontmatter: pi strips the frontmatter when it expands /skill:review-pr, so the model would never see it there. Each PR that changes a copy raises its version.
Review (level 3)Reviewed at level 3 with anthropic/claude-opus-5-5, thinking xhigh. Revisions: head Verdict: request changes, for one blocking finding (B1): when one table's commit fails, the rows of the other tables in that receive pass are stored twice. Title and description
Suggested body, assuming B1 ends as a recorded, sized tradeoff: BlockingB1. A failed table commit duplicates the other tables' rows for the rest of the receive pass
ModerateM1. The redo's per-frame record keeps its peak size for the life of the connection context
M2. The zero-allocation test doesn't catch a 16-byte allocation per frame
M3. A writer close that throws leaves the table's schema parse reachable
Minor
Adjacent (not blocking, already on base)
Coverage
Summary
|
A QWP table entry whose writer close threw kept its column mapping and its schema slot with the parse. WalTableUpdateDetails.close() cleared the two fields only after super.close() returned, and closing the writer of a dropped or distressed table does file I/O (rollback, msync) that can throw out of TableUpdateDetails.closeNoLock(). Every QWP free path goes through this close(), and a freed entry can stay reachable, for example from a table map that does not clear its values. close() now clears the fields in a finally around super.close(). The writer still closes first, and its failure reaches the caller unchanged: reset(), clear(), releaseIdleWriters() and the UDP commit loops still log it, commitAll() and commitIfMaxUncommittedRowsReached() still propagate it, and a failed lookup still adds it to its own failure as suppressed. The new QwpIngressProcessorStateTest test testEntryFreedByFailingWriterCloseKeepsNoQwpCache frees an entry through each of the nine free paths while msync fails. It asserts the failure each path reports, that the freed entry holds no QWP cache, and that the same cache then writes and commits the re-created table.
The QWP and ILP/HTTP table caches free every table entry when a connection ends or a cache is distressed, then clear their table map. LowerCaseUtf8SequenceObjHashMap.clear() blanked only the keys and the key list, so its value array kept every freed entry reachable, with the closed writer of an entry whose writer close threw, until a later table name took the same slot or the map grew. Both caches live in the pooled HTTP connection context, so the heap kept the entries of past connections. clear() now also fills the value array with null, as erase() does for one removal and as IntObjHashMap.clear() and LowerCaseCharSequenceObjHashMap.clear() already do. The fill allocates nothing and keeps the arrays. No lookup reads a value through a blank key, so every lookup returns what it returned before. LowerCaseUtf8SequenceObjHashMapTest#testClearReleasesValues asserts, for maps of 0 to 10,000 keys, after removals and on reuse, that no slot that held a key still holds a value after clear(). testClearAllocatesNothing asserts that clearing a map grown to 10,000 keys allocates nothing. testResetAndDistressedClearReleaseFreedEntries, in QwpIngressProcessorStateTest and in the new LineHttpTudCacheTest, frees 17 entries through reset() and through the distressed clear(), then asserts through weak references and full collections that none stays reachable. The QWP test also fails every writer close and asserts that the closed writers do not stay reachable either.
testInterleavedTablesAllocateNothingOnceWarm measured 3,000 frames against a 64 KiB bound, which leaves 21.8 bytes per frame. The smallest Java object takes 16 bytes, so a frame path that allocated one object per frame still passed: with a new Object() stored on every joinCommitBatch() call, the window allocated 55,392 bytes. The bound has to absorb one-off JIT work that HotSpot charges to the test thread when it queues a method for C2, 5 to 21 KB in the runs measured, and that work does not grow with the frame count. A per-frame allocation does, so the test now measures 30,000 frames against the same bound: one 16-byte object per frame adds 480,000 bytes. The warm-up grows to 3,000 frames and the batch commits every 300 frames instead of every 30, so the window keeps its 100 batch commits and the test makes no more WAL commits than before. Under the ADAPTIVE commit mode each WAL commit fdatasyncs, so the commit count sets the test's duration: about 2.0 s, against 1.6 s before. Each table now ends with 11,000 rows. Mutations show the test fails: a new Object() on every joinCommitBatch() call fails it at 487,344 bytes, on every prepareMessage() call at 486,504 bytes, and a 1,000-byte array on every commitBatch() call at 107,616 bytes. Unmutated, the window allocates 5,680 bytes under ADAPTIVE and 8,344 bytes under NOSYNC, and 19,192 and 21,080 bytes under the JaCoCo agent.
Two earlier commits on this branch each added a test, with its Javadoc, between an existing comment and the test that comment describes, which left each comment above the wrong test. In QwpIngressCommitBatchTest the Javadoc of testRedefiningSymbolDictionaryEndsCommitBatch sat stacked on top of the Javadoc of testRecreatedTableAfterFailedEvictionSharesOneBatch. It now sits directly above its own test, which had no comment. In QwpIngressProcessorStateTest the comment of testZeroRowDeferredFrameIsNeverAckCovered sat above the Javadoc of testWideSchemaFramesJoinCommitBatch. It now sits directly above its own test again, as on the base branch. The move keeps the wording of both comments and changes no code.
When a frame failed or could not join after a run of batched frames, the server rolled the batch back and redid its frames one at a time, committing each as its own WAL transaction. A pass of K one-row frames followed by one bad frame turned one commit into K: at K=5,000 the I/O worker stalled for 31 s under the sync commit mode (base: 0.06 s). A first block that failed without writing a row took the same redo, and an unexpected error looking up a table new to the batch distressed the table cache and refused the whole batch at its first, good frame. joinCommitBatch() now answers BOUNDARY exactly when the batch can commit as it stands: no row of the frame sits in a writer, the table cache is healthy, no batch writer is left mid columnar write, and the frame's first block is still prepared. Every other frame that fails or cannot join is a REDO. A frame that failed in its preparation is classified the same way, and the redefined-symbol-dictionary check runs first, so a redo never re-decodes frames against redefined ids. redoCommitBatch() now joins the batch's frames again and commits each table once; a frame that no longer joins ends that batch, commits what joined, and goes on its own, and the frames after it start a new batch. prepareMessage() gains a documented test hook, onSymbolDictApplied(). Existing expectations that change, because the redo now commits each table once instead of once per frame: testRedoAnswersFrameThatFailed- WhileJoining and testMissingTableAfterAppendRedoesBatch expect a at aBefore + 1 instead of aBefore + 2, and tests that expected the batch's first frame to be NACKed after an unexpected lookup error now expect the batch to commit and the failing frame to be NACKed, as on base. Tests: QwpIngressCommitBatchTest covers failing first and later blocks of every stage (lookup, Phase 1, append) for LONG, timestamp, VARCHAR, STRING, BINARY, DOUBLE[] and SYMBOL columns; distress and columnar- write faults; a real ALTER, DROP and demote during the redo; durable ack seqTxns; sync and nosync; the commit-batch ack-contract fuzz gains an invariant that bounds each table's txn count. QwpIngressProcessor- StateTest, QwpAckSeqTxnCoverageBlackBoxTest, QwpIngressUpgradeProcessor- ResumeRecvTest and QwpIngressOracleFuzzTest are adjusted to match.
QuestDB commits each table separately, so a commit batch that held rows of several tables could commit some of them and then fail on another. The ack then stopped before the failed table's first frame, and the client's replay stored the committed tables' later rows a second time: for 1,000 alternating frame pairs of a and b, 999 duplicate rows in a, against none on base. A concurrent DROP TABLE of a batch table, a demote landing between two tables' commits, or a refused insert permission all produced it. commitBatch() now takes the role-switch read lock once and holds it across every table's commit, so a demote can no longer fall between two tables. Before the first commit it checks every table the way a commit would refuse it: the node is writable, no table was dropped (the table's sequencer txn tracker), and the table's insert permission holds (TableUpdateDetails.authorizeCommit(), now public). Any refusal there refuses the whole batch with nothing committed, and the NACK names the batch's first frame, so the replay sends no row twice. What can still fail between two tables is a commit that hits a storage fault; that case keeps the existing partial-commit handling. Changed expectations, because these triggers now refuse the whole batch: testDroppedTableStopsBatchCommitAtIt becomes testDroppedTableRefusesWholeBatch (a commits nothing, NACK at frame 0), and testDemoteAtSecondTableCommitAcksFirstTableBeforeClose becomes testDemoteAtBatchCommitCommitsNoTable (only the CLOSE, no table commits). The tests of the remaining partial-commit path (testCommitFailureOnLaterTableAcksFramesBeforeItsFirstFrame, assertCommitFailureAtTable, QwpIngressUpgradeProcessorResumeRecvTest's testBoundaryFailureResumesPrefixAckBeforeErrorWithoutAppendingTail) now fail the table's own commit after the batch's checks passed, standing in for a storage fault, and keep their expectations. denyCommitsAfter- First now denies once the table has committed a transaction, instead of counting insert checks, which a batch now runs twice per table. New: testPermissionRefusalOfAnyTableCommitsNoTable refuses each of three tables in turn. Without the checks, it and testDroppedTableRefusesWholeBatch fail; without the lock held across the loop, testDemoteAtBatchCommitCommitsNoTable fails.
A commit batch keeps three longs per frame so that a redo can find the batch's frames in the receive buffer again. The record grows with the largest receive pass a connection processes: a 2 MiB pass of one-row frames from the Java client takes 1 MiB of heap, of 18-byte frames 4 MiB. LongList.clear() keeps the capacity, and the record lives on the connection's QwpIngressProcessorState, which the pooled HTTP connection context keeps after the connection ends. The upgrade processor's copy for a redo kept its size on the worker's processor the same way. onDisconnected() now restores the record's initial capacity, three longs for 64 frames, when it grew past it, and redoCommitBatch() restores its copy once the redo ends. A live connection still holds the record at the size of its largest pass, as it holds its receive buffer. Tests: testFrameRecordShrinksWhenConnectionEnds grows the record with a 300-frame batch, checks onDisconnected() shrinks it, and that the pooled state then takes a new connection; testFullReceivePassRedo- CommitsEachTableOnce now checks the processor's copy shrinks after a 24,000-frame redo. Both fail without the trims (1,536 and 72,000 longs).
commitBatch() skips the role-switch lock and the batch checks when the batch holds no table: a run of frames that name no table. No test sent only such frames, so that branch never ran. The new test sends three zero-table frames in one receive pass and checks that a single ack covers all three and no frame gets an error response.
When a frame cannot join the commit batch, handleBinaryMessage commits the batch first. If a table that joined after the batch's first frame fails its commit there, the NACK must name that table's first frame, where the client's replay starts, and the ack must stop just before it. No deterministic test reached that line with the failed table being other than the batch's first: the existing partial-failure tests commit at the end of the receive pass, through commitPendingBatch. testBoundaryCommitFailureOnLaterTableNacksItsFirstFrame sends a, b, a and a frame that needs a new column on a, and makes b's commit fail after a's. It fails when the NACK names the batch's first frame. The commit also moves the javadoc of testClientClosePipelinedBeforeRoleChangeCloseIsNotTheEcho back onto that test; it had ended up above testCaseVariantBlocksWithDifferentColumns- JoinCommitBatch.
The javadoc of assertBatchLostToReplacement had ended up on the new assertLookupEvictsDistressedWriter, and the SameParameterValue suppression of assertEmptySymbolDiff on the new assertCommitRefusalCheckMatchesCommit, when the new helpers were inserted above them.
A commit batch held the role-switch read lock across every table's commit, and checked every table's sequencer before the first commit. A demote then waited for the whole batch, 2.7 s for 1,000 tables under adaptive commit, while every other write on the node waited behind it. Every batch also paid two sequencer lookups and lock round trips per table, for refusals that are rare. commitBatch() now takes the lock once and reads the node's role once under it. Between two tables' commits it hands the lock to a pending role switch and reads the role again after it. A demote now waits for one table's commit. One that lands mid-batch stops the commits as a storage fault does: the replay can duplicate rows of the tables that committed, bounded by one receive pass. The tables commit through the new TableUpdateDetails.commitUnderRoleSwitchFence(), which neither re-enters the lock nor reads the role again. The pre-commit refusal loop is gone: a DROP, RENAME, SUSPEND WAL or REVOKE that lands while a table waits in the batch fails that table's commit instead. The check as a table joins reads the table's cached sequencer tracker and a rename flag that the cache lookup sets, instead of opening the sequencer, so TableSequencerAPI.throwIfCommitRefused() goes away. The batch's integrity assert checks the entry's slot, which closing an entry clears, instead of a hash lookup per table.
A commit batch handed the role-switch read lock to a pending role switch between two tables' commits. A demote that landed there left the earlier tables committed, while frames past the next table's first frame still carried their rows: alternating and multi-table frames put a table's rows on both sides of that frame. The ack stopped before it, and the client's replay stored those rows a second time. With N alternating one-row frames of two tables, N - 1 rows of the first table came back twice, where master stores none twice. commitBatchEntries() now holds the read lock across every table's commit and reads the node's role once, so a role switch lands before the batch or after it, never between two tables. A read-only node still refuses the whole batch with nothing committed. A demote now waits for each connection's batch in flight instead of one table's commit. CairoEngine.hasPendingRoleSwitchWriter() and RoleSwitchReadWriteLock.hasPendingWriter() lose their only caller and go away. The authorizeCommit() Javadoc no longer mentions the pre-commit check of every table, which an earlier commit removed. RoleSwitchingEngine in QwpIngressCommitBatchTest queues a real role switch on the write lock as a batch reads the node's role, before any table commits. Three tests check, for interleaved and multi-table frames, that the switch gets the lock only after the last table, that one ack covers the batch, and that after a demote the replay of every frame above the ack stores each row once. They replace the tests of the hand-over.
|
Reviewing PR #8195 at level 3 with anthropic/claude-opus-5-5, thinking xhigh, review-pr 4.0. Verdict: approve. Nothing blocks. The body needs one bullet shortened. Title and description
BlockingNone. ModerateNone. MinorNone. Adjacent findingsNone. Coverage map
Summary
|
The PR's tests had grown to about 7,200 net lines, much of it fixture boilerplate repeated per test and multi-paragraph Javadoc that restated each scenario. - Each test Javadoc the PR added becomes one line, or goes where the test name says it all. The fuzz test keeps its Javadoc: it defines the invariants I1-I4 that its assertion messages name. - QwpIngressCommitBatchTest gains an httpConfig field, createTables() and ingestInOnePass() overloads that build the processor. They replace 79 local default configs, 54 CREATE TABLE lines and 36 processor arguments. - Four pairs of near-identical tests share a helper each: symbol redefinition, the every-column-type rewind, the later table's commit failure, and a table change between passes that still joins. No test and no assertion goes. Two merged tests gain the checks of their twin. The 685 tests of the 16 affected classes pass, and the PR's production lines keep their coverage: 454/454 lines and 205/208 branches.
|
Reviewing PR #8195 at level 3 with anthropic/claude-opus-5-5, thinking xhigh, review-pr 4.0. Verdict: approve. Nothing blocks. One body bullet is too long, and there are three Minor items. Title and description
BlockingNone. ModerateNone. Minor
Adjacent findingsNone. Coverage map
Summary
|
…commit_batch # Conflicts: # java-questdb-client
handleBinaryMessage read the error text of every frame that left a pending commit batch, allocating a String even for a frame that did not fail and so has no error. It now reads the text only for a failed frame, the only case that uses it. QwpIngressCommitBatchTest moves V_100_TO_109 into the static field block and puts the Javadoc of assertTableChangeBetweenPassesNacks- ItsFirstFrame back on that method, from assertSymbolRedefinition- CommitsOnce.
CI failed two checks on this branch.
WalSequencingCallSiteTest scans the main sources for ".nextTxn(" and
allows it only in TableSequencerAPI and WalWriterBase. A comment in
QwpIngressProcessorState.throwIfJoinRefused() named
TableSequencerImpl.nextTxn() that way; it now reads "TableSequencerImpl's
nextTxn()".
The formatting check runs the IntelliJ formatter over the changed files
and fails on any change. This commit applies its output to the test
files: one-line Javadocs become three lines, and one long case arm
wraps. Nothing else changes.
testCommitBatchAckContractFuzz failed in CI under nosync: a frame that closes a deferred group commits its tables on the normal path, and a demote that lands at its second table's commit leaves the first table's rows above the ack. A durable-ack connection then defers the role-change CLOSE until that work is durable, at most for the upload grace, so the pass sends no NACK and no CLOSE, and the fuzz oracle's I2 check reported that the client is never told to replay. Under adaptive the work is durable at once and the CLOSE goes out in the pass. The server refuses every frame while the close is deferred, so the deferred close tells the client too. ingestInOnePass now reports whether the pass left the close deferred, and I2 counts that. testDemoteBetweenTablesOfGroupClosingFrame* pin the shape: the first table commits, the second does not, and the server sends only the CLOSE, or, on a durable-ack connection, nothing yet with the close deferred.
|
Reviewing PR #8195 at level 3 with anthropic/claude-opus-5-5, thinking xhigh, review-pr 4.0. Verdict: approve with comments. Nothing blocks. One Moderate item: a role switch now waits for the whole batch commit, and the body doesn't say so or how long it takes. Title and description
BlockingNone. ModerateM1. A role switch waits for every table commit of a QWP batch
MinorNone. Adjacent findingsNone. Coverage map
Summary
|
|
Correction to the review above: M1's sync and adaptive numbers were inflated. The mechanism and the verdict stand. The falsifier measured them with load average 7-11 from parallel builds, and on the first synced pass after a no-fsync setup, which pays a one-time flush. Re-measured on an idle machine (Samsung 990 PRO, consumer NVMe, ext4): one-row WAL commits to 300 distinct tables, one per table, which is what the batch commits under the role-switch lock.
What changes in M1:
M1 stays Moderate. It needs an explicit sync or adaptive commit mode with replication (Enterprise defaults to nosync when replication is configured), hundreds of distinct tables per receive pass, and a demote at that moment. The refusal is clean and can be retried with a larger |
WalWriter.prepareCommit() runs the part of a plain commit that comes before sequencing: it appends the DATA event record and flushes the writer's column, event and segment metadata files at the commit mode's grade, including what nextSeqTxn() would flush first. The commit() that follows only sequences the record, so a caller can do the file syncs outside a lock and hold the lock only for the sequencing. A QWP commit batch uses it in the next commit. commit0() is split into appendAndFlushCommit() and sequenceCommit(); the plain path keeps head's behaviour and catch blocks. A prepared commit sequences through the same helper, with a row guard first, and any failure once the sequencer is asked distresses the writer. Under a group window the due group flush is left to flushDueGroupCommit(), which the caller runs once it holds no lock; cleanupBeforeClose() clears that flag on every path. A prepared record is never withdrawn. rollback() and close abandon it: they distress the writer and decrement lastSegmentTxn once, on the transition of isCommitPrepared, so the segment's closure reports only sequenced records. A pool tenant's close runs cleanupBeforeClose() twice and the REVOKE path runs rollback() and then a close; neither decrements twice. A prepare that fails after its append restores lastSegmentTxn for the same reason. DeferredCommitInterceptor gets a default onSequencedBeforeDurablePublish hook for the W=0 publish re-check, read from the same once-per-commit local. Methods of WalWriter audited for the prepared guard, which refuse a prepared writer with a critical error and leave it prepared: - guarded: prepareCommit, markSavepoint, rollbackToSavepoint, addColumn (8-arg), apply(AlterOperation, boolean), apply(UpdateOperation), truncateSoft, rollSegment, commitMatView, commitLiveView, commitLiveViewWithReplaceRange, commitWithParams, commitWithParamsIfSeqTxn, appendCustomEvent, resetMatViewState, commitRebaseSeed, goActive(long), and WalColumnarRowAppender.beginColumnarWrite through the writer; - reached only through a guarded one: addColumn (3-arg) and renameTable (apply), goActive() (goActive(long)), the columnar finish, cancel and server-timestamp methods (beginColumnarWrite), newRow's segment roll (rollSegment; never pending while rows are uncommitted); - dispatch to the prepared sequencing: commit, ic, ic(long); - abandon the prepared commit: rollback, close and cleanupBeforeClose; - left unguarded on purpose: the inner MetadataWriterService and MetadataValidatorService and applyMetadataChangeLog, which the NO_TXN replay of a concurrent DDL runs on a prepared writer; truncate() always throws; commit0 asserts it never sees a prepared writer. WalWriterPreparedCommitTest covers the lifecycle under SYNC, ADAPTIVE with and without a group window, NOSYNC and ASYNC: equality with a plain commit, abandon by rollback and by close, segment closure accounting, refusals, the row guard, faults before and after the assignment, concurrent DDL replay, segment rollover, purge and the deferred group flush. WalFlushCountingFacade counts WAL file flushes and whether they ran under a role-switch lock.
Under SYNC and ADAPTIVE, each table of a QWP commit batch fsynced its private WAL files under the role-switch read lock, so a demote waited for every table's file syncs: seconds for 1,000 tables. The batch now prepares every table first (WalWriter.prepareCommit), outside the lock, and holds the lock only for the role check and the sequencing. The batch reads the configured commit mode, not the resolved one; NOSYNC and ASYNC keep the single-phase commit. A prepare failure refuses the whole batch with nothing committed, at the batch's first frame, with the text "commit batch prepare failed [table=<name>]: <message>". Head committed the tables before the failed one and the replay stored their rows twice. After a demote has published REPLICA the refusal is the role-change close, read from the live role on this branch only; a wrong answer there yields a reconnect-eligible close, never a terminal error. A group flush that falls due while the tables sequence runs after the unlock. Its failure never fails the batch: the rows are sequenced, the writer is distressed, and the table's next frame is refused once. The abandoned paths need no cleanup under the lock: the prepared writers are distressed, and the clear() that follows every failure exit discards them. QwpTudCache.clear() asserts that no prepared writer survives a healthy clear.
resumeSend drains buffered frames through the frame loop, which can commit a batch. An unexpected throwable there escaped to the HTTP dispatcher and left the connection's WAL writers checked out, among them the prepared writers of an abandoned batch. resumeSend now logs it and disconnects, as resumeRecv does, so onConnectionClosed frees the writers. The three checked exceptions it declares pass through unchanged.
QwpIngressCommitBatchTest pins the commit mode and group window of its hook engines (pinnedConfiguration, DemotableEngine(configuration)), which DefaultTestCairoConfiguration otherwise takes from the CI shard, and adds tests of the prepare phase: it runs only under SYNC and ADAPTIVE; a demote after it or during it commits nothing and discards the prepared writers; a REVOKE abandons its table under the lock with no file work; a prepare failure at the first, middle or last table refuses the whole batch at its first frame with the truncated text, and the replay stores every row once; after a demote it is the role-change close; a fatal one poisons the engine; the due group flush runs after the unlock, and its failure keeps the ack and refuses the table's next frame once; an unexpected error after the prepare phase loses the batch at the boundary and disconnects on the receive and the send path. A flush count test shows that a batch flushes no writer-private file under the lock, and as many files per pass as plain commits of the same rows. The ack-contract fuzz runs per commit mode and draws prepare faults from a Rnd of its own. QwpCommitBatchPrepareRealClientTest drives the real client through a demote after the prepare phase, with and without durable ack, and a prepare fault: the client replays and every row is stored once. QwpCommitBatchPrepareCrashTest takes a power loss at a table's prepare, between the prepare phase and the lock, after the first table sequenced and inside the deferred group flush, under SYNC and ADAPTIVE with and without a window. AdaptiveQwpCommitBatchCrashSweepTest sweeps every durability op of two batches in the nightly job. FuzzRunner can send a share of the WAL commits and rollbacks through prepareCommit(); WalWriterPreparedCommitFuzzTest runs the WAL fuzz so and pins that it sequences and abandons prepared commits. QwpCommitBatchLockHoldMeasurementTest, gated by -Dquestdb.measure.e1=true, measures the lock hold, the refusal path and the discard cost, with nothing the base lacks, so it measures the base too.
commitPendingBatch's Javadoc names every exit through which a failed commit batch frees its prepared WAL writers, none of them under the role-switch lock: the abandon and clear() of a failure return, the frame loop's abandon and the disconnect of resumeRecv or resumeSend for an unexpected throw, and handleBinaryMessage's catch and finally for its boundary commit. testFatalDueGroupFlushFailurePoisonsEngineAndKeepsTheAck: a data-sync failure of the group flush that runs after the unlock stops the engine, the sequenced rows keep their ack, and the durable frontier stays behind them.
QwpIngressMultiTableBenchmark connected to fixed ports 9000 and 8812, so it could only measure a server on those ports. -Dqdb.bench.http.port and -Dqdb.bench.pg.port now name others, with the old ports as defaults. The Javadoc now says how to measure each commit mode: one server per mode, configured with cairo.commit.mode and cairo.adaptive.commit.group.window, since the benchmark cannot set them; and a root on a real disk for SYNC and ADAPTIVE, whose cost is the device flush that tmpfs skips.
WalWriterPreparedCommitTest covers what the coverage run left out: a prepare that fails flushing the new segment's _meta, which sequencing would otherwise flush, with and without a data-sync failure, does not count its record at close; a data-sync failure of the sequencer-log flush while a prepared commit sequences stops the engine and distresses the writer; and the plain path keeps its behaviour through the helpers commit0 is now split into: the fence rejection, replace ranges with and without rows, and the commits without rows.
Five cases of three repeats each overran the 20-minute test timeout in one method. Each case now has its own test method, so a run of one fits it and can be repeated alone when the host gets busy.
AdaptiveQwpCommitBatchCrashSweepTest checked only upper bounds on the rows each crash point recovers. It stayed green when the prepare path let a sequencer record become durable ahead of its WAL column data: recovery suspended the tables and they read back empty, which every upper bound accepts. The oracle now requires each table to come back unsuspended, holding exactly the rows of the transactions the sequencer recorded. The sweep also requires that the recovered rows never drop as the crash point moves later, that the first crash point loses rows and that the last recovers every row, as the sibling sweeps do. Dropping the column sync from WalWriter.prepareCommit() now fails both methods at k=16, where the table is suspended. Skipping the sequencer log fdatasync in the deferred group flush fails the group-window method at the last crash point, which recovers no rows. Both regressions left the old sweep green.
The ack-contract fuzz draws prepare faults that often never fire, yet it treated every iteration with a drawn fault as commit-faulty. That switched off the I2 check that no row above the ack is committed. NOSYNC and ASYNC never prepare, so every fault drawn there was inert. Nothing checked that a fault ever fired: breaking the trigger of PrepareFaultFacade left every test green. PrepareFaultFacade now counts the faults it throws, and those it throws inside WalWriter.prepareCommit(). A prepare fault makes an iteration commit-faulty only once it fired, and only SYNC and ADAPTIVE draw prepare faults. Each leg logs its drawn, fired and in-prepare counts. testCommitBatchFuzzPrepareFaultsFireSync and its ADAPTIVE twin replace the generator-only test. They script a fault into the iterations of fixed seeds and require transient and persistent faults to fail a prepare in a multi-table batch. A dead trigger, or a prepare that no longer flushes the column file, fails both.
QwpIngressProcessorState read cairo.commit.mode itself to decide whether a commit batch prepares its tables before the role-switch lock. That duplicated WalWriter.walCommitMode(), the grade every WAL writer commits at, and added a configured-mode read that DataSyncFailureModePolicyTest had not classified, which failed the test in CI. WalWriter.isCommitSyncing() now answers whether its commits wait for WAL file syncs, under SYNC and ADAPTIVE, and the batch asks the writer of its first entry. The decision stays once per batch and the modes that prepare are unchanged: testCommitBatchPreparesOnlyUnderSyncAndAdaptive pins them in all five modes.
Enterprise main now overrides SqlCompilerImpl.getShowExecutionRequirements(), which master added in #8118, so its CI no longer compiles against this branch without master.
[PR Coverage check]😍 pass : 680 / 684 (99.42%) file detail
|
http.send.buffer.sizeno longer drop the connection and duplicate rows.Tandem: questdb/java-questdb-client#122.
The server's commit batch now spans all tables of a receive pass: plain frames append to their tables' WAL writers, and each table commits once when the pass ends or at the first frame that cannot join. Before a frame's first block of a table appends, the table's WAL writer marks a savepoint. When the frame fails or cannot join, each writer it touched rolls back to its savepoint, the batch commits as it stands, and the frame is answered on its own. If that rollback fails, or the frame's failure leaves the writers in doubt, the server commits nothing of the batch and NACKs the failed frame, as master does.
QuestDB commits each table separately, so the server checks a table for what would refuse its commit (insert permission, a dropped, renamed or suspended table) as the table joins the batch: a refused table's first frame ends the batch, the frames before it commit, and that frame gets the refusal. The batch sequences its tables under the role-switch lock, which it holds across all of them, and reads the node's role once: a demote lands between batches and never splits one, and a read-only node refuses the whole batch. Under the sync and adaptive commit modes, each table first appends its WAL commit record and syncs its WAL files outside the lock (
WalWriter.prepareCommit()), so only the sequencer appends, and without a group window their log syncs, run under it. A failed WAL file sync therefore fails the batch before any table commits. A batch refused or failed after its tables prepared discards their writers, which the replay reopens. An adaptive group flush that falls due in a batch runs after the lock is released; when it fails, the batch keeps its ack and the table's next frame is NACKed. A storage fault while sequencing, or a DROP, RENAME, SUSPEND WAL or REVOKE that lands while a table waits in the batch, can still fail a table after others committed: the ack then stops before the failed table's first frame, and the client's replay sends the committed tables' rows from the rest of that receive pass (http.recv.buffer.size, 2 MiB by default) again.An ack names every table it covers and could outgrow the send buffer, which dropped the connection. The server now stages such a frame in connection memory and sends it in send-buffer-sized pieces.
A WAL writer that failed while it held no uncommitted rows, for example on a segment roll, stayed in the QWP table cache, because neither the commit nor the rollback path touches such a writer. The table lookup now evicts a distressed writer and the next frame or datagram gets a fresh one.
Rows/s on one connection: server pinned to 12 cores of a Ryzen 9 9950X, client on 4 other cores, tmpfs, loopback, JDK 25. Raw replays pre-encoded frames (
QwpIngressMultiTableBenchmark, server only); Java is the same benchmark with Java client 1.3.12.* Raw and Java: one 1 s warmup and three 1 s measurements on a fresh server, because the default run fills a 45 GB tmpfs; expect ±20% noise.