Repository navigation
Every announced ship gate phase ends with a verdict line (sc-3326) - #280
Conversation
## Why A docs-only `devkit ship` printed gate-phase headers with nothing after them. An agent nearly reported the PR as reviewer-checked when no domain reviewer had run (sc-3326). Three of the story's five criteria were already on main: - the zero-reviewer notice (#764) - the Sentry gate's verdict line (#1195) - the completeness cached-PASS wording This PR closes the two gaps that were left. ## What changes 1. **Zero-scope review line carries the scan token.** The line is now `guard-review: SKIPPED — no domain reviewer ran; none of the N staged file(s) is in review scope (...)`. The end-of-run banner tells readers to scan for `SKIP / BYPASSED / DEGRADED /⚠️ `, and the old line had none of those. "domain" keeps the line from contradicting completeness's own `guard-review: completeness — PASS` in the same log. 2. **Every `▶ deterministic: <label>` header ends with one verdict line**, printed by the runner. It is built from the same facts as the gate's `gate_step` row, so no gate needed editing. The possible lines: - `✓ passed` - `✗ failed` - `✗ could not run` (unexpected exit, timeout, did not spawn) - `⚠ opted out — did NOT run` - `⚠ BYPASSED — verified nothing` - `⚠ passed (DEGRADED)` Before this, `guard-comments`, `guard-decision-checks`, `guard-size`, `knip`, `git-pathspec` and similar gates printed nothing when they passed. The line is indented and never starts with `▶` or ends with `...`, so the ship's timeout attribution still names the gate header. A full prefix-cache hit runs no gate, so it prints no header and no verdict. 3. Decision note on `review-gate-in-chain` recording both. ## Scope narrowed on purpose - **AC4 (digest lists SKIPPED phases):** the ship digest stays silent on `not_selected`. That silence is a recorded ruling: the gate-verdict-attribution silence clause, 2026-09-01 and 2026-10-16 notes, locked by `cli/lib/ship/digest/gate-digest-skips.test.mts:82`. The banner token from change 1 is what makes the skip findable. - **AC5 (docs-only ship fixture):** a plain `--dry-gates` skips domain review (sc-2346/sc-2531), so such a fixture would never reach the zero-scope line. Unit tests cover both changes instead, and the proof below runs the real dry-gates ship. - **AC1's "N selected: a, b, c" line:** not added. Each selected reviewer already prints its own PASS or cached line. Fixes [sc-6172]: a passing run's log now names every gate it executed, extraGates included, each with its own verdict. Per-gate `gate_step` telemetry rows already existed. ## Proof `bun run playground --change <dir holding one new docs/*.md>` (docs-only change, real dry-gates ship): ``` ▶ deterministic: guard-size ✓ guard-size passed ▶ deterministic: guard-fanout ✓ guard-fanout passed ✓ Ship dry gates passed; no commit, branch, push, or PR was kept. ``` devkit's own `ship --dry-gates` of this PR printed a verdict for all 20 gates. That included a real `⚠ guard-dup passed (DEGRADED)`, from embeddings missing on this machine, and a real `✗ guard-anti-slop failed`, which I fixed before shipping. Tests: - `gate-engine/deterministic/__tests__/gate-start-lines.test.mts`: exact line order, plus fail, could-not-run, opt-out, timeout, BYPASSED and DEGRADED cases. Reverting the two `console.log(gateVerdictLine(...))` calls fails 3 of them. - `gate-engine/review/__tests__/empty-roots.test.mts`: the new wording. - `bun run test:run gate-engine/deterministic/__tests__ gate-engine/review/__tests__/empty-roots.test.mts cli/__tests__/gate-capture-stale-jobs.test.mts`: 19 files, 470 tests passed. ## Follow-ups (not in this PR) - In overlay mode the 🧱 eslint overlay phase header (`cli/lib/husky/husky-block.mts:206`) still prints no verdict when it passes. Fixing it needs a hook-fragment change, `doctor --fix` regeneration in consumers and parity tests. - The per-reviewer skip lines (`<reviewer> skipped (GUARD_REVIEW_SKIP)`, `(… is empty in guard.config.json)`) are lowercase, so they also miss the banner's SKIP scan. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
## Why A docs-only `devkit ship` printed gate-phase headers with nothing after them. An agent nearly reported the PR as reviewer-checked when no domain reviewer had run (sc-3326). Three of the story's five criteria were already on main: - the zero-reviewer notice (#764) - the Sentry gate's verdict line (#1195) - the completeness cached-PASS wording This PR closes the two gaps that were left. ## What changes 1. **Zero-scope review line carries the scan token.** It now reads `guard-review: SKIPPED — no domain reviewer ran; none of the N staged file(s) is in review scope (...)`. The end-of-run banner tells readers to scan for `SKIP / BYPASSED / DEGRADED /⚠️ `, and the old line had none of those. "domain" keeps it from contradicting completeness's own `guard-review: completeness — PASS`. The per-reviewer skip lines now print `SKIPPED` too: `<r> SKIPPED (GUARD_REVIEW_SKIP)` and `<r> SKIPPED (<key> is empty …)`. Before, when every selected reviewer was dropped, the phase ended on lowercase lines that the scan missed. 2. **The reviewer gate names what it selected** on one line, e.g. `guard-review: 5 reviewer(s) selected: api-security-reviewer, …` (AC1). 3. **Every `▶ deterministic: <label>` header ends with one verdict line**, printed by the runner from the same facts as the gate's `gate_step` row: - `✓ passed` - `✗ failed` - `✗ could not run` - `⚠ SKIPPED — opted out, did NOT run` - `⚠ BYPASSED — verified nothing` - `⚠ passed (DEGRADED)` Before, `guard-comments`, `guard-decision-checks`, `guard-size`, `knip`, `git-pathspec` and others printed nothing when they passed. The line is indented and never starts with `▶` or ends with `...`, so the ship's timeout attribution still names the gate header. A full prefix-cache hit prints neither the header nor a verdict. 4. **A replayed completeness PASS says where it came from** (AC3). The PASS now stores `ship_id` (`DEVKIT_SHIP_ID`). On replay the line ends `· judged earlier in this ship run`, meaning this ship's parallel pre-commit judge, or `· judged in an earlier ship run`. 5. A decision note on `review-gate-in-chain` recording all of the above. ## Scope narrowed on purpose - **AC4 (digest lists SKIPPED phases):** the ship digest stays silent on `not_selected`. That is a recorded ruling: the gate-verdict-attribution silence clause, notes of 2026-09-01 and 2026-10-16, locked by `cli/lib/ship/digest/gate-digest-skips.test.mts:82`. The SKIPPED tokens above are what make a skip findable in the log. - **AC5 (docs-only fixture):** built as a docs-only staged repo run through the real `runReviewGate`, in `gate-engine/review/__tests__/empty-roots.test.mts`. It asserts exactly one SKIPPED line, no roster line and no judge spawned. A ship-level fixture was not added: a plain `--dry-gates` skips domain review, and the `--with-reviewers` suite stubs the hook. - Verdict vocabulary is `passed` / `failed` rather than `PASS` / `FAIL`, matching the deterministic stage's existing lines. Fixes [sc-6172]: a passing run's log now names every gate it executed, extraGates included, each with its verdict. A two-arm test covers it: declared → `✓ git-pathspec passed`; undeclared → absent. Per-gate `gate_step` telemetry rows already existed. ## Proof `bun run playground --change <dir holding one new docs/*.md>` (docs-only change, real dry-gates ship): ``` ▶ deterministic: guard-size ✓ guard-size passed ▶ deterministic: guard-fanout ✓ guard-fanout passed ✓ Ship dry gates passed; no commit, branch, push, or PR was kept. ``` devkit's own `ship --dry-gates` of this PR printed a verdict for all 20 gates. That included a real `⚠ guard-dup passed (DEGRADED)`, from embeddings missing on this machine, and a real `✗ guard-anti-slop failed`, which I fixed before shipping. Tests: - `gate-engine/deterministic/__tests__/gate-start-lines.test.mts`: exact line order, plus these cases: - fail, could-not-run, opt-out and timeout - a strict-refused opt-out reads could not run - BYPASSED for an `--extra`, a `guard-`-prefixed registry and the coverage label - DEGRADED not leaking to the next gate - sc-6172's declared/undeclared extraGate - `empty-roots.test.mts`: the SKIPPED wording, the roster, and the docs-only `runReviewGate` run. - `completeness-pass-findings.test.mts`: the same-run and earlier-run cache source. - Mutation checks: reverting the verdict print, the per-gate DEGRADED delta, the roster, the `ship_id` save or the SKIPPED prefix each fails its test. - `bun run test:run gate-engine/review/__tests__ gate-engine/deterministic/__tests__ cli/lib/ship/digest`: 112 files, 2,828 tests passed. - Diff coverage: 20/20 added executable lines. ## Follow-ups (not in this PR) - In overlay mode the 🧱 eslint overlay phase header (`cli/lib/husky/husky-block.mts:206`) still prints no verdict when it passes. Fixing it needs a hook-fragment change, `doctor --fix` regeneration in consumers and parity tests. - With `noLlm` the reviewer gate returns before selection and prints no verdict line after its header (`gate-engine/review/run-review.mts` ~209). - `guard-review --only <r>` on a docs-only diff prints the SKIPPED line and then exits 1, since an empty recheck is never a PASS. - The banner's `⚠️ ` (with VS16) doesn't byte-match the `⚠` that deterministic lines use, so a literal grep misses them. Reviewer notes declined: - commit-guard, `gate-engine/deterministic/command-gates.mts` gateVerdictLine: the bypass lookup strips `guard-` from the label. `labelSchema` rejects `guard-` config labels, and `emitGateStep` strips the label the same way, so the verdict line and the gate_step row cannot diverge. All three label shapes are now tested. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Complete against acceptance criteria: NO — AC4 (end-of-run summary lists SKIPPED phases): the ship digest stays silent on not_selected under the recorded gate-verdict-attribution silence clause (locked by gate-digest-skips.test.mts:82), and flipping that ruling is an owner decision
## Why A docs-only `devkit ship` printed gate-phase headers with nothing after them. An agent nearly reported the PR as reviewer-checked when no domain reviewer had run (sc-3326). Three of the story's five criteria were already on main: - the zero-reviewer notice (#764) - the Sentry gate's verdict line (#1195) - the completeness cached-PASS wording This PR closes the two gaps that were left. ## What changes 1. **Zero-scope review line carries the scan token.** It now reads `guard-review: SKIPPED — no domain reviewer ran; none of the N staged file(s) is in review scope (...)`. The end-of-run banner tells readers to scan for `SKIP / BYPASSED / DEGRADED /⚠️ `, and the old line had none of those. "domain" keeps it from contradicting completeness's own `guard-review: completeness — PASS`. The per-reviewer skip lines now print `SKIPPED` too: `<r> SKIPPED (GUARD_REVIEW_SKIP)` and `<r> SKIPPED (<key> is empty …)`. Before, when every selected reviewer was dropped, the phase ended on lowercase lines that the scan missed. 2. **The reviewer gate names what it selected** on one line, e.g. `guard-review: 5 reviewer(s) selected: api-security-reviewer, …` (AC1). 3. **Every `▶ deterministic: <label>` header ends with one verdict line**, printed by the runner from the same facts as the gate's `gate_step` row: - `✓ passed` - `✗ failed` - `✗ could not run` - `⚠ SKIPPED — opted out, did NOT run` - `⚠ BYPASSED — verified nothing` - `⚠ passed (DEGRADED)` Before, `guard-comments`, `guard-decision-checks`, `guard-size`, `knip`, `git-pathspec` and others printed nothing when they passed. The line is indented and never starts with `▶` or ends with `...`, so the ship's timeout attribution still names the gate header. A full prefix-cache hit prints neither the header nor a verdict. 4. **A replayed completeness PASS says where it came from** (AC3). The PASS now stores `ship_id` (`DEVKIT_SHIP_ID`). On replay the line ends `· judged earlier in this ship run`, meaning this ship's parallel pre-commit judge, or `· judged in an earlier ship run`. 5. A decision note on `review-gate-in-chain` recording all of the above. ## Scope narrowed on purpose - **AC4 (digest lists SKIPPED phases):** the ship digest stays silent on `not_selected`. That is a recorded ruling: the gate-verdict-attribution silence clause, notes of 2026-09-01 and 2026-10-16, locked by `cli/lib/ship/digest/gate-digest-skips.test.mts:82`. The SKIPPED tokens above are what make a skip findable in the log. - **AC5 (docs-only fixture):** built as a docs-only staged repo run through the real `runReviewGate`, in `gate-engine/review/__tests__/empty-roots.test.mts`. It asserts exactly one SKIPPED line, no roster line and no judge spawned. A ship-level fixture was not added: a plain `--dry-gates` skips domain review, and the `--with-reviewers` suite stubs the hook. - Verdict vocabulary is `passed` / `failed` rather than `PASS` / `FAIL`, matching the deterministic stage's existing lines. Fixes [sc-6172]: a passing run's log now names every gate it executed, extraGates included, each with its verdict. A two-arm test covers it: declared → `✓ git-pathspec passed`; undeclared → absent. Per-gate `gate_step` telemetry rows already existed. ## Proof `bun run playground --change <dir holding one new docs/*.md>` (docs-only change, real dry-gates ship): ``` ▶ deterministic: guard-size ✓ guard-size passed ▶ deterministic: guard-fanout ✓ guard-fanout passed ✓ Ship dry gates passed; no commit, branch, push, or PR was kept. ``` devkit's own `ship --dry-gates` of this PR printed a verdict for all 20 gates. That included a real `⚠ guard-dup passed (DEGRADED)`, from embeddings missing on this machine, and a real `✗ guard-anti-slop failed`, which I fixed before shipping. Tests: - `gate-engine/deterministic/__tests__/gate-start-lines.test.mts`: exact line order, plus these cases: - fail, could-not-run, opt-out and timeout - a strict-refused opt-out reads could not run - BYPASSED for an `--extra`, a `guard-`-prefixed registry and the coverage label - DEGRADED not leaking to the next gate - sc-6172's declared/undeclared extraGate - `empty-roots.test.mts`: the SKIPPED wording, the roster, and the docs-only `runReviewGate` run. - `completeness-pass-findings.test.mts`: the same-run and earlier-run cache source. - Mutation checks: reverting the verdict print, the per-gate DEGRADED delta, the roster, the `ship_id` save or the SKIPPED prefix each fails its test. - `bun run test:run gate-engine/review/__tests__ gate-engine/deterministic/__tests__ cli/lib/ship/digest`: 112 files, 2,828 tests passed. - Diff coverage: 20/20 added executable lines. ## Follow-ups (not in this PR) - In overlay mode the 🧱 eslint overlay phase header (`cli/lib/husky/husky-block.mts:206`) still prints no verdict when it passes. Fixing it needs a hook-fragment change, `doctor --fix` regeneration in consumers and parity tests. - With `noLlm` the reviewer gate returns before selection and prints no verdict line after its header (`gate-engine/review/run-review.mts` ~209). - `guard-review --only <r>` on a docs-only diff prints the SKIPPED line and then exits 1, since an empty recheck is never a PASS. - The banner's `⚠️ ` (with VS16) doesn't byte-match the `⚠` that deterministic lines use, so a literal grep misses them. Reviewer notes declined: - commit-guard, `gate-engine/deterministic/command-gates.mts` gateVerdictLine: the bypass lookup strips `guard-` from the label. `labelSchema` rejects `guard-` config labels, and `emitGateStep` strips the label the same way, so the verdict line and the gate_step row cannot diverge. All three label shapes are now tested. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Complete against acceptance criteria: NO — AC4 (end-of-run summary lists SKIPPED phases): the ship digest stays silent on not_selected under the recorded gate-verdict-attribution silence clause (locked by gate-digest-skips.test.mts:82), and flipping that ruling is an owner decision
Why
A docs-only
devkit shipprinted gate-phase headers with nothing after them. An agent nearly reported the PR as reviewer-checked when no domain reviewer had run (sc-3326).Three of the story's five criteria were already on main:
This PR closes the two gaps that were left.
What changes
Zero-scope review line carries the scan token. It now reads
guard-review: SKIPPED — no domain reviewer ran; none of the N staged file(s) is in review scope (...). The end-of-run banner tells readers to scan forSKIP / BYPASSED / DEGRADED / ⚠️, and the old line had none of those. "domain" keeps it from contradicting completeness's ownguard-review: completeness — PASS. The per-reviewer skip lines now printSKIPPEDtoo:<r> SKIPPED (GUARD_REVIEW_SKIP)and<r> SKIPPED (<key> is empty …). Before, when every selected reviewer was dropped, the phase ended on lowercase lines that the scan missed.The reviewer gate names what it selected on one line, e.g.
guard-review: 5 reviewer(s) selected: api-security-reviewer, …(AC1).Every
▶ deterministic: <label>header ends with one verdict line, printed by the runner from the same facts as the gate'sgate_steprow:✓ passed✗ failed✗ could not run⚠ SKIPPED — opted out, did NOT run⚠ BYPASSED — verified nothing⚠ passed (DEGRADED)Before,
guard-comments,guard-decision-checks,guard-size,knip,git-pathspecand others printed nothing when they passed. The line is indented and never starts with▶or ends with..., so the ship's timeout attribution still names the gate header. A full prefix-cache hit prints neither the header nor a verdict.A replayed completeness PASS says where it came from (AC3). The PASS now stores
ship_id(DEVKIT_SHIP_ID). On replay the line ends· judged earlier in this ship run, meaning this ship's parallel pre-commit judge, or· judged in an earlier ship run.A decision note on
review-gate-in-chainrecording all of the above.Scope narrowed on purpose
not_selected. That is a recorded ruling: the gate-verdict-attribution silence clause, notes of 2026-09-01 and 2026-10-16, locked bycli/lib/ship/digest/gate-digest-skips.test.mts:82. The SKIPPED tokens above are what make a skip findable in the log. (sc-8908)runReviewGate, ingate-engine/review/__tests__/empty-roots.test.mts. It asserts exactly one SKIPPED line, no roster line and no judge spawned. A ship-level fixture was not added: a plain--dry-gatesskips domain review, and the--with-reviewerssuite stubs the hook.passed/failedrather thanPASS/FAIL, matching the deterministic stage's existing lines.Fixes [sc-6172]: a passing run's log now names every gate it executed, extraGates included, each with its verdict. A two-arm test covers it: declared →
✓ git-pathspec passed; undeclared → absent. Per-gategate_steptelemetry rows already existed.Proof
bun run playground --change <dir holding one new docs/*.md>(docs-only change, real dry-gates ship):devkit's own
ship --dry-gatesof this PR printed a verdict for all 20 gates. That included a real⚠ guard-dup passed (DEGRADED), from embeddings missing on this machine, and a real✗ guard-anti-slop failed, which I fixed before shipping.Tests:
gate-engine/deterministic/__tests__/gate-start-lines.test.mts: exact line order, plus these cases:--extra, aguard--prefixed registry and the coverage labelempty-roots.test.mts: the SKIPPED wording, the roster, and the docs-onlyrunReviewGaterun.completeness-pass-findings.test.mts: the same-run and earlier-run cache source.ship_idsave or the SKIPPED prefix each fails its test.bun run test:run gate-engine/review/__tests__ gate-engine/deterministic/__tests__ cli/lib/ship/digest: 112 files, 2,828 tests passed.Follow-ups (not in this PR)
cli/lib/husky/husky-block.mts:206) still prints no verdict when it passes. Fixing it needs a hook-fragment change,doctor --fixregeneration in consumers and parity tests. (sc-8909)noLlmthe reviewer gate returns before selection and prints no verdict line after its header (gate-engine/review/run-review.mts~209). (sc-8910)guard-review --only <r>on a docs-only diff prints the SKIPPED line and then exits 1, since an empty recheck is never a PASS. (sc-8911)⚠️(with VS16) doesn't byte-match the⚠that deterministic lines use, so a literal grep misses them. (sc-8912)Reviewer notes declined:
gate-engine/deterministic/command-gates.mtsgateVerdictLine: the bypass lookup stripsguard-from the label.labelSchemarejectsguard-config labels, andemitGateStepstrips the label the same way, so the verdict line and the gate_step row cannot diverge. All three label shapes are now tested.gate-engine/review/completeness.mtsship_id: an emptyDEVKIT_SHIP_IDstoresship_id: "". Inside a ship that can't happen: both minting sites use${DEVKIT_SHIP_ID:-…}, which replaces an empty value (cli/lib/ship/ship-branch.sh:760,cli/lib/ship/commit-with-gate-capture.sh:84). An empty id therefore only comes from a non-ship run, and when a later ship replays that entry, "judged in an earlier ship run" is still true.reportNonRuns,runCompleteness): fixed. The roster moved to a module-scopereportSelected, and theship_idassignment no longer adds a branch.🤖 Generated with Claude Code
Complete against acceptance criteria: NO — AC4 end-of-run summary lists SKIPPED phases (sc-8908): the ship digest stays silent on not_selected under the recorded gate-verdict-attribution silence clause, and relaxing that ruling is an owner decision
Automated evidence
Merge verdict: 🟠 policy · advisory, at
b1e7e41ea602✅ Regression evidence · captured: 16 of 16 red-failing tests passed on green (setup 24s, red 112s, green 104s, cleanup 4s)
Head
b1e7e41ea602· base50ddac1af8b7Failed on red (16 tests)
runDeterministic — a line names each gate as it starts prints one line per gate kind, each immediately before that gate spawns:AssertionError: expected [ '▶ deterministic: guard-size', …(7) ] to deeply equal [ Array(12) ]runDeterministic — a verdict line closes each gate header names a failure, an opt-out and a timeout, never as passed:AssertionError: expected [] to deeply equal [ …(4) ]runDeterministic — a verdict line closes each gate header names a declared extraGates entry on a passing run, and drops it once undeclared:AssertionError: expected [ …(2) ] to include ' ✓ git-pathspec passed'runDeterministic — a verdict line closes each gate header marks only the gate that left a withheld note as DEGRADED, not the gates after it:AssertionError: expected [] to deeply equal [ …(2) ]runDeterministic — a verdict line closes each gate header reports an opt-out that strict mode refuses as could not run, never as SKIPPED:AssertionError: expected [ '▶ deterministic: guard-dup', …(1) ] to include ' ✗ guard-dup could not run'runDeterministic — a verdict line closes each gate header marks an --extra gate that passed through its own bypass flag as BYPASSED:AssertionError: expected [ Array(2) ] to include ' ⚠ hook-parity BYPASSED — verified n…'runDeterministic — a verdict line closes each gate header marks a registry gate that passed through its own bypass flag as BYPASSED:AssertionError: expected [ …(2) ] to include ' ⚠ guard-decision-checks BYPASSED — …'runDeterministic — a verdict line closes each gate header marks the coverage gate that passed through its own bypass flag as BYPASSED:AssertionError: expected [ …(2) ] to include ' ⚠ guard-coverage BYPASSED — verifie…'a completeness PASS with findings names which ship run judged a replayed PASS (replayed under ship-completeness-findings):AssertionError: expected 'guard-review: completeness — cached P…' to match / · judged earlier in this ship run$/a completeness PASS with findings names which ship run judged a replayed PASS (replayed under a-later-ship):AssertionError: expected 'guard-review: completeness — cached P…' to match / · judged in an earlier ship run$/Test command
Captured runs, not proof of causality. Dependencies are the shipping checkout's installed packages. Stale unless the PR head is
b1e7e41ea602b011bc244a0a7987cdb3a0a69ab4.💬 Reviewer notes · 2 notes at
b1e7e41ea602· 1 at earlier commitsgate-engine/review/completeness.mtsDEVKIT_SHIP_IDis set to an empty string,""is now stored, where the old guard dropped it. A later ship replaying that entry prints "judged in an earlier ship run" with no real ship behind it, becausecachedShipRunin base-context.mts only checks forundefined. Guard with|| undefined. · correctnessDEVKIT_SHIP_IDto an empty string now storesship_id: "", which the old truthy guard skipped. A later ship then replays that entry and reports "judged in an earlier ship run" for a run that never had a ship ID. · correctness1 note at earlier commit
3ba8c024baa1gate-engine/deterministic/command-gates.mtsgateVerdictLinederives the bypass check by strippingguard-from the label, so a gate whose label doesn't follow that prefix convention but has a bypass flag could be mislabelled✓ passedinstead of BYPASSED. · commit-guard