From 39387e944a62e4ff761d3e8ed5123cbb3710cf54 Mon Sep 17 00:00:00 2001 From: norvalbv Date: Fri, 20 Nov 2026 03:01:54 +0000 Subject: [PATCH 1/2] devkit ship --wait-ci names gh's error on an unavailable verdict (sc-7637) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Problem On #1389, `devkit ship --wait-ci` tallied `ci: 3 pass`, then printed `ci: gh-failed โ€” retrying (1/3)`, `(2/3)`, and ended `ship: ci-outcome=unavailable pr=1389 reason=gh-failed` on a PR whose checks were all green. The gh stderr was captured in `WaitCiResult.unavailableMessage` but printed nowhere: not on the verdict line, not on the retry lines, not in the `ship_ci` row. So the failing call could not be identified or classified. Under `--wait-ci-strict` the run exits 5. ## Change - An `unavailable` verdict line now ends with `detail=`. It comes after `reason=`, so token-wise readers are unaffected, and is left off when gh said nothing. - The per-strike line names the same text: `ci: gh-failed โ€” retrying (1/3): `. The gh-unreachable first-sighting line now uses the same helper; it was previously unredacted and capped at 120 chars. - The `ship_ci` telemetry row gains `detail` (null unless unavailable). - All three go through one helper, `failureDetail` in `cli/lib/ship/wait-ci/classify.mts`. It collapses whitespace, applies `redactSecrets`, then caps at 200 code points. The two pre-redaction `raw.slice(0, 200)` cuts in `ghChecksPoller` and head-runs `readApi` are removed. With pretty-printed output, they could leave a `ghp_` stub too short for the redaction regex inside the visible detail. - Docs: `ship --help`, `skills/using-devkit/SKILL.md` (synced), and a dated note on the `ship-ci-wait-is-observation-not-intent` decision axis. ## Not built: "return passed once checks were terminal-green" This acceptance criterion is declined because it contradicts ruling (4) of `docs/decisions/ship-ci-wait-is-observation-not-intent.md`: a terminal tally must hold for the wall-clock settle grace, not merely for N polls. Once the grace holds, the wait already returns passed, so no later gh failure can override it. The remaining case is two green polls (0s, 10s) followed by three gh failures. In that case nothing was observed after the 30s grace, and returning passed would be a false pass that a `--wait-ci-strict && merge` script acts on. The wait stays fail-closed. The fix for a real transient is to classify its now-visible text, so the wait keeps polling to a real confirmation. A regression test locks this: `stays unavailable on a green set the settle grace never confirmed, and names gh's words`. ## Tests - `wait.test.mts`: - the incident sequence: fail-closed, with detail on the verdict and retry lines - one line, `ghp_` redaction, 200 cap, detail last - a token past 200 chars of unparseable output stays redacted - an emoji is never split at the cap - whitespace-only stderr adds no detail - `telemetry.test.mts`: `detail` is redacted, and null when not unavailable. - `ship-wait-ci.test.mts`: the spawned ship asserts the full `detail=` on the real verdict line. - Updated exact-line expectations in `ci-verdict.test.mts` and `head-runs.test.mts`. - Revert proof: dropping `detail=` fails the new verdict tests. coverage-diff: 9/9 added executable lines covered. ## Follow-ups (not in this PR) - Classify the real gh error text once a recurrence prints it: add it to TRANSPORT, or move to status-code classification in the octokit plugin-retry style. - `redactSecrets`' assignment pattern over-redacts diagnostic text after words like `secret`/`token`, e.g. a check named `secret-scan: โ€ฆ`. - A spawned test that reads the `ship_ci` row's `detail` end-to-end. Story: sc-7637 ๐Ÿค– Generated with [Claude Code](https://claude.com/claude-code) Complete against acceptance criteria: NO โ€” return passed when a gh failure follows a terminal green tally: declined, it contradicts ruling (4) of ship-ci-wait-is-observation-not-intent (settle grace, not N polls) and needs an owner decision to reopen --- .agents/skills/using-devkit/SKILL.md | 2 +- .claude/skills/using-devkit/SKILL.md | 2 +- .cursor/skills/using-devkit/SKILL.md | 2 +- .devkit/skills-manifest.json | 8 +- cli/__tests__/ci-verdict.test.mts | 8 +- cli/__tests__/ship-wait-ci.test.mts | 5 +- cli/commands/ship.mts | 2 + cli/lib/ship/wait-ci/classify.mts | 8 ++ cli/lib/ship/wait-ci/head-runs.mts | 2 +- cli/lib/ship/wait-ci/head-runs.test.mts | 2 +- cli/lib/ship/wait-ci/telemetry.mts | 2 + cli/lib/ship/wait-ci/telemetry.test.mts | 33 ++++++ cli/lib/ship/wait-ci/wait.mts | 18 ++-- cli/lib/ship/wait-ci/wait.test.mts | 100 ++++++++++++++++++ .../ship-ci-wait-is-observation-not-intent.md | 1 + skills/using-devkit/SKILL.md | 2 +- 16 files changed, 177 insertions(+), 20 deletions(-) diff --git a/.agents/skills/using-devkit/SKILL.md b/.agents/skills/using-devkit/SKILL.md index 7556c80f8..b9e827147 100644 --- a/.agents/skills/using-devkit/SKILL.md +++ b/.agents/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship "" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. | diff --git a/.claude/skills/using-devkit/SKILL.md b/.claude/skills/using-devkit/SKILL.md index 7556c80f8..b9e827147 100644 --- a/.claude/skills/using-devkit/SKILL.md +++ b/.claude/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship <branch> "<title>" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. | diff --git a/.cursor/skills/using-devkit/SKILL.md b/.cursor/skills/using-devkit/SKILL.md index 7556c80f8..b9e827147 100644 --- a/.cursor/skills/using-devkit/SKILL.md +++ b/.cursor/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship <branch> "<title>" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. | diff --git a/.devkit/skills-manifest.json b/.devkit/skills-manifest.json index 7dba3292f..7788c25f8 100644 --- a/.devkit/skills-manifest.json +++ b/.devkit/skills-manifest.json @@ -35,7 +35,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "0b869c7c47ecaadddd3f02ddad525ab0daaa55882f139b0b6727a8d56af0ddd5" + "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" }, "providers": { "claude": { @@ -72,7 +72,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "0b869c7c47ecaadddd3f02ddad525ab0daaa55882f139b0b6727a8d56af0ddd5" + "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" } }, "codex": { @@ -109,7 +109,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "0b869c7c47ecaadddd3f02ddad525ab0daaa55882f139b0b6727a8d56af0ddd5" + "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" } }, "cursor": { @@ -146,7 +146,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "0b869c7c47ecaadddd3f02ddad525ab0daaa55882f139b0b6727a8d56af0ddd5" + "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" } } } diff --git a/cli/__tests__/ci-verdict.test.mts b/cli/__tests__/ci-verdict.test.mts index 82cfe8245..c8797ab09 100644 --- a/cli/__tests__/ci-verdict.test.mts +++ b/cli/__tests__/ci-verdict.test.mts @@ -100,7 +100,9 @@ describe('devkit ci-verdict', () => { throw Object.assign(new Error('spawn gh ENOENT'), { code: 'ENOENT' }); }; expect(await ciVerdict(['985'], '/r', { exec, wait: fastWait() })).toBe(2); - expect(out).toEqual(['ship: ci-outcome=unavailable pr=985 reason=gh-missing']); + expect(out).toEqual([ + 'ship: ci-outcome=unavailable pr=985 reason=gh-missing detail=the `gh` CLI is not on PATH', + ]); }); it('prints the attributed failed verdict on stdout and exits 0', async () => { @@ -168,6 +170,8 @@ describe('devkit ci-verdict', () => { return args[1] === 'view' ? PR_JSON : JSON.stringify([pass]); }; expect(await ciVerdict(['985'], '/r', { exec, wait: fastWait() })).toBe(2); - expect(out).toEqual(['ship: ci-outcome=unavailable pr=985 reason=merge-conflict']); + expect(out).toEqual([ + 'ship: ci-outcome=unavailable pr=985 reason=merge-conflict detail=the PR has a merge conflict, so GitHub starts no pull_request run on abc', + ]); }); }); diff --git a/cli/__tests__/ship-wait-ci.test.mts b/cli/__tests__/ship-wait-ci.test.mts index 016f96535..e797d5bec 100644 --- a/cli/__tests__/ship-wait-ci.test.mts +++ b/cli/__tests__/ship-wait-ci.test.mts @@ -287,7 +287,10 @@ describe('ship-branch.sh โ€” --wait-ci', () => { const log = argvLog(); const { r } = ship(['feat/badjson', 't', '--wait-ci', 'note.txt'], log, 'not json'); expect(r.status, r.stderr).toBe(0); - expect(r.stderr).toContain('ci-outcome=unavailable pr=42 reason=gh-json-unparseable'); + // gh's own words ride the verdict line, so a gh-failed on a green PR names the failing call. + expect(r.stderr).toContain( + 'ci-outcome=unavailable pr=42 reason=gh-json-unparseable detail=gh pr checks returned JSON this reader cannot use: not json', + ); }); }); diff --git a/cli/commands/ship.mts b/cli/commands/ship.mts index 0a420cdd5..a71f531dc 100644 --- a/cli/commands/ship.mts +++ b/cli/commands/ship.mts @@ -182,6 +182,8 @@ Usage: a head with no pull_request run passes only when mergeable, not stacked, and run-free. A red that ends the wait while any check still runs says attribution=unknown reason=run-in-progress: the unfinished run may still add failures. + On unavailable, a trailing detail=<โ€ฆ> carries gh's error (one line, secrets + redacted, 200 chars at most), so the failing call is named. The verdict NEVER reaches the exit code (unless --wait-ci-strict): a red PR is not a failed ship, and an agent reading non-zero would retry --resume against a record the push deleted. diff --git a/cli/lib/ship/wait-ci/classify.mts b/cli/lib/ship/wait-ci/classify.mts index 4d8b0cf7d..d5121cad0 100644 --- a/cli/lib/ship/wait-ci/classify.mts +++ b/cli/lib/ship/wait-ci/classify.mts @@ -1,4 +1,5 @@ /** Maps a failed `gh pr checks` exec to the PollResult the wait loop in `wait.mts` reads. */ +import { redactSecrets } from '../redact-secrets.mts'; import type { PollResult } from './wait.mts'; /** What `execFileSync` throws. Every field optional: this is read to CLASSIFY, never to trust. */ @@ -51,3 +52,10 @@ export function classifyChecksFailure(e: ExecFailure): PollResult { return { kind: 'unavailable', reason: 'gh-unreachable', message: msg }; return { kind: 'unavailable', reason: 'gh-failed', message: msg }; } + +/** gh's words for a failed poll, one line, redacted before the ONLY cap (callers pass it uncut) + * so no half-token leaks; the cap counts code points, so an emoji is never split. */ +export const failureDetail = (message: string): string => + Array.from(redactSecrets(message.replace(/\s+/g, ' ').trim())) + .slice(0, 200) + .join(''); diff --git a/cli/lib/ship/wait-ci/head-runs.mts b/cli/lib/ship/wait-ci/head-runs.mts index 8f0d3e711..12812e8ea 100644 --- a/cli/lib/ship/wait-ci/head-runs.mts +++ b/cli/lib/ship/wait-ci/head-runs.mts @@ -119,7 +119,7 @@ function readApi<T>( } const parsed = schema.safeParse(json); if (parsed.success) return { ok: true, value: parsed.data }; - const message = `gh ${args.join(' ')} returned JSON this reader cannot use: ${raw.slice(0, 200)}`; + const message = `gh ${args.join(' ')} returned JSON this reader cannot use: ${raw}`; return { ok: false, result: { kind: 'unavailable', reason: 'gh-json-unparseable', message } }; } diff --git a/cli/lib/ship/wait-ci/head-runs.test.mts b/cli/lib/ship/wait-ci/head-runs.test.mts index 20dd97c7e..a2469f0cf 100644 --- a/cli/lib/ship/wait-ci/head-runs.test.mts +++ b/cli/lib/ship/wait-ci/head-runs.test.mts @@ -199,7 +199,7 @@ describe('headRunsPoller โ€” a head with no workflow run', () => { polls: 1, }); expect(verdictLine(result, '42')).toBe( - 'ship: ci-outcome=unavailable pr=42 reason=merge-conflict', + 'ship: ci-outcome=unavailable pr=42 reason=merge-conflict detail=the PR has a merge conflict, so GitHub starts no pull_request run on abc123def456', ); }); diff --git a/cli/lib/ship/wait-ci/telemetry.mts b/cli/lib/ship/wait-ci/telemetry.mts index 9ac88cd14..b2d02cd58 100644 --- a/cli/lib/ship/wait-ci/telemetry.mts +++ b/cli/lib/ship/wait-ci/telemetry.mts @@ -1,6 +1,7 @@ /** The `ship_ci` telemetry row `wait.mts` appends after its verdict line. */ import { devkitVersion } from '../../../../gate-engine/devkit-version.mts'; import { appendShipRow } from '../telemetry/row.mts'; +import { failureDetail } from './classify.mts'; import type { WaitCiResult } from './wait.mts'; /** @@ -22,6 +23,7 @@ export function recordCiEvent(result: WaitCiResult, pr: string, env = process.en waited_s: Math.round(result.elapsedMs / 1000), polls: result.polls, reason: result.unavailableReason ?? null, + detail: result.unavailableMessage ? failureDetail(result.unavailableMessage) : null, required: result.required ?? false, required_contexts: result.requiredContexts ?? null, attribution: result.attribution?.token ?? null, diff --git a/cli/lib/ship/wait-ci/telemetry.test.mts b/cli/lib/ship/wait-ci/telemetry.test.mts index 2cd650a89..2df8def74 100644 --- a/cli/lib/ship/wait-ci/telemetry.test.mts +++ b/cli/lib/ship/wait-ci/telemetry.test.mts @@ -126,3 +126,36 @@ describe('recordCiEvent attribution', () => { expect(none).toMatchObject({ attribution: null, inherited: null, new_failures: null }); }); }); + +describe('recordCiEvent detail', () => { + it('records ghโ€™s redacted words on an unavailable verdict, and null otherwise', () => { + const file = join(mkdtempSync(join(tmpdir(), 'ci-events-')), 'events.jsonl'); + const env = { DEVKIT_GATE_EVENTS: file, DEVKIT_SHIP_ID: 'ship-1' }; + const base = { + polls: 3, + elapsedMs: 1_000, + counts: summarise([]).counts, + failures: [], + pending: [], + cancelled: [], + }; + const token = `ghp_${'b'.repeat(36)}`; + recordCiEvent( + { + ...base, + outcome: 'unavailable', + unavailableReason: 'gh-failed', + unavailableMessage: `HTTP 403\n${token}`, + }, + '1', + env, + ); + recordCiEvent({ ...base, outcome: 'passed' }, '2', env); + const [unavailable, passed] = readFileSync(file, 'utf8') + .trimEnd() + .split('\n') + .map((l) => JSON.parse(l)); + expect(unavailable).toMatchObject({ reason: 'gh-failed', detail: 'HTTP 403 [REDACTED]' }); + expect(passed).toMatchObject({ reason: null, detail: null }); + }); +}); diff --git a/cli/lib/ship/wait-ci/wait.mts b/cli/lib/ship/wait-ci/wait.mts index 06b1de272..d4b5b8d7e 100644 --- a/cli/lib/ship/wait-ci/wait.mts +++ b/cli/lib/ship/wait-ci/wait.mts @@ -18,7 +18,7 @@ import { } from './checks-json.mts'; export { CHECK_BUCKETS, CHECKS_JSON_FIELDS, parseChecksJson, type CheckBucket, type CheckRow }; -import { classifyChecksFailure, type ExecFailure } from './classify.mts'; +import { classifyChecksFailure, type ExecFailure, failureDetail } from './classify.mts'; import { headRunsPoller } from './head-runs.mts'; import { requiredSetPoller } from './required.mts'; import { withFailedSteps } from './steps.mts'; @@ -162,8 +162,11 @@ export function verdictLine(result: WaitCiResult, pr: string): string { return `${head} cancelled=${linked(result.cancelled)}`; case 'timed-out': return `${head} pending=${named(result.pending)} waited=${Math.round(result.elapsedMs / 1000)}s`; - case 'unavailable': - return `${head} reason=${result.unavailableReason ?? 'gh-failed'}`; + case 'unavailable': { + // Free text, so it goes last: a token-wise reader of reason= never sees it. + const detail = failureDetail(result.unavailableMessage ?? ''); + return `${head} reason=${result.unavailableReason ?? 'gh-failed'}${detail ? ` detail=${detail}` : ''}`; + } default: return `${head} checks=${Object.values(result.counts).reduce((a, b) => a + b, 0)}`; } @@ -277,9 +280,10 @@ export async function waitForChecks(options: WaitCiOptions): Promise<WaitCiResul const unreachable = result.reason === 'gh-unreachable'; if (PERMANENT.has(result.reason) || (!unreachable && ++transient >= transientBudget)) return done('unavailable', result); - if (!unreachable) say(`ci: ${result.reason} โ€” retrying (${transient}/${transientBudget})`); - else if (failStreak === 1) - say(`ci: gh unreachable (${oneLine(result.message).slice(0, 120)})`); + const detail = failureDetail(result.message); + if (!unreachable) + say(`ci: ${result.reason} โ€” retrying (${transient}/${transientBudget}): ${detail}`); + else if (failStreak === 1) say(`ci: gh unreachable (${detail})`); else say('ci: gh unreachable โ€” retrying until the deadline'); } else { transient = 0; @@ -358,7 +362,7 @@ export function ghChecksPoller(options: ChecksPollerOptions): () => PollResult { return { kind: 'unavailable', reason: 'gh-json-unparseable', - message: `gh pr checks returned JSON this reader cannot use: ${raw.slice(0, 200)}`, + message: `gh pr checks returned JSON this reader cannot use: ${raw}`, }; }; return () => { diff --git a/cli/lib/ship/wait-ci/wait.test.mts b/cli/lib/ship/wait-ci/wait.test.mts index 141ec5a7c..3111aaded 100644 --- a/cli/lib/ship/wait-ci/wait.test.mts +++ b/cli/lib/ship/wait-ci/wait.test.mts @@ -20,6 +20,7 @@ import { verdictLines, waitForChecks, } from './wait.mts'; +import { failureDetail } from './classify.mts'; import { recordCiEvent } from './telemetry.mts'; const row = (bucket: CheckBucket, name: string, link = ''): CheckRow => ({ @@ -696,3 +697,102 @@ describe('attribute', () => { }); }); }); + +describe('gh failure detail', () => { + const ghFailed: PollResult = { + kind: 'unavailable', + reason: 'gh-failed', + message: 'GraphQL: Could not resolve to a node\n(statusCheckRollup)', + }; + + it('stays unavailable on a green set the settle grace never confirmed, and names ghโ€™s words', async () => { + // Greens at 0s and 10s, then three failures: no poll observed the set after the 30s grace. + const harness = drive([rows(row('pass', 'gate')), rows(row('pass', 'gate')), ghFailed], { + intervalMs: 10_000, + settleGraceMs: 30_000, + timeoutMs: 3_600_000, + }); + const result = await harness.run(); + expect(result).toMatchObject({ outcome: 'unavailable', unavailableReason: 'gh-failed' }); + expect(verdictLine(result, '1389')).toBe( + 'ship: ci-outcome=unavailable pr=1389 reason=gh-failed detail=GraphQL: Could not resolve to a node (statusCheckRollup)', + ); + expect(harness.lines.filter((l) => l.startsWith('ci: gh-failed'))).toEqual([ + 'ci: gh-failed โ€” retrying (1/3): GraphQL: Could not resolve to a node (statusCheckRollup)', + 'ci: gh-failed โ€” retrying (2/3): GraphQL: Could not resolve to a node (statusCheckRollup)', + ]); + }); + + it('keeps the verdict one line, redacts secrets and caps the detail, last on the line', () => { + const token = `ghp_${'a'.repeat(36)}`; + const line = verdictLine( + { + polls: 3, + elapsedMs: 1_000, + counts: summarise([]).counts, + failures: [], + pending: [], + cancelled: [], + outcome: 'unavailable', + unavailableReason: 'gh-failed', + unavailableMessage: `HTTP 401\nAuthorization: token ${token}\n${'x'.repeat(500)}`, + }, + '7', + ); + expect(line).not.toContain('\n'); + expect(line).not.toContain(token); + const [head, detail] = line.split(' detail='); + expect(head).toBe('ship: ci-outcome=unavailable pr=7 reason=gh-failed'); + expect(detail).toMatch(/^HTTP 401 Authorization: token \[REDACTED\] x+$/); + expect(detail).toHaveLength(200); + }); + + it('redacts a token that sits past 200 chars of unparseable gh output', () => { + // Cutting gh's stdout to 200 chars BEFORE redaction left ghp_ + 35 chars unmatched, and the + // indentation of pretty-printed output collapses so that stub lands inside the capped detail. + const token = `ghp_${'a'.repeat(40)}`; + const raw = `${' '.repeat(100)}${'x'.repeat(60)} ${token}`; + const poll = ghChecksPoller({ pr: '7', cwd: '/r', exec: () => raw }); + const result = poll(); + expect(result.kind).toBe('unavailable'); + const line = verdictLine( + { + polls: 1, + elapsedMs: 0, + counts: summarise([]).counts, + failures: [], + pending: [], + cancelled: [], + outcome: 'unavailable', + unavailableReason: 'gh-json-unparseable', + unavailableMessage: result.kind === 'unavailable' ? result.message : '', + }, + '7', + ); + expect(line).not.toMatch(/ghp_a/); + }); + + it('never splits an emoji at the cap', () => { + const detail = failureDetail(`${'x'.repeat(199)}๐Ÿš€ deploy`); + expect(detail).toBe(`${'x'.repeat(199)}๐Ÿš€`); + expect(detail.isWellFormed()).toBe(true); + }); + + it('adds no detail when gh said nothing', () => { + const line = verdictLine( + { + polls: 1, + elapsedMs: 0, + counts: summarise([]).counts, + failures: [], + pending: [], + cancelled: [], + outcome: 'unavailable', + unavailableReason: 'gh-failed', + unavailableMessage: ' \n ', + }, + '7', + ); + expect(line).toBe('ship: ci-outcome=unavailable pr=7 reason=gh-failed'); + }); +}); diff --git a/docs/decisions/ship-ci-wait-is-observation-not-intent.md b/docs/decisions/ship-ci-wait-is-observation-not-intent.md index fe3148097..15f1bbc70 100644 --- a/docs/decisions/ship-ci-wait-is-observation-not-intent.md +++ b/docs/decisions/ship-ci-wait-is-observation-not-intent.md @@ -47,3 +47,4 @@ created: 2026-09-03 - 2026-11-17 โ€” sc-2676 โ€” the accepted gap 'a signal during the wait exits 130 on a PR that did open' is closed, converging on ruling (2): ship exits 0 whenever the PR opened. Two layers forced the 130: the managed wrapper replaced any child exit with the latched signal's status, and the wait still ran under gate-signal-handoff's exiting traps. Now ship_settled_signal_traps (wait-ci/args.sh) swaps those for record-only traps (first signal wins) once the PR is open, armed before ship_end_record 0 so a recorded 0 cannot become a 130, and runs the node wait in the background so a trapped signal ends the wait builtin at once, even one that reached only the shell or landed before node started; an interrupted wait prints ci-outcome=not-run reason=interrupted and exits 0, or the signal's status (129/130/131/143) under --wait-ci-strict, whose verdict it leaves unknown. Holding signal.lock for the wait, the original proposal, was rejected: the wrapper neither records nor forwards a signal while the lock is held, and the child is detached, so Ctrl-C would do nothing for up to 7200s. - 2026-11-17 โ€” sc-6506 โ€” the Revisit-when 'a --wait-ci run reports a terminal verdict that a later check contradicts' fired four times: passed on a CodeRabbit-only rollup while the head's own Actions run was queued and not yet listed, never created after a force-push, or never created because the PR conflicted (GitHub starts no pull_request run on a conflicting PR) or was stacked. gh cannot list a scheduled check (cli/cli#7401), so the default all-checks wait (and devkit ci-verdict) now reads the head itself, only on a poll that would settle passed or no-checks: gh api pulls/<n> then actions/runs?head_sha=. Per workflow and event the deciding run is the newest one that is not fully skipped: unfinished, conclusion-less or action_required holds the wait as pending, cancelled is cancel, and any other non-green conclusion is a fail row, which catches startup_failure, a run with no check runs at all. Every settling poll also holds each context the base branch requires that has not reported (contextsForBase, the --wait-ci-required reader, read once per wait), so a protected base never passes on a review bot or an unrelated green workflow. Only a pull_request run that is not fully skipped counts as the PR's own CI. Without one, a conflicting PR ends unavailable reason=merge-conflict (permanent, so the wait ends at once); unknown mergeability, a head where Actions ran other events (so the PR's run is due), or a base that is another open PR's head (a stack, whose runs GitHub keys to the stack base) stays pending and ends timed-out; a mergeable, unstacked head with no run at all passes once the base's required checks have reported. actions/runs 404 reads as no runs; other gh trouble is classified as for gh pr checks and is never a pass. --wait-ci-required does not read head runs: its contract is that unrequired checks never hold the wait. Rejected (d)'s premise that devkit's own base is unprotected is stale (main now requires the 'required' context); that ruling is not reopened here. Known gaps: on an unprotected base, a run GitHub creates later than the settle grace can still pass early, and a repo whose PR CI is push-triggered only holds every PR until the budget ends (devkit's and frink's PR CI run on pull_request). - 2026-11-18 โ€” sc-6506 review follow-up (PR #240): the head-run guard now scopes runs to the polled PR (a run whose pull_requests names another PR on the same head SHA never decides this one; fork runs, which GitHub leaves unassociated, still count), reads every page of actions/runs (gh api --paginate --slurp) before concluding no pull_request run exists, takes native-stack membership from the pulls response's own stack field instead of guessing from another open PR's head, and keeps a known red when the protection read fails. The settle grace is now keyed by PR head as well as outcome: a green tally on a new head restarts the grace rather than inheriting the previous head's confirmations. Required contexts stay read once per wait, and a push-only-CI repo still holds every PR, as recorded above. +- 2026-11-20 โ€” sc-7637 โ€” an unavailable verdict now names gh's own error. The verdict line ends with detail=<message> (last, after reason=, so token-wise readers are unaffected), the per-strike 'ci: <reason> โ€” retrying (n/m):' line carries it, and the ship_ci row gains a detail field; all three use one helper (wait-ci/classify.mts failureDetail): whitespace collapsed, redactSecrets applied before a 200-char cap. PR #1389 had ended unavailable reason=gh-failed after three unclassified strikes on a green set, and the stderr was captured but printed nowhere, so the failing call could not be classified. Declined: returning the held green outcome once the gh-failed budget runs out. Under ruling (4) a tally must hold for the wall-clock settle grace; greens at 0s and 10s then three failures leave nothing observed after the grace, and under --wait-ci-strict that would be a false pass a merge script acts on. The wait stays fail-closed; the fix for a real transient is to classify its now-visible text so it keeps polling to a confirmation. diff --git a/skills/using-devkit/SKILL.md b/skills/using-devkit/SKILL.md index 7556c80f8..b9e827147 100644 --- a/skills/using-devkit/SKILL.md +++ b/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship <branch> "<title>" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. | From a7601ad3f0aca1e4a496a54e1a6a008ca397cfe2 Mon Sep 17 00:00:00 2001 From: norvalbv <benjinorval@gmail.com> Date: Fri, 20 Nov 2026 03:09:37 +0000 Subject: [PATCH 2/2] devkit ship --wait-ci names gh's error on an unavailable verdict (sc-7637) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Problem On #1389, `devkit ship --wait-ci` tallied `ci: 3 pass`, then printed `ci: gh-failed โ€” retrying (1/3)`, `(2/3)`, and ended `ship: ci-outcome=unavailable pr=1389 reason=gh-failed` on a PR whose checks were all green. The gh stderr was captured in `WaitCiResult.unavailableMessage` but printed nowhere: not on the verdict line, not on the retry lines, not in the `ship_ci` row. So the failing call could not be identified or classified. Under `--wait-ci-strict` the run exits 5. ## Change - An `unavailable` verdict line now ends with `detail=<gh's error>`. It comes after `reason=`, so token-wise readers are unaffected, and is left off when gh said nothing. - The per-strike line names the same text: `ci: gh-failed โ€” retrying (1/3): <detail>`. The gh-unreachable first-sighting line now uses the same helper; it was previously unredacted and capped at 120 chars. - The `ship_ci` telemetry row gains `detail` (null unless unavailable). - All three go through one helper, `failureDetail` in `cli/lib/ship/wait-ci/classify.mts`. It collapses whitespace, applies `redactSecrets`, then caps at 200 code points. The two pre-redaction `raw.slice(0, 200)` cuts in `ghChecksPoller` and head-runs `readApi` are removed. With pretty-printed output, they could leave a `ghp_` stub too short for the redaction regex inside the visible detail. - Docs: `ship --help`, `skills/using-devkit/SKILL.md` (synced), and a dated note on the `ship-ci-wait-is-observation-not-intent` decision axis. ## Not built: "return passed once checks were terminal-green" This acceptance criterion is declined because it contradicts ruling (4) of `docs/decisions/ship-ci-wait-is-observation-not-intent.md`: a terminal tally must hold for the wall-clock settle grace, not merely for N polls. Once the grace holds, the wait already returns passed, so no later gh failure can override it. The remaining case is two green polls (0s, 10s) followed by three gh failures. In that case nothing was observed after the 30s grace, and returning passed would be a false pass that a `--wait-ci-strict && merge` script acts on. The wait stays fail-closed. The fix for a real transient is to classify its now-visible text, so the wait keeps polling to a real confirmation. A regression test locks this: `stays unavailable on a green set the settle grace never confirmed, and names gh's words`. ## Tests - `wait.test.mts`: - the incident sequence: fail-closed, with detail on the verdict and retry lines - one line, `ghp_` redaction, 200 cap, detail last - a token past 200 chars of unparseable output stays redacted - an emoji is never split at the cap - whitespace-only stderr adds no detail - `telemetry.test.mts`: `detail` is redacted, and null when not unavailable. - `ship-wait-ci.test.mts`: the spawned ship asserts the full `detail=` on the real verdict line. - Updated exact-line expectations in `ci-verdict.test.mts` and `head-runs.test.mts`. - Revert proof: dropping `detail=` fails the new verdict tests. coverage-diff: 9/9 added executable lines covered. ## Follow-ups (not in this PR) - Classify the real gh error text once a recurrence prints it: add it to TRANSPORT, or move to status-code classification in the octokit plugin-retry style. - `redactSecrets`' assignment pattern over-redacts diagnostic text after words like `secret`/`token`, e.g. a check named `secret-scan: โ€ฆ`. - A spawned test that reads the `ship_ci` row's `detail` end-to-end. Story: sc-7637 ## Reviewer advisories Fixed: completeness (LOW): `detail=` also carries devkit's own reason (`gh-missing`, `merge-conflict`), so `ship --help` and the using-devkit skill now say "gh's own error, or devkit's reason" instead of "gh's error". Reviewer notes declined: - backend-performance: `failureDetail` collapses and redacts a large unparseable gh output on every failed poll. Measured on 0.6 MB of pretty-printed JSON: about 6 ms per call. A wait makes at most 3 strikes plus 1 verdict call and 1 telemetry call, against a 10 s poll interval. Capping before redaction would bring back the half-token leak this PR removes. - commit-guard: `failureDetail` import order in `wait.test.mts`. The ship's lint and oxfmt passes ran green, and the file already imported `./telemetry.mts` after `./wait.mts`. ๐Ÿค– Generated with [Claude Code](https://claude.com/claude-code) Complete against acceptance criteria: NO โ€” return passed when a gh failure follows a terminal green tally: declined, it contradicts ruling (4) of ship-ci-wait-is-observation-not-intent (settle grace, not N polls) and needs an owner decision to reopen --- .agents/skills/using-devkit/SKILL.md | 2 +- .claude/skills/using-devkit/SKILL.md | 2 +- .cursor/skills/using-devkit/SKILL.md | 2 +- .devkit/skills-manifest.json | 8 ++++---- cli/commands/ship.mts | 4 ++-- skills/using-devkit/SKILL.md | 2 +- 6 files changed, 10 insertions(+), 10 deletions(-) diff --git a/.agents/skills/using-devkit/SKILL.md b/.agents/skills/using-devkit/SKILL.md index b9e827147..dbf7181fc 100644 --- a/.agents/skills/using-devkit/SKILL.md +++ b/.agents/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship <branch> "<title>" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<why>` (one line, secrets redacted, at most 200 chars): gh's own error for a gh failure, naming the call that failed, or devkit's reason (e.g. the merge conflict). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. | diff --git a/.claude/skills/using-devkit/SKILL.md b/.claude/skills/using-devkit/SKILL.md index b9e827147..dbf7181fc 100644 --- a/.claude/skills/using-devkit/SKILL.md +++ b/.claude/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship <branch> "<title>" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<why>` (one line, secrets redacted, at most 200 chars): gh's own error for a gh failure, naming the call that failed, or devkit's reason (e.g. the merge conflict). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. | diff --git a/.cursor/skills/using-devkit/SKILL.md b/.cursor/skills/using-devkit/SKILL.md index b9e827147..dbf7181fc 100644 --- a/.cursor/skills/using-devkit/SKILL.md +++ b/.cursor/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship <branch> "<title>" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<why>` (one line, secrets redacted, at most 200 chars): gh's own error for a gh failure, naming the call that failed, or devkit's reason (e.g. the merge conflict). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. | diff --git a/.devkit/skills-manifest.json b/.devkit/skills-manifest.json index 7788c25f8..75ea5602d 100644 --- a/.devkit/skills-manifest.json +++ b/.devkit/skills-manifest.json @@ -35,7 +35,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" + "using-devkit/SKILL.md": "0efe6aa6ad2a31717e8cc9acfa8fa305d18262c0ec4081462710daf11b5380c7" }, "providers": { "claude": { @@ -72,7 +72,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" + "using-devkit/SKILL.md": "0efe6aa6ad2a31717e8cc9acfa8fa305d18262c0ec4081462710daf11b5380c7" } }, "codex": { @@ -109,7 +109,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" + "using-devkit/SKILL.md": "0efe6aa6ad2a31717e8cc9acfa8fa305d18262c0ec4081462710daf11b5380c7" } }, "cursor": { @@ -146,7 +146,7 @@ "upstream-sync/SKILL.md": "1007c8377ac15393dcc9db5384d80bd38487c85aa465921ae0b49ba5a91bad70", "upstream-sync/scripts/sync.mjs": "218aa7ad7e96f280f281a38f99919f2ce55eace902ed95b1119a0099ddbb3c76", "upstream-sync/scripts/verify-merge.mjs": "afb2977ab8b7a96fac2e93e29a68ae432508ba7e63d03c580fcf03229fcd0676", - "using-devkit/SKILL.md": "8b73ed50b71b522404fa4738ea6058c3557c00e5e5fb83650ed5295651651d9f" + "using-devkit/SKILL.md": "0efe6aa6ad2a31717e8cc9acfa8fa305d18262c0ec4081462710daf11b5380c7" } } } diff --git a/cli/commands/ship.mts b/cli/commands/ship.mts index a71f531dc..65c46ed18 100644 --- a/cli/commands/ship.mts +++ b/cli/commands/ship.mts @@ -182,8 +182,8 @@ Usage: a head with no pull_request run passes only when mergeable, not stacked, and run-free. A red that ends the wait while any check still runs says attribution=unknown reason=run-in-progress: the unfinished run may still add failures. - On unavailable, a trailing detail=<โ€ฆ> carries gh's error (one line, secrets - redacted, 200 chars at most), so the failing call is named. + On unavailable, a trailing detail=<โ€ฆ> says why: gh's own error, naming the + failing call, or devkit's reason (one line, secrets redacted, 200 chars max). The verdict NEVER reaches the exit code (unless --wait-ci-strict): a red PR is not a failed ship, and an agent reading non-zero would retry --resume against a record the push deleted. diff --git a/skills/using-devkit/SKILL.md b/skills/using-devkit/SKILL.md index b9e827147..dbf7181fc 100644 --- a/skills/using-devkit/SKILL.md +++ b/skills/using-devkit/SKILL.md @@ -58,7 +58,7 @@ devkit command. | The work is **not ready for review** โ€” you want a visible PR running CI, but no reviewers pinged yet | `devkit ship <branch> "<title>" --draft -- <paths>` | ship opens a **ready-for-review** PR by default, which requests review the moment it lands; `--draft` opens a draft instead. The bit is recorded with the invocation, so a gate-blocked draft ship still opens a draft under `--resume`. To make **every guard-suggested** ship in a repo a draft, set `.devkit/config.json` โ†’ `{ "ship": { "command": "devkit ship", "extraArgs": ["--draft"] } }` โ€” the `command` key is required or `extraArgs` is ignored, and the guard drops `--draft` from its `--pr` suggestions, where it cannot apply | | You are shipping a **layer on top of another open PR** (stacked PRs) | `devkit ship <branch> "<title>" --base <lower-pr-branch> -- <paths>` โ€” nothing else | ship joins the new PR to the lower PR's **GitHub native stack** by default: it creates the stack when the lower PR targets the default branch, appends when it tops one, and prints `ship: stacked #N on #M`. No `gh stack link` and no extension needed. It never fails the ship: a mid-stack base, a lower PR on another non-default base, or a refusal from GitHub prints why plus the command to run by hand. Stacked PRs merge bottom-up, and merging one also merges those below it, so pass `--no-stack` (recorded for `--resume`) for a PR you want to merge on its own | | A **draft** PR is finished and you are pushing the final commit | `devkit ship <branch> "<title>" --pr --ready -- <paths>` | one command instead of a re-push followed by a hand-run `gh pr ready`. The flip runs **last**, so a gh failure never costs the pushed commit โ€” it names the exact `gh pr ready` to re-run. `--ready` requires `--pr` (a new ship is ready already); to go the other way, `gh pr ready --undo <branch>` | -| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<gh's error>` (one line, secrets redacted, at most 200 chars) naming the call that failed. On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | +| You need the PR's **CI verdict**, not just its URL โ€” an end-to-end status before handing off | `devkit ship <branch> "<title>" --wait-ci -- <paths>` (add `--wait-ci-timeout <60..7200>`, default 3600) | ship otherwise returns while the checks are still pending. The wait runs **last**, after every artifact is durable, so it cannot cost the ship: it prints progress only when the tally changes plus a liveness line each minute, then one verdict line on stderr โ€” `ship: ci-outcome=<passed\|failed\|cancelled\|no-checks\|timed-out\|unavailable\|not-run> pr=<n> โ€ฆ` (`not-run` when the wait was requested but could not run โ€” a failed `--ready` flip, an unresolvable PR number, `SHIP_DRY_RUN`, or a signal that ended the wait (`reason=interrupted`, still exit 0); it always carries a `reason=`). An `unavailable` line ends with `detail=<why>` (one line, secrets redacted, at most 200 chars): gh's own error for a gh failure, naming the call that failed, or devkit's reason (e.g. the merge conflict). On `failed` it adds `attribution=<inherited-only\|new\|unknown> inherited=<n> new=<m> new-failing=<files> new-steps=<job โ€บ step> base-run=<id> base-ci=<red\|green>`: which failures the base the PR was merged onto already had, read from CI's per-file test summaries, never from logs. `new` means not failing on that base run; `unknown` names a `reason=` or an `unsure=` count. Only `inherited-only` says none of the red is yours. A still-running Actions job whose step has already failed counts as red, so the wait ends about 40s after that step rather than at job end; it adds `failed-steps=<job โ€บ step>`. A green set also needs the head's own Actions runs, since a review bot can report green on a head CI never ran on: a queued run holds the wait, a run that failed before reporting any check (`startup_failure`) is red, a conflicting PR whose head has no `pull_request` run ends `unavailable reason=merge-conflict` at once (GitHub starts no `pull_request` run on a conflicting PR; a run made before the conflict still decides), every check the base branch requires must have reported, and a head without a `pull_request` run that ran jobs passes only when the PR is mergeable, is not stacked on another open PR, and has no workflow run at all; a fully skipped run or another event's run (e.g. `push`) is not the PR's run but still holds the wait, so it stays pending and ends `timed-out`. Any red that ends the wait while a check is still running, a step-level one included, reports `attribution=unknown reason=run-in-progress`: the per-file summary is not uploaded yet, and the unfinished run may still add new failures. Grep that line; by default the verdict **never** reaches the exit code, so a red PR is not a failed ship and must not be retried with `--resume`. To branch on it in a script, add `--wait-ci-strict`: ship then exits **5** for any verdict but `passed` (`not-run` and a crashed wait included; a signal that ends the wait exits its own 129โ€“143) โ€” the PR is open, so 5 is never a cue to `--resume`; an exit 1 still wins. On `timed-out` the next line names the `gh pr checks` one-shot. Run a `--wait-ci` ship in the background with a tool timeout covering the gates plus the wait. Valid with `--pr` too. **Not** replayed by `--resume` โ€” re-pass it on a retry. For a PR that is already open, use `devkit ci-verdict` (next row) rather than re-shipping; `devkit baseline-status --pr <n> [--json]` reads an open PR's checks once, without waiting, and prints the same `attribution=` fields (a pending or empty check set is `unknown`, nothing failing is `attribution=none`); for a one-off check without waiting, `gh pr checks <n> --json bucket,name,state,link` is the quiet one-shot (`gh run watch` needs a run id and redraws) | | A PR is **already open** and its check is **red** โ€” is the red yours or inherited from the base it merges onto? | `devkit ci-verdict <pr-number\|branch>` (add `--timeout <60..7200>`, default 900) | prints the same `ship: ci-outcome=โ€ฆ attribution=<inherited-only\|new\|unknown> โ€ฆ base-run=<id>` line `--wait-ci` does, on stdout, without pushing or taking a ship slot โ€” `ship --pr --wait-ci` with nothing to push only says `not-run`. It compares CI's per-file test summaries of the PR run and the base run; hand-diffing `gh run view --log-failed` output compares grepped names instead. Exit 2 means no verdict (no gh, no such PR, or `reason=merge-conflict`); a red or timed-out verdict exits 0. Checks still running: run it in the background with a `--timeout` that outlasts them, since a killed wait prints nothing | | Another ship may be **running on this machine** (parallel agents, a second session) | just run `devkit ship โ€ฆ`; see who is running and who is next with `devkit ship --queue` | every ship waits for a machine-wide slot, first come first served, before its first gate, and prints its queue position once. There is one slot by default; the machine owner may allow more with `~/.devkit/ship-queue/config.json` `{"slots": N}` (1..4) โ€” never change it yourself, since too many concurrent ships exhaust memory. Do not wrap ship in a `pgrep` wait loop: it has no order, can wait on a shell that merely mentions ship, and can double-start. The slot frees itself when the ship and its process group are gone, and is handed on before `--wait-ci` polls. There is no way to skip the queue: the old `DEVKIT_SHIP_NO_QUEUE` and `DEVKIT_SHIP_QUEUE_DIR` are ignored. If a running ship looks stuck, `devkit ship --queue` names it with the `ps` command that inspects it; stop it yourself once you have checked โ€” never delete the slot by hand while its ship lives. Every attempt prints one `attempt <id> <branch>: queued Ns ยท setup Ns ยท gates Ns ยท <start> โ†’ <end> ยท outcome=<pass or the blocked gate> ยท log <path>` line just above its gate digest, so queue wait never reads as gate time; find a past attempt with `grep -h '^attempt ' ~/.devkit/telemetry/logs/*.log | grep <branch>` | | An existing PR conflicts after its base moved, and you have **already resolved it locally** | First rebase/merge the actual PR base, then `devkit ship <branch> "<title>" --pr --base <actual-pr-base> -- <paths...>` | Brief every path you changed and every old-PR path the base does not already hold identically: paths a squash-merged parent PR already landed may be omitted. devkit gates one replacement commit and rewrites only under an exact head-OID lease. It publishes the caller-prepared resolution; it does not perform the rebase or merge. The local rebase/merge is all it needs: **no** raw `git push` to the PR branch and **no** `gh pr update-branch` first โ€” GitHub's PR base OID is a stale snapshot and is not checked. A base that moves on during gates is fine; a rewound one is refused. |