Repository navigation
ci: the step budget rests on the step's own clock - #9
Open
noidee-dev wants to merge 2 commits into
Open
noidee-dev wants to merge 2 commits into
noidee-dev wants to merge 2 commits into
Conversation
The Unit-Tests budget goes 16 -> 20 minutes, the maintainer's decision of 2026-10-31, and the derivation under it is replaced rather than extended: it had been stitched together from two different runs. Withdrawn, quoted verbatim: "16 minutes (960 s) is 2.04x of 392 s + 78 s = 470 s -- this image's measured compile plus the OTHER image's measured run, because this image's run figure does not exist yet: the step was killed before the suite finished." That paragraph promised its own replacement by the first green run on this image, and keeping the promise exposed the shape of the error rather than just the missing addend: a step budget bounds a step's wall clock, and adding one run's compile to another run's suite duration bounds a step that neither run is. That wall clock is a field pair of the Actions API, present on every run that reached the step. Measured 2026-10-31: 38225383495 macos-15 329 s success 40933631584 macos-latest 493 s failure -- the 480 s budget's own kill 41045404989 macos-latest 581 s success 41047833837 macos-latest 517 s success 41048445836 macos-latest 640 s success 640 s is the maximum of a step that finished, so 1200 s is 1.88x. The two derivations this comment has carried landed at 2.09x and 2.04x, and 1.88x is the first under 2 -- but neither of them states 2x as a requirement, so the comment now says in as many words that "the 2x floor" is a reading of them and not a quote, and gives 22 minutes (1320 s, 2.06x) as the smallest whole-minute budget that would hold that reading. 20 was chosen while the best figure on offer was the 558.79 s the withdrawn stitching produced, against which 1200 s read as 2.15x; the series above was measured after the choice, not before it. The spread matters more than the maximum, and one pair isolates it: 517 s against 640 s is one commit apart, and that commit adds 23 lines of CLAUDE.md, which no target compiles. The compile half alone moved 345.71s -> 492.86s across it. The job budget above is unaffected: 753 s and 937 s for the two green runs against its 2100 s, and a step running the full 1200 s would put the worse of those at 1497 s, 0.71x. Two citations named heads that the merge rebase rewrote. Both now name the develop commit beside the head the run actually had, because replacing the hash alone would have claimed a run ran on a commit it never saw. `git show <h> | git patch-id --stable` gives one id per pair: `9c8d3673`/`a774ac4d` in ci.yml, `c8041d19`/`bcd0bc61` in ConnectMainActorLivenessTests. docs/BACKLOG.md: two rows corrected in place with their wording quoted, and two rows opened. The row "Every layout figure the guards hold was measured on Darwin 27" asked for "the first push or pull request that triggers a run, and then reading the checks' verdict". PR 8 was that run, and it answers half of the row: both green runs passed `Suite "Bounded view layout guard"` and `Suite "Resizable container floors"` on macOS 26.6.2, so the local Darwin 27 figures survive a Darwin 26 runner across all 33 bounded views and the resizable floors. By how much is still unmeasured -- both reporters are gated, so the runs produced a verdict and no figures, and a pass is not a margin. The row's macos-15 premise is withdrawn with the quote, as is the now-false title half of the row it was split from: the gate has not run macos-15 since `a774ac4d`. Opened: the three-way toolchain split -- the gate on Swift 6.3.3, this Mac on 6.4, the retired image on 6.1.2 -- whose first instance was the `allLines` warning that reddened run 41045404989 at 26 occurrences on a tree every macos-15 run had built clean. And `claude/nice-leakey-5da0b9`, which a local branch sweep surfaced: 26 of 32 local branches were content-identical to develop or main and were deleted, and that one carries `ed0b70af`, a finished relaunch fix dated 2026-09-05 that sits on neither develop nor main. Every command written into the comment and into the rows was extracted from the committed text and run in that form. No local suite run: the only Swift change is four doc-comment lines, parse-checked with `swiftc -parse` against a positive control that reported 2 errors on a broken file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…he runner CI run 41244948849 (2026-10-31, `e4e5d054`) went red on `ConnectionDiagnosticsTests.theSSHDialNeverPutsTheSecretInTheReport` with "a closed port did not fail the SSH dial: .timedOut". The dial never failed: its step was settled by its own deadline before the probe was given a thread. The case handed the runner `stepTimeout: .seconds(10)`, and in that run 8184 of the 8307 cases carrying a duration line reported a wall clock AT OR ABOVE 10 s (the pipeline compares `>= 10` on the integer seconds) against a run total of 116.701 s -- so the budget bounded scheduling, not the dial. This is `docs/BACKLOG.md`'s "Wall-clock ceilings still in the tree", fifth shape, arriving through a production parameter the test chooses. The same shape had already been bought one file at a time, twice: `61850a9c` (2026-09-19) moved the jump suite's twin off `.seconds(10)` for exactly this reason, in a commit touching one file, and `ResolveNamesTests` had already named its own 600 s `roomy`. The five literals in this file were never swept and fell 42 days later. - A named `roomy` of 600 s carrying the measurement and the command that produces it, with all five call sites through it. Raising it costs those five nothing, which is what makes 600 s safe here: every target either answers at once (the rig) or refuses at once (a closed loopback port), and the skipped case never dials, so none can park on the larger budget. - Both diagnostics suites' notes withdraw the clause "it is deliberately far larger than any step budget below", quoted in place. It is false of this file by construction now, and it was false of the jump file on the day it was written: `git blame` dates both of that file's 600 s sites to `61850a9c` and the sentence's heading to `ae19a53a`, nine days later. - `docs/BACKLOG.md` records the shape as the eighth, what was fixed, and the six occurrences in `ICMPEchoTests` and `NetworkTraceTests` deliberately NOT swept -- five send real echo or trace probes where a roomy budget's cost is unmeasured, and the sixth is `NetworkTraceTests.traceStep`, whose own comment calls 5 s generous and records the 5.004 s behind it. A second `docs/BACKLOG.md` row, "The gate's toolchain and the maintainer's are two versions apart", is answered in part by the full local run this commit was verified with. It withdraws "nothing has been run to find what Swift 6.4 diagnoses that 6.3.3 does not" as far as "nothing has been run" goes, and records what was found instead: not a diagnostic but a rendering divergence. `ViewTestabilitySpike`'s "Varying only isRegex leaves the pixels unchanged" fails on this Mac (macOS 27.0.1, Swift 6.4.2) and passed on the gate in run 41244948849 itself. Pre-existing and environmental: the spike is in `macSCPAppKitTests` and nothing here touches that target. Not fixed in this commit, and the row says so. Red first, in two parts. The observed red is the CI run above. The mechanism reproduces at a keyboard with the budget planted at `.nanoseconds(1)`, where the deadline cannot lose: the SSH case recorded the identical sentence, `a closed port did not fail the SSH dial: .timedOut`, and the S3 case its own twin. Two issues in that run and three in the FIRST one, same plant -- which case varied is not established, because that earlier run's per-case lines were lost to a `tail` in the command that captured it. Green at 600 s: `swift test --filter 'ConnectionDiagnostics|ICMPEcho|NetworkTrace|ResolveNames'` is 154 tests in 6 suites, 0 issues. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #8, which moved the gate to
macos-latestand left three things owed: a step budget resting on a figure stitched from two runs, two citations naming heads the merge rebase rewrote, and a backlog row whose next step that PR had quietly fulfilled.The budget: 16 -> 20 minutes
Withdrawn, quoted verbatim: "16 minutes (960 s) is 2.04x of 392 s + 78 s = 470 s -- this image's measured compile plus the OTHER image's measured run, because this image's run figure does not exist yet: the step was killed before the suite finished."
That paragraph promised its own replacement by the first green run on this image. Keeping the promise exposed the shape of the error rather than just the missing addend: a step budget bounds a step's wall clock, and adding one run's compile to another run's suite duration bounds a step that neither run is. That wall clock is a field pair of the Actions API, present on every run that reached the step:
640 s is the maximum of a step that finished, so 1200 s is 1.88x — the first of this comment's derivations to land under 2, after 2.09x and 2.04x. Neither of those states 2x as a requirement, so the comment now says in as many words that "the 2x floor" is a reading of them and not a quote, and names 22 minutes (1320 s, 2.06x) as the smallest whole-minute budget holding that reading. 20 is the maintainer's decision of 2026-10-31, taken while the best figure on offer was the 558.79 s the withdrawn stitching produced, against which 1200 s read as 2.15x. The series was measured after the decision, not before it.
The spread matters more than the maximum, and one pair isolates it: 517 s against 640 s is one commit apart, and that commit adds 23 lines of
CLAUDE.md, which no target compiles. The compile half alone moved 345.71s -> 492.86s across it.The job budget is unaffected: 753 s and 937 s for the two green runs against its 2100 s, and a step running the full 1200 s would put the worse of those at 1497 s, 0.71x.
Two heads the rebase rewrote
Both citations now name the develop commit beside the head the run actually had. Replacing the hash alone would have claimed a run ran on a commit it never saw.
git show <h> | git patch-id --stablegives one id per pair:9c8d3673/a774ac4dinci.yml,c8041d19/bcd0bc61inConnectMainActorLivenessTests.docs/BACKLOG.md
Two rows corrected in place with their wording quoted, two opened.
The row "Every layout figure the guards hold was measured on Darwin 27" asked for "the first push or pull request that triggers a run, and then reading the checks' verdict". #8 was that run, and it answers half the row: both green runs passed
Suite "Bounded view layout guard"andSuite "Resizable container floors"on macOS 26.6.2, so the local Darwin 27 figures survive a Darwin 26 runner across all 33 bounded views and the resizable floors. By how much is still unmeasured — both reporters are gated, so the runs produced a verdict and no figures, and a pass is not a margin. The macos-15 premise is withdrawn with the quote, as is the now-false title half of the row it was split from.Opened: the three-way toolchain split (gate on Swift 6.3.3, the maintainer's Mac on 6.4, the retired image on 6.1.2), whose first instance was the
allLineswarning that reddened run 41045404989 at 26 occurrences on a tree everymacos-15run had built clean. Andclaude/nice-leakey-5da0b9, surfaced by a local branch sweep in which 26 of 32 local branches were content-identical to develop or main and were deleted: it carriesed0b70af, a finished relaunch fix dated 2026-09-05 that is on neither develop nor main.Verification
Every command written into the comment and into the rows was extracted from the committed text and run in that form; all six reproduce their stated figures. Two false claims were caught in the new comment before committing — "two commits apart" (it is one) and "one test-function annotation" (it is 23 lines of Markdown) — and the correction strengthens the variance argument rather than weakening it.
No local suite run: the only Swift change is four doc-comment lines, parse-checked with
swiftc -parseagainst a positive control that reported 2 errors on a broken file. This PR is what tests the new budget —ci.ymltriggers only on a push tomain/developand onpull_request, so the branch alone starts nothing.20fbe7f2— the first run on the new budget went red, and not on the budgetRun 41244948849 took 597 s for
Unit-Tests, comfortably inside the new 1200 s, so nothing was killed. It went red on one test:ConnectionDiagnosticsTests.theSSHDialNeverPutsTheSecretInTheReportrecorded "a closed port did not fail the SSH dial: .timedOut". The dial never failed — its step was settled by its own deadline before the probe was given a thread. The case handed the runnerstepTimeout: .seconds(10), and in that run 8184 of the 8307 cases carrying a duration line reported a wall clock at or above 10 s, against a run total of 116.701 s.That is
docs/BACKLOG.md's "Wall-clock ceilings still in the tree", fifth shape, reaching the test through a production parameter the test itself chooses. It had already been bought one file at a time, twice:61850a9c(2026-09-19) moved the jump suite's twin off.seconds(10)for exactly this reason — one file, 10 insertions, 2 deletions — andResolveNamesTestshad already named its own 600 sroomy. The five literals inConnectionDiagnosticsTestswere never swept and fell 42 days later.roomyof 600 s carrying the measurement and the command that produces it, with all five call sites through it. Raising it costs those five nothing, which is the argument that makes 600 s safe here: every target either answers at once (the rig) or refuses at once (a closed loopback port), and the skipped case never dials.git blamedates both of that file's 600 s sites to61850a9cand the sentence's heading toae19a53a, nine days later.ICMPEchoTestsandNetworkTraceTests. Five send real echo or trace probes, where a roomy budget's cost is unmeasured; the sixth isNetworkTraceTests.traceStep, whose own comment calls 5 s generous and records the 5.004 s behind it. Two more route through per-file factories shared with cases where a tight budget may be load-bearing. Each needs classifying before any value moves — which is how 2026-09-19 produced this red.Red first, in two parts
The observed red is the CI run. The mechanism reproduces at a keyboard with the budget planted at
.nanoseconds(1), where the deadline cannot lose: the SSH case recorded the identical sentence, and the S3 case its own twin. Two issues in that run and three in the first one, same plant — which case varied is not established, because the earlier run's per-case lines were lost to atailin the command that captured it. Green at 600 s: 154 tests in 6 suites, 0 issues.One pre-existing red found by the full local run, not fixed here
ViewTestabilitySpike's "Varying only isRegex leaves the pixels unchanged" fails on this Mac (macOS 27.0.1, Swift 6.4.2) and passed on the gate in run 41244948849 itself, after 1.265 s. The case's doc comment states as measured that aToggle's checked state "stays out of the bitmap"; on macOS 27 it does not. Recorded in the toolchain-split row, which withdraws its own "nothing has been run to find what Swift 6.4 diagnoses that 6.3.3 does not" as far as "nothing has been run" goes — what was found is a rendering divergence, not a diagnostic one, so the row stays open. Untouched here by construction: the spike is inmacSCPAppKitTestsand this commit touches onlymacSCPCoreTestsanddocs/.Full local suite on the committed tree: 5091 tests in 408 suites passed, 2884 in 242 suites with that one pre-existing issue.
🤖 Generated with Claude Code