Skip to content

Comment fixes, eval-test environment split, phase 6 re-scope, action pin updates - #28

Merged
irvingpop merged 5 commits into
mainfrom
next-code-pr
Nov 6, 2026
Merged

irvingpop merged 5 commits into
mainfrom
next-code-pr

Conversation

@irvingpop

Copy link
Copy Markdown
Member

Summary

  • Summary comment: the TIP box now names the signing step when unsigned commits exist; the unsigned-commit row uses correct singular/plural forms; the "Dependency graph disabled" setting label needs one narrow affirmative phrase (negated text no longer matches); collect.py catches http.client.IncompleteRead and still fails closed.
  • Eval environment split: a no-permission select-environment job picks eval only for exactly refs/heads/main; every other ref (including MAIN, tags, PR refs) gets eval-test. Federation values are per environment; the eval-test rule id is unset, so branch evals fail closed until the owner creates the rule. Boundary lint updated.
  • Phase 6 scope (owner decisions 2026-11-05): phase 4 exit evidence = 5 real PRs, 2 weekly reports, 7 clean days; START_HERE is not a target (to be archived, Move START_HERE content into the org .github repo and archive START_HERE #27); the pilot bypass stays (gate switch is follow-up work); security configs recorded as live, S1 skipped; final acceptance checks rewritten for this shape.
  • Pins: harden-runner v2.22.2 -> v2.22.3 (audit mode kept; Deferred: re-test harden-runner v2.22.3 in block mode (phase 5) #24 stays open, the westus2 fix is not released yet); setup-uv v10.3.2 -> v10.3.3 with releases.astral.sh added to the setup-uv job allowlists.

Review

Adversarial review (Codex) found 3 Important + 1 Minor; all fixed in a8a8de4 and closed by a scoped re-review. Unit tests: 2928 -> 3016 passed. All commits signed.

After merge (owner)

  1. Create environment eval-test (required reviewer, prevent self-review).
  2. Create the Anthropic Console federation rule for the eval-test subject.
  3. Small PR that sets the eval-test rule id.
  4. Release (comment, collect and pin changes reach the pilot repos only after a release).

🤖 Generated with Claude Code

irvingpop and others added 5 commits November 5, 2026 16:58
…tion, IncompleteRead

- The TIP callout no longer says "Nothing to do" when the unsigned-commit
  row shows; it names the signing step. Verdict and risk do not change, and
  the row stays informational. When the row does not fit, the TIP keeps
  "Nothing to do".
- The unsigned-commit row uses "commit" for a one-commit PR.
- The Dependency graph setting-problem match needs the state words in the
  same clause, with no negation, clause join or "enabled" between them.
  Display only.
- collect.default_transport treats an error body that cannot be read
  (http.client.HTTPException, such as IncompleteRead) as empty. The status
  still fails the caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- eval.yml picks the environment from the ref with one fixed expression
  (main -> eval, any other ref -> eval-test) and sets EVAL_ENVIRONMENT from
  the same expression.
- evals.py holds the federation IDs per environment. Only eval and
  eval-test are accepted; a missing or other value fails closed before any
  token is minted. The eval-test rule is not created yet, so it is unset and
  the runner fails closed with "eval-test federation rule not configured".
- Workflow lint: eval.yml must use exactly that expression for the
  environment and EVAL_ENVIRONMENT; no other workflow may use eval or
  eval-test. New eval-boundary fixtures and edits for each case.
- Spec P6-4 and build plan: the two environments, who approves and why,
  the live eval state, and the open owner steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lot bypass stays

- Phase 4 exit evidence: 5 real PRs reviewed, 2 weekly metric reports,
  7 days with no unexplained failure.
- START_HERE is not a review target (it will be archived, issue #27): it
  moves from candidate to the excluded list in supported-repos.md.
- The bypass switch (org admins only, rename to claude-review) is follow-up
  work, done only when someone asks the owner for a gate. The review stays
  an advisory signal for maintainers.
- Security configurations: both live and read back; one planned change is
  skipped by owner decision. Details stay in the owner's records.
- New phase 6 final acceptance test for this shape. Spec P6-5.
- Build plan: P6-1 comment format shipped in 364058d; P6-3 fixes are a
  release item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fest host

- step-security/harden-runner v2.22.2 -> v2.22.3
  (46b06cea07f9e64621fdf066ba77ac6915163d43, released 2026-10-27) in every
  job. Audit mode stays: the safe-by-default fix for issue 708 is not
  released yet (.github issue 24 stays open).
- astral-sh/setup-uv v10.3.2 -> v10.3.3
  (68559e99cc807c1eb7e9238fd43a15bba460eec2, released 2026-10-28) in every
  job. It fetches the uv version manifest from releases.astral.sh first
  (GitHub raw is the fallback), so that host joins the allowlist of every job
  that runs setup-uv (not measured). The uv download stays on GitHub.
- Tag refs checked with the GitHub API: both lightweight tags point at
  these commits.
- Fixture edits, release-test allowlists, workflow comment, spec K-14, K-15
  and the build plan updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ity checks

- I1: Actions expressions compare strings without case, so a branch named
  MAIN picked eval. A select job with no permissions now picks the
  environment with an exact shell test; the evals job takes its one output
  for the environment and EVAL_ENVIRONMENT. The lint requires the job, its
  fixed script and the needs link; tests run the real script for MAIN, Main,
  tags, PR refs and others.
- I2: the Dependency graph setting match needs one narrow affirmative phrase,
  so "is neither disabled nor turned off" is no longer evidence.
- I3: the phase 6 security check reads the features and attachment sets,
  not only attachment and defaults. The skipped change stays skipped.
- M1: the release eval on main needs no approval; the release steps now say
  so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@irvingpop
irvingpop requested a review from a team as a code owner November 6, 2026 01:22
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

🔴 Hold deployment: Claude of Duty identified 1 item requiring attention

Mission: Review the code. Protect the branch. Deploy the future.

Caution

Out of range. This change is too large for the model review. File checks ran, but model checks did not. A maintainer can waive this.
Next step for the author: split the change, or ask a maintainer for a waiver (under "Waiver" below).

Severity Effect Where Finding Found by
⚠️ Not run Blocks, waivable Model passes Too large for automatic review. File checks ran, model checks did not. Workflow (code, no model)
File checks Review Security Supply chain Dependency review Coverage
✅ ok, 7 signals ⚠️ skipped, too large ⚠️ skipped, too large ⚠️ skipped, too large ✅ ok ✅ complete

Run 42143423528, attempt 2 for head a8a8de4 into main. The required workflow in the merge box is the result that counts.

Claude of Duty never approves a PR. A human reviewer gives the final approval.

🔑 Waiver: how a maintainer clears this

This PR fails for waivable reasons only. A person with write or admin access, who is not the PR author, posts this exact line as a new comment and replaces <reason> with the reason. Then they re-run the newest run of the required workflow through the GitHub REST API (POST .../actions/runs/{run_id}/rerun). Do not edit the comment afterwards. Do not close and reopen the PR to apply the waiver.

/oc-review override a8a8de4c8d23f8e27cbacc0e5652f7bc41a2419f 364058df01f84ffe70d1c9ea6cd60a7930fa81b7 <reason>

A waiver covers the code at this exact head and merge base. Any new push needs a new waiver.

📋 Full review record: reasons, checks, signals, finding details, model summaries, token usage

Reviewed snapshot

Head SHA Merge base SHA Base branch Run
a8a8de4c8d23f8e27cbacc0e5652f7bc41a2419f 364058df01f84ffe70d1c9ea6cd60a7930fa81b7 main 42143423528, attempt 2

Reasons the gate failed

Reason Waivable? Detail
Change too large for the model review ✅ yes too large for automatic review

Checks (code, no model)

Check Result Detail
Hard blocks ✅ none
Coverage ✅ complete files in comparison 40, files collected 40, trees complete
Dependency changes ✅ none
Dependency review job ✅ success

Signals (context only, never block)

  • ⚠️ Sensitive path changed: change to workflow, in .github/workflows/claude-review.yml
  • ⚠️ Sensitive path changed: change to workflow, in .github/workflows/eval.yml
  • ⚠️ Sensitive path changed: change to workflow, in .github/workflows/test.yml
  • ℹ️ Very long line: lines longer than 500 characters, in docs/build-plan.md, lines 61 110 246 247
  • ℹ️ Text aimed at an AI reviewer: pattern reviewer-address, in docs/claude-review-design.md, lines 706
  • ℹ️ Very long line: lines longer than 500 characters, in docs/claude-review-design.md, lines 43 199 211 611 614 615 693 694 711 762 and more
  • ℹ️ Credential-like string: credential-assignment (value redacted), in review/tests/unit/test_workflows.py, lines 942

Dependency changes

None.

Finding details

Model output from untrusted input. Check it before you act on it. Verified means only that the quoted text is at that place. It does not mean the claim is true.

Blocking is decided by code from category and severity, never by the model. A finding in a blocking category blocks only at high or critical severity.

No findings from the passes that ran. The Review, Security, Supply chain pass did not run, so this is not a clean result.

Model summaries

Model output from untrusted input. Check it before you act on it. Verified means only that the quoted text is at that place. It does not mean the claim is true.

Pass Status Model Summary
Review ❌ skipped-size No summary. Reason: bundle status skipped-size
Security ❌ skipped-size No summary. Reason: bundle status skipped-size
Supply chain ❌ skipped-size No summary. Reason: bundle status skipped-size

Token usage

No token usage was recorded for any pass.

@irvingpop

Copy link
Copy Markdown
Member Author

/oc-review override a8a8de4 364058d because

@irvingpop
irvingpop merged commit 8992adb into main Nov 6, 2026
7 of 9 checks passed
@irvingpop
irvingpop deleted the next-code-pr branch November 6, 2026 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant