Skip to content

feat(dev): measure per-stage model and effort on the defect probe set (CORE-112) - #456

Merged
PrzemekGalarowicz merged 17 commits into
mainfrom
feat/core-112-stage-model-tuning
Nov 17, 2026
Merged

PrzemekGalarowicz merged 17 commits into
mainfrom
feat/core-112-stage-model-tuning

Conversation

@PrzemekGalarowicz

@PrzemekGalarowicz PrzemekGalarowicz commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Per-stage model and effort arms for the defect probe set, the pairing reader that scores them, and a kit fix without which a model arm would not have run the model it names. Apparatus only: nothing under the product surface changes, SKILLS_VERSION is unchanged, the CHANGELOG entry is [Unreleased], and no shipped default changes.

Depends on #455 (CORE-90) and must merge after it. This branch is cut from feat/core-90-defect-probe-set. When #455 merges, merge origin/main into this branch and drop nothing.

  • Kit fix (.dev/floor/defect-probe-kit.mjs). Each kit session is a direct claude -p '/pharn-<stage> …', and Claude Code takes a directly invoked command's model and effort from that command's model: / effort: frontmatter for the turn (pharn/floor/check-model-config.mjs, "TURN SCOPE"). /pharn-grill and /pharn-plan say model: opus, so --model grill=sonnet would most likely have run opus and been scored as sonnet. A stage named by --model / --effort is now written into each run project's command frontmatter and pharn.config.json models.stages. The build then refuses unless this repository's check-model-config.mjs agreement passes over the project; the arm's own copy is never executed. A control kit changes no byte.
  • Pairing reader (.dev/floor/stage-model-pairs.mjs, new). It pairs the i-th variant run with the i-th control run for each probe. Per pair it reports the catch (via the scorer's scoreRun), stage minutes, USD, weighted units and the served model, read from each <stage>.session.json. It then gives the arm one reading from a closed set by a fixed first-match rule: cannot-decide (arm-not-applied, fewer-than-six-pairs or control-caught-none), does-not-support (catch-lost or no-saving), or supports-change. A test shows the control arm's own sessions cannot satisfy the served-model check.
  • Record (.dev/measurements/stage-model-tuning-2026-11-17.md, pending). Pre-registers four kits before any run: plan-sonnet with control-spec, then grill-sonnet, then grill-medium, which runs only if at least USD 29 of the USD 70 budget is left. Also pre-registered: the manifest sha256, the arm commit, the exact kit and pairing commands, the reading rule, the budget and stop rule, and the bounds. Those bounds are: no randomization, served effort not recorded, all cache writes weighted as 5-minute writes, and E1 out of scope.
  • Docs: the probe README and .dev/guides/floor-checks.md. Ignore lists: the record's future runs/ directory.

Results (the user's runs, USD 29.94 of USD 70)

One reading per arm, on the probe chain only:

arm reading pairs catches lost chain USD chain minutes
grill model opus → sonnet supports-change 8 comparable of 8 0 −43% −24%
plan model opus → sonnet cannot-decide (fewer-than-six-pairs) 4 comparable of 6 (two plan-stage stops cite no anchor) 0 −23% −19%
grill effort high → medium does-not-support (catch-lost) 8 comparable of 8 1 (p4/r2) −14% −17%

Every variant session on the varied stage was served the expected model, so the frontmatter override took effect. No
shipped default changes: each default needs E1's whole-loop pairs in the measured project and the human's call. The
record's "Defaults" table says so row by row.

  • New since the first push: CORE-90's caught-at-plan / stopped-at-plan-unanchored outcomes are merged from feat(dev): defect probe set, per-finding scorer and run kit (CORE-90) #455, and the reader pairs plan-stage stops on a valid plan session. A pair holding an unanchored stop is costed but is not comparable for the catch rules. The record states this amendment as written before any pairing.
  • origin/main (7.21.0) is merged. CORE-90's [Unreleased] entry is re-dated to 2026-11-17 so it is not older than the 7.21.0 section.
  • Run files and each arm's pair JSON are committed under .dev/features/stage-model-tuning/runs/, added with -f because the repo-wide runs/ ignore pattern matches this directory. Session files are not committed: they carry session ids.

Pipeline

/pharn-dev-ship stage-model-tuning: plan, grill, build, regress, verify, review. GATE 1 was decided by the orchestrator under the user's delegation; GATE 2 was a model decision under delegation. See .dev/features/stage-model-tuning/SHIP.md.

Not covered

The whole-loop experiment E1 on the measured project.

Linear: CORE-112

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Updated repository checks to skip development-only probe outputs.
    • Added changelog notes covering internal evaluation tools and experiment readings.
    • No changes to shipped product behavior or defaults.

PrzemekGalarowicz and others added 10 commits November 16, 2026 22:36
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…E-90)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…it (CORE-90)

Six probes over a small fixture project, each with one planted defect (four PLAN
probes, two SPEC probes), pinned by sha256 in manifest.json. The scorer counts a
run as caught when a valid finding's file matches an anchor and the valid triage
does not give it proceed; it reads no free text. The kit builds one fresh project
and one claude -p command line per run, with model and effort per stage, for the
user to run. The catch count on main 7.19.0 is pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ORE-112)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…et, and make kit overrides take effect (CORE-112)

A directly invoked /pharn-<stage> runs on its command's model:/effort: frontmatter, so the
kit's --model flag alone did not set a stage's model. The kit now writes an overridden
stage into each run project's frontmatter and models.stages and refuses the build unless
check-model-config agreement passes. stage-model-pairs.mjs pairs a variant arm with the
control and gives it one closed reading. The record pre-registers three arms; results pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-112)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(CORE-112)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The changelog records defect-probe and stage-model-tuning tooling and reported experiment readings. Markdown lint, Prettier, and ESLint now exclude specified probe data and run outputs.

Changes: Development probe apparatus

Layer / File(s) Summary
Record probe and tuning apparatus
CHANGELOG.md
The changelog describes defect-probe fixtures, a pinned manifest, anchor-based scoring, and a run kit. It also reports stage-model-tuning readings, including results that remain undecided or pending. It states that shipped defaults and SKILLS_VERSION do not change.
Exclude probe and run data from tooling
.markdownlint-cli2.jsonc, .prettierignore, eslint.config.mjs
The configurations exclude .dev/probes and specified defect-probe and stage-model-tuning run outputs. Comments describe the contents as hashed or byte-sensitive.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Other

Possibly related PRs

  • pharn-dev/pharn-oss#455: Adds the defect-probe set, scorer, and run kit that this PR’s changelog documents and tooling exclusions cover.
Merge Risk: Minimal · up to 7827b

This change adds changelog entries and excludes development-only probe data from lint and formatting tools. It does not alter shipped behavior, and no merge-blocking risk is evident.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: a development-only measurement of per-stage model and effort on the defect probe set, tracked as CORE-112. The changeset adds the probe kit, a pairing reader, and the …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: a development-only measurement of per-stage model and effort on the defect probe set, tracked as CORE-112. The changeset adds the probe kit, a pairing reader, and the recorded experiment results under .dev, plus ignore-list updates for Prettier, markdownlint, and ESLint. The title is specific and accurate. It does not mention that the change is development-only, but it does not need to.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 744e3cd and 7827bdf.

Files ignored due to path filters: 96

  • .dev/features/defect-probe-set/GRILL.md is excluded by !**/.dev/**
  • .dev/features/defect-probe-set/PLAN.md is excluded by !**/.dev/**
  • .dev/features/defect-probe-set/REGRESSION.md is excluded by !**/.dev/**
  • .dev/features/defect-probe-set/REVIEW.md is excluded by !**/.dev/**
  • .dev/features/defect-probe-set/SHIP.md is excluded by !**/.dev/**
  • .dev/features/defect-probe-set/VERIFY.md is excluded by !**/.dev/**
  • .dev/features/defect-probe-set/regression-report.json is excluded by !**/.dev/**
  • .dev/features/defect-probe-set/verify-report.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/GRILL.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/PLAN.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/REGRESSION.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/REVIEW.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/SHIP.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/VERIFY.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/regression-report.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/control-spec/p6/r1/PLAN.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/control-spec/p6/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/control-spec/p6/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p1/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p1/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p1/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p1/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p2/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p2/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p2/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p2/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p3/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p3/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p3/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p3/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p4/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p4/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p4/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-medium/p4/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p1/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p1/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p1/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p1/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p2/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p2/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p2/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p2/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p3/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p3/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p3/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p3/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p4/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p4/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p4/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/grill-sonnet/p4/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/pairs/grill-medium.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/pairs/grill-sonnet.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/pairs/plan-sonnet.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r1/PLAN.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r1/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r1/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r2/PLAN.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r2/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r2/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r3/PLAN.md is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r3/grill-findings.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/runs/plan-sonnet/p6/r3/grill-triage.json is excluded by !**/.dev/**
  • .dev/features/stage-model-tuning/verify-report.json is excluded by !**/.dev/**
  • .dev/floor/defect-probe-kit.mjs is excluded by !**/.dev/**
  • .dev/floor/defect-probe-kit.test.mjs is excluded by !**/.dev/**
  • .dev/floor/defect-probe-score.mjs is excluded by !**/.dev/**
  • .dev/floor/defect-probe-score.test.mjs is excluded by !**/.dev/**
  • .dev/floor/stage-model-pairs.mjs is excluded by !**/.dev/**
  • .dev/floor/stage-model-pairs.test.mjs is excluded by !**/.dev/**
  • .dev/guides/floor-checks.md is excluded by !**/.dev/**
  • .dev/measurements/defect-probes-main-7.19.0.md is excluded by !**/.dev/**
  • .dev/measurements/stage-model-tuning-2026-11-17.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/README.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/README.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/package.json is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/src/audit.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/src/auth.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/src/invoices.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/src/keys.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/src/money.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/src/server.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/src/store.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/test/auth.test.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/test/helpers.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/test/invoices.test.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/test/money.test.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/fixture/test/store.test.js is excluded by !**/.dev/**
  • .dev/probes/defect-probes/manifest.json is excluded by !**/.dev/**
  • .dev/probes/defect-probes/probes/p1/PLAN.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/probes/p2/PLAN.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/probes/p3/PLAN.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/probes/p4/PLAN.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/probes/p5/SPEC.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/probes/p6/SPEC.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/specs/api-key-rotation/SPEC.md is excluded by !**/.dev/**
  • .dev/probes/defect-probes/specs/invoice-refunds/SPEC.md is excluded by !**/.dev/**
Files selected for processing: 1

  • CHANGELOG.md
Files skipped from review as they are similar to previous changes: 1

  • CHANGELOG.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

pharn

🛡️ pharn-ci · ⚠️ Warn

Warning

Warn · Diff coverage of this change is 100% against a threshold of 100%; 10 test-integrity findings; 1 content finding; 106 agent setup files found.

Verdict Diff coverage Test integrity Agent setup
⚠️ Warn 100% (threshold 100%) ℹ️ 10 findings 106 files

🧪 Diff coverage

▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰ 100%

100% of changed lines covered (1104/1104, threshold 100%)

Changed files (3)
File Coverage Covered Uncovered lines
.dev/floor/defect-probe-kit.mjs 100% 323/323
.dev/floor/defect-probe-score.mjs 100% 491/491
.dev/floor/stage-model-pairs.mjs 100% 290/290

🔍 Test integrity

  • ℹ️ TI15 .dev/floor/defect-probe-kit.mjs:103 — The condition at ".dev/floor/defect-probe-kit.mjs:103" compares with a 7-character string literal that the test file ".dev/floor/defect-probe-kit.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-kit.mjs:104 — The condition at ".dev/floor/defect-probe-kit.mjs:104" compares with a 7-character string literal that the test file ".dev/floor/defect-probe-kit.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-kit.mjs:104 — The condition at ".dev/floor/defect-probe-kit.mjs:104" compares with a 8-character string literal that the test file ".dev/floor/defect-probe-kit.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-kit.mjs:107 — The condition at ".dev/floor/defect-probe-kit.mjs:107" compares with a 7-character string literal that the test file ".dev/floor/defect-probe-kit.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-kit.mjs:108 — The condition at ".dev/floor/defect-probe-kit.mjs:108" compares with a 8-character string literal that the test file ".dev/floor/defect-probe-kit.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-kit.mjs:109 — The condition at ".dev/floor/defect-probe-kit.mjs:109" compares with a 7-character string literal that the test file ".dev/floor/defect-probe-kit.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-score.mjs:151 — The condition at ".dev/floor/defect-probe-score.mjs:151" compares with a 8-character string literal that the test file ".dev/floor/defect-probe-score.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-score.mjs:281 — The condition at ".dev/floor/defect-probe-score.mjs:281" compares with a 7-character string literal that the test file ".dev/floor/defect-probe-kit.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-score.mjs:297 — The condition at ".dev/floor/defect-probe-score.mjs:297" compares with a 6-character string literal that the test file ".dev/floor/defect-probe-score.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/floor/defect-probe-score.mjs:298 — The condition at ".dev/floor/defect-probe-score.mjs:298" compares with a 14-character string literal that the test file ".dev/floor/defect-probe-score.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.

🧬 Content

  • ⚠️ PI03 .dev/features/stage-model-tuning/REVIEW.md:9 — phrase PH01
    • Hint: Remove the phrase. If the file documents prompt injection on purpose, keep such samples in test files or under fixtures/.

📦 Dependencies

0 added or changed dependencies checked · registry lookups off

No dependency findings.

  • ⚠️ Not checked: 1 manifest or lockfile file could not be read, so dependency safety did not check it: `.dev/probes/defect-probes/fixture/package.json` (unavailable).
    • Hint: Keep manifests and lockfiles valid and under 5 MiB, and run on a checkout of the pull request head with fetch-depth: 0, so they can be compared.

💬 Messages (2)

  • ℹ️ COVERAGE_PATHS_PARTIALLY_UNMATCHED 126 file paths in the coverage report are not tracked files in the repository; their coverage is not used.
    • Hint: Nothing to do if those files are generated or outside the repo; if changed files are missing from the table, check the coverage root.
  • ⚠️ AGENT_CONFIG_SECRET_UNSCANNED Agent config files were not scanned for committed secrets: `pr-number mode: the checkout is not the pull request head, so its committed agent config files were not read`.
    • Hint: Keep agent config files text, under 512 KiB and inside the repository, and run on a checkout of the pull request head (not PR-number mode or a compare-API diff), so they can be scanned for committed secrets.

🤖 Agent setup (106 files)

Agent setup: 106 files · claude-code

Agent setup files (106)
Path Kind
.claude/agents/pharn-review-lens.md claude-agent
.claude/agents/pharn-stage.md claude-agent
.claude/commands/pharn-build.md claude-command
.claude/commands/pharn-dev-build.md claude-command
.claude/commands/pharn-dev-eval.md claude-command
.claude/commands/pharn-dev-grill.md claude-command
.claude/commands/pharn-dev-memory-promote.md claude-command
.claude/commands/pharn-dev-plan.md claude-command
.claude/commands/pharn-dev-regress.md claude-command
.claude/commands/pharn-dev-review.md claude-command
.claude/commands/pharn-dev-ship.md claude-command
.claude/commands/pharn-dev-verify.md claude-command
.claude/commands/pharn-grill.md claude-command
.claude/commands/pharn-loop-close.md claude-command
.claude/commands/pharn-loop-quick.md claude-command
.claude/commands/pharn-loop.md claude-command
.claude/commands/pharn-memory-promote.md claude-command
.claude/commands/pharn-plan.md claude-command
.claude/commands/pharn-regress.md claude-command
.claude/commands/pharn-review.md claude-command
.claude/commands/pharn-ship-close.md claude-command
.claude/commands/pharn-ship-quick.md claude-command
.claude/commands/pharn-ship-stages.md claude-command
.claude/commands/pharn-ship.md claude-command
.claude/commands/pharn-spec.md claude-command
.claude/commands/pharn-supervise.md claude-command
.claude/commands/pharn-test.md claude-command
.claude/commands/pharn-triage.md claude-command
.claude/commands/pharn-verify.md claude-command
.claude/hooks/enforce-writes-scope.cjs claude-hook
.claude/hooks/enforce-writes-scope.test.cjs claude-hook
.claude/hooks/hook-wiring.test.cjs claude-hook
.claude/hooks/protect-trusted-paths.cjs claude-hook
.claude/hooks/protect-trusted-paths.test.cjs claude-hook
.claude/hooks/require-loop-record.cjs claude-hook
.claude/hooks/require-loop-record.test.cjs claude-hook
.claude/hooks/set-writes-scope.cjs claude-hook
.claude/hooks/set-writes-scope.test.cjs claude-hook
.claude/hooks/writes-scope-release.test.cjs claude-hook
.claude/settings.json claude-settings
.claude/skills/phargus-diagnose/SKILL.md claude-skill
.claude/skills/phargus-diagnose/agents/openai.yaml claude-skill
.claude/skills/phargus-diagnose/references/compatibility.md claude-skill
.claude/skills/phargus-diagnose/references/data.md claude-skill
.claude/skills/phargus-diagnose/references/diagnosis.md claude-skill
.claude/skills/phargus-diagnose/references/tools.md claude-skill
.claude/skills/phargus-diagnose/scripts/capture.mjs claude-skill
.claude/skills/phargus-diagnose/scripts/operations.mjs claude-skill
.claude/skills/phargus-diagnose/scripts/phargus.mjs claude-skill
.claude/skills/phargus-diagnose/scripts/records.mjs claude-skill
…and 56 more files

pharn-ci 0.10.0 · report schema v2 · coverage format lcov

PrzemekGalarowicz and others added 4 commits November 17, 2026 09:31
…stopped-at-plan-unanchored (CORE-90)

Both p5 plan sessions refused to plan, naming the planted contradiction, and
left no PLAN.md; the pre-registered rule could only score that findings-missing.
A plan-stage stop is now caught-at-plan when the plan session's result cites an
anchored SPEC.md line, else stopped-at-plan-unanchored (neither a catch nor a
miss). Both read model free text and are advisory; the floor part is the line
overlap. The catch count is caught + caught-at-plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ORE-112)

caught-at-plan is a catch; stopped-at-plan-unanchored is neither, so a pair holding one
is costed but not comparable, and the six-pair, control-catch and catch-lost rules count
comparable pairs only. Grill and triage sessions are optional for a plan-stopped run.
The record states this amendment before any pairing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PrzemekGalarowicz and others added 3 commits November 17, 2026 10:08
grill model sonnet: supports-change on the probe chain (8 comparable pairs, 0 lost, chain
-43% USD). plan model sonnet: cannot-decide (4 comparable pairs). grill effort medium:
does-not-support (1 catch lost). No shipped default changes; each needs E1's loop pairs.
Run files and the reader's pair JSON are committed; session files are not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…E-112)

Added with -f: the repo-wide runs/ ignore pattern also matches this feature's runs/
directory. Findings, triage, a SPEC probe's PLAN.md and the reader's JSON; no session files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PrzemekGalarowicz
PrzemekGalarowicz merged commit f8a7a74 into main Nov 17, 2026
8 checks passed
@PrzemekGalarowicz
PrzemekGalarowicz deleted the feat/core-112-stage-model-tuning branch November 17, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant