Skip to content

docs(dev): CORE-113 loop-driver spike — a driven quick loop measured, verdict no-go - #457

Merged
PrzemekGalarowicz merged 8 commits into
mainfrom
spike/loop-driver-core-113
Nov 17, 2026
Merged

PrzemekGalarowicz merged 8 commits into
mainfrom
spike/loop-driver-core-113

Conversation

@PrzemekGalarowicz

@PrzemekGalarowicz PrzemekGalarowicz commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

What this is

The CORE-113 spike: a node driver that sequences one /pharn-loop --quick by running the loop's own pinned floor
lines itself, with one headless claude -p session per model stage. It also adds the kit the maintainer ran from a
terminal, and the record written from those runs. It is dev apparatus only, under .dev/features/loop-driver-spike/,
plus one [Unreleased] CHANGELOG entry. No product path changes and no SKILLS_VERSION bump. No driver ships.

Record: .dev/features/loop-driver-spike/RECORD.md.

Result: no-go

  • Driven run (lg3-remove-files, measure base on 7.19.0):
    • It ran every floor line in order, with 10 sessions and no model request of its own.
    • It ended STOP_CAP after 3 iterations.
    • The closeout's check-loop-record and check-loop-decision were both GREEN, and the ledger was emitted.
    • Wall time was 124.5 min, and 18.66 M weighted units excluding the probes.
  • Five spike gates: none failed.
    • G1, served model matches config: passed for every routed stage. Effort was not observable, and the inline grill
      was served its frontmatter model.
    • G2, hooks fire: the writes-scope guard denied an out-of-scope Write inside a driven pharn-stage session.
    • G3, stages complete: all 10 sessions exited 0.
    • G4, question round-trip: passed for the transport only. --resume kept the session, and the answer word was typed
      in advance.
    • G5, no permission bypass: all sessions ran in auto mode with no skip flag or allow list.
  • AC5: the ledger attributes no token of a driven run. It reports coverage: unavailable, "no session id
    available", and 0 requests; it keeps only the minutes derived from the markers.
  • Comparison: both /pharn-loop --quick runs stopped at the spec stage with refused S6c (not-quick), on the
    same intent bytes the driven spec accepted. So the orchestrator's share after Epics 1 and 2 is measured only up to
    that stop: 58.6% and 65.7%. That is not a whole-run share, and it is not comparable with the baseline's
    27.7% / 30.2%.
  • Why no-go: the epic's start condition (a whole post-change share above the baseline's threshold) is unmet. The
    benchmark cannot end green by either route: a project test needs CLAUDE.md / .claude/rules edits that the intent
    puts out of scope. RECORD.md §7 names what would reopen the decision.
  • Contradictions a real driver would cause are listed in RECORD.md §8 for a human. Nothing trusted was edited.

Chain

/pharn-dev-ship:

  • plan, then grill: check-plan-lessons exit 0;
  • build: validate exit 0;
  • regress: no-regressions, 6,906 tests passed at head, new_failures: [];
  • verify: PASS, all 7 gates, run twice. The second run came after the GATE-2 fixes.

GATE 1 and GATE 2 were decided by the orchestrating agent under the user's delegation, as recorded in PLAN.md and
SHIP.md.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added a changelog entry recording the workflow evaluation, which stopped after three iterations with validation checks passing.
    • Recorded the decision not to proceed with building the driver, along with unavailable effort and token attribution and limitations in the comparison runs and benchmark scope.

PrzemekGalarowicz and others added 7 commits November 16, 2026 22:36
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… hook entries

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ledger's reader

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ecord

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Explain Change →

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters: 7

  • .dev/features/loop-driver-spike/RECORD.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/SHIP.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/VERIFY.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/driver-core.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/loop-driver.test.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/measure.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/verify-report.json is excluded by !**/.dev/**

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The Unreleased changelog documents the CORE-113 loop-driver spike, its reported checks and limitations, and the decision not to build the driver. It also notes that no SKILLS_VERSION bump was made.

Changes: CORE-113 spike record

Layer / File(s) Summary
Document spike results
CHANGELOG.md
The Unreleased entry records the run, checks, limitations, no-go decision, and lack of a SKILLS_VERSION bump.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Other

Possibly related PRs

Merge Risk: Minimal · up to 36ee7

Merge Risk: ⚪ Minimal · up to 36ee7

This change only adds a documentation entry for a development-only spike and does not alter product behavior. It is ready to merge.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the change type (docs) and scope (dev) and the CORE-113 loop-driver spike. It also states the measured outcome and the no-go verdict. The changeset adds a development-only spike record…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the change type (docs) and scope (dev) and the CORE-113 loop-driver spike. It also states the measured outcome and the no-go verdict. The changeset adds a development-only spike record and a CHANGELOG entry that reports a no-go decision, so the title matches the main change. It is somewhat long at 83 characters, but it is specific and accurate.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 155d120 and d5e8fd1.

Files ignored due to path filters: 20

  • .dev/features/loop-driver-spike/GRILL.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/PLAN.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/RECORD.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/REGRESSION.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/REVIEW.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/SHIP.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/VERIFY.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/driver-core.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/loop-driver.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/loop-driver.test.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/measure.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/driver/token-exit.mjs is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/kit/README.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/kit/intent.md is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/kit/preflight.sh is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/kit/run-driver.sh is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/kit/run-loop.sh is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/kit/setup.sh is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/regression-report.json is excluded by !**/.dev/**
  • .dev/features/loop-driver-spike/verify-report.json is excluded by !**/.dev/**
Files selected for processing: 1

  • CHANGELOG.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

pharn

🛡️ pharn-ci · ⚠️ Warn

Warning

Warn · Diff coverage of this change is 100% against a threshold of 100%; 10 test-integrity findings; 106 agent setup files found.

Verdict Diff coverage Test integrity Agent setup
⚠️ Warn 100% (threshold 100%) ℹ️ 10 findings 106 files

🧪 Diff coverage

▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰ 100%

100% of changed lines covered (1441/1441, threshold 100%)

Changed files (3)
File Coverage Covered Uncovered lines
.dev/features/loop-driver-spike/driver/driver-core.mjs 100% 1078/1078
.dev/features/loop-driver-spike/driver/measure.mjs 100% 299/299
.dev/features/loop-driver-spike/driver/token-exit.mjs 100% 64/64

🔍 Test integrity

  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/driver-core.mjs:930 — The condition at ".dev/features/loop-driver-spike/driver/driver-core.mjs:930" compares with a 7-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/driver-core.mjs:960 — The condition at ".dev/features/loop-driver-spike/driver/driver-core.mjs:960" compares with a 7-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/driver-core.mjs:1062 — The condition at ".dev/features/loop-driver-spike/driver/driver-core.mjs:1062" compares with a 8-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/driver-core.mjs:1068 — The condition at ".dev/features/loop-driver-spike/driver/driver-core.mjs:1068" compares with a 7-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/driver-core.mjs:1068 — The condition at ".dev/features/loop-driver-spike/driver/driver-core.mjs:1068" compares with a 9-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/loop-driver.mjs:114 — The condition at ".dev/features/loop-driver-spike/driver/loop-driver.mjs:114" compares with a 8-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/measure.mjs:81 — The condition at ".dev/features/loop-driver-spike/driver/measure.mjs:81" compares with a 6-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/measure.mjs:82 — The condition at ".dev/features/loop-driver-spike/driver/measure.mjs:82" compares with a 9-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/measure.mjs:86 — The condition at ".dev/features/loop-driver-spike/driver/measure.mjs:86" compares with a 11-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.
  • ℹ️ TI15 .dev/features/loop-driver-spike/driver/measure.mjs:88 — The condition at ".dev/features/loop-driver-spike/driver/measure.mjs:88" compares with a 6-character string literal that the test file ".dev/features/loop-driver-spike/driver/loop-driver.test.mjs" also adds, so the code may special-case the test's input.
    • Hint: Check that the code handles the general case and not only the test's value, and add a test with another input.

💬 Messages (2)

  • ℹ️ COVERAGE_PATHS_PARTIALLY_UNMATCHED 126 file paths in the coverage report are not tracked files in the repository; their coverage is not used.
    • Hint: Nothing to do if those files are generated or outside the repo; if changed files are missing from the table, check the coverage root.
  • ⚠️ AGENT_CONFIG_SECRET_UNSCANNED Agent config files were not scanned for committed secrets: `pr-number mode: the checkout is not the pull request head, so its committed agent config files were not read`.
    • Hint: Keep agent config files text, under 512 KiB and inside the repository, and run on a checkout of the pull request head (not PR-number mode or a compare-API diff), so they can be scanned for committed secrets.

🤖 Agent setup (106 files)

Agent setup: 106 files · claude-code

Agent setup files (106)
Path Kind
.claude/agents/pharn-review-lens.md claude-agent
.claude/agents/pharn-stage.md claude-agent
.claude/commands/pharn-build.md claude-command
.claude/commands/pharn-dev-build.md claude-command
.claude/commands/pharn-dev-eval.md claude-command
.claude/commands/pharn-dev-grill.md claude-command
.claude/commands/pharn-dev-memory-promote.md claude-command
.claude/commands/pharn-dev-plan.md claude-command
.claude/commands/pharn-dev-regress.md claude-command
.claude/commands/pharn-dev-review.md claude-command
.claude/commands/pharn-dev-ship.md claude-command
.claude/commands/pharn-dev-verify.md claude-command
.claude/commands/pharn-grill.md claude-command
.claude/commands/pharn-loop-close.md claude-command
.claude/commands/pharn-loop-quick.md claude-command
.claude/commands/pharn-loop.md claude-command
.claude/commands/pharn-memory-promote.md claude-command
.claude/commands/pharn-plan.md claude-command
.claude/commands/pharn-regress.md claude-command
.claude/commands/pharn-review.md claude-command
.claude/commands/pharn-ship-close.md claude-command
.claude/commands/pharn-ship-quick.md claude-command
.claude/commands/pharn-ship-stages.md claude-command
.claude/commands/pharn-ship.md claude-command
.claude/commands/pharn-spec.md claude-command
.claude/commands/pharn-supervise.md claude-command
.claude/commands/pharn-test.md claude-command
.claude/commands/pharn-triage.md claude-command
.claude/commands/pharn-verify.md claude-command
.claude/hooks/enforce-writes-scope.cjs claude-hook
.claude/hooks/enforce-writes-scope.test.cjs claude-hook
.claude/hooks/hook-wiring.test.cjs claude-hook
.claude/hooks/protect-trusted-paths.cjs claude-hook
.claude/hooks/protect-trusted-paths.test.cjs claude-hook
.claude/hooks/require-loop-record.cjs claude-hook
.claude/hooks/require-loop-record.test.cjs claude-hook
.claude/hooks/set-writes-scope.cjs claude-hook
.claude/hooks/set-writes-scope.test.cjs claude-hook
.claude/hooks/writes-scope-release.test.cjs claude-hook
.claude/settings.json claude-settings
.claude/skills/phargus-diagnose/SKILL.md claude-skill
.claude/skills/phargus-diagnose/agents/openai.yaml claude-skill
.claude/skills/phargus-diagnose/references/compatibility.md claude-skill
.claude/skills/phargus-diagnose/references/data.md claude-skill
.claude/skills/phargus-diagnose/references/diagnosis.md claude-skill
.claude/skills/phargus-diagnose/references/tools.md claude-skill
.claude/skills/phargus-diagnose/scripts/capture.mjs claude-skill
.claude/skills/phargus-diagnose/scripts/operations.mjs claude-skill
.claude/skills/phargus-diagnose/scripts/phargus.mjs claude-skill
.claude/skills/phargus-diagnose/scripts/records.mjs claude-skill
…and 56 more files

pharn-ci 0.10.0 · report schema v2 · coverage format lcov

…ts the tests found

The pharn-ci diff-coverage gate on PR #457 read 89.31%. Tests now execute every added line of
driver-core.mjs and measure.mjs in process. They found two defects in paths driver-1 never took:
duplicate step ids on the Step 6d path (the SPEC revert was silently replaced) and a spent step
budget that skipped the close steps. Both are fixed and pinned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PrzemekGalarowicz
PrzemekGalarowicz merged commit 3b09299 into main Nov 17, 2026
8 checks passed
@PrzemekGalarowicz
PrzemekGalarowicz deleted the spike/loop-driver-core-113 branch November 17, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant