Repository navigation
Conversation
Checkpoint of scripts/finish_fable52.py, forked from finish_terra.py, with the release constants, the 20261027 base pins and the no-engine-change block that replaces Terra's reference install, plus docs/fable52/seed_digest.csv. The driver's remaining upgrade paths, the spec, the freeze, the dating script and the tests follow in later commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This release changes no engine, so the driver drops finish_terra's install-references step and the gates that served only it. Prepare checks the base's engine pins and the audit's publication record in their place, and every later step re-checks them through stage_upgrade. The export keeps the scope check from Haiku 5.5's and Terra's drivers: with release 20261027's exclusion record put back, every incumbent's modelStats must equal the base's byte for byte. A new check, exclusion_scope_problems, compares the release payload with that scope export and refuses any difference the ruled records do not explain, the new model's cells included. An early rehearsal stages the release's record under a provisional stage.json, as Terra's staged its build. bind-seed before prepare now only checks the seed and grounding against their pins, because prepare binds the seed itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/fable52/proposed_exclusions.json holds the one record Max's d2180 rules: scenario_122's Minnesota state_refundable_credits, whose $0 reference rests on a Certificate of Rent Paid the prompt never states (reference_audit/2026-10-29-mn-renters-crp). It uses the newest record field order, that of the 2026-10-10 wave. docs/fable52/spec.json pins that file and quotes the rulings: the engine (no change; the references stay 20261027's on policyengine-us 2.60.0), d2180 and r24 as Terra's spec states it. The spec carries the ruled adjudication (prompt_ambiguity, unlisted_input, state_local_rule), the derivation sentence, the wave sentence and empty triage adjudications. snapshot_date and adjudications_written_on hold [LEAD:] placeholders that the driver and the freeze refuse until the lead fills them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The doc covers mechanics only: the base and input hashes (each recomputed), what the driver keeps from Terra's, what goes with the install and what checks the base's state instead, the ruled exclusion as release 20261010's mechanics apply it, the two disclosures the release note carries, the lead's command sequence with real paths, the steps the lead does by hand, and [LEAD: ...] placeholders for every result. It marks the export's exclusion_scope_problems check as new code and explains why the publication check takes the committed build as its snapshot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/freeze_fable52.py forks freeze_terra.py for a release that installs no reference build. It keeps every gate Terra's no-upgrade branches run and replaces the install gates with verify_no_upgrade: a release stage (require_release_stage), no installed build (driver.stage_upgrade, which re-checks the base's engine pins and the audit's publication record), and every staged scoring source at release 20261027's pins with the release's exclusion record. The live description counts the ruled record on policyengine-us 2.60.0 after the base's three engines, and the record's engines must be exactly those four. The record gate passes the spec's triage decisions, as triage does. The freeze states the references' base pins in the frozen manifest, checks the snapshot's us_reference_outputs.csv and the explanations pin, requires real YYYY-MM-DD days in the spec, and preflights the freezer's engine-setup and provenance reads before writing any destination. --finalize re-gates a rerun publication check through stage_upgrade. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release adds a wave, Max's 2026-11-01 ruling (d2180), so the evidence in reference_audit/2026-09-28/verification/judge_verdicts.json needs it: scripts/date_fable52_judge_verdicts.py takes Terra's restatement file and Argon's uncommitted-wave update. Release 20261027 decided no wave, so its seven waves all name their commits and nothing waits to be filled. The new wave joins uncommitted with the spec's adjudications_written_on, and its decision gains the stage verdict it names. The script requires the record's new waves to be exactly the spec's ruled wave (and a triage wave only when the spec lists triage decisions), each deciding exactly the spec's outputs on a real YYYY-MM-DD day on or after the wave. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Synthetic stages, verdicts and mock boards; the real exporter runs the scope check on a three-household MOCK board. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The step tests moved driver.ROOT before reading the base's exclusion record; they now read it once under the checkout's root, as Terra's tests did. The mock boards keep a parsed answer in every model's output group, since the exporter's heatmap MAE is NaN without one. The any-one-output property now states when the scope check refuses an exclusion: one that reorders the exporter's groups, drops programStats' mape or excludes an eligibility output. The ruled output does none of these, which a new test checks on the committed references. The scope check reports a changed field list on its own line instead of folding it into the n/nParsed message; it refuses the same payloads. The dating script's docstring no longer spells the release tag, which the drivers define once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The heatmap and programStats bullet now says the group's entries keep their fields. The paragraph after the bullets names the three kinds of exclusion the check refuses (a group's first scored row, a group's last parsed answer on a nonzero reference, an eligibility output), says that d2180's output makes none of them, and names the tests that check it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two fixture docstrings in the Fable 5.2 driver tests named the real model and the real ruled output, so a reader of the public repository could take them for the unpublished result. They now say "the new model", "MOCK answers" and "MOCK verdict". The fixtures' data stays synthetic and unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven tests pinned the working tree as it stands before the lead fills the spec, freezes and dates, so they would fail on the release PR's last head. - The committed-spec test now accepts each day as the lead's placeholder or a real UTC day on or after the ruling's, and any triage list (the gates hold its form). The placeholder refusals run on filled copies of the spec: the record gate's test, and a parametrized freeze test that replaces the one reading the committed spec. - Each suite's filled spec carries no triage decision, so a triage decision the lead adds leaves the ruled-wave tests as they are. - The dating tests read release 20261027's adjudication record and evidence from git at the base commit, since the freeze and the dating step rewrite the working tree's copies. No gate changes. With the spec's days filled in the working tree, the dating and freeze suites and the spec-reading driver tests pass too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--finalize gated the publication record again only when sweep_timing.json had changed since export, so the 2026-10-29 record (policyengine-us 2.78.0) bound the release whenever the lead skipped the publication check or the check stopped before it wrote. What --finalize now refuses that it accepted before: - a stage whose sweep_timing.json is unchanged since export (it finalized such a stage without re-gating the record); - a publication record whose engine is not PyPI's newest at the check's read (pypi.newest_at_read), or whose PyPI read is not a UTC stamp on or after the spec's snapshot_date: the committed 2026-10-29 record stops it; - a missing or placeholder snapshot day. verify_publication_record takes the day as checked_on_or_after, and only finalize passes it, so every earlier step gates the record as before. The freeze's main hands finalize the spec's snapshot_date. A record of the reference engine itself (PyPI's newest still 2.60.0) passes, as the check writes it. Tests: the finalize test that expected no re-gate is now a refusal test; finalize passes the day to the gate, stops on the base's record through the real gate, and refuses a missing day; the driver's currency check has parametrized cases on the base's record from git and a Hypothesis property (any read day against any snapshot day). Z 112 passed; the driver's 39 publication-record and engine tests passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
validate_verdicts alone refuses a new or re-opened verdict whose judge was not Claude Opus 5.5, did not run through scripts/run_audit_claude.sh, or is not stamped in UTC, and no test of this driver's copy reached that block: removing it left the suite green. New tests, each on an otherwise valid MOCK verdict: - 16 identity defects (the new model requested or reported, both models reported, none, another runner, a stamp without an offset or off UTC, a bare day, no stamp): a new verdict goes pending, and remove_invalid sets it aside with the identity reason; - the same defects on a case judged anew whose wrong models are incumbents alone (the not-carried arm), and on a seed-carried verdict that names the new model (the other arm), which stops the step; - 12 isolation defects (no judge_isolation, another login, no transcript, a second or another user message, an answer other than verdict.json, a second accepted answer, another tool call, a git working directory, a turn at another effort): pending at validate_verdicts, and refused by verify_judge_provenance, which export runs; - UTC stamps written with Z and with +00:00 pass. No gate changes. The judge, verdict and isolation subset: 91 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t run) Review r1's M3: no test ran the adjudication gate this release changed. This ports Haiku 5.5's tests of ruled_adjudication_problems to the d2180 entry the adjudicate-exclusions step writes, and runs triage's record gate (stage_adjudications) and the freeze's (verify_adjudication_record) without a stub: - the step appends one entry and keeps every base entry's bytes and place (64 decisions, 50 exclusions), and excludes exactly the release record's 50 outputs; - the gate passes the exact built entry and refuses it missing or edited by hand (a Hypothesis property over a value, the key order, an added and a dropped field), and each of 13 named field edits; - stage_adjudications passes the step's record with and without the ruled case re-opened, and refuses the ruled entry edited after the step with "Staged adjudications of the ruled outputs", a changed, dropped, reordered or added base decision, a re-opened case's decision field and a decision the spec does not list; - the freeze's record gate passes the step's record and refuses the edited entry and a record that disagrees with the staged exclusion record; - the one-to-one check refuses a spec with no adjudication for its record, the record decided twice, or an adjudication with no record, and the restated-list check refuses a restatement of an undecided output. Part 2 ports Terra's triage_stage tests through driver.triage itself. The Mac ran on battery, so these tests have not run yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t 2; not yet run) Review r1's M3 asked for Terra's triage_stage tests with the d2180 entry. The triage_stage fixture takes the stage after --step adjudicate-exclusions, installs the release's exclusion record, re-opens the ruled case and gives driver.triage MOCK collected annotations, one MOCK wrong row for each of the 64 decided cases. driver.triage then runs its real record gates: - it accepts the step's own record unchanged, rewrites the ruled case's note to the decided class and leaves no flag or unresolved row; - it refuses an exclusion record the decisions do not make (the base's 49), a decision the spec does not list, judge fields written by hand on a re-opened case, any other change to a recorded decision, a dropped decision, an edited date convention, the ruled entry edited after the step, and a duplicate key the entry gate cannot see; - it stops on a pending verdict, a missing or hedged one, and a case whose wrong_model_count disagrees with its rows. The Mac ran on battery, so these tests have not run yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…et run)
Review r1's M2: the equal-treatment gates ran in no test of this copy.
(a) A differential source test. For each function the build report lists
as kept byte for byte (driver 60, freeze 17, dating 4) the Fable 5.2
source must equal Terra's line for line. For each one kept with names
only (driver 20, freeze 7), its code, docstrings aside, must equal Terra's
after a committed rename map: ndiff.py's map plus two temporary-folder
prefixes and two verify_restatements messages that now name the base by
its role. Terra's suites then cover these bodies here, and an edit to one
fails until it gets tests of its own. A guard test shows the comparison
catches a one-token edit (">= 0" to "> 0" in verify_usage).
(c) Two of the report's names-only functions are off the lists:
finalize, which M6 changed and tested, and provenance_note, whose note
now gives the new model as a case's only reason to re-open. Terra's
round-trip property of the note is ported for it, with a test of the
reason sentence.
(b) A spy test that export runs require_combined_predictions,
verify_verdict_inventory, stage_upgrade, both verify_reference_pins (the
committed pins with the release's hash once the pointer names this
release), verify_parse_failure_flags, verify_new_model_inputs and
verify_judge_provenance before build_payload, with their arguments; that
an early or partial export skips only its named gates; and that a
refusal by any gate leaves the payload unbuilt and nothing written.
The Mac ran on battery, so these tests have not run yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… run) Terra's verify_restatements names its base as 20261014; the M2 rename map named it by the 20261027 spelling ndiff prints after its own renames, so the differential test would have failed on verify_restatements. A read of the three scripts against the lists (no test run) now finds every listed function equal: 81 byte for byte, and 27 equal in code after the map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s (M4, part 1) Review r1's M4: no test reached the freeze's write half, verify_frozen or rebuild_payload, and the no-build path had never run in any release. This ports Terra's end-to-end harness to that path. A module-scoped template writes a MOCK checkout from git at BASE_COMMIT and a MOCK stage as prepare, bind-seed and install-exclusions leave it (the real write_release_exclusions), then runs the driver's real adjudicate-exclusions, triage, write_judge_provenance and export on it. Its MOCK data: Claude Fable 5.2 right on every output but the ruled one, an incumbent wrong on the 63 decided cases with seed verdicts classed as the base record's judges, the ruled case's Opus 5.5 v2 verdict with its sidecar and transcript, synthetic boards from the two exporters on which both scope checks hold, and a stand-in for freeze_snapshot.main that writes what the freezer writes. The first test runs release.main with --dry-run: every gate passes and nothing is written. Part 2 adds the write half: the freeze, a second freeze, verify_frozen's refusals of the freezer's defects, and the refusals before any write. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M4. Through the real main, on the MOCK stage part 1 built: - a freeze writes the release: the pointer, the version list's live entry, release 20261027's references with the release's exclusion record, the committed copy, the annotations (the staged record and CSVs, the base's explanations and amendments), the serving rows, the manifest's statements, the predictions asset and the cache; - a second freeze writes the same bytes, and the rebuild reads a scratch copy of the bound bundle; - verify_frozen refuses each of Terra's freezer defects the no-build path still has, and the base's 49-record exclusion record frozen in place of the release's, each pinned to its own check; finalize then refuses the stage, which records no completed freeze; - verify_frozen refuses another window, date, run, artifact, the base's exclusion pin, a short incumbent roster and an amendments record the freeze did not write; - the refusals that must come before any write do, in the freeze and its dry run, with the freezer never called: Terra's receipt, evidence, rehearsal, provenance, payload, scope (checks a and b), validator, roster, verdict, annotation, record (the ruled entry's included), held reference, incumbent row, run state, treatment, window, prepare-time hash and pin refusals, and the no-build path's own: a stage that records a build, a changed repository input or historical version entry, and a staged exclusion record that is not the release's. Tests only; no gate changes. tests/test_freeze_fable52.py: 236 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M7 (D P3-1). Finish_terra held its installed build to code-held
pins at every step; this driver checked the staged files at export only.
Now:
- main, at every later step (bind-seed, install-exclusions, judge,
adjudicate-exclusions, provenance, triage, export), runs
verify_prepared_inputs, then holds stage/scoring and bundle/us to their
code-held pins with the exclusion record the stage records installed
(release 20261027's until install-exclusions writes the release's), and
the bundle's explanations to release 20261027's bytes
(verify_staged_explanations, new);
- prepare checks the explanations right after prepare_inputs, before
prepare_cases renders a prompt and before an early rehearsal exports;
- export checks them in every mode before it builds or writes.
Gate change. What each now refuses that it accepted before:
- every later step but export: a stage.json whose files are empty or omit
a prepared input; a model-provenance.json that disagrees with the staged
run; a bundle reference off its pin with stage.json edited to follow;
- every later step, export included: a scoring-source reference off its
pin (no step after prepare read the scoring source), an exclusion record
in either copy other than the one the stage records installed, and
bundle explanations that are not release 20261027's, or missing (only
the freeze checked them);
- prepare: explanations a stopped prepare left edited in the bundle, which
prepare_inputs keeps (it rendered judge prompts from them);
- export: explanations off the base's bytes, which it bound into
release-ready.json, in every mode (early and partial included).
Tests: _resumable builds a complete prepared stage (the files-{} stage the
later-step tests used is refused now); the export spy lists the new gate.
tests/test_finish_fable52.py 580 passed; tests/test_freeze_fable52.py 236
passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M8 (D P3-2). No driver test reached the code-held reference pins where export and prepare call them; deleting either call passed the suite. Now, with the real verify_reference_pins: - export passes the pinned references to build_payload, with the live pointer at the base and at this release; - export refuses, before build_payload runs, each of the bundle's five reference files one byte off its pin and a bundle scoring on release 20261027's record; - export refuses each committed reference one byte off its pin, under both pointer arms, and the committed record the pointer does not name (the release's before the freeze, the base's after it); - prepare's resolve_base refuses each committed reference one byte off its pin before it reads the live pointer and before prepare stages anything, and reads the pointer once they pass. prepare_inputs' own pin call stays covered by M2's byte-for-byte test. Tests only; no gate changes. tests/test_finish_fable52.py: 606 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…M17) The coordinator's request, as review r1's M17: the repository is public, and these comments, docstrings and one MOCK prompt string in the freeze tests named Claude Fable 5.2 while stating what it answers on the MOCK board. Each now states it of the new model on the MOCK board: - the harness section comment on the MOCK run and its first request; - _h_judge_ruled_case's docstring and the two comments in _h_write_stage (the ruled case, and the new model right everywhere but the ruled output); - the docstrings of the payload-edit, kept-case-claim and bundle-row tests; - the line the kept-case-claim test appends to a prompt, now "MOCK: one more wrong answer joined this case." (the test asserts on the refusal only). A grep of the three test files and the scripts finds no other sentence that states what Claude Fable 5.2 answers, gets, misses or scores. Text only; no gate changes. The nine freeze tests these touch pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M9 (S P3-1). Check (a) held only modelStats to release 20261027's. base_restoration_problems now also compares the scope export, built with release 20261027's exclusion record, with the live payload: - country, scenarios, policyengineBundles, globalWeights and referenceExclusions, serialized as the payload serializes them, so key order counts; - each incumbent's heatmap rows, keyed by model and variable; - each incumbent's scenarioPredictions cells, keyed by scenario, variable and model. On a case this release re-judged, the annotation fields (release_20261006.TEXT_CELL_FIELDS | CLASS_CELL_FIELDS) may differ. Export passes rejudged_cases(stage) in release mode and no case in early and partial modes, where the bundle holds the base's annotations. The freeze's rebuild passes rejudged_cases(stage), as export did. A caller that passes nothing holds every case to the base. Gate change. Check (a), in export's build and in the freeze's rebuild, now refuses a scope export whose country, scenarios, policyengineBundles, globalWeights or referenceExclusions differ from release 20261027's; an incumbent heatmap row that differs, or that only one side holds; and an incumbent cell that differs, or that only one side holds, outside the annotation fields of a re-judged case. Before, each of these passed whenever modelStats matched. Tests: tests/test_finish_fable52.py 624 passed; tests/test_freeze_fable52.py 239 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M10 (S P3-3, P3-4). Tests only; no gate changes. S P3-3, the scope check's branches: - PAYLOAD_EDITS gains the nine edits the review lists, through export: the ruled cell scored again, its excludedReason, its excludedInput, a field added to it, one model's ruled cell removed, the ruled group's programStats n, the correctPct of a segment that holds no ruled household, a modelStats entry's field order, and the ruled exclusion entry. - SCOPE_EDITS runs nineteen edits straight through exclusion_scope_problems on the MOCK pair and names the line each must report. Sixteen reach one branch alone and must report exactly that line: among them the key-order clause (scored moved), the scope export's scored flag, the modelStats n and nParsed clause behind the RELEASE_SCORED gate, the heatmap and programStats counts, a segment that loses nothing, and the ruled referenceExclusions entry. Three more name their branch among the lines they cause: "its models changed", "lacks a cell" and the seen check. - A ruled segment may lose only its ruled rows. - The single-cell perturbation runs over every field of every ruled cell by parametrize (51 cases), and a test holds the field lists to the MOCK pair's cells. S P3-4, the properties: - incumbent_drift: a value that turns from int to float (1935 to 1935.0, and a drawn integral value), which == calls equal and json.dumps tells apart. - Excluding any one output: every open row runs, by parametrize. - The fold's property drives the driver's prepare_inputs on a MOCK complete run, with Terra's real-preparation setup. It varies the incumbent count, missing answers, an optional column and the run's key set (an output dropped, repeated or renamed): prepare refuses a changed key set before it publishes; otherwise the fold adds exactly 1,984 rows on the reference key set, the missing answers stay missing, the cells keep the run's text, and every incumbent row is kept. Two examples always run, and each example clears and removes one folder under the basetemp. - The manifest property's oracle reads STATED_CHANGES and STATED_PINS, spelled out in the test, and a new test holds MANIFEST_CHANGES and MANIFEST_PINNED to them. Tests: tests/test_finish_fable52.py 751 passed; the manifest tests in tests/test_freeze_fable52.py 15 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M12 (Z P3-1). release_versions transformed the working tree's live description, which export binds only by hash and verify_repository_inputs frees, and held it to three regex shapes. It now reads release 20261027's description from git at BASE_COMMIT, transforms that text (the exclusion count by engine and the model count), and requires the working tree's to be release 20261027's text or that result, so a second freeze passes. count_exclusions keeps its form checks. Gate change. The freeze, in every mode, now refuses a working-tree live description that is neither release 20261027's text nor this release's form of it: for example the review's case, a claim inserted after "same household facts", where each pattern still matches once. Before, any text that matched the three patterns passed, and the freeze published the working tree's words. It also refuses a live version whose id is not release 20261027's live version. Tests (tests/test_freeze_fable52.py): either allowed text gives the release's description; the claim is refused in both forms, directly and end to end before any write (the receipt bound to the edited file); the text comes from git's copy; any text inserted anywhere is refused (Hypothesis); a reworded description is refused, and count_exclusions still refuses a reworded form. 246 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M13 (Z P3-2). verify_frozen's two reference checks move into verify_committed_references: the frozen run's reference files must be release 20261027's pins and its exclusion record the release's, and the snapshot's us_reference_outputs.csv release 20261027's. verify_frozen calls it as before, and finalize now calls it on every run, with driver.release_exclusions_sha256(), right after the receipt shows a completed freeze and before any other repository read. Gate change. Finalize now refuses a frozen run whose reference_outputs.csv, scenarios.csv or scenarios.csv.meta.json differs from release 20261027's pin, a frozen exclusion record that is not the release's (release 20261027's put back, say), and a snapshot us_reference_outputs.csv that is not release 20261027's: the review's case, an edit between the freeze and --finalize. Before, finalize hashed only the frozen run's reference_outputs.csv.meta.json (verify_base_engine, through stage_upgrade) and passed every other edit until CI ran. Tests (tests/test_freeze_fable52.py): the finalize fixture records the check; a unit test holds its arguments and its place (a refusal stops finalize before stage_upgrade, the publication gate and the stage checks); end to end, after a real freeze, each of six edits stops --finalize and writes nothing, and the untouched freeze passes the check. The finalize and freeze tests: 35 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, M14 (Z P3-3, P3-4, P3-5).
Z P3-3. base_wave_releases now compares each of release 20261027's seven
waves with that release's evidence file, read from git at BASE_COMMIT
(base_evidence), and verify_base_cases holds every case outside this
release's waves, and the file's note and other fields, to the same file. A
case this release's waves decide keeps the base's fields besides the two
the step writes ('current', and 'published', which it drops).
Z P3-5. new_waves now requires each new wave's written day, which it takes
from the spec, to be the day the record's date conventions state for that
wave (recorded_written_day parses the wave's own sentence, singular or
plural, as driver.release_date_conventions writes it).
Z P3-4. Tests now run restatements: a restatement with its verdict, a
decision that disagrees with its verdict, and the empty set. The review's
fix for the empty set, to let it through with a note, would make the step
accept a state it refuses now; the brief says no gate may get weaker, so
require(cases) stays and the test pins it. The relaxation waits on a
ruling.
Gate change. The dating step now refuses:
- a record of a base wave that differs from release 20261027's in its
commit, release, pull request or commit day (the review's case:
2026-10-14's commit moved to another non-null string). Before, it
checked the waves' order and that each named some commit;
- a case outside this release's waves that is not release 20261027's
(edited, dropped or added), a changed note or added field, and a
ruled-wave case carrying a field the base lacks. Before, each passed and
was written back;
- a spec whose written day differs from the one the frozen record states
for the wave, and a record that states none (the review's case: frozen
with 2026-11-10, the spec moved to 2026-11-11 before dating). Before, the
step wrote the spec's day as it stood at dating.
Tests: tests/test_date_fable52_judge_verdicts.py 71 passed; the inherited
source tests in tests/test_finish_fable52.py 118 passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…M15)
The driver writes a wave of one record as "The 2026-11-01 wave's decision
... was written on <day> UTC". tests/test_adjudications.py read each later
wave's sentence with startswith("... wave's decisions") and a split on
" were written on ", so after the dating step it would stop on that
sentence once the lead updated its wave pins.
- tests/test_adjudications.py: _wave_written_day reads either form. The
plural path is the old reader unchanged; the singular path pairs
"decision" with "was". The test now accepts the singular form, which only
this release's driver writes, and refuses a sentence whose noun and verb
disagree in number. test_a_wave_sentence_states_its_written_day_in_either_number
pins both forms and four refusals.
- tests/test_finish_fable52.py: a differential test runs the shared
reader over the conventions release_date_conventions writes for three
written days and reads every later wave's day. With the old reader all
three cases fail.
- docs/fable52/design.md: the by-hand step names the pins to update and
the exception a re-opened scenario_122 case needs in
test_each_decision_records_the_verdict_it_reviewed_by_its_wave_release
(its (2026-11-01, judged day) pair beside ("2026-09-22", "2026-09-23"),
and later_than_wave rising by one).
No driver, freeze or dating gate changes.
Tests: tests/test_adjudications.py 47 passed; F date-convention subset 14
passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The coordinator ruled on Z P3-4: restatements keeps require(cases), and the relaxation stays a saved patch outside the repository. The design note's hand items now say what the lead does if the dating step stops on "no committed decision is on a re-opened case": rule on the saved patch after an independent review, with no edit to the dating script before that ruling. Docs only; no gate changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, E F3 and F4. The record's date conventions say a recorded verdict can postdate adjudicated_on only up to the last day its wave's decisions were written, and they print the triage wave's written day as the spec spells it. Gates that change, and what each now refuses that it accepted before: - release_date_conventions, triage wave (F3): a triage_adjudications_written_on that is not a real YYYY-MM-DD day (utc_day): the "[LEAD: ...]" placeholder, "2026-11-1", "2026-13-01", "12 November 2026", " 2026-11-12". Before, any non-blank string passed. It also refuses a written day before the triage wave's own day (the review's case: decided 2026-11-12, written 2026-11-02). - adjudicate_exclusions (F4), through the new verify_written_days: a ruled or triage wave's written day before the day of a verdict its decisions record (judge_rejudged_on, else judged_on_utc), such as the ruling's own day 2026-11-01 for a verdict judged 2026-11-10; a written day after today's UTC date (utc_today); and a decision of the wave whose verdict names no day. Before, it took any real day on or after the ruling's. It writes nothing when a day fails. - freeze_fable52.release_spec (F3): a spec that lists triage decisions without a real triage_adjudications_written_on. Before, the freeze never read that day. - freeze_fable52.verify_adjudication_record (F4): a staged record whose wave was written before the day of a verdict its decisions record, even when the record's date conventions and the spec agree on that day. Before, the freeze took it. No gate accepts anything it refused before. Tests. F: utc_today; the triage placeholder spellings (7); a Hypothesis property that the triage day passes exactly on or after its wave, with the review's case as an example; a Hypothesis property that the ruled wave passes exactly when its verdict's day <= written <= today, an unrelated entry never counting, with the review's case as an example; triage-wave cases (5); an undated ruled entry; adjudicate_exclusions refusing four days without writing, and taking the verdict's day up to today. The adjudication_stage fixture and Z's h_template fix today's date, so no test reads the clock. Z: release_spec's triage spellings (6) and its scope; the record gate on the MOCK H stage refuses a written day of 2026-11-08 for the verdict judged 2026-11-09 and takes 2026-11-09. Reversals: with each new call or check taken out, 4, 1, 6 and 6 of these tests fail. docs/fable52/design.md states the new limits where the lead sets each day. F 776 passed, Z 262 passed, D 71 passed (all full runs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, E F6. The record's values existed only as typed text in the proposal and in one test's literal (0.0, 1921.0), so an edit to the proposal's alternative_value, with the spec re-pinned and the literal changed, passed every test. test_the_ruled_record_states_what_its_audit_computed reads the 2026-10-29 audit's sweeps and traces from release 20261027's commit and requires the working tree's copies to be those bytes. For each engine (2.60.0 and 2.41.0) it checks: - the sweep's engine is the trace's, its harness matches its pin, and its as-built pass reproduces every reference; - the sweep's moved list holds the ruled output, scored, and nothing else, moving from the record's frozen_value to its alternative_value; - the trace gives frozen_value as built and alternative_value with a qualifying CRP, with eligibility the only switch and the household income, rent constituting property taxes and exemption flag unchanged. It also checks that the record's engine is the 2.60.0 sweep's, that the 2026-10-22 build's row and the committed row both hold frozen_value, and that every dollar figure in the record's alternative_reading and note and in the spec's reasoning is the trace's ($65,346.18, $79,330, $4,800, $1,921), the record's ($0) or the prompt's rent ($28,235, from the household description built from the committed scenario, which names no CRP). Reversal: the review's edit (alternative_value 1912.0, its text and the spec's sha256 and reasoning changed to match) fails the new test at the sweep's with_crp value, while the existing spec test still passes. Tests only; no gate changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut (M19) Review r1, E F7. sweep_timing.py check counts scenario_122's output as scored, so a newer engine that moves it stops --finalize. The design note now says the gate stays strict: the lead stops and queues a decision for Max under his latest-engine rule, with no change to verify_publication_record. The coordinator chose this option over accepting a scored_differ that holds only the ruled outputs. Docs only; no gate changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1, S P3-5 names a mutant that removes verify_usage's validity require. No test in the three new files reached that check, so the mutant would survive. test_preparation_refuses_a_row_with_invalid_usage_or_call_provenance gives one MOCK run row a negative cost, an infinite token count, a negative live call count or a live time over its elapsed time, and requires prepare_inputs to stop with verify_usage's or verify_call_provenance's own message before the treatment check and before it publishes anything (Terra's preparation test, on one row). _prepare_and_check takes the row defect as an optional argument. Tests only; no gate changes. The mutant run follows in the report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_a_record_that_differs_from_the_releases_is_refused_at_later_steps composed up to three record mutations, and two of them found the ruled record by position (index -2). After add_another_output inserts a record before the last one, drop_the_ruled_record popped the added record, so the "mutated" record matched the release's and verify_reference_pins rightly passed it. Hypothesis drew that pair during M11's survival run on 2026-11-10; the test has had the gap since 0de026d. The two ruled-record mutations now find the record by its key, so no combination cancels. The test asserts its premise (the written record differs from the release's) before it calls the gate, and two @example rows pin the pairs that cancelled by position: add then drop, and add, edit, then drop. With the positional drop restored, the premise assertion fails on the first row. Tests only. verify_reference_pins is unchanged, and no gate accepts anything it refused before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rerunning round 1's mutants (M11) at 2d85f69, the one that deletes new_waves' require(wave <= day) survived its listed test, test_any_real_day_on_or_after_the_wave_is_accepted_and_no_other. Since M14, new_waves also requires the spec's day to equal the day the frozen record states, and the property's early-day branch kept the record's default day (2026-11-10), so the new check refused those cases on its own. Only test_a_wave_other_than_the_spec_says_is_refused's 2026-10-31 case still caught the mutant, and only by its message. The early-day branch now edits the record to state the spec's early day, as a hand edit would (the driver refuses to write such a day), so the wave's own check is the only one that can stop it, and each branch matches its refusal's message: "names no day" for a string that is not a real day, "was written before it" for a real day before the wave. Three @example rows always run: 2026-10-31, the wave's own day and 2026-11-1. With the require deleted, the property now fails. Tests only. new_waves is unchanged, and no gate accepts anything it refused before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2, N1 (P1). Since c812dd1, check (a) held every incumbent cell of the scope export to release 20261027's payload and let annotation fields differ only on the cases this release re-opened. Triage applies d2180's decision to the ruled case whether or not the new model joins its wrong models (apply_adjudications), so when the release carries that case over, the incumbents' failureSource, failureSubtype, caseAnnotation, caseFailureSources and caseFailureSubtypes there are not release 20261027's, and export and the freeze's rebuild both stopped, while the freeze's verify_annotation_amendments requires those same fields. The error began in round 1's wording of the fix (S P3-1). The gate change, as the lead directed under N1: - Check (a) now accepts, on a case this release decides and did not re-open (decided_cases: the spec's ruled records and its triage items), differences in CLASS_CELL_FIELDS and caseAnnotation. It refused them before; that refusal was the bug. - On those cases it still refuses a row annotation, a referenceExplanation or any other field that is not release 20261027's. Every other kept case still keeps every field, and a re-opened case keeps its exemption of every annotation field. - Export passes decided_cases() beside rejudged_cases() in release mode and none in a rehearsal; the freeze's rebuild passes the same. Tests: - On real data from git (F): the entry adjudicate-exclusions builds (MOCK verdict judge fields), applied with apply_adjudications to release 20261027's committed annotations, moves nine incumbent cells of the ruled output in the class fields and caseAnnotation only; the frames first reproduce release 20261027's own cells byte for byte. Check (a) passes them with no case re-opened, passes them re-opened, and refuses all nine with nothing exempted. - The same entry moved to an undecided kept case (MOCK) still fails; a row annotation or referenceExplanation edit on the decided case fails unless re-opened; a prediction edit fails either way. - Through the real exporter on the MOCK board: a MOCK decision applied to the release board's annotations on a carried case passes decided or re-opened and fails with neither; the same decision on an undecided case, or a reworded row annotation, fails. - Z: a second template carries the ruled case over (MOCK incumbent wrong on it, the new model right on it and opening one undecided case, release 20261027's payload built from the base's annotations). Every gate passes and the freeze writes; with no decided cases check (a) refuses it; a base row annotation or referenceExplanation still binds. The re-opened template keeps its tests. - The property covers none, this and another case in each of the two sets; decided_cases has unit tests on the committed spec and on a spec with a MOCK triage decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(N2) Review r2, N2 (P2). The M2 differential compared the inherited definitions' bodies with Terra's but not the module-level values those bodies read, so widening JUDGE_ATTACHMENTS, JUDGE_EVENT_TYPES, ACCOUNT_KEY, EMAIL_ADDRESS or JUDGE_PROMPT_NUDGE failed no test. Both of the fixes the review names, tests only: - The differential now takes every module-level name a listed definition loads, and every name such a value loads in turn, bound in either script: 53 in the driver, 11 in the freeze, none in the dating step. After the rename map each must be Terra's, save an allow-list of the seven by-design constants (BASE_COMMIT, BASE_ENGINE, BASE_MODELS, NEW_EXCLUSIONS, BASE_SHA256, SEED_DIGEST_SHA256, BASE_REFERENCE_SHA256), which earlier tests pin; a test requires the values that differ to be exactly those seven, and one shows the comparison sees "skill_listing" added to JUDGE_ATTACHMENTS. - Terra's holders, ported: the runner's ATTACHMENTS, EVENT_TYPES, account and e-mail patterns and PROMPT_NUDGE equal this gate's sets, and the transcript and provenance values equal finish_terra's. - Four isolation defects, one per widened set or pattern: an unlisted attachment, an unlisted event type, an e-mail address and an account key in a context attachment. Each is refused by validate_verdicts and by verify_judge_provenance. F: 869 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2, N3 (G P3-1). write_release_exclusions writes the scoring copy first and the bundle copy second, and install_exclusions writes stage.json last. An install stopped between the two writes leaves the release's record in the scoring source with no install recorded, and the pin M7 added (5077a68) then refused the rerun with "this release revises no reference value". This narrows a refusal M7 added, to two exact byte states. At install-exclusions with no install recorded, main now accepts the scoring copy's exclusion record when it holds exactly release 20261027's pinned bytes (as before) or exactly build_release_exclusions' bytes (release_exclusions_sha256()). The rerun of a half-written install, which M7 refused, now passes main's pins and finishes the install. Still refused, as before: - any other bytes in the scoring copy's record, at install-exclusions and at bind-seed (an edit to either record); - the release's record in the scoring copy at bind-seed; - any other scoring reference off its pin; - the bundle copy off release 20261027's pin (verify_reference_pins, and the staged-input check against stage.json). So an install stopped after both writes and before stage.json still stops with "staged input changed: ...; prepare a new stage". It accepts nothing where an install is recorded: a reinstall (which later steps ask for when the spec's record no longer matches the one installed) stopped between its writes is still refused at the scoring copy's pin. Tests (F 876 passed): - test_before_the_install_the_staged_record_is_the_bases adds the release's record edited, and accepts the release's exact record at install-exclusions only; - test_an_install_stopped_after_its_first_write_can_be_run_again stops a MOCK install at the bundle copy, reruns it through main and runs the next step's pins; - test_a_half_written_install_takes_no_other_byte_state holds the bundle copy (with and without stage.json restamped) and another scoring reference beside the half-written state; - test_a_stopped_reinstall_is_still_refused pins the scope: no acceptance where an install is recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y run
Review r2, N4 (G P3-2, R P3-4). The relaxation of require(cases) stays
unapplied. Two additions, neither weaker than before:
1. The dating step's restatements now reads docs/fable52/judge_provenance.json
through bound_provenance(stage): the stage's release-ready.json must hold a
"judge_provenance" entry whose path is JUDGE_PROVENANCE_PATH and whose
sha256 is that of the bytes the step parses. Export writes the entry, and
the freeze and finalize rewrite the receipt with it kept.
It now refuses, where before it accepted:
- a provenance record that changed after export, whatever cases it names
(before, require(cases) stopped one only when it shared no case with
the committed record);
- a stage without release-ready.json, a receipt with no
"judge_provenance" entry, an entry naming another path or another
sha256, and a missing record.
It accepts nothing it refused before.
2. The freeze's dry run ends with a read-only "Dating step:" line
(dating_preview): whether the record the freeze commits (the staged
record, which verify_frozen holds byte for byte) holds a decision on a
re-opened case (rejudged_cases, which verify_judge_provenance holds the
provenance record to). It names the cases, or says the dating step stops
on none and points to the hand step. It computes the list with the
dating step's own reopened_decisions, which restatements now uses too.
It refuses nothing new.
docs/fable52/design.md's hand step now says the dry run warns before the
freeze writes, so the ruling on the patch can come first, and names the
binding. It points at runs/driver-fix-r3/m14-p3-4-empty-restatements.patch,
round 2's patch refreshed so it applies after these changes (round 2's
copy stays where it was).
Tests: D 82 passed, Z 269 passed, F 876 passed.
- D: the MOCK receipt binds the MOCK provenance (bind_provenance);
test_the_step_reads_only_the_provenance_export_bound[6 defects],
test_the_bound_provenance_is_the_record_export_bound,
test_reopened_decisions_are_the_cases_restatements_restates[4].
- Z: the releasable dry run's line names the ruled case;
test_the_dry_runs_dating_line_is_the_dating_steps_own_finding[reopened|kept]
runs the dating step's restatements on the record the freeze commits and
the bound provenance, and checks the line against it in both states;
test_the_freeze_writes_the_release checks the frozen receipt keeps the
binding.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP. This branch will carry the board release that adds Claude Fable 5.2 as the 52nd model and applies one exclusion Max ruled on 2026-11-01. It holds no result yet: scores, the note and the frozen snapshot arrive with the freeze commits.
What the release does
dashboard-data-20261027.state_refundable_creditsfrom scoring for every model (decision d2180). Its $0 reference rests on a Certificate of Rent Paid input that no prompt states (reference_audit/2026-10-29-mn-renters-crp/). The exclusion record goes from 49 records to 50, and each model's scored outputs from 1,935 to 1,934.In this branch so far
scripts/finish_fable52.py, forked fromscripts/finish_terra.py, with the 20261027 base pins and a no-engine-change block in place of Terra's reference install (checkpoint; unfinished).docs/fable52/seed_digest.csv, the digest of release 20261027's 681 judged cases, which the driver binds as its audit seed.Plan
scripts/freeze_fable52.py, the dating script,docs/fable52/spec.jsonanddocs/fable52/design.md, with tests (Hypothesis properties included).Invariants the driver must hold (tests to follow)
modelStatsequals the base's byte for byte.Who wrote and who reviews
A Claude Opus 5.5 agent writes the driver. The session that carries the release runs on Claude Fable 5.2, the model this release adds, so a reviewer from another model reads every reference, exclusion, adjudication or wording decision that could move that model's row.
🤖 Generated with Claude Code