Skip to content

Spec 017: the shifted-prior gap, test record on seed 344, panel review, write-up and slides - #35

Merged
IanMayo merged 5 commits into
mainfrom
claude/keen-wozniak-77td7j
Nov 14, 2026
Merged

IanMayo merged 5 commits into
mainfrom
claude/keen-wozniak-77td7j

Conversation

@IanMayo

@IanMayo IanMayo commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Summary

Spec 017's test on seed 344 (scored once, plan tl-f.json as registered), the five-reviewer panel, and the two posts. PR 34 merged the study's code and registration. A registered study, not a gate; nothing trained, nothing spent.

Record (experiments/20261114-tl5/; shifted split, 451 scored periods):

  • 0.37 of true manoeuvres add less than 20 to the χ² when left out (Annex B: 0.16).
  • Control ÷ reference told every manoeuvre: 2.27 [2.04, 2.55].
  • Not being told the ones under 20 costs 1.58; control ÷ that reference 1.43 [1.28, 1.61].
  • Attainable share 0.65 / 0.44 / 0.14 at thresholds 10 / 20 / 40.
  • Control ÷ the search under the targets' own prior: 1.17 [1.06, 1.29].
  • By the logic fixed in the decision record, a proposer study is not recommended (one of two conditions held).

Panel, round 1 (review.md): five approve with findings; every number reproduces. The reading is narrowed:

  • The pooled split is two kinds of period: where a manoeuvre is hidden the control is 1.16 times the reference not told it; where none is, 2.65. Under a third of periods carry 83% of the attainable gap.
  • The logic's second condition counted manoeuvres, not cost, and turns on the match tolerance: the "no" is the rule's output, not the evidence's.
  • The attainable part is at least half a prior problem (of 138 detectable manoeuvres the control misses, the right prior finds 66).
  • The detectability measure is not additive and the references are not bounds.
  • Corrections to the first record are listed at the end of the README (the tuning/test "disagreement", the IMM/EKF ratios, shifted 2).

Also: write-up site/blog/2026-11-14-shifted-gap.md, slides post and deck, figures with sidecars, scripts tl5_{supplement,tracks,charts}.py, AGENTS.md (results, gotchas, next fresh seed 345), tasks ticked.

Recommended follow-on (owner's choice): a manoeuvre prior that adapts to the period, classical and on CPU, registered against the control with the true-prior search as its ceiling; no GPU proposer now.

Not delivered: the split under each way of fitting the reference; course and speed in the estimands; a joint detectability; unit tests for detect.py and run5.py.

No SRD, annex or constitution change in this PR.

Test plan

  • uv run pytest -q tests/unit/test_timelate5.py (tables regenerate from the summary)
  • uv run ruff check . and ruff format --check .
  • Site build with the new posts
  • CI green

🤖 Generated with Claude Code

https://claude.ai/code/session_019erPajyAwQ6mBQ2Xz4xNHs

…tracks

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019erPajyAwQ6mBQ2Xz4xNHs
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019erPajyAwQ6mBQ2Xz4xNHs
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019erPajyAwQ6mBQ2Xz4xNHs
…ecord's corrections

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019erPajyAwQ6mBQ2Xz4xNHs
@IanMayo IanMayo changed the title Spec 017: the shifted-prior gap, test record on seed 344 (panel and write-up in progress) Spec 017: the shifted-prior gap, test record on seed 344, panel review, write-up and slides Nov 14, 2026
@IanMayo
IanMayo merged commit e6fc853 into main Nov 14, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants