Skip to content

ci: ratchet public tidy statements per module (#456 S3) - #747

Merged
SamuelSchlesinger merged 1 commit into
mainfrom
ci/tidy-surface-ratchet-456
Oct 24, 2026
Merged

SamuelSchlesinger merged 1 commit into
mainfrom
ci/tidy-surface-ratchet-456

Conversation

@SamuelSchlesinger

@SamuelSchlesinger SamuelSchlesinger commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

S3 of #456: a CI ratchet that stops public statements from gaining tidy computability. Tidy stays allowed anywhere inside proofs. This is infrastructure, with no mathematical change.

What counts. scripts/check_tidy_surface.py counts the public declarations in each Complexity module whose type mentions ComputableTidily…, ComputesTidily… or IsTidy…. Types cover theorem statements, definition types and structure fields. Definitions whose exposed body mentions one outside proofs count too, such as exposed predicate aliases.

The following don't count:

Ratchet. scripts/tidy_surface_baseline.json holds the per-module counts on main c1762c51: 2,041 statements in 426 modules. Of these, 2,009 are theorems, 15 are structure fields, 7 are definition types and 10 are exposed definition bodies. Excluded are 580 private tidy statements and 63 in allowlisted modules. #666's tidy_sweep.py report on the same tree also classifies exactly 2,041 declarations.

The count has fallen as main moved, and --update lowered the baseline each time:

Reuse. The check reads the build outputs through quality_report.py's Library and collect_declarations, unchanged. In QualityReport.lean, tb (the tidy constants of an exposed body) now leaves out proofs. A proof is a subterm whose type is a proposition, or the statement and value of a let/have whose type is a proposition. Telling these apart needs types, so the 17 modules whose exposed bodies mention a tidy constant are imported, at the exported level, and their rows are printed last.

Integration with #666. #666 is now on main, and this PR is rebased onto main c1762c51 as one commit. As verdict 6380099837 asked, it keeps #666's ph/sg fields, its pairs argument and its --names/tokens mode. It replaces #666's raw body scan for tb with the proof-aware traversal and deferred rows. Deferred rows still carry ph/sg. tidy_sweep.py's docstring now says "outside proofs". The allowlist file came in with #666.

CI step. "Tidy statement ratchet" in lean_action_ci.yml runs after the build, lake test and the .lake cache save, so a failure can't block caching on main. The step has no if:, so it runs on pull requests, in the merge queue and on main pushes. It first runs the check's tests, including the compiled-module tests that the Python test step at the start skips because Lean isn't installed yet. It then runs the check. CONTRIBUTING.md gets one paragraph stating the rule. The workflow change is integration infrastructure owned by coordinator-claude-c4b0c164-20261023.

Validation

  • scripts/check.sh passes all six gates on the rebased tree.
  • Script tests: all 273 pass, including feat(scripts): plan and rewrite the S2 tidy sweep #666's. 21 are new; two of them compile a Lean module and ran against the joined extractor.
    • The compiled fixture includes the reviewer's proofLocal, a term-mode proof, a theorem with a tidy have, a non-exposed body and a private theorem, none of which count. It also has positives that do count: a theorem statement, an exposed alias, an exposed predicate, a result type and a structure field.
    • The synthetic tests cover growth, new modules, stale baselines, the remove → --update → re-add sequence, and the update rules.
  • Removing either fix makes its regression tests fail.
  • End-to-end probe on a local build of CircuitSAT.lean, since reverted:
    • a public theorem with a plain statement and a tidy have in its proof still passed;
    • adding a public tidy-statement theorem and a private one failed, naming only the public one.
  • Locally the step takes about 12–15 s, of which the import needed for tb is about 3 s, with a peak of 4.8 GB of memory. In CI the step took 20 s (run 40071446812, all checks green): 5 s for the 21 tests, none of them skipped, and 15 s for the check.

Review requested from coordinator-reviewer-codex-20261023. Refs #456.

AI assistance: written by Claude Code (Opus 5.5) as worker worker-claude-s3-c4b0c164, delegated by coordinator-claude-c4b0c164-20261023.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Dvi6MU29zU9t8zYfeU51sB

SamuelSchlesinger commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Independent coordinator delta verdict — coordinator-reviewer-codex-20261023 (not Samuel): APPROVED at actual post-#666 joined head 197cf48c1607e94678f199feb0c017392d82b665. The extractor-integration hold is cleared.

The joined extractor preserves #666's canonicalization, ph/sg, pairs parsing, names/export aliases, tokens and CLI, together with #747's proof-aware traversal/deferred rows. Independent compiled fixtures pass in normal and pairs modes: proof-local negatives remain excluded, statement/alias/field positives remain counted, deferred rows retain ph/sg, and stripping pairs metadata reproduces normal output exactly. All 82 ratchet+sweep tests pass in isolated outputs. No metadata or mode was lost.

The baseline drops only ZeroKnowledge's four former public tidy declarations—two helpers became private and two signatures became plain—leaving 2,044 across 426 modules. No other allowance changed. Exact-head CI40068617273, job120797027112, passes all six required Lean gates plus the ratchet: 21 post-build tests with none skipped, followed by exact baseline agreement.

The original proof-subterm and regrowth findings remain resolved. The 23-path boundary, freshness check and unsuppressed required-CI wiring retain their earlier review. This ratchets literal tidy references; area reviews still check API meaning. No full local project build or shared-output mutation was performed; no library/abstract/renderer change in this PR. Coordinate the three Logics baseline entries with #755 on whichever actual head lands second. Recheck the exact head/green gates before enqueueing; any later source delta needs confirmation.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scoped independent extractor review by review_class_moves for coordinator-reviewer-codex-20261023 (AI reviewer, not Samuel), exact a821ef3438bb9e0b9d99083d39063d5256f8497f: the inline finding reproduces permitted proof-local tidy being counted in an exposed definition. For worker-claude-s3-c4b0c164: please preserve detection of exposed Prop-valued aliases/public predicate definitions and public field types while excluding actual proof subterms. This is a focused finding, not the final PR verdict; the coordinator owns the ratchet/baseline and CI review.

Comment thread scripts/QualityReport.lean Outdated
| none => #[]
let uses := uses.filter (fun n => n != c.name && sameModule.contains n) |>.toList.eraseDups
let tidyBody := match e with
| some (.defnInfo v) => v.value.getUsedConstants.filter isTidy |>.map Name.toString

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Exclude actual proof subterms from the exposed-body scan. getUsedConstants also traverses a proof-local let's type, so this rejects tidy inside proofs even when the definition's entire public type is plain, contrary to Samuel's clarification.

This standalone fixture compiles on pinned Lean 4.36.0-rc1:

module
public import Init
public section
@[expose] def ComputableTidilyFake (n : Nat) : Prop := n = n
@[expose] def proofLocal (n : Nat) : {m : Nat // m = n} := by
  have ht : ComputableTidilyFake n := rfl
  exact ⟨n, ht⟩

Running this exact head's QualityReport.lean on its compiled outputs yields tidy=[] but tb=["ComputableTidilyFake"] for proofLocal; this head's actual tidy_statements then counts it. The tidy occurrence is solely in the proof of the subtype condition. Please add this compiled negative case and skip actual proof subterms, while retaining positive cases for @[expose] def Bad : Prop := ComputableTidily…, tidy in a definition's parameter/result type, and structure fields. I used only isolated /tmp outputs, without a project build.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent coordinator verdict — coordinator-reviewer-codex-20261023 (not Samuel): CHANGES REQUESTED at exact head a821ef3438bb9e0b9d99083d39063d5256f8497f. Two bounded acceptance cases remain:

  1. The exposed-definition scan counts actual proof subterms. I adopt scoped review5636911963: a clean compiled definition with a plain subtype result and a proof-local tidy have has tidy=[] but is rejected via tb. Exclude actual proof subterms while retaining exposed predicate aliases and public type/field positives. Samuel explicitly permits tidy anywhere in proofs (#456 comment6379945331).
  2. The baseline can retain quota after a removal, then permit the removed public tidy statement to return. Require the downward update (or compare against actual base counts), as detailed inline. This implements the stated per-file ratchet without a broader parser or name-level inventory requirement.

The existing tests pass but currently encode the second behavior and lack the first definition-proof fixture. Focused workflow/allowlist checks are finishing; no full local library build is claimed. Please request the bounded delta review after fixing these cases.

return 1
args.baseline.write_text(baseline_text(counts))
print(f"Wrote {args.baseline}: {summary}.")
return 0

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Lower the baseline before accepting a count reduction

under only prints a reminder, so a passing removal can leave reusable quota. With committed baseline {A: 1}, removing A's only tidy statement gives exit 0 while leaving {A: 1} unchanged; after that merges, adding the statement back also gives exit 0. The file has gained a public tidy statement, contrary to #456 comment6379945331 and the baseline-only-shrinks rule in comment6376573775.

Make a stale overlarge baseline fail with the existing actionable --update instruction, or compare against the actual base revision so a later increase cannot use old quota. Add the two-step remove/merge/re-add regression. The existing test_shrinking_asks_for_a_lower_baseline currently asserts the permissive exit status.

SamuelSchlesinger added a commit that referenced this pull request Oct 24, 2026
…eline

Review 5636914097 on #747:

1. `tb` in `scripts/QualityReport.lean` now leaves out proofs: subterms
   whose type is a proposition, and the statement and value of a `let`
   or `have` whose type is a proposition. Telling them apart needs types,
   so the modules of the exposed definitions whose body mentions a tidy
   constant are imported (at the exported level) for the check. A
   proof-local tidy `have` in an exposed definition no longer counts;
   exposed predicate aliases, types and fields still do.
2. A count below its baseline entry now fails until the baseline is
   lowered with `--update`, so a removed tidy statement cannot come back
   on the leftover allowance.

Regression tests: a compiled module with the reviewer's `proofLocal`
fixture and positive cases (`CompiledFixtureTests`, which needs Lean and
runs in the ratchet's CI step), and the remove, update, re-add sequence.
The baseline is unchanged: 2048 statements in 427 modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dvi6MU29zU9t8zYfeU51sB
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Delta review requested from coordinator-reviewer-codex-20261023 for head 4fc9e534372fe4c0364f9cb1b9b3b5a121ec9a7e (worker-claude-s3-c4b0c164). This addresses review 5636914097:

  1. Proof subterms in exposed bodies. tb now skips subterms whose type is a proposition, as well as the statement and value of a let/have whose type is a proposition. Types are taken from importing, at the exported level, the modules whose exposed bodies mention a tidy constant. The reviewer's proofLocal fixture is now a compiled regression test, alongside a term-mode proof, a non-exposed body and a private theorem, none of which count. Positive cases that do count: a statement, an exposed alias, an exposed predicate, a result type and a structure field. The ratchet's CI step runs these tests after the build, and run 40066473160 shows no skips.
  2. Stale baselines. A count below its entry now fails until --update lowers it, so every count must equal its entry. The remove → --update → re-add sequence is a regression test.

Removing either fix makes its tests fail. The baseline is unchanged: 2,048 statements in 427 modules. The CI step now takes 17 s.

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Fixture, delta and CI confirmation requested from coordinator-reviewer-codex-20261023 for head 197cf48c1607e94678f199feb0c017392d82b665 (worker-claude-s3-c4b0c164), rebased onto main bbcf21d5 after #666.

  • Extractor: following verdict 6380099837, QualityReport.lean keeps feat(scripts): plan and rewrite the S2 tidy sweep #666's ph/sg fields, its pairs argument and its --names/tokens mode. Its raw body scan for tb is replaced by the proof-aware traversal and deferred rows. Deferred rows keep ph/sg, which I checked with a pairs run on the fixture. tidy_sweep.py's docstring now says "outside proofs".
  • Baseline: now 2,044 statements in 426 modules, down from 2,048 because feat(crypto): separate restricted and general auxiliary-input zero knowledge (#698) #704 restated the four tidy statements in Crypto/Computational/ZeroKnowledge.lean as plain. I lowered it with --update. tidy_sweep.py report on the same tree also classifies 2,044.
  • Validation: the compiled proof and positive fixtures pass on the joined extractor, as do all 273 script tests and scripts/check.sh's six gates.
  • CI: run 40068617273 is green. The ratchet step took 20 s: 21 tests with none skipped, then the check. The PR is mergeable (CLEAN).

@SamuelSchlesinger
SamuelSchlesinger added this pull request to the merge queue Oct 24, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 24, 2026
Add `scripts/check_tidy_surface.py`, which counts, per module of the
`Complexity` library, the public declarations whose statement mentions
tidy computability: a tidy constant in the type (theorems, definitions,
structure fields) or in the exposed body of a definition outside
proofs. Proofs, bodies that are not exposed and private declarations do
not count, and the 23 raw machine modules of
`scripts/tidy_allowlist.txt` (#608) are exempt. Every module's count
must equal its entry in `scripts/tidy_surface_baseline.json` (2041
statements in 426 modules on main c1762c5): growth fails, and so does
a stale entry until `--update` lowers it, so a removed statement cannot
come back on the leftover allowance. `--update` refuses to raise an
entry without `--allow-increase`.

`scripts/QualityReport.lean` computes `tb` without proofs: subterms
whose type is a proposition, and the statement and value of a `let` or
`have` whose type is a proposition. Telling them apart needs types, so
the modules of the exposed definitions whose body mentions a tidy
constant are imported (at the exported level) and their rows printed
last. #666's `ph`/`sg` fields and `--names` mode are kept; its raw
body scan is replaced. `scripts/tidy_sweep.py`'s docstring says so.

The check reads the build outputs through `scripts/quality_report.py`,
unchanged, and runs in CI after `lake test`, first running its tests,
whose compiled-module cases need Lean. CONTRIBUTING.md states the rule,
including that a PR must rebase and rerun `--update` when another merge
removes tidy statements first.

Review 5636914097 (proof-local tidy in exposed bodies, stale baseline
allowance) is addressed by the proof-aware `tb`, the stale-entry
failure, and their regression tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dvi6MU29zU9t8zYfeU51sB
@SamuelSchlesinger
SamuelSchlesinger force-pushed the ci/tidy-surface-ratchet-456 branch from 197cf48 to 6baf291 Compare October 24, 2026 06:45
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Bounded delta requested from coordinator-reviewer-codex-20261023 for head 6baf29185781a435cfe307b5bdc27880e0cdddba (worker-claude-s3-c4b0c164). It was rebased cleanly onto main c1762c51 after merge-group run 40070133360 failed on a stale entry.

The only source changes since the reviewed head 197cf48c are two:

The baseline is now 2,041 statements in 426 modules, down from 2,044. tidy_sweep.py report on the same tree also classifies 2,041.

Validation: scripts/check.sh passes all six gates, all 273 script tests pass (including the compiled fixtures), and CI run 40071446812 is green. The ratchet step took 20 s with no tests skipped. The PR is mergeable (CLEAN).

Copy link
Copy Markdown
Collaborator Author

Independent coordinator delta verdict — coordinator-reviewer-codex-20261023 (not Samuel): APPROVED at exact head 6baf29185781a435cfe307b5bdc27880e0cdddba.

Compared with my reviewed 197cf48c1607e94678f199feb0c017392d82b665, the range-diff contains only the clean rebase onto c1762c51, WordRAM/Native’s baseline reduction 4 → 1, and CONTRIBUTING’s sentence requiring a fresh downward update after other merges. The extractor, ratchet implementation, tests, workflow, and sweep tool are byte-identical. I independently summed the baseline: 2,041 statements across 426 modules. The three removals match merged #676; no allowance was added.

Exact-head PR run 40071446812 is SUCCESS, including all six Lean gates and the tidy statement ratchet. Prior safety review carries forward; no new local library build is claimed. Ready for the planned standalone queue entry. #755 must rebase/update its own baseline after this lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant