[juaristi22]

Follows Vahid's review of #1381 (#1359). Rebased onto main (d4d66a9), and retargeted from #1381's merged branch. It also carries Vahid's two round-3 nits on #1426, which merged without them (see "#1426's nits", below).

Microcosm's regime-gated QRF decides whether each recipient row is zero or positive (or negative) with a boosted classifier, the sign gate. The gate is calibrated only on the donors it was fitted to, so on every other row it draws a class at the wrong rate. On the LCFS donors it draws rare outcomes far too rarely; on the SPI stage 2 it over-draws the rarest benefit reports.

This PR adds an opt-in that calibrates each gate out of fold, and turns it on for every UK fit that has a gate. The US fits keep today's gate, and their draws do not move.

How the QRF model changes

RegimeGatedQRF(gate_calibration=...) takes a name from the new GateCalibration constants. in_sample, the default, is the gate as before, byte for byte. cross_fitted works in four steps:

1. It cuts the training rows into five folds, stratified by class. With groups, it deals whole groups, such as households, to folds.
2. For each fold, it fits a gate the same way, with the same classifier and weights, on the other four folds. That gate scores the fold it left out.
3. It fits one weighted calibration on the pooled out-of-fold scores: a slope and a per-class offset on the gates' log-probabilities. For two classes this is Platt scaling. At the fit, the calibrated out-of-fold probabilities' weighted mean equals each class's weighted rate. The balance holds on the pooled donor scores, not on any set of recipients.
4. At draw time, each row's probability is the mean of the five mapped fold gates.

Fallbacks, each recorded:

• A target keeps the in-sample gate when a class has fewer rows, or groups, than folds (in_sample_thin_class).
• It also keeps it when a fold's training rows carry no weight for a class (in_sample_weightless_fold).
• The calibration checks its score equations at exit, with damped steps where Newton stalls. If the balance still fails, or the out-of-fold ranking is worse than none, every row gets the class rates (cross_fitted_class_rates).

FittedRegimeGatedQRF.gate_outcomes() names what each target got, and every opted-in stage writes it to its evidence under gate_outcomes.

What does not change:

• The fold assignment and the fold seeds derive from the one seed the in-sample gate takes, so every magnitude forest, bootstrap and keyed uniform is identical.
• In a chain, a sign that changes moves later targets' amounts through their predictors.
• Checkpoints name the calibration only when it is cross-fitted, and the grouping only when given, so in-sample checkpoints are byte-identical.
• The fit.qrf@1 graph kernel takes the option too.

Where it runs

Each stage declares gate_calibration: cross_fitted on its operation, and the schema allows it as an enum:

• LCFS consumption: the chain, the prepayment-meter flag and the zero-road-fuel redraw's petrol share;
• ETB services and WAS wealth, both chains;
• SPI income: the tape forest, the FRS reference forests its rank transport reads (they take stage 1's declaration, so the gate that places a twin's observed sign and the gate that reads it agree), and stage 2;
• the SPI housing shell, NTS bus use, salary sacrifice and the UC reporter redraw.

Fits on people or benefit units pass their households as fold groups. The ETB VAT rate is positive on every donor, so its fit has no gate. The gate is bypassed, and nothing moves, in gambling participation, the property components, the LISA amounts and the LCFS gas payers.

Gate level: every gated UK fit, out of fold on its own donors

Method:

• A hook on a full spine build from this branch recorded each gated fit's donors: features, signs, weights and the stage's fold groups. That is 177 fits across eight stages.
• Five outer folds hold donors out, whole households where the stage folds by household.
• On each outer training set, the in-sample gate and the cross-fitted gate are both fitted as RegimeGatedQRF fits them.
• Both are scored on the held-out donors.
• The figure is the weighted mean probability of each fit's rarest class over its weighted rate, out of fold.

Nine fits whose rarest class has fewer rows or households than folds are left out, since the build keeps the in-sample gate there anyway. No fit below rests on fewer than three donors in its rarest class.

Within 10% of the rate, in sample → cross-fitted, and fits with the lower log loss when cross-fitted:

• LCFS consumption, 25 fits: 9 → 25, and 25 of 25 have the lower log loss. The in-sample median is 0.85 of the rate. Its worst are solid fuel, drawn at a tenth of its rate, and the rare zeros of the near-universal divisions, at between a fiftieth and a quarter of theirs.
• WAS wealth, 17: 14 → 17; 16 of 17.
• SPI housing shell, 12: 11 → 12; 12 of 12.
• SPI tape forest, 21: 17 → 21; 21 of 21.
• SPI FRS fits (the reference forests' folds and stage 2), 87: 65 → 86; 85 of 87. In sample, stage 2 over-draws the rarest benefit reports about five to seven times.
• ETB services 3, NTS bus use 1, salary sacrifice 1, UC reporter redraw 1: all within 10% either way, and all have the lower log loss when cross-fitted.

Overall:

• within 10% of the rate: 122 of 168 fits in sample, 167 of 168 cross-fitted;
• the lower log loss: 165 of 168 cross-fitted.

The LCFS gates are the most overfit: with fewer than 10,000 donors, the classifier has no early stopping.

Frame level: two full spines from main, control against this head

Both builds use the same recipe, on main 4a36edc and on this branch as it stood on it. The branch has since been rebased onto d4d66a9. None of the 286 commits between touches microcosm-fit. The first 152, up to 79a533c, touch no stage this PR opts in either: 119 touch none of the UK build or microcosm-data, and the 33 after c8984c4 change the UK build's diagnostics, staging and publication code, the data contract's diagnostics checks and the frame's concept mapping. Of the 134 since 79a533c, #1426's change two stages this PR opts in: lcfs_consumption (the zero-emission share by income fifth, each road fuel levelled to its household share, and the vehicle excise duty inputs) and hmrc_spi_income_spine (the stage-1 forests' age band and the employment pay floor). The spines were not rebuilt on any of them, so the figures below are this PR's move on 4a36edc. Shares are of frame households or people, at prior weights, with the donor's weighted rate in brackets. The control passes all 55 spine gates. The head passes all 55 with the gambling stake-scale floor at 0.3 (see "The gambling stake scale", below).

LCFS consumption, with top_up_gambling_participation in place:

• Rare spending:

  • solid fuels: 0.30% → 2.17% (2.47%);
  • liquid fuels: 3.10% → 3.91% (4.54%);
  • education: 4.29% → 6.71% (5.83%);
  • tobacco: 8.17% → 11.04% (10.34%);
  • gambling stakes: 14.49% → 16.76% (17.03%);
  • National Lottery: 10.44% → 10.91% (11.42%);
  • betting and gaming: 1.95% → 2.70% (3.58%).

• The near-universal divisions' zeros are now drawn at their rate. Incidence moves to the donor's:

  • food: 99.67% → 99.05% (99.15%);
  • recreation: 98.91% → 97.02% (97.27%);
  • communication: 91.30% → 89.39% (89.13%);
  • household furnishings: 93.47% → 92.19% (92.19%);
  • miscellaneous: 98.48% → 97.41% (97.31%).

• Totals:

  • the twelve divisions' prior-weighted total falls 1.5%;
  • the six divisions held at the shared frame-to-donor ratio fall with it, as the ratio goes from 1.254 to 1.230;
  • restaurants and hotels fall 4.4%.

• Excise inputs. They keep their national levels; who carries them changes:

  • tobacco duty is 1.80 times tobacco spending, from 2.21, on 11.0% of households;
  • alcohol duty is 0.44 times alcohol spending, from 0.40;
  • Lottery Duty is 0.63 times National Lottery stakes, from 0.58;
  • the other gambling duties are 5.28 times betting and gaming stakes, from 4.91.

  The last two keys are carried at the top-up's participation, 62.7% and 29.8% of households, in both runs. The betting and gaming multiple now rises, where on the pre-top-up stage it fell: the top-up sizes the added players' stakes to the seen households' mean, and that mean falls.

The other stages:

• WAS wealth: student loan balances 2.86% → 3.79% (6.66%), cash ISAs 34.50% → 35.48% (33.28%). The other targets move by less than a point.
• ETB services: rail support 14.15% → 15.89% (16.52%). Education and bus support barely move.
• SPI housing shell, SPI households: subletting 0.88% → 0.30% (0.32%); the rest move by under a point.
• SPI income, SPI adults: small moves, the largest an owned company's income share, 0.95% → 0.58% (0.65%). The stage-2 benefit reports that fall back to the class rates are mostly reports that spi_benefit_coherence then zeroes or restores from the twin for SPI rows.
• NTS bus use, salary sacrifice and the UC reporter redraw: under 0.1 points.
• Rows. The head frame has 0.4% fewer households, with the same weight total. The CGT incidence clones key on the SPI-drawn incomes, so slightly fewer households carry gains to clone.

The gambling stake scale

One gate needed a new bound: uk_stage_lcfs_consumption_gambling_participation. Betting and gaming's amount scale is 0.41 in the head, against 0.63 in the control and the old floor of 0.5. The floor is now 0.3, on María's ruling of 2026-11-03. The scale is the chain's level times the walk's selection:

• the level: the seen households' stakes over the stake model's mean for the same households. It goes 0.79 → 0.53;
• the selection: the stake model's mean for the seen households over its mean for the added players. It goes 0.79 → 0.77.

The stake model, and the walk that picks players, barely move. What moves is the chain's level: it now draws betting and gaming for 38% more households, with smaller stakes each.

Lowering the floor hides no move away from the official figures, because neither depends on the scale:

• Incidence. The top-up brings both builds exactly to the household shares that GSGB 2025's adult rates imply: 53.2% over four weeks and 62.7% over twelve months for the National Lottery, and 20.6% and 29.8% for betting and gaming. The gate's own participation checks pass.
• Amounts. Every duty base is levelled to HMRC's 2024-25 figure in both builds: £14.47bn across the twelve betting-and-gaming bases, and £7.77bn of National Lottery ticket sales. So the base per playing household is the same in both. The scale pins the added players' mean to the seen households', so each class's share of a base follows its count.

What the scale compares is the chain's diary-level stakes against the stake model's draws for the same households.

• The head's chain gives its betting-and-gaming players about the LCFS donors' own stake per player, where the control's gave about 1.5 times it.
• The stake model runs about twice the donors' per-player stake in both builds.
• So the scale falls. A draw that puts the added players' mean more than about 3.3 times the seen players' still fails.

VAT coverage

Max's review pointed out that this moves the COICOP totals the VAT coverage is derived on. It does, by about 1.7%.

policyengine-uk divides household VAT by survey_consumption_coverage, 0.67, times the household share of receipts, 0.7. Its base is the twelve COICOP divisions times each household's full-rate share. On a6299504, the cut policyengine.py certifies now, 0.67 puts 2024-25 VAT 3.0% above receipts, and Max's open PolicyEngine/policyengine-uk#2634 re-derives it there as 0.69.

On the two spines, at prior weights:

• consumption falls 1.5%, from £994.4bn to £979.2bn;
• the full-rate VAT base falls 1.7%, from £400.1bn to £393.3bn. Households' average full-rate share barely moves (0.402 both times), so the fall comes from the consumption totals, not the mix;
• most of it is restaurants and hotels (−£1.5bn), transport (−£1.3bn) and recreation (−£1.3bn), the near-universal divisions whose rare zeros the in-sample gates under-drew;
• the coverage this base implies goes from 0.676 to 0.665.

So read the move as a ratio. At any given coverage, VAT on a release built with this PR comes out about 1.7% lower, and the coverage that matches receipts falls by about the same ratio. The 0.676 and 0.665 are these spines' own levels, at prior weights, not a certified cut's. On that basis, if the 1.7% carries over, VAT at 0.69 on the first cut built with this PR comes out about £2.8bn below 2024-25 receipts, and re-deriving there gives about 0.68. The parameter's note asks for it to be re-derived when the dataset it matches changes, so that starts from whatever value is then current. obr.vat stays out of the calibration objective, so calibration does not pull it back, though the bound road-fuel row may recover part of the transport share. The exact figure needs a calibration of each spine.

Read before reviewing

• Re-pins.

  • The byte edits of qrf.py and kernels.py move:

    • the fit.qrf graph-parity pins;
    • the spec-loader golden;
    • the US seed protocol and seed map, with the F0 coverage report;
    • the pool tool's US bundle hash.

    Only CI's engine-us and engine-workflow tiers check the US ones. The seed pins, the F0 report and the pool hash are recomputed on 79a533c. Main re-pinned its own after the ACS local stages (Repin the US spec-engine contracts after the ACS local SSI and immigration stages #1496), and its tree matches them, so these carry the gate calibration's move alone. The commit that re-pinned main's stale ones on c8984c4 (5ed3468) is dropped. On d4d66a9 the seed digests still recompute to the pins and the F0 report's --check passes, so none of the three moves.

  • The stake-scale floor moves the UK gate battery's mirrored digests and the spine certification part's. They are recomputed on main's gates.json, which UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 changed too.

  • The UK declarations move source_stages.json, the coverage manifest and the H2 fixture.

• What remains is the chain's own. The LCFS chain conditions each target on its own earlier draws, so liquid fuels and betting and gaming still sit below the donor. No gate calibration reaches that.
• The US keeps the in-sample gate. Its donors and engine aren't on my machine, so it is Max's to measure if we want it.

#1426's nits

Vahid's third round on #1426 left two nits on spi_benefit_coherence, and #1426 merged without them, so this PR carries them, in its last commit. Neither changes a build's data.

• Pension payments. Neither contributory award is reduced for the copy's pension payments, though the copies carry the SPI tape's pension income (private_pension_income, from PENSION). JSA Regs 1996 reg 81 takes every pound above £50 a week off contribution-based JSA, and ESA Regs 2008 reg 74 half of every pound above £85 a week off contributory ESA. B10 tests the copy's work alone, so the awards stand. The stage's receipt now shows how much rests on the omission: for each award, the support and weighted award of the copies whose pension payments are above its threshold, the most a reduction could take off. A figure is withheld where one or two source households carry it or the rest of the award. JSA's £50 is read from policyengine-uk's gov.dwp.JSA.contrib.pension_disregard, with the other JSA limits. ESA's £85 is a constant, because policyengine-uk has no parameter for it. Reducing the awards would take them beyond B10's ruling, so the receipt measures that move first. The node now declares private_pension_income, which moves the H2 fixture.
• Part-year workers. The stage's docstring now says that its weekly work tests average the tape's annual pay over 52 weeks, which understates part-year claimants, above all of contribution-based JSA, which lasts at most 182 days (Jobseekers Act 1995 s. 5(1)).

Cost

A full spine build takes 21.7 minutes against 12.2. Each gate is now five gates, and the SPI tape forest's gates fit on 100,000 rows.

Tests

Run locally, targeted to the touched surfaces, and counted with JUnit, with no failures at the end and no skips but those named:

• microcosm-fit's QRF and kernel tests, with the UK stage tests of every opted-in stage: 538. Five test fakes lacked the new arguments; with them fixed, their three files pass, 61 tests;
• the spec engine, country spec, source stages, coverage, stage health and the fit-weight roster: 522;
• the stake-scale floor: the gate battery, its contract pins, the data contract, stage health and LCFS, 700;
• after the notes: source stages, coverage, the spec loader, re-emission and bundles, test_uk_qrf_gates, and H2 UK spine parity on the regenerated fixture (engine tier), 307;
• after the rebase onto c6fdfc3, 565: the spec loader, inventory coverage and seeds, the gate battery pins and the data contract, UK source stages and coverage, the fit gate and kernel tests, and the graph parity pins;
• after the rebase onto c8984c4, 587;
• for Vahid's second round, every UK test file that imports a stage this PR opts in, or its test fakes, main's new ones included: 1,068 engine-free and 43 engine-tier. Three skip, because the licensed LCFS, ETB and WAS tabs and the pinned SPI donor aren't staged in this worktree. After the NTS guard, the 13 files that use the NTS stage again: 465, one skipped for the same tabs. CI runs engine-free and engine-uk with --maxfail=1, so at 5ed3468 each stopped at its first failure. A third stale fake, in the E6 identity receipt test, failed only here;
• after the rebase onto 79a533c, 1,185: the set run after the earlier rebases, plus the transport evidence kernels (the count Count the two modules #1342 added to the transport bindings' identity, and merge main's US re-pin (#1496) #1495 fixed), staging v2, five of main's changed UK test files (the dense release assembler, full build CLI, graph terminal, rowwise national role and diagnostics tests), and the three files whose fakes Vahid's second round fixed;
• after the rebase onto d4d66a9, with UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426's nits, 3,101: every UK test file that imports a stage this PR opts in, or its test fakes, main's changed UK test files, the spec, source-stage, coverage, data-contract and gate-battery pins, the fit gate and kernel tests, and the engine-tier UK files that import an opted-in stage, with H2 UK spine parity on the fixture regenerated at each commit that moves it. Eight skip, because the licensed E6, WAS and SPI donor inputs and the pinned Chronicle feeds aren't staged in this worktree;
• ci_test_plan.py verify, the coverage manifest's --check and spec_engine_coverage.py --check.

Not run here, because they need the US engine: engine_contract/us/test_spec_engine_coverage_tool.py and engine_workflow/us/test_us_multispine_pool_tool.py.

🤖 Generated with Claude Code

[MaxGhenis]

Thanks María. This is a clear diagnosis, and calibrating the gate out of fold is the right fix. I'd land it after the Budget rather than in Monday's cut, since it moves 22 of the 24 excise inputs and the COICOP totals that the VAT coverage is being derived on.

Two changes before merge, and one wording fix:

1. Fall back when a fold has too little weight. _can_cross_fit counts rows, so a target with zero-weight rows can pass it and still leave a training fold with no weight. sklearn then raises ZeroDivisionError: Weights sum to zero. A reproducer: ten donors, five per class, eight of them at weight zero. The in-sample gate fits; the cross-fitted one crashes. Please check each fold's training weight per class, and keep the in-sample gate when it falls short. The LCFS weights should all be positive, but this matters before other stages opt in.
2. Check convergence before returning the calibration. When the out-of-fold scores are saturated, the Newton step is huge and the line search hits its 1e-10 floor. The loop then returns the identity map with no warning, so the rate balance fails silently. A reproducer: ten rows, five per class, every logit 29. That gives a positive-class mean of about 1 against a rate of 0.5. Please check the weighted residuals at exit, and fall back to an offsets-only fit or the class rates, or raise. Real gates probably never get there, and your LCFS figures show the balance held, but it should fail loudly.
3. Wording. The balance is exact on the pooled out-of-fold donor scores, not on recipients. Also, in the chain, a sign that changes moves later targets' amounts through their predictors, even though the forests and uniforms are the same. The changelogs and the module docstring should say both.

Please also add Hypothesis tests for the invariants:

• the weighted rate balance, including saturated and constant scores;
• the sparse-weight fallback;
• base-versus-head byte identity on the default path.

The two reviews (one by Claude, one by GPT-6.1 Sol) checked these, all on synthetic data:

• The default path is byte-identical to the base, and so are 36 checkpoint byte hashes across all seven regimes.
• Resuming under a different calibration is refused.
• LCFS is the only opt-in.
• The 22 UK manifest re-pins and the US seed-protocol and seed-map pins recompute.
• On held-out data the cross-fitted gate lands at about 1.0 of the true rate with 100 or more positives; the in-sample gate lands at 0.05 to 0.5.

I can send you the reproducers and the proposed property tests.

[vahid-ahmadi]

Review of 807e024

What I checked: staleness against main (0b1eecf), the cross-fit code in qrf.py (folds, seeds, calibration fit, draw-time use, thin and one-signed classes), the LCFS wiring, tests, changelog fragments and CI. I ran the 10 new fit tests, the whole microcosm-fit suite, and the engine-free LCFS and spec-loader tests at head (all pass), plus synthetic probes of the gate. No survey data; the frame figures below are yours from the PR body.

Max's review of 24 Oct still applies: there are no commits since it.

Must

1. The PR is stale against main and its gambling figures no longer describe the stage. The branch is 628 commits behind main and mergeable is CONFLICTING. Its base, uk-excise-quantities-1359, merged as Write policyengine-uk's alcohol, tobacco and gambling duty inputs from the LCFS stage #1381 on 26 Oct with ten commits that this branch does not have, among them top_up_gambling_participation, which now brings the National Lottery and betting and gaming keys to GSGB participation before level_excise_quantities shares the bases. So the gambling lines in the PR body, changelog.d/1359-lcfs-cross-fitted-gates.changed.md:8-16 and the rewritten notes at uk/spec/sources.yaml:4210 (Lottery Duty 0.61 times stakes, other gambling duties 5.5 times, 10.9% and 2.6% of households, and "the rest of the gap is the chain's own") come from a pipeline main no longer runs; part of that gap is now the top-up's. Main has no gate calibration work of its own, so the method is not superseded. I cherry-picked the three new commits onto main: the qrf.py and lcfs_consumption.py changes apply cleanly, and the conflicts are pins and spec text only (test_spec_engine_loader.py golden vector, inventory_coverage.py and us-f0-coverage.json digests, test_us_multispine_pool_tool.py bundle hash, release_input_coverage_manifest.json, source_stages.json, the sources.yaml notes and the uk_spine parity fixtures). Suggested fix: retarget to main, rebase onto it, re-pin, rerun the stage comparison with the top-up in place, and rewrite the gambling figures in the notes, changelog and PR body.

2. Max's items 1 and 2 reproduce at head. One addition to item 1: the zero-weight crash depends on the seed. Ten rows, five per class, only one row per class with positive weight: across 40 seeds, 6 raise ZeroDivisionError in a fold gate and 34 fit. So a fixed-seed test can pass while a build with another seed fails; the per-fold, per-class training-weight check he asked for, falling back to the in-sample gate, covers it. Item 2 also reproduces: with every logit at 29, _fit_gate_calibration (qrf.py:330) returns slope 1.0 and a positive-class mean of 1.0 against a rate of 0.5, with no warning. I agree with his wording point too.

Should

3. Folds are dealt by row, with no grouping key. _stratified_folds (qrf.py:319) deals rows, and _cross_fit_gate (qrf.py:409) takes no groups. That is fine for this opt-in: the LCFS chain fits one row per household (household_weight, recipients keyed on household_id). But the PR body proposes measuring person-level fits next (salary_sacrifice, uc_reporter_redraw and others), and there household mates who share household-level features end up on both sides of a fold. In a synthetic probe (4 people per household, an unobserved household effect, two household-level features with no signal), row folds kept incidence on fresh households (0.0299 against 0.0303 with household folds, true 0.0301) but fitted a calibration slope of 0.41 against 0.15 with household folds. The offsets absorb the level, but the per-row probabilities come out too sharp, so the rare class lands on the wrong rows. Suggested fix: before any person-level stage opts in, add a groups argument that deals whole households across folds, or refuse cross_fitted when the fit's rows are not one per group.

4. The thin-class fallback is silent. At qrf.py:2010, a target whose rarest class has fewer than five rows keeps the in-sample gate, the very bias this PR fixes (in a probe with 4 positives in 3,000 rows, in-sample mean probability 0.00033 against a rate of 0.00133), and nothing records it. Suggested fix: note in the chain's fit records or the stage receipt which targets fell back, so a later opt-in can see it.

Nit

5. fit.qrf@1 in kernels.py reads only n_estimators, zero_atol and max_samples_leaf, so a fit run through the graph kernel cannot opt in. That doesn't matter for LCFS, which calls the constructor directly, but the stage-by-stage plan will need it.

Checked and fine

• Rate recovery on fresh rows (5,000 donors, 100,000 fresh rows): true rates 0.0218, 0.0541 and 0.1224 gave cross-fitted means of 0.0220, 0.0549 and 0.1226, against in-sample means of 0.0091, 0.0417 and 0.1112.
• Thin and one-signed classes: 4 positives falls back to in-sample; 5 gives slope 0, the class rate with no features, as documented; a perfectly separable rare class stays finite and on rate. One-signed targets have no gate, so they are untouched.
• Seeds: the fold assignment and the fold gates' seeds spawn from the single draw the in-sample gate takes, so the forests and uniforms do not move. The new test covers this.
• At draw time, every path reads classes_ and predict_proba from _CrossFittedGate, and the mean of the mapped fold gates keeps the out-of-fold mean.
• The checkpoint names gate_calibration only when it is cross-fitted, and resuming under a different calibration is refused.
• LCFS is the only opt-in, and the declaration changes the stage's hash through source_stages.json. The LCFS wiring test checks both the declared and undeclared paths.
• There are two changelog fragments, one added (model option) and one changed (LCFS outputs); only the second needs the refresh from item 1.

CI: none at this head. The PR is stacked on a branch that has since merged, so no checks have run; they will start once it is retargeted to main.

Verdict: the method is sound and the LCFS evidence is convincing, but it needs a rebase onto main with the gambling figures re-measured, and Max's two fixes, before it can merge.

🤖 Generated with Claude Code

[juaristi22]

Thanks Max. All three are in 37df309, on the branch now rebased onto main:

1. A fold with too little weight. The fold assignment is checked for each class's training weight before any fold gate is fitted. A target that falls short keeps the in-sample gate, recorded as in_sample_weightless_fold. Your reproducer is a test. As Vahid found, whether a fold falls short depends on the seed, so a Hypothesis property covers any seed and any number of weightless rows.
2. Convergence. The calibration checks its score equations at exit. Where Newton's steps stall, as on saturated or constant scores, damped (Levenberg–Marquardt) steps take over. If the balance still fails, every row gets the class rates, recorded as cross_fitted_class_rates. A negative out-of-fold slope, a ranking worse than none, gets the class rates too. Your all-29 reproducer is a test.
3. Wording. The changelog and the module docstring now say the balance is exact on the pooled out-of-fold donor scores, not on recipients. They also say that in a chain, a sign that changes moves later targets' amounts through their predictors.

The Hypothesis tests cover:

• the rate balance on ordinary, saturated and constant scores;
• the sparse-weight fallback;
• grouped folds never splitting a group;
• the default gate being exactly the classifier it always fitted.

Byte identity of the default path against main held on my machine: one digest over draws, chain checkpoints and positional predicts, across all seven regimes and two seeds. I haven't committed it, because those bytes are platform-specific.

With the Budget behind us, every other gated UK fit now opts in too, measured the same way. The figures are in the body. The US fits stay on the in-sample gate: their donors and engine aren't on my machine, so they're yours to measure if you want them.

🤖 Generated with Claude Code

[juaristi22]

Thanks Vahid. Taking your points in order:

1. Stale against main. The branch is rebased onto main (now c6fdfc3) and retargeted to it. The pins are re-pinned on main: the spec-loader golden, the seed protocol and seed map with the F0 report, the pool tool's bundle hash, and the fit.qrf graph pins. Main's own qrf.py matched main's pins first, so the move is the gate change alone. The LCFS figures are re-measured on full spines built from main (4a36edc), with top_up_gambling_participation in place, control against head. The body, the changelog and the notes now carry the new figures. With the top-up in place, the National Lottery and betting and gaming keys sit at GSGB participation in both runs. So the cross-fitting moves who carries them, not how many: Lottery Duty goes 0.58 → 0.63 times stakes, and the other gambling duties 4.91 → 5.28 times betting-and-gaming stakes. The chain's own residual is unchanged: it draws betting and gaming on 2.70% of households against the donor's 3.58%. One gate needed a new bound: the top-up's stake-scale floor is now 0.3, on María's ruling of 3 November. The body's section on the gambling stake scale shows why neither the incidence nor the duty bases move with it.
2. Max's items 1 and 2. Both are fixed, as in my reply to him. The weightless-fold check is per fold and per class, before any fold gate is fitted, and a property test covers any seed.
3. Grouped folds. fit, start_chain, fit_draw_next and the fit helper take groups, which deal whole groups to folds. A chain locks its grouping and refuses a step that changes it. Every person- and benefit-unit-level fit that now opts in passes its households:
   • the NTS bus-use band;
   • salary sacrifice;
   • the Universal Credit reporter redraw;
   • the SPI FRS reference forests and stage 2.
4. Silent thin-class fallback. FittedRegimeGatedQRF.gate_outcomes() names what each target got. Every opted-in stage writes it to its evidence under gate_outcomes, so a target that stayed in sample, or fell back to the class rates, is visible. In the head build, counting the SPI reference forests' folds once per target:
   • 119 of 133 gated targets cross-fit;
   • eight SPI targets fall back to the class rates: five stage-2 benefit reports and three tape targets. One more does in three of its five reference folds;
   • five keep the in-sample gate because a class is thin.
5. The graph kernel. fit.qrf@1 takes gate_calibration, and writes the calibration and outcome into its receipt only when it is cross-fitted.

🤖 Generated with Claude Code

[vahid-ahmadi]

Round 2 at 5ed3468

What I checked: every commit since 807e024, both replies, the qrf.py and kernel changes, each of the 11 operations that now declare cross_fitted, the stake-scale floor, and CI. I ran the 59 gate-calibration and QRF tests and the 10 kernel tests at head (all pass), re-ran my round 1 probes on synthetic data, and ran a base-against-head byte check of the default path. No survey data; the frame figures are yours.

Round 1 items

1. Stale against main: fixed. Head sits on main c8984c4 with nothing behind, and is MERGEABLE. The control/head spines were built on 4a36edc, and none of the 119 commits from there to c8984c4 touch microcosm-fit, microcosm-data or the UK build. The gambling figures are re-measured with the top-up in place, and the changelog and notes no longer carry the old 0.61 / 5.5 / 10.9% / 2.6% lines.
2. Max's items 1 and 2: fixed. On the 10-row case with one positive-weight row per class, all 40 seeds now return in_sample_weightless_fold, with no crash (6 of 40 crashed in round 1). With two such rows per class, it is cross_fitted or cross_fitted_class_rates, and nothing raises. When every logit is 29, the calibration now converges (slope 0.001, means 0.5/0.5 against a rate of 0.5). Saturated three-class scores and an inverted ranking both fall back to cross_fitted_class_rates, at the rates.
3. Grouping: fixed. I re-ran the round 1 probe: 1,500 households of 4, an unobserved household effect, and household-level features with no signal. Row folds fit a slope of 0.336 and household folds 0.150. Mean probability on fresh households is the same either way (0.049), and expected log loss is lower with household folds (0.1875 against 0.1885). Across 20 seeds, _stratified_folds never split a household, and a chain refuses a step whose grouping changed.
4. Silent thin-class fallback: fixed. gate_outcomes() reaches each opted-in stage's evidence under gate_outcomes, pooled over the SPI reference folds.
5. Graph kernel: fixed. fit.qrf@1 reads and validates gate_calibration and adds it to the receipt only when it is cross-fitted.

Must

1. CI fails at head because two test fakes lack the new arguments. Both fakes are wrong, not the code:

   • engine-uk (3.13 and 3.14): packages/microcosm-build/tests/engine/uk/test_uk_spi_income.py:324. The local CapturingQRF.fit(self, frame, predictors, targets, *, weights) gets groups= from fit_reference (spi_income.py:1131-1136), which now passes it on every path. The result is TypeError: ... got an unexpected keyword argument 'groups' in test_stage2_pension_bridge_uses_observed_and_drawn_receipt (1 failed, 222 passed).
   • engine-free (3.14): packages/microcosm-build/tests/engine_free/uk/test_uk_company_cars.py:976. _NoSacrificeForest.__init__(self, *, n_estimators, seed) is patched in as salary_sacrifice.QRF, and salary_sacrifice.py:617-629 now constructs it with gate_calibration= and fits it with groups=. The result is TypeError in test_salary_sacrifice_runs_on_the_stage_s_output (1 failed, 18,249 passed).

   The salary-sacrifice fakes in test_uk_salary_sacrifice.py were updated, but this one in the company-cars file was not, and the engine-tier fake runs only in CI. Fix: add gate_calibration=... to the _NoSacrificeForest constructor and groups=None to both fit signatures, or take **kwargs.

Should

2. Name the VAT coverage basis against policyengine-uk#2634. The body says that with 0.67 kept, VAT comes out about 1.7% lower and re-deriving gives about 0.66. But Max's open policyengine-uk#2634 moves survey_consumption_coverage from 0.67 to 0.69, derived on a6299504, a cut without this PR. If the 1.7% fall in the full-rate base carries over, the value after this PR lands would be about 0.68 on that basis, not 0.66. Suggested fix: in the body's VAT section and the LCFS changelog, state the move as a ratio (about −1.7% of the base) and cite #2634, so whoever re-derives on the first cut built with this PR starts from the current value.

Nit

3. nts_bus_travel.py:1230: groups=donor["household_id"] ... if "household_id" in donor else None falls back to row folds without saying so. The production donor has the column (renamed at :795), so nothing is wrong today. But a donor without it would silently re-open item 3 on a person-level fit. Suggest raising, or recording the grouping in the receipt.

New scope, checked

• Which fits cross-fit, and how they fold:
  • one row per household, so no grouping is needed: the LCFS chain, prepayment meter and zero-road-fuel redraw; ETB services; the WAS chains; the SPI housing shell;
  • one row per taxpayer record, with no household to group by: the SPI tape forest;
  • grouped by household: the NTS bus band, salary sacrifice, the UC reporter redraw (benefit units by their household), and the SPI FRS reference forests and stage 2.
• Left in sample, correctly, because each draws only through the positive forest (predict_positive_from_uniforms): the gambling stake model, property components, LISA balances and LCFS gas payers. The ETB VAT rate has no gate.
• test_uk_qrf_gates.py pins the declared set to exactly these 11 operations.
• Gambling stake-scale floor: it is a sanity bound on the stake draw, not a fitted target. The head's 0.41 sits inside [0.3, 2.5]. As the body says, most of the distance from 1 is the stake model running about twice the donors' per-player stake, in both builds. The chain's own level is now about the donors'. Neither incidence nor the duty bases depend on the scale.
• Gambling against HMRC, aggregates from uk/excise_facts.json. Lottery Duty receipts of £931.9m are 12% of the £7.77bn ticket-sales base. The six other gambling duty lines (general and pool betting, bingo, gaming, machine games, remote gaming) total £2.68bn, which at their headline rates implies a base of roughly £14bn, consistent with the £14.47bn levelled base. Both bases are levelled, so national duty inputs match HMRC by construction. The 0.63× and 5.28× multiples are duty over the diary's own stakes, so they rise as diary stakes per player fall. They say how short the LCFS diary records gambling, not that the model is off HMRC.

Checked and fine

• Default path is byte-identical. One digest over fit/predict draws, start_chain/fit_draw_next raw draws and both checkpoints' JSON, on two seeds, matched between c8984c4's qrf.py and head's. In-sample checkpoints omit gate_calibration and groups_sha256.
• Fold seeds spawn from the one draw the in-sample gate takes. Thin classes are counted in groups when grouping is given, and a weightless training fold is caught before any fold gate fits.
• The US fits are untouched, and only their pins move.

CI at 5ed3468: lint, wheels, select-countries and integration-uk pass. engine-uk fails on both Pythons and engine-free fails on 3.14 (Must 1). engine-free 3.13 and engine-us were still running when I posted.

Verdict: the round 1 items are all closed and the wider opt-in is well covered. Once the two test fakes are fixed and CI is green, it is ready to merge.

🤖 Generated with Claude Code

[juaristi22]

Thanks Vahid. All three are in, at 5c412e8:

1. The test fakes. Fixed in 283632c. _NoSacrificeForest takes gate_calibration and groups, and its fitted model now answers gate_outcomes(), which the stage's evidence reads next. CapturingQRF.fit takes groups and passes it on. A third fake had the same gap: the E6 identity receipt's chained model, which ETB services builds. CI runs both tiers with --maxfail=1, so at 5ed3468 it stopped before reaching that one. I then ran every UK test file that imports an opted-in stage or its fakes, main's new ones included: 1,068 engine-free and 43 engine-tier, with no failures. Three skip, because the licensed LCFS, ETB and WAS tabs and the pinned SPI donor aren't staged in that worktree.
2. The VAT basis. The LCFS changelog now gives the move as a ratio (5c412e8). The full-rate base falls 1.7%, so a coverage re-derived on the first cut built with this PR moves by about that ratio from wherever it starts: about 0.68 from the 0.69 that Move the VAT survey coverage to 0.69 on Microcosm a6299504  policyengine-uk#2634 derives on a6299504. The body's VAT section says the same and cites it. This corrects my earlier line that, with 0.67 kept, VAT would land about £2.9bn below receipts. On a6299504, 0.67 sits 3.0% above them, so a 1.7% fall leaves VAT above receipts. A gap below them holds only against a coverage that matches receipts: about £2.8bn at 0.69.
3. The NTS fallback. In 22c6179, a cross-fitted band fit on a donor without household_id raises instead of falling back to row folds, and a test covers it. The production donor has the column, so nothing moves.

CI at 5c412e8: engine-uk now passes on both Pythons, as do integration-uk, lint and wheels. engine-free fails on one test, test_transport_evidence_kernels.py::TestImplementationHashes::test_the_batterys_binding_identity_adds_only_the_leaf_and_its_imports, which fails on main at 2d41a20 too. #1342 added two modules to the transport battery's binding identity, so it holds 222 against the 220 the test pins; this branch adds none. engine-us is still running.

🤖 Generated with Claude Code

[vahid-ahmadi]

Round 3 at 66a3f08

What I checked: the rebase onto main 79a533c (range-diff against 5ed3468), the three new commits, the PR body and LCFS changelog, and CI. I ran the NTS bus, company-cars and E6 identity-receipt test files at head (80 pass). Code reading and CI otherwise; no survey data.

Round 2 items

1. CI red from stale test fakes: fixed (ee56199). CapturingQRF.fit takes groups and passes it on, _NoSacrificeForest takes gate_calibration and groups and answers gate_outcomes(), and the third fake you found, the E6 receipt's chained model, takes gate_calibration.
2. VAT coverage basis: fixed (66a3f08 and the body). Both now state the 1.7% fall as a ratio and cite policyengine-uk#2634: about 0.68 from 0.69 on a6299504. The arithmetic holds (393.3 / 400.1 = 0.983, and 0.69 × 0.983 = 0.678). The corrected "0.67 sits 3.0% above receipts" line matches 0.69 / 0.67 = 1.030.
3. NTS silent fallback: fixed (f10be40). A cross-fitted band fit on a donor without household_id now raises NTSBusTravelError. The default path still passes groups=None exactly as before. The new test fails on the pre-fix file, so it guards the change.

Rebase and default path

• Commits 1, 2 and 4 to 7 are patch-identical to round 2. Commit 3 only re-pins the US seed and spec digests from main's new values. The old pin-only commit 8 is folded into it.
• packages/microcosm-fit is byte-identical to 5ed3468, and main did not touch it between c8984c4 and 79a533c. So round 2's base-against-head byte check of the default path still holds.
• Against 5ed3468, the only other changes in the PR's files are the three new commits and main's own contract.py changes. The contract.py gate-battery digests the PR re-pins pass in CI.

No new findings.

CI at 66a3f08: all green. engine-free, engine-uk and engine-us pass on 3.13 and 3.14, as do integration-uk, lint, select-countries and wheels. The transport-battery binding-identity failure you mentioned is gone now that main has #1495.

Verdict: all three round 2 items are closed and CI is green. Ready to merge from my side.

🤖 Generated with Claude Code