Skip to content

Fill CBO total AGI for 2015-2018 with IRS SOI actuals - #10114

Merged
MaxGhenis merged 10 commits into
mainfrom
fix-cbo-agi-2016-2018-soi
Nov 5, 2026
Merged

MaxGhenis merged 10 commits into
mainfrom
fix-cbo-agi-2016-2018-soi

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

What changes

This PR sets calibration.gov.cbo.income_by_source.adjusted_gross_income (total AGI) to IRS Statistics of Income actuals for 2015-2018. Its per-capita sibling, adjusted_gross_income_per_capita (total ÷ calibration.gov.census.populations.total), uprates dollar inputs that have no series of their own and deflates income for Medicare IRMAA's two-year lookback.

Year Main This PR Source
2015 9,033,840,000,000 10,210,310,102,000 SOI Table 1.1, TY2015
2016 (holds 2015) 10,225,938,479,000 SOI Table 1.1, TY2016
2017 (holds 2015) 11,009,900,155,000 SOI Table 1.1, TY2017
2018 (holds 2015) 11,643,439,106,000 SOI Table 1.1, TY2018
2019 on CBO CBO, unchanged

Each value takes the "All returns" row's "Adjusted gross income less deficit" amount and converts the table's thousands of dollars to dollars. Each parameter value cites its own tax year's workbook.

Why

Main's 2015 value comes from a Tax Foundation page whose Table 1 reports tax-year 2013 AGI on returns that exclude dependent filers. SOI reports 9,093,628,703 thousand for all returns in tax year 2013 (13in11si.xls) and 10,210,310,102 thousand for 2015. Main has no values for 2016-2018, so the parameter holds that 2015 value through 2018, and the per-capita index drifts down with population and then jumps 31.7% into 2019. The model therefore uprates a dollar input dated 2018 or earlier past the gap by the wrong factor. The gap and the wrong 2015 level each account for part of the jump: with the right 2015 total and the gap left in, the index would still rise 16.5% from 2018 to 2019.

SOI's tax-year 2019 total, 11,966,873,976 thousand (19in11si.xls), exceeds the series' CBO 2019 value of 11,962,000,000,000 by about 0.04%, so the series moves onto CBO's values nearly level.

Year Per capita, main Per capita, this PR Growth into the year, main This PR
2015 27,704.54 31,312.48
2016 27,461.49 31,085.28 −0.9% −0.7%
2017 27,275.18 33,241.35 −0.7% +6.9%
2018 27,085.95 34,910.24 −0.7% +5.0%
2019 35,672.43 35,672.43 +31.7% +2.2%

Effect

Parameters

A dump of every leaf parameter after system init, at main (abc67b98de) and at this branch, differs only in the AGI total and its per-capita sibling, at 2015-2018. Both series keep main's year-on-year ratios from 2019 on, and the map from variables to series doesn't change.

Household inputs on the default series

policyengine-core multiplies an uprated input by index(target) ÷ index(source), using the latest earlier input, and holds the index at its first value before the index begins (_uprating_index_value in policyengine_core/simulations/simulation.py). On main, an input dated 2018 or earlier and read in 2019 or later exceeds this PR's result by index_new(source) ÷ index_old(source) − 1:

Input year Main's excess
2015 or earlier 13.0%
2016 13.2%
2017 21.9%
2018 28.9%

Inside 2016-2018, main's factors fall short of this PR's by up to 12%. For example, main multiplies an input dated 2015 and read in 2018 by 0.978, and this PR multiplies it by 1.115.

Medicare IRMAA lookback

medicare_irmaa_magi_two_years_prior gives a household with inputs only for benefit year Y that year's MAGI times index(Y−2) ÷ index(Y), with the index held at its 2015 value before 2015:

Benefit year Main This PR
2016 1.009 1.007
2017 1.016 0.942
2018 1.014 0.890
2019 0.765 0.932
2020 0.721 0.929
2021 on unchanged unchanged

Dataset extension

extend_single_year_dataset grows each year from the dataset's base year forward, so a dataset based in 2019 or later reads none of the changed values.

impact: none on the default dataset from 2021 on. It holds 2024 records, so its inputs read the series from 2024 on, and the IRMAA lookback factors match main's from benefit year 2021. For household calculations, the PR changes inputs dated 2018 or earlier and lookback factors for benefit years 2016-2020. The default dataset holds no data from before 2024, so we report no aggregate for those years.

Tests

We computed expected values by hand from the SOI totals above and the population series, not from the model.

  • medicare_irmaa_magi_two_years_prior.yaml: Case 2 (500,000 → 503,654.40) and Case 7 (100,000 → 100,730.88) move. New Case 8 pins the 2018 benefit year (500,000 → 445,217.19; main gave 506,932.37). New Case 9 pins the 2019 benefit year (500,000 → 465,924.93; main gave 382,300.59).
  • senior_household_2015_2016_integration.yaml: Case 2 moves (10,000 → 10,073.09).
  • household/expense/charitable/charitable_cash_donations.yaml (new): the model multiplies a 2018 input read in 2019 by 1.0218 (main 1.3170) and a 2015 input read in 2018 by 1.1149 (main 0.9777).
  • tests/core/test_uprating_series.py::test_default_series_has_a_value_for_every_year: the default series has a January 1 value for every year from its first to its last. It fails on main (2016-2018 missing).
  • CI: the latest Pull request run, at e0ba755386 (https://github.com/PolicyEngine/policyengine-us/actions/runs/41984504030), concluded success: all 36 jobs passed.

Invariants

  1. The default total has a January 1 value for every year from its first to its last. A missing year holds the previous value, so the per-capita index drifts down with population and then jumps. The guard checks every year between them. IRMAA Cases 2 and 7 and senior-household Case 2 fail if someone deletes the 2015 value or restores main's 2016 value.
  2. Each per-capita series equals its total over population at every instant. The existing test_every_per_capita_series_is_its_total_over_population checks this.
  3. An input projected from year X to year Y grows by index(Y) ÷ index(X). Core implements this, the existing simulation test checks it from 2024, and the new YAML cases pin it with hand-computed values across 2015-2019.

This PR adds no Hypothesis test: it changes data and no logic, and invariant 1 ranges over a finite set of years, so the guard enumerates them. A simulation property test would compute its expected factor from the same parameter core reads, and it would pass on main too.

Overlap with open PRs

axiom: n/a: this PR corrects historical AGI calibration data and the uprating and income estimates built on it. In the Medicare IRMAA variable it changes only comments and documentation, and the revised test expectations follow the corrected index.

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 11, 2026 15:12
calibration.gov.cbo.income_by_source.adjusted_gross_income had a 2015
value of $9.03T, cited to a Tax Foundation page whose table is tax-year
2013 AGI excluding dependent filers, and no values for 2016-2018, so the
series and its per-capita sibling (the default uprating index for dollar
inputs) held that figure through 2018 and then jumped 31.7% into 2019.

Replace 2015 and add 2016-2018 with IRS SOI Publication 1304 Table 1.1,
all returns, adjusted gross income less deficit: $10.210T, $10.226T,
$11.010T and $11.643T. CBO values from 2019 on are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CBO AGI total, which every dollar input without its own series
follows per capita, must have a value for every year from its first to
its last: a missing year holds the previous value, so the per-capita
index drifts down with population and then jumps. The check fails on
main (2016-2018 missing) and passes with the SOI fill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The node description now says the by-source series are CBO's, with SOI
actuals for total AGI in 2015-2018. The changelog also covers inputs
projected to a year within 2016-2018, which the old flat series grew
up to 12% too little.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

The CI run on this pull request was cancelled at 23:27Z on 2026-10-11 to free the organization's shared GitHub runners for an urgent fix in another repository. The US + core hub will re-run it automatically (cancelled jobs only; jobs that had passed are kept). Nothing in this pull request changed. Please don't push or reopen just to restart CI.

@DTrim99

DTrim99 commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

PR Review: Fill CBO total AGI for 2015-2018 with IRS SOI actuals

PR #10114 (author: MaxGhenis, head aeabda50bc, merge-base ae3782f4d7). 3 files, 3 commits.

Summary

The PR fixes the early years of calibration.gov.cbo.income_by_source.adjusted_gross_income, the default uprating series.

  • Old data. The 2015 value was tax-year 2013 AGI excluding dependent filers, and there were no values for 2016-2018.
  • New data. 2015 is replaced and 2016-2018 are added, all from IRS SOI Table 1.1 ("All returns", adjusted gross income less deficit).
  • Why it matters. The derived adjusted_gross_income_per_capita series is the default uprating index for dollar inputs. It drifted down through 2018 and then jumped 31.7% into 2019.
  • New guard. A test now checks that the default series has a value for every year.

The data change itself is correct:

  • All four values match the SOI files to the thousand dollars.
  • The units match the CBO series.
  • The SOI-to-CBO splice is level: SOI tax year 2019 is within 0.04% of CBO's 2019 value.
  • Every number in the PR body's per-capita, growth, factor and IRMAA tables reproduces.
  • No parameter that is uprated by this series changes, and microsimulation aggregates don't move.

One blocking problem comes from timing:

Critical (must fix)

1. Once merged with main, three Medicare IRMAA lookback expectations fail, and a docstring on main becomes false.

  • Why the series now matters for IRMAA. Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106 (merged 2026-10-12 01:03 UTC) added LOOKBACK_DEFLATOR = "calibration.gov.cbo.income_by_source.adjusted_gross_income_per_capita" to medicare_irmaa_magi_two_years_prior.py.
    • A household with inputs only for benefit year Y gets that year's MAGI times index(Y−2)/index(Y).
    • So the 2015-2018 values this PR changes now feed household results for benefit years 2016-2020.
  • Reproduced. I built the merge with git merge-tree --write-tree PolicyEngine/main aeabda50bc and ran the IRMAA YAML files on it with policyengine-core 3.33.3. Result: 3 failed, 24 passed.
    • policyengine_us/tests/policy/baseline/gov/hhs/medicare/eligibility/medicare_irmaa_magi_two_years_prior.yaml:33 (Case 2): [503654.4] differs from 504425.34375 (margin 0.01).
    • Same file :91 (Case 7): [100730.88] differs from 100885.0703125. The Part D surcharge stays 0, because 100,730.88 is still under the 103,001 threshold.
    • policyengine_us/tests/policy/baseline/gov/hhs/medicare/senior_household_2015_2016_integration.yaml:56 (Case 2): [10073.089] differs from 10088.509765625 (margin 1).
  • Hand check. The new index is 10,210,310,102,000 / 326,078,000 = 31,312.48 for 2015 and 10,225,938,479,000 / 328,964,000 = 31,085.28 for 2016, a ratio of 1.0073088.
    • 500,000 × 1.0073088 = 503,654.40
    • 100,000 × 1.0073088 = 100,730.88
    • 10,000 × 1.0073088 = 10,073.09
  • Stale text on main once this lands:
    • policyengine_us/variables/gov/hhs/medicare/eligibility/medicare_irmaa_magi_two_years_prior.py:62-67: "CBO AGI has no values for 2016 to 2018 and is held at its 2015 value ... one entered for 2019 or 2020 alone gets that year's income times 0.765 or 0.721." With this PR those factors are 0.932 and 0.929, and 2016-2018 get 1.007, 0.942 and 0.890 rather than "slightly more than that year's income".
    • medicare_irmaa_magi_two_years_prior.yaml:5-12: the header shows 2015: 9,033,840,000,000 / 326,078,000 = 27,704.54 and 2016: ... = 27,461.49.
    • The per-case comments at :22, :32, :90 and senior_household_2015_2016_integration.yaml:35, :55 use the same old index values.
  • The Python IRMAA tests pass on the merged tree. They compute the deflator from the parameter. 59 passed: test_irmaa_first_modeled_year.py, the new guard, and the per-capita invariant.
  • Fix. Merge main into the branch, then:
    • Set the three expectations to 503,654.40, 100,730.88 and 10,073.09 (or 10,073 within the 1-dollar margin).
    • Update the index comments to 31,312.48 (2015) and 31,085.28 (2016).
    • Rewrite the docstring sentence. For example: "CBO AGI uses IRS SOI actuals for 2015-2018, so a household entered for 2017, 2018, 2019 or 2020 alone gets that year's income times about 0.942, 0.890, 0.932 or 0.929." Or drop the numbers.

Should address

1. The changelog and the PR body's test-impact analysis predate #10106.

  • Changelog. changelog.d/fix-cbo-agi-2016-2018-soi.fixed.md describes only uprated dollar inputs. On main the change also moves the imputed Medicare IRMAA lookback MAGI (and Part B/D IRMAA) for households entered for a single benefit year from 2016 to 2020. The 2019 factor goes from 0.765 to 0.932 and the 2020 factor from 0.721 to 0.929.
  • PR body. The "Tests" section says the only way the fix can move a YAML case is forward uprating, "since no variable formula reads a later period". It also says the IRMAA lookback cases read stored arrays, "so nothing is projected".
    • On main, medicare_irmaa_magi_two_years_prior reads this index directly for earlier years, and that is how the three cases in Critical 1 move.
    • The "Local runs ... queued under the heavy-test lock (results to follow)" line was never filled in.
  • Fix.

Suggestions

  1. Pin the largest IRMAA move with a test. No case covers a benefit year of 2017-2020 entered alone, where the deflator moves most. An example: a 66-year-old with only 2019 wages of 500,000.
    • Expected IRMAA MAGI two years prior: 500,000 × (11,009,900,155,000 / 331,211,000) / (11,962,000,000,000 / 335,329,000) = 500,000 × 0.931850 ≈ 465,924.93 (hand-computed, not run).
    • It would pin the household-level effect of the splice, which the new guard checks only at the parameter level.
  2. Sequence with draft Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128 ("Fill IRS SOI uprating series gaps with Publication 1304 actuals"). It also edits policyengine_us/tests/core/test_uprating_series.py, presumably to generalize the new guard as the PR body's "Follow-up" describes. Land this PR first, or fold the guard into Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128's generalized version, to avoid a conflict.
  3. Optional wording in the guard's docstring. test_uprating_series.py:127-130 ("CBO AGI once had nothing for 2016 to 2018, so inputs dated 2018 came out 29% too high by 2019") is accurate: 1.317 / 1.022 = 1.289. Consider also naming the IRMAA lookback as a reader of the series, since Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106 made it one.

Verified correct

  • Values against IRS SOI Table 1.1 ("All returns" row, column 3 "Adjusted gross income less deficit, Amount"; the header says "money amounts are in thousands of dollars"):
  • Units and concept. Both series are in dollars and cover all returns, dependent filers included. SOI TY2019 is 11,966,873,976 thousand (https://www.irs.gov/pub/irs-soi/19in11si.xls), against CBO's 2019 value of 11,962,000,000,000: a 0.04% gap, so the splice is level.
  • The old value was wrong, as the PR says.
    • The cited Tax Foundation page is "Summary of Federal Income Tax Data, 2013": "$9,033,840" million AGI on 138,313,155 returns, and "Does not include dependent filers".
    • SOI's TY2013 all-returns figure is 9,093,628,703 thousand (https://www.irs.gov/pub/irs-soi/13in11si.xls).
  • PR-body tables reproduced from the YAML values and the CBO population series:
    • Per capita, old to new:
      • 2015: 27,705 to 31,312
      • 2016: 27,461 to 31,085
      • 2017: 27,275 to 33,241
      • 2018: 27,086 to 34,910
      • 2019: 35,672 (unchanged)
    • Growth into 2019 drops from +31.7% to +2.2%.
    • Factor to 2026:
      • from 2015: 1.927 to 1.705 (+13.0%)
      • from 2016: 1.944 to 1.718 (+13.2%)
      • from 2017: 1.958 to 1.606 (+21.9%)
      • from 2018: 1.971 to 1.529 (+28.9%)
    • 2015 to 2018 goes from ×0.978 to ×1.115.
    • IRMAA deflators:
      • 2017: 1.016 to 0.942
      • 2018: 1.014 to 0.890
      • 2019: 0.765 to 0.932
      • 2020: 0.721 to 0.929 (the PR body rounds 0.92950 up to 0.930)
  • Parameters uprated by this series don't move. Core parameter uprating extends only after a parameter's last value:
    • calibration.gov.irs.soi.{alimony,farm,farm_rent,rental}_income: last value 2021
    • calibration.gov.irs.soi.agi.total_agi bands: last value 2020
    • gov.hhs.smi.amount: last value 2026-10-01
  • Microsimulation is unaffected.
    • Dataset extension runs forward from the 2024 base year.
    • The IRMAA deflator for 2024 and 2025 benefit years reads 2022-2024 CBO values, which are unchanged.
  • Tests that read the series compute expectations from the parameter rather than pinning 2015-2018 values:
    • test_per_capita_uprating.py (the per-capita invariant at 2015 and 2017)
    • test_foreign_earned_income_exclusion_leaves.py (2025-2026)
    • the IRMAA Python tests
  • The new guard (test_uprating_series.py:126-139) checks parameter structure, which YAML can't express, so a Python test is justified. It fails on main (years 2015, 2019-2036) and passes at the head.
  • References. There is one reference per value, each pointing to its own tax year's SOI workbook. The workbooks are XLS files, so no #page anchor applies, and the titles contain no page text.
  • Description and changelog.
    • The node description now names CBO for 2019-2036 and SOI for 2015-2018.
    • The changelog fragment is top-level and typed fixed.

Axiom line assessment

Valid n/a. The line reads axiom: n/a: calibration uprating series, not a legal provision.

CI status

All 26 checks pass (gh pr checks 10114), but the run doesn't cover current main.

Branch status

Verdict: REQUEST_CHANGES

@DTrim99 DTrim99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All four values match SOI Table 1.1 exactly, and the splice into CBO is smooth. One critical (details in the comment above): #10106, merged 10/12, deflates IRMAA two-year-lookback MAGI by this series, so merged with main the PR breaks three of its expectations (two_years_prior.yaml:33 and :91, senior_household_2015_2016_integration.yaml:56), and the medicare_irmaa_magi_two_years_prior docstring and index comments become false. Please merge main and update them. One should item: the changelog and PR body should state the IRMAA lookback effect (2019 factor 0.765 → 0.932, 2020 0.721 → 0.929).

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Picking this up for the US hub. I'll post a hand-off comment here when it's ready.


Generated by Claude Code

Main now has #10106, whose IRMAA two-year lookback deflates income by the
per-capita CBO AGI series this branch fills for 2015-2018, so the branch
must be tested against it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The IRMAA two-year lookback from #10106 deflates benefit-year income by
the per-capita AGI index this branch fills for 2015-2018, so its 2016
cases move (500,000 -> 503,654.40, 100,000 -> 100,730.88, 10,000 ->
10,073.09) and the docstring's account of the 2016-2018 gap no longer
holds. New 2018 and 2019 benefit-year cases pin the largest moves,
where the deflator went from 1.014 to 0.890 and from 0.765 to 0.932.
The changelog now states the lookback effect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Household cases show a default-uprated input crossing the SOI-to-CBO
splice (2018 to 2019: x1.0218, was x1.3170) and staying inside the SOI
years (2015 to 2018: x1.1149, was x0.9777). Property tests check that an
input moves by the index ratio for any pair of years from 2015 to 2036,
and that the index never moves more than 25% in a year, which a missing
year would break (the 2016-2018 gap made 2018 to 2019 +31.7%). The guard
docstring now names the IRMAA lookback as a reader of the series.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An independent review found that the comments on the old IRMAA and
donation results said the index was held at its 2015 value. It was the
total that was held, so the per-capita index fell with population, which
is what the quoted numbers reflect. The module comment and changelog now
say the index is SOI for 2015-2018 and that only inputs on the default
series were affected, and the guard docstring separates the gap's share
of the error (14%, 0.864) from the wrong 2015 level's.

The Hypothesis file computed its expected factor from the same parameter
core reads, so it passed on main as well, and its 25% bound does not
detect a gap once the 2015 level is right. The guard and the hand-computed
YAML cases cover what it claimed, so it is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per-capita AGI fell from 2015 to 2016, so for a household entered for
2016 alone the lookback multiplies income by about 1.007. "Deflated"
read as a reduction; the docstring now says the result is slightly
more than that year's income, and why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Picking this up for the US hub. I'm continuing an earlier session's work on this PR, which stopped before its hand-off; its five commits from ab1b3699d9 to 574fe07eb5 stay on the branch, and I'm checking each of them. I'll post a hand-off comment here when it's ready.


Generated by Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Picking this up for the US hub. I'm continuing earlier sessions' work on this PR: the five commits from ab1b3699d9 to 574fe07eb5 stay on the branch, and I'm checking each one. I'll post a hand-off comment here when it's ready.


Generated by Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review

Head: 7ef425eccd8e74764b77bc287f1a03ef403fe26d

This continues two earlier sessions on this brief. The first pushed ab1b3699d9 to 574fe07eb5 and posted no hand-off. The second pushed nothing. I checked each earlier commit and kept all of them. The only new commit is 7ef425eccd, a merge of main at abc67b98de, which had no conflicts. Nobody else committed, commented or reviewed here after 2026-11-03T16:30Z.

Earlier commits: checked and kept

Commit What it does Check
ab1b3699d9 Merges main (brings in #10106) Superseded by today's merge. No conflicts.
79f8a56588 Updates the three IRMAA expectations; adds Cases 8 and 9 All five values recomputed by hand (below)
2acbee077d New charitable_cash_donations.yaml (2 cases) Both recomputed by hand
a4814e2dc4 Corrects the docstring and comments on the old AGI value; drops a Hypothesis file Kept; see judgment call 2
574fe07eb5 Docstring: for 2016 alone, the result lands slightly above income 31,312.48 / 31,085.28 = 1.0073 > 1, so this is true

The review from DTrim99 (2026-10-13), item by item

  • Critical 1: three IRMAA expectations and stale text. Fixed.
    • medicare_irmaa_magi_two_years_prior.yaml:39 is 503,654.40.
    • The same file at :97 is 100,730.88.
    • senior_household_2015_2016_integration.yaml:57 is 10,073.09.
    • The index header at medicare_irmaa_magi_two_years_prior.yaml:5-19 lists 2015-2019 values.
    • The docstring at medicare_irmaa_magi_two_years_prior.py:62-67 no longer describes the gap and quotes no factors.
  • Should 1: changelog and body.
    • changelog.d/fix-cbo-agi-2016-2018-soi.fixed.md gives the lookback factors for 2016-2020.
    • The body's Tests, IRMAA and Effect sections are rewritten.
    • The "results to follow" line is gone.
  • Suggestion 1: pin the largest move. Case 8 (2018 benefit year) and Case 9 (2019 benefit year, 465,924.93, which matches the reviewer's number).
  • Suggestion 2: sequence with Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128. See below.
  • Suggestion 3: guard docstring. tests/core/test_uprating_series.py:153 now names the IRMAA lookback.

Hand computations

These use the PR's AGI totals and calibration.gov.census.populations.total, computed in plain Python, not from the model.

Per-capita index, main → this PR:

Year Main This PR
2015 27,704.54 31,312.48
2016 27,461.49 31,085.28
2017 27,275.18 33,241.35
2018 27,085.95 34,910.24
2019 35,672.43 35,672.43 (unchanged)

Expected values:

Case Formula Result
IRMAA Case 2 500,000 × 31,312.48 / 31,085.28 503,654.40
IRMAA Case 7 100,000 × the same ratio 100,730.88
Senior household Case 2 10,000 × the same ratio 10,073.09
IRMAA Case 8 500,000 × 31,085.28 / 34,910.24 445,217.19 (main: 506,932.37)
IRMAA Case 9 500,000 × 33,241.35 / 35,672.43 465,924.93 (main: 382,300.59)
Charitable, 2018 input read in 2019 10,000 × 35,672.43 / 34,910.24 10,218.33 (main: 13,170.09)
Charitable, 2015 input read in 2018 10,000 × 34,910.24 / 31,312.48 11,148.99 (main: 9,776.72)
  • Lookback factors for 2016-2020: 1.0073, 0.9420, 0.8904, 0.9318 and 0.92950 (more precisely 0.9294956), which match the changelog's rounding. The 2020 factor rounds to 0.929, not 0.930. That settles the reviewer's rounding note.
  • Guard docstring numbers. With the right 2015 total and the gap left in:
    • inputs dated 2018 come out 14.0% too high by 2019;
    • a 2019 lookback factor of 0.864 results;
    • main's error is 28.9%.

Source values

  • Values not re-read here. www.irs.gov and web.archive.org are blocked from this session's network, so I couldn't open 15in11si.xls to 18in11si.xls.
  • 2018 corroborated. A search excerpt of SOI's preliminary-versus-final table for tax year 2018 shows 11,643,439,106 thousand, the PR's value.
  • 2015-2017 rest on the review. DTrim99's review reports that all four values match Table 1.1 "All returns", "Adjusted gross income less deficit", to the thousand.
  • The hub should open the four workbooks before landing:

Code claims re-read here

  • Core's uprating factor. policyengine-core's _uprating_index_value holds the index at its first value before it begins. The factor is index(target) ÷ index(latest earlier input).
  • Parameter dump. I dumped every leaf parameter (125,817) at this head, then again with main's income_by_source.yaml swapped in. They differ only in adjusted_gross_income and adjusted_gross_income_per_capita, at 2015-2018.

Tests (local, one file per process)

File Result
medicare_irmaa_magi_two_years_prior.yaml 23 passed
senior_household_2015_2016_integration.yaml 2 passed
charitable_cash_donations.yaml 2 passed
tests/core/test_uprating_series.py 131 passed
tests/core/test_per_capita_uprating.py 51 passed
test_irmaa_first_modeled_year.py 57 passed
test_irmaa_lookback_and_switch.py 23 passed (needs the dev extra's hypothesis)
tests/core/test_soi_agi_band_keying.py 6 passed
  • test_default_series_has_a_value_for_every_year fails with main's income_by_source.yaml, and passes here.
  • ruff format --check and ruff check pass on the changed Python files.
  • I ran no full suites and no microsimulations.

Mutants (each in its own process; the file was restored after each)

  • A: delete the 2017 value. Killed by the guard (1 failed) and by IRMAA Case 9 (1 failed of 23).
  • B: change the 2018 value to 11,743,439,106,000. Killed by IRMAA Case 8 and by both charitable cases. The guard passed, as expected, since no year is missing.

#10128 guard check (scratch merge, not pushed)

Overlap with #10132 (not touched)

CI

Push at 20:41Z. The Pull request run at 7ef425eccd: still in progress at 21:41Z, 60 minutes after the push (run 41714937303). 34 jobs succeeded, including Microsimulation, every Baseline shard, the Python shards, Household API Partners and PR validation. Two jobs were still running: Contrib other-shard-1 and other-shard-3. None had failed or been cancelled.

Judgment calls

  1. Readings. This is a calibration data series, not a legal provision, and no source conflict arises. Splicing SOI actuals for 2015-2018 onto CBO from 2019 is level: SOI's tax-year 2019 total is within 0.04% of CBO's 2019 value, a figure taken from the review and the PR body, not re-read here.
  2. No Hypothesis file. The earlier session dropped one in a4814e2dc4. The change is data, not logic. The guard enumerates every year, so the invariant is checked exhaustively. A simulation property test would compute its expected factor from the same parameter that core reads. I kept that decision. The body's Invariants section states three invariants and the test that covers each.
  3. Push target. I pushed only to this PR's branch, fix-cbo-agi-2016-2018-soi.

Proposed follow-up

None.

impact: the hub will measure
axiom: (hub files pe-parity before landing)


Generated by Claude Code

@MaxGhenis

MaxGhenis commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Hub review in progress

Superseded: the full review is posted below as a separate comment that starts Ready for hub review (verdict APPROVE at 7ef425eccd8e74764b77bc287f1a03ef403fe26d).


Generated by Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review
VERDICT: APPROVE
7ef425e
METHODOLOGY_CHOICE: no

This is an independent review of the head above, which is unchanged since the hand-off. Main has moved four commits past the merged abc67b98de, to adea7035ea. All four are California state supplement changes or version bumps, and none touches this PR's files. git merge-tree --write-tree origin/main pr-10114 merges cleanly.

No P1 or P2 findings. The data are right and every hand value reproduces. The defect reproduces at current main. Every changed test passes, and each mutant is killed. The P3 items below are optional.

Findings

P3-1. The guard can't see a missing first year or a wrong value

  • Where: policyengine_us/tests/core/test_uprating_series.py:153-170.
  • What it checks. The test asserts that the January years run without a gap from the first to the last. Deleting 2015 makes the series start in 2016, and the test still passes. Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128's test_series_has_a_january_value_for_every_year uses the same years[0] logic.
  • Mutants. Each ran in its own processes on a scratch edit of income_by_source.yaml, and I restored the file after each one.
    • C, the brief's: main's 2016 value put back (10_225_938_479_000 → 9_033_840_000_000). The guard passes. Killed by IRMAA Cases 2, 7 and 8 and senior-household Case 2:
      medicare_irmaa_magi_two_years_prior@2016: [570116.25] differs from 503654.40625
      medicare_irmaa_magi_two_years_prior@2016: [114023.26] differs from 100730.8828125
      medicare_irmaa_magi_two_years_prior@2018: [393315.6] differs from 445217.1875
      (senior household) medicare_irmaa_magi_two_years_prior@2016: [11402.326] differs from 10073.08984375
      
    • D, mine: delete the 2015 entry. The guard passes (1 passed). Killed by IRMAA Cases 2 and 7 (504,425.3 and 100,885.07, the old main values), senior-household Case 2, and charitable Case 2 (11,131.949 against 11,148.99).
  • Failure scenario. A later edit drops the first year, or puts back a stale level. The guard stays green, and only the household YAML cases catch it.
  • Suggestion (optional). The YAML cases already kill both mutants, and the hand-off plans to delete this guard once Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128 lands. So if anything changes, it belongs in Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128's generalized guard: assert that the default series starts at 2015, for example assert years[0] == 2015. Not blocking.

P3-2. "Within 0.04%" is a rounding of 0.041%

  • Where: the PR body ("Why", second paragraph) and the hand-off's judgment call 1.
  • The numbers. The hub read SOI's TY2019 total as 11,966,873,976 thousand. Against CBO's 11,962,000,000,000, that is 0.0407% above:
    python3: (11966873976*1000/11962000000000 - 1)*100 = 0.04074549...
    
  • Replacement text: "is about 0.04% above the series' CBO 2019 value of 11,962,000,000,000". The splice is still level, so this is a wording fix only.

P3-3 (follow-up, the same on main). Part B and Part D IRMAA brackets for years before 2024 are backdated 2024 values

  • Where: all five filing-status files under policyengine_us/parameters/gov/hhs/medicare/part_d/irmaa/, and the same five under part_b/irmaa/. Every file's earliest date is 2024-01-01, so backdate_parameters carries the 2024 schedule back to 2015.
  • Witness. IRMAA Case 7 (medicare_irmaa_magi_two_years_prior.yaml:86-98) tests a 2016 benefit year "under the first threshold of 103,001" and expects income_adjusted_part_d_premium_surcharge: 0. 103,001 is the 2024 threshold.
  • Same on main. Main's Case 7 has the same comment, with MAGI 100,885.07. This PR only updates the MAGI value. I couldn't open the 2016 CMS fact sheet or 42 U.S.C. 1395r(i) from this VM, so I don't state the 2016 threshold here.
  • Proposed follow-up issue: "Part D and Part B IRMAA brackets have no values before 2024". The brackets start at 2024 and are backdated, so 2016-2023 benefit years use 2024 thresholds and amounts. Add the CMS values for 2015-2023, and recheck IRMAA Case 7.

Checked and sound

Sources.

  • irs.gov is blocked from this VM: curl https://www.irs.gov/pub/irs-soi/15in11si.xls returned CONNECT tunnel failed, response 403 for 15-19. uscode.house.gov, govinfo.gov, ecfr.gov and cms.gov returned the same 403.
  • So the SOI values rest on the hub's read of 15in11si.xls to 19in11si.xls (Table 1.1, "All returns", "Adjusted gross income less deficit", thousands): 2015 10,210,310,102; 2016 10,225,938,479; 2017 11,009,900,155; 2018 11,643,439,106; 2019 11,966,873,976.
  • The YAML at the head (income_by_source.yaml:24-47) holds each 2015-2018 value × 1,000 exactly, and each value's reference points to its own tax year's workbook.
  • The body's TY2013 figure (9,093,628,703 thousand, 13in11si.xls) is outside the hub's read. It rests on DTrim99's review alone. It supports only the "why the old value was wrong" narrative, not any parameter value.

Hand values. I computed these in plain Python from the SOI totals above and calibration/gov/census/populations/total.yaml, before reading the tests.

Year Per capita, old Per capita, new Growth into year, old → new
2015 27,704.54 31,312.48
2016 27,461.49 31,085.28 −0.88% → −0.73%
2017 27,275.18 33,241.35 −0.68% → +6.94%
2018 27,085.95 34,910.24 −0.69% → +5.02%
2019 35,672.43 35,672.43 +31.70% → +2.18%
  • Lookback factors, benefit years 2016-2020. New: 1.0073088, 0.9419738, 0.8904344, 0.9318499, 0.9294956. Old: 1.0088506, 1.0157416, 1.0138647, 0.7646012, 0.7211713. The changelog's 1.007, 0.942, 0.890, 0.932, 0.929 and 1.009, 1.016, 1.014, 0.765, 0.721 are correct roundings. 2020 is 0.92950, so 0.929 is right.

  • The seven expected values:

    • IRMAA Case 2: 503,654.40
    • IRMAA Case 7: 100,730.88
    • Senior-household Case 2: 10,073.09
    • IRMAA Case 8: 445,217.19 (main 506,932.37)
    • IRMAA Case 9: 465,924.93 (main 382,300.59)
    • Charitable, 2018 input read in 2019: 10,218.33 (main 13,170.09)
    • Charitable, 2015 input read in 2018: 11,148.99 (main 9,776.72)

    All match the YAML.

  • Other body and changelog figures:

    • Input dated 2015/2016/2017/2018 and read in 2019 or later was too high by 13.02%, 13.20%, 21.87% and 28.89%.
    • The worst under-uprating within 2016-2018 is ×0.877, which supports "up to 12%".
    • With the right 2015 total and the gap kept, the 2018→2019 step is +16.53%, a 2018 input comes out 14.04% too high, and the 2019 lookback factor is 0.8642. That supports the guard docstring's "14% and 0.864" and the body's "+16.5%".

The defect reproduces at current main (adea7035ea). I ran the head's charitable and IRMAA YAML files with main's code:

charitable_cash_donations@2019: [13170.089] differs from 10218.330078125
charitable_cash_donations@2018: [9776.719] differs from 11148.990234375
medicare_irmaa_magi_two_years_prior@2016: [504425.34] differs from 503654.40625
medicare_irmaa_magi_two_years_prior@2016: [100885.07] differs from 100730.8828125
medicare_irmaa_magi_two_years_prior@2018: [506932.38] differs from 445217.1875
medicare_irmaa_magi_two_years_prior@2019: [382300.6] differs from 465924.9375

Main's outputs are exactly the body's "main gave" values. The new guard fails on main: assert [2015, 2019, ...] == [2015, 2016, ...].

Tests at the head, one file per process. Commands were uv run python -m policyengine_core.scripts.policyengine_command test <file> -c policyengine_us and uv run pytest -q -p no:cacheprovider <file>.

File Result
medicare_irmaa_magi_two_years_prior.yaml 23 passed
senior_household_2015_2016_integration.yaml 2 passed
household/expense/charitable/charitable_cash_donations.yaml 2 passed
tests/core/test_uprating_series.py 131 passed
tests/core/test_per_capita_uprating.py 51 passed
test_irmaa_first_modeled_year.py 57 passed
test_irmaa_lookback_and_switch.py 23 passed (after uv sync --extra dev; without hypothesis it errors at import)
tests/test_foreign_earned_income_exclusion_leaves.py 25 passed (also needs hypothesis; it skips without it)
tests/core/test_soi_agi_band_keying.py 6 passed

Each new or changed case asserts the hand-computed value itself, not a weaker bound. The cases span benefit years 2016, 2018 and 2019, and reads across 2015→2018 and 2018→2019, so both the SOI years and the SOI-to-CBO splice are pinned.

Mutants. Each ran on a scratch edit, and I checked git status was clean after each.

  • A (hand-off): delete 2017. Killed by the guard (1 failed) and IRMAA Case 9 ([432748.66] differs from 465924.9375).
  • B (hand-off): 2018 → 11,743,439,106,000. The guard passes. Killed by IRMAA Case 8 ([441426.] differs from 445217.1875) and both charitable cases (10,131.313 and 11,244.741).
  • C and D: see P3-1.

Readers of adjusted_gross_income and adjusted_gross_income_per_capita in 2015-2019.

  • Dollar inputs on the default series. tools/default_uprating.py:132 assigns the total, and tools/per_capita_uprating.py:217-219 repoints it to the per-capita sibling. Core's _uprating_index_value (policyengine_core/simulations/simulation.py:87-114, core 3.34.2) holds the index at its first value before 2015. A household input read in another year within 2015-2019 moves toward the SOI growth path, which is the right direction.

  • medicare_irmaa_magi_two_years_prior (LOOKBACK_DEFLATOR, line 14). Benefit years 2016-2020 move as tabled above. Benefit years 2021 and later read 2019 and later, which is unchanged.

  • foreign_earned_income legacy carry-over (foreign_earned_income.py:38-41). It reads the same per-capita index through _uprating_index_value, so it moves like the default series, in the right direction.

  • Parameter uprating: readers. These are calibration.gov.irs.soi.{farm_income,farm_rent_income,alimony_income,rental_income} (last value 2021), the soi.agi.total_agi bands (last value 2020) and gov.hhs.smi.amount (per capita, start_instant: 2027-01-01). Core's uprate_parameters only appends values after the last instant (uprate_parameters.py:339-370), so none of them reads 2015-2018.

  • Parameter dump. I dumped all 125,817 leaf parameters after system init at the head, then again with abc67b98de's income_by_source.yaml swapped in. They differ only in adjusted_gross_income and adjusted_gross_income_per_capita, at 2015-01-01 through 2018-01-01. That confirms the body's claim.

  • Others.

    • uprating_extensions.py:1186-1193 extends from 2036.
    • extend_single_year_dataset loops from the base year forward (economic_assumptions.py:126-136), so the 2024-based dataset reads none of the changed values.
    • reforms/ssa/trustees_2026.py mentions the path only in its docstring.

    No reachable result moves the wrong way.

DTrim99's review (2026-10-13), item by item, at the head.

  • Critical 1. The three expectations are now medicare_irmaa_magi_two_years_prior.yaml:39 (503_654.40), :97 (100_730.88) and senior_household_2015_2016_integration.yaml:57 (10_073.09). The index header (:5-19) and the per-case comments use the new values, and the docstring (medicare_irmaa_magi_two_years_prior.py:62-67) no longer describes the gap. A repo-wide grep finds no other mention of 0.765, 0.721, 27,704, 504,425 or 100,885 outside unrelated files.
  • Should 1. The changelog has the IRMAA sentence. The body's Tests and IRMAA sections are rewritten, and the "results to follow" line is gone.
  • Suggestion 1. Cases 8 and 9 were added. Case 9 = 465,924.93, which matches the reviewer's figure.
  • Suggestion 2. Sequencing with Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128 is addressed below.
  • Suggestion 3. The guard docstring names the IRMAA lookback.

#10128 (head 664ed5ebcc).

#10132 (head aada9740fa).

CI. The Pull request run 41714937303 at 7ef425eccd, attempt 1, completed with all 36 jobs success, none cancelled or skipped. That includes Microsimulation, every Baseline and Contrib shard, Household API Partners and PR validation. The hand-off's two outstanding jobs (Contrib other-shard-1 and -3) have since passed.

Scope. No file under .github/ or policyengine_us/tests/policy/baseline/partners/. A search for open PRs on income_by_source returns only this PR. The known overlaps are #10128 and #10132, described above.

Body and docs.

  • The impact: the hub will measure and axiom: (hub files pe-parity before landing) lines are present.
  • Every numeric claim checked above holds, except the P3-2 wording.
  • The changelog fragment is top-level and typed fixed.
  • Statements that rest on the hub's SOI read:
    • the "What changes" table (2015-2018);
    • "within 0.04%" (TY2019; see P3-2);
    • the hand-off's "Source values" section, which says 2015-2017 weren't re-read. That is now superseded by the hub's read.
  • Suggested body addition (in "Why", after the splice sentence): "The hub opened 15in11si.xls to 19in11si.xls on 2026-11-03 and read the same 2015-2019 totals, to the thousand."

Effect at baseline.

  • What moves. The diff moves household-level outputs only, for periods that read the index across 2015-2019:
    • IRMAA lookback MAGI, and Part B and Part D IRMAA through it, for single-year households in benefit years 2016-2020;
    • default-series dollar inputs read in a year other than the input year within that span;
    • the FEIE legacy carry-over.
  • What doesn't. Microsimulation on the 2024-based dataset reads none of the changed values, as the parameter dump and the extension loop show. No population check is needed beyond confirming that the 2024+ aggregates are unchanged, which CI's Microsimulation job is consistent with.

Methodology. The PR splices SOI actuals onto CBO's projection for a calibration series. No statute, regulation or legal reading is involved, so METHODOLOGY_CHOICE: no.

The brief against the PR. The build brief asked for Hypothesis property tests for logic with outputs people rely on. The PR has none, and the body gives the reason: the change is data and the guard enumerates every year. I accept that. The brief also asked the build to check every SOI value against the table. The build couldn't reach irs.gov, and the hub's read now covers 2015-2019.

What I couldn't check

  • The SOI workbooks themselves. irs.gov returned 403 through this VM's proxy. I relied on the hub's read for 2015-2019. Nobody in this round re-read TY2013 (9,093,628,703 thousand).
  • The 2016 Part D IRMAA threshold for P3-3. cms.gov, uscode.house.gov, govinfo.gov and ecfr.gov were all blocked.
  • Population-data microsimulation. Not run, per the brief. CI's Microsimulation job passed.

Generated by Claude Code

@MaxGhenis
MaxGhenis dismissed DTrim99’s stale review November 5, 2026 07:02

Round 1 of the hub review approved this PR, and these changes answer this review: main merged in; the three IRMAA expectations updated (medicare_irmaa_magi_two_years_prior.yaml Cases 2 and 7, senior_household_2015_2016_integration.yaml Case 2); the docstring and index comments updated. The changelog and PR body now state the lookback effect (2019 factor 0.765 → 0.932, 2020 factor 0.721 → 0.929).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 2598688 into main Nov 5, 2026
36 checks passed
@MaxGhenis
MaxGhenis deleted the fix-cbo-agi-2016-2018-soi branch November 5, 2026 11:30
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

US + core hub merge audit, us#10114 at e0ba755386688b9dbbb1ec8538668c665bb21924 (squash): Fill CBO total AGI for 2015-2018 with IRS SOI actuals.

  • Review: Round 1 approved 7ef425eccd, with METHODOLOGY_CHOICE no. The hub's merge of main changes none of this PR's files, and the hub applied the findings to the body:
  • Earlier review: the hub dismissed DTrim99's review with a note that maps each request to the change that answers it.
  • Gates checked live at merge:
    • gh pr checks exits 0;
    • the latest Pull request run at the head concluded success (41984504030, 36 of 36 jobs), on main merged in at 2b61205a18;
    • MERGEABLE;
    • not a draft;
    • no reviewer's latest review requests changes;
    • the semantic-overlap guard;
    • head pinned.
  • Impact: none on the default dataset from 2021 on, by a code read. It holds 2024 records, so its inputs read the series from 2024 on, and the IRMAA lookback factors match main's from benefit year 2021. No parameter that uprates on the series stores its last value before 2020.
  • Axiom: n/a, since the PR corrects historical calibration data.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants