[anth-volk]

Fixes #4211

Stacked on #4210 (remove household_options). This pull request's base is remove-household-options, so its diff shows only its own changes. Merge #4210 first.

Summary

The maintainer decided on 4–5 November 2026 that US 2022 and 2023 are dropped everywhere: household and society-wide calculations share one list of years per country, defined once in the Public API, US 2024–2035 and UK 2024–2032. A society-wide (economy) request for any other year is refused with a 400, even when a result for that year is stored. Household calculations are not refused; only the menus stop offering those years.

One constant per country

• policyengine_api/economy_years.py now holds US_ECONOMY_YEARS (2035 down to 2024) and UK_ECONOMY_YEARS (2024 up to 2032), with ECONOMY_YEARS mapping each country to its tuple. It replaces:
  • the hand-written US list in PolicyEngineCountry.build_microsimulation_options (country.py);
  • uk_economy_years, which read the projection end from the installed policyengine-uk;
  • the values of METADATA_TIME_PERIODS in data/v2/catalog/extraction.py, which is now ECONOMY_YEARS itself.
• UK values checked first: before hard-coding them, uk_economy_years was called with the installed policyengine-uk 2.151.0 (PolicyEngine.py 6.12.0). It returned [2024, 2025, 2026, 2027, 2028, 2029, 2030, 2031, 2032]: PROJECTION_END_YEAR is 2032, and the capital gains index sets no earlier end.
• Order: both responses already ordered each country the same way, so each keeps it. The US is newest first in /metadata and in the catalog; the UK is oldest first in both.
• Effect on /metadata: economy_options.time_period for the US loses 2022 and 2023. The UK list is unchanged (2024–2032).
• Effect on the v2 catalog: the US years drop 2022 and 2023 (2035–2024). The UK years move from 2024–2030 to 2024–2032.
• Dependency check baseline: uk_economy_years was the package's only import of policyengine_uk. The declared-dependencies guard then failed with "baseline entry no longer applies: DEP003 policyengine_uk". The entry is removed from rules/baselines/deptry.txt, the count of api.dependencies.declared-and-used goes from 15 to 14 in rules/registry.toml, and docs/generated/rules.md is regenerated. These are rule machinery files, so a core developer other than the author needs to approve them (api.process.rule-machinery-code-owned).

The 400 for other years

A society-wide request whose year isn't in its country's constant gets a 400 with the code DATASET_YEAR_UNAVAILABLE. This is the Public API's own code: the simulation service doesn't send it and refuses no years. No existing code had this meaning in a response the Public API produces. The name already appears in worker_spm.WORKER_INPUT_ERROR_CODES and the SPMValidationError docstring on master, which expect the simulation service to send it; those references are outside this pull request and are left as they are. For a US request for 2022, the body is:

{
  "status": "error",
  "message": "US results are available for 2024 to 2035. 2022 is outside the years the current US model and data cover.",
  "result": null,
  "errors": [{"code": "DATASET_YEAR_UNAVAILABLE", "message": "US results are available for 2024 to 2035. 2022 is outside the years the current US model and data cover."}]
}

• The body matches what the app reads. The message pattern is the one in UK reports request years after 2032, and a failed year shows the raw API response policyengine-app-v2#1452. Keep report years within the model's data, and show a failed year's error readably policyengine-app-v2#1453 (app/src/libs/flagship/calculationError.ts) treats a failure as "year not covered" only when errors holds an item with "code": "DATASET_YEAR_UNAVAILABLE", and shows the top-level message as written. The first and last years in the message are read from the country's constant.
• How it's built: like the SPM 400s and the region and policy 400s (Return 404 when an economy request names a policy its country lacks #4116, Refuse UK economy regions .py's region registry can't resolve #4118, Refuse new policies whose period keys or values simulations cannot read #4142). EconomyService._validate_economy_year (in economy_simulation_requests.py) raises SPMValidationError("DATASET_YEAR_UNAVAILABLE", message). The route's existing _bad_request_response then returns _make_error_response(error, 400, result=None, errors=[{"code": ..., "message": ...}]). SPMValidationError is the existing error type for typed 400s with a code in errors, not only SPM ones.
• Where it's checked: first, before anything is read from storage or sent to the simulation service. That means before region normalisation and validation, the SPM capability request, the stored-result lookup and any submission.
  • GET /{country}/economy/{policy_id}/over/{baseline_policy_id} (get_economic_impact). A stored result for the year is not returned.
  • DELETE on the same URL (delete_failed_economic_impact), which validates its query exactly as the GET does.
  • GET .../budget-window (get_budget_window_economic_impact), on the start year, after the existing window-size and end-year checks.
• Budget windows:
  • A window whose start year is outside the list is refused. A window that starts in the list cannot include a year before the list's first year, so that case needs no check of its own.
  • Not changed: a window that starts in the list and ends after its last year, such as US 2034 with window_size=5 (2034–2038). Windows may still end as late as 2099 (BUDGET_WINDOW_MAX_END_YEAR).
• Household endpoints are unchanged.
• The canonical-spm.md query rules and the route docstrings describe the 400.

Open question: budget windows that end after the list's last year. Today the Public API accepts such a window and submits one batch job for every year to the simulation service, which refuses no years. It either computes the later years or fails on them, for example on a UK year after 2032, the last year policyengine-uk 2.151.0 projects its dataset to. The route answers with whatever that produces:

• the computed window, when every year is computed;
• a 200 with status failed and the per-year progress, when the batch fails. The years inside the list are not returned then either.

Should a window be refused when it ends after the list's last year, or limited to it?

Catalog publication replaces a stored version's year list

The catalog stores each version's years (metadata_time_periods), keyed by country and PolicyEngine.py version. assert_country_matches raised CatalogPublicationError when they differed from the staged ones. So the release after this change would have failed publication for the already-stored 6.12.0 version, whose US list still has 2022 and 2023.

• _comparison_pairs leaves metadata_time_periods out of the model-version comparison, for both the pre-publication check and the final validation.
• New update_stored_time_periods (publication_reconciliation.py) reads the stored and staged lists. When they differ, a change of order included, it sets the staged list on the stored version in the publication's transaction. publication._reconcile_stored_versions (formerly _check_stored_versions) calls it only after every stored version has passed the comparison of every other column and table. So any other difference still fails before anything is changed.
• Logging and evidence: each replacement logs a warning in the same form as the kept-regions warning, for example replaced the time periods of the stored us catalog version 6.12.0: [2035, ..., 2022] became [2035, ..., 2024], with the country, version and both lists in policyengine_attributes. PublicationEvidence gains time_period_updates, printed as updated_time_periods (country_id, policyengine_version, previous_time_periods, time_periods). The evidence then still describes what is stored.
• Regions keep their current behaviour: the stored regions are kept, with a warning.
• Other stored versions: publication stages only the installed PolicyEngine.py version, so other stored versions keep the years they were published with. For example, an older US version still lists 2022 and 2023 on /v2/economy-options?policyengine_version=<older>.
• Docs:
  • docs/migration/stage-9-v2-metadata.md no longer says that publication never rewrites a stored version or that a version never changes once published. It gains a "Time-period options" section, an evidence entry, and the time periods as the second exception next to regions.
  • The docstrings of SelectedCatalogStore, MetadataResponseCache and EXPLICIT_VERSION_MAX_AGE_SECONDS made the same claim, and now give the exception. Processes of the previous release keep the years they already read until they stop. HTTP caches can keep a response with the old years for up to five minutes, or a day for a request naming policyengine_version.
• New integration test file: tests/integration/test_v2_catalog_publication.py is at 1,453 of the 1,500 lines a test file may have. The new database tests are therefore in tests/integration/test_v2_catalog_publication_time_periods.py, which .github/workflows/v2-integration-check.yml now runs in the same step.

Testing

Run on Python 3.12 (UV_PYTHON=3.12 uv run --frozen --extra dev pytest), each test selected by its node ID. All pass.

New:

• tests/unit/test_economy_years.py::test_us_years_are_2024_to_2035_newest_first, ::test_uk_years_are_2024_to_2032_oldest_first, ::test_catalog_extraction_uses_the_same_years, ::test_each_extracted_catalog_version_stores_its_countrys_years (us, uk), ::test_the_unavailable_year_message_names_the_years_and_the_request (US 2022, UK 2033).
• tests/unit/test_country.py::test__metadata_offers_the_countrys_economy_years_in_their_order (us, uk).
• tests/unit/routes/test_economy_unavailable_years.py:
  • ::test_an_annual_year_outside_the_countrys_years_gets_400_and_nothing_runs: GET and DELETE, for US 2022, 2023 and 2036 and UK 2023 and 2033. The test checks the whole body, including the top-level message and errors[0].code, and that the service's collaborators got no call.
  • ::test_a_budget_window_starting_outside_the_countrys_years_gets_400: US 2023 with a window of 3, US 2036, and UK 2023.
  • ::test_a_stored_result_for_an_unavailable_year_is_not_returned: a finished US impact is stored for 2022. The GET answers 400, the simulation service stand-in gets no call, and the stored record is unchanged.
  • ::test_a_stored_result_for_an_offered_year_is_returned: the same setup for 2024 answers 200 from the stored record.
• tests/unit/services/test_economy_service_budget_window_requests.py::TestEconomyService::TestGetBudgetWindowEconomicImpact::test__given_window_ending_after_the_countrys_years__still_submits_it: 2034–2038 is submitted as before.
• tests/unit/v2/test_catalog_publication.py::test_stored_versions_are_compared_without_their_time_periods, ::test_time_period_update_warning_prints_country_version_and_years, ::test_matching_time_periods_log_nothing.
• tests/integration/test_v2_catalog_publication_time_periods.py, run against a disposable local PostgreSQL 17 container:
  • ::test_stored_version_takes_the_staged_time_periods_and_logs_them: a version is stored with US 2035–2022. Republishing stores 2035–2024, keeps every row ID, logs the warning and fills updated_time_periods. Republishing again reports nothing.
  • ::test_stored_version_with_changed_time_periods_still_fails_on_other_tables: a changed variable label still fails with "differs from the staged catalog in its variables", and the stored years are unchanged.

Changed:

• tests/unit/test_country.py::TestUKCountryMetadata::test__uk_years_are_years_the_model_projects (was test__uk_years_run_to_the_models_last_projected_year). It now checks that the UK years are UK_ECONOMY_YEARS and are all years the installed policyengine-uk projects to. It therefore fails if a policyengine-uk update projects to fewer years, instead of requiring the constant to follow the model.
• tests/unit/routes/test_spm_year_worker_polling.py::test_typed_worker_error_is_terminal_and_replays_after_service_recreation, ::test_a_dataset_year_error_returns_400_and_replays_from_storage and ::test_an_uncertifiable_stored_result_becomes_terminal_instead_of_repeating. Their annual requests asked for US 2036, which is now refused before the worker is polled. They now ask for 2035, the last US year, which still reaches the worker. The budget windows (2035–2036) are unchanged.
• tests/unit/v2/test_catalog_publication.py::test_completion_evidence_contains_only_reviewed_non_secret_fields: adds updated_time_periods.
• tests/unit/test_alembic_workflows.py::test_reusable_v2_integration_check_uses_disposable_databases_and_coverage: checks that the workflow runs the new integration file.

Without the service change, the 400 tests failed (14 cases). Without the publication change, both integration tests failed.

No other test expected US 2022 or 2023 in economy_options or the catalog, or the old publication failure for a changed year list.

Other checks:

• uv run --frozen --extra dev ruff format --check and ruff check on the 21 changed Python files: pass (ruff 0.15.12, the version uv.lock pins).
• python scripts/run_quality_guards.py: every guard passes.
• lint-imports: 5 contracts kept.
• mypy: no issues.
• python scripts/export_migration_contracts.py: no change.
• BASE_SHA=remove-household-options bash .github/scripts/check_changelog_fragment.sh: 2 fragments, us-years-from-2024.removed.md and catalog-time-periods.changed.md.

Environment variables

No environment variables are added, removed or changed.

.github/workflows/v2-integration-check.yml gains one test file in an existing step. The file reads only V2_MIGRATION_DATABASE_URL, and through alembic-v2.ini also V2_ALEMBIC_DISPOSABLE_TEST, as the step's other publication tests do. Both are literal values in the catalog-runtime-integration job's env: block, so they reach every step of the job. Callers run the workflow through uses: and pass no secrets, and it needs none.

No inline suppressions are added.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 94.82759% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.12%. Comparing base (39a4de4) to head (5b93037).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
..._api/data/v2/catalog/publication_reconciliation.py	70.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4212      +/-   ##
==========================================
- Coverage   92.12%   92.12%   -0.01%     
==========================================
  Files         296      296              
  Lines       16652    16696      +44     
  Branches     2712     2722      +10     
==========================================
+ Hits        15341    15381      +40     
- Misses        798      800       +2     
- Partials      513      515       +2     

Flag	Coverage Δ
integration	52.41% <74.13%> (+0.08%)	⬆️
unit	90.18% <63.79%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.