Skip to content

Revert "Refuse dataset years the country model does not project to (#730)" - #742

Merged
anth-volk merged 1 commit into
mainfrom
revert-dataset-year-refusal
Nov 6, 2026
Merged

anth-volk merged 1 commit into
mainfrom
revert-dataset-year-refusal

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #741

Reverts #730 (issue #729), released in .py 6.13.0 and 6.14.0. The supported dataset years will be defined as constants in the Public API, which refuses any other year with a 400 response before a job starts, so .py goes back to creating year files for any requested year.

Summary

git revert 18581729 applied to main without conflicts. Every file #730 changed is now identical to its state before #730, except #730's two changelog fragments, which the 6.13.0 release had already removed.

Removed:

  • policyengine.provenance.dataset_years, with supported_years(country_id, dataset_path), refuse_unsupported_years and UnsupportedDatasetYearError, and their exports from policyengine.provenance;
  • the allow_unsupported_years argument of the UK and US create_datasets and ensure_datasets, and the year check those functions ran before creating year files;
  • the "Supported years" section of docs/microsim.md;
  • tests/test_dataset_years.py, and the allow_unsupported_years keyword in the create_datasets mocks in tests/test_release_manifests.py and tests/test_us_year_file_stamp.py.

Added changelog.d/741.removed.md.

Behaviour change for .py users

  • create_datasets and ensure_datasets no longer raise UnsupportedDatasetYearError. Code that imports it, calls supported_years or passes allow_unsupported_years has to drop those.
  • US: year files are created for any requested year again, including years before a dataset's data year. For populace_us_2024 those are 2022 and 2023, for which the certified dataset holds almost no data (see Refuse dataset years the country model does not project the dataset to #729). Years after the end of policyengine-us's projection are also created again, from inputs policyengine-core carries forward or uprates.
  • UK: a year outside policyengine-uk's projection fails again with policyengine-uk's own No dataset found for year ... instead of UnsupportedDatasetYearError.
  • The UK qualification check (uk_candidate.projected_years_from_model and the supported_years check that uses it) and execution receipts (receipt._projected_years) go back to their own year rules from before Refuse dataset years the country model does not project to #730. Like Refuse dataset years the country model does not project to #730's UK rule, they build the projection with policyengine-uk's extend_single_year_dataset and keep the years from the data year onwards, so the years they record do not change.

supported_years still appears in qualification.py, uk_candidate.py, receipt.py, the execution-receipt and replay-receipt schemas, docs/receipts.md, replay_policy.md and their tests and fixtures. All 157 matching lines in src/, scripts/, docs/ and tests/ are identical to the tree before #730, so none of them came from #730.

Related

#735 (issue #734) builds on #730 and will be closed.

Environment variables

No environment variables are added or changed, and no workflow, Dockerfile or deployment file changes. The deleted tests/test_dataset_years.py was the only reader of the optional POLICYENGINE_UK_DEFAULT_DATASET and POLICYENGINE_US_DEFAULT_DATASET, which no workflow sets.

Tests

Run locally in the locked environment (uv sync --frozen --extra dev, Python 3.14.6), each test selected by node ID:

  • tests/test_release_manifests.py::TestReleaseManifests::test__given_us_ensure_datasets_without_dataset__then_uses_certified_default: passed. The revert removed allow_unsupported_years=False from its expected create_datasets call.
  • tests/test_us_year_file_stamp.py::test_ensure_cuts_again_exactly_when_load_would_refuse: passed. It checks that ensure_datasets recreates year files exactly when load_datasets would refuse them, and it is the only user of _create_current_files, the create_datasets stand-in whose allow_unsupported_years parameter the revert removed.

Other checks:

  • python -c "import policyengine; import policyengine.provenance" succeeds. policyengine.provenance no longer has supported_years or UnsupportedDatasetYearError, and policyengine.provenance.dataset_years no longer exists.
  • uv run ruff format --check on the seven changed Python files passes. uv run ruff check (locked ruff 0.12.11) on them reports only the 42 UP038 findings in receipt.py that main also has; ruff check --ignore UP038 passes.
  • towncrier check --compare-with origin/main finds changelog.d/741.removed.md, and .github/check-agent-notes.sh HEAD passes.

tests/test_dataset_years.py is deleted, so it was not run. No other tests were run locally.

🤖 Generated with Claude Code

)"

This reverts commit 1858172.

The supported dataset years will be defined as constants in the Public
API, which refuses any other year with a 400 response before a job
starts, so .py no longer checks requested years itself (#741).

This removes policyengine.provenance.supported_years,
UnsupportedDatasetYearError, refuse_unsupported_years and the
allow_unsupported_years argument of the UK and US create_datasets and
ensure_datasets. Those functions create year files for any requested
year again, including US years before a dataset's data year, such as
2022 and 2023 for populace_us_2024, which holds almost no data for
them. The UK qualification check and execution receipts go back to
their own copies of the UK year rule, and docs/microsim.md no longer
has a "Supported years" section.

Adds changelog.d/741.removed.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as ready for review November 6, 2026 13:36
@anth-volk
anth-volk merged commit aec5f09 into main Nov 6, 2026
14 checks passed
@anth-volk
anth-volk deleted the revert-dataset-year-refusal branch November 6, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Revert the dataset-year refusal from #730

1 participant