[anth-volk]

Fixes #4161

Warning

Deploying this PR changes the staging and production v2 database schemas and table privileges. The release's v2 migration job adds four tables, moves the execution columns off simulations, re-keys the result tables by run and adds the economy output's missing columns. It needs the changed tables empty, which they are while no managed role can write them (see "Data"). It also declares two new database roles, which an operator creates (see "What this waits for before merging"), and changes four rules' entries in the rules registry (the three privilege rules, and the allowed table-model imports of api.v2.pure-modules-no-database for the new run_results module), so it needs a core developer's review.

Summary

API v2's simulation tables recorded one execution per simulation. This PR makes every run a row, with the versions it used, so re-running a simulation or report keeps the earlier results, and prepares the tables for the simulation API (PolicyEngine/policyengine-sim-api) and its Modal workers to write them.

• simulations keeps only what to calculate. Its execution columns (status, error_message, started_at, completed_at, household_result) move to runs; output_dataset_id is replaced by the run's output file.
• simulation_runs (new): one row per run of a simulation for one year, with the catalog version (tax_benefit_model_version_id), country_package_version and policyengine_version it used, the dataset and region it resolved, its status (pending, running, succeeded, failed, reusing the v2_simulation_status type), Modal call ID, error, timings, household_result for a household run, and output_uri with output_sha256 for an economy run's output file. Checks: a finished run has completed_at; the output URI and digest are set together; the digest is 64 characters; the year is 1900–2200. Indexes for the newest successful run of a simulation in a year on given versions, and for runs by status and start time.
• report_runs (already one row per run with both versions) names the two simulation runs it compares: baseline_simulation_run_id (required) and reform_simulation_run_id (empty for a household report with only a baseline), each a foreign key to simulation_runs with RESTRICT. It gains modal_call_id, an index by status and start time, and the parts of an economy comparison that have no result table: the baseline's and reform's cliff gap and share (four numbers, set together or not at all), the SPM receipt (spm_provenance, jsonb) and the data release (data_version).
• labor_supply_responses and labor_supply_decile_responses (new): the labour supply response of a report run, one row per report run (the earnings changes from the income and substitution effects and their total, the revenue change, the two relative responses, and baseline, reform and changed weekly hours split into the two effects), and one row per report run and income decile (each decile's income and substitution responses, per household and relative to earnings).
• Budget windows. reports.budget_window_size (1 to 75 years; an economy report with a year, ending by 2200) marks a budget-window report. budget_window_runs (new) holds one row per calculation of one: its report, catalog version, trigger, whether its years calculate their simulations again (recalculate), status (reusing v2_report_run_status and v2_report_run_trigger), error and times. Each year is a report run naming budget_window_run_id and year, at most one per year of a window run; a composite foreign key keeps the window run to the same report. What starts a window's later years is left to the simulation service's scheduling work.
• Results keyed by run. poverty and inequality describe one simulation, so they belong to a simulation run (simulation_run_id, cascade). The other result tables (budget_summary, program_statistics, decile_impacts, intra_decile_impacts, congressional_district_impacts, constituency_impacts, local_authority_impacts) still belong to a report run and name the two simulation runs it compares (baseline_simulation_run_id, reform_simulation_run_id) instead of two simulations; each row is unique within its report run. aggregates and change_aggregates are unchanged.
• Each result row says what it measures. decile_impacts.decile_variable names the variable that groups households when it isn't the income variable's own deciles (UK wealth deciles: household_wealth_decile), and poverty.filter_group names the group a filter variable selects (age: child, adult, senior), as PolicyEngine.py's outputs name them; both join their table's unique key, where missing values count as equal (NULLS NOT DISTINCT).
• The economy output's missing fields. congressional_district_impacts gains the winner, loser and no-change shares; constituency_impacts and local_authority_impacts gain the UK disclosure fields (disclosure_tier, checked to be reported or approximate, and the four _lower/_upper interval bounds); program_statistics.entity and is_tax become nullable, because the detailed budget names neither. Every field of the simulation API's single-year economy output now has a column.
• household_jobs calculates one household for one year under one policy: it gains household_id, year (1900–2200), the catalog and package versions, modal_call_id, updated_at and the same completion check; request_data now holds the calculation's options, such as axes. Null policy_id means current law.
• Foreign keys between the two services' tables stay. The simulation API and its workers write simulations, reports, budget_window_runs, household_jobs, the run tables and the results; the Public API writes users, saved items, policies, dynamics, households and the catalog. Each service writes only its own tables, and the foreign keys between them stay, so PostgreSQL checks every row a simulation-side row names when it is written. The new columns that name the Public API's rows get keys with the delete rules the same columns have on simulations: a simulation run's catalog version and dataset (RESTRICT) and region (SET NULL), a household job's household and catalog version (RESTRICT), and a budget-window run's catalog version (RESTRICT). reports.user_id gains an index, which account deletion's SET NULL searches. No privilege is needed for the checks: PostgreSQL runs them with the referenced table's owner's privileges.
• API v1's alpha create_report_run takes the simulation runs a new report run compares, with its package versions, as one ReportRunInputs.
• simulation_runs is also a v1 Cloud SQL table name; the v2 metadata validation now treats it as a name the two databases share, as it does simulations.

No table is dropped. No route, request or response changes: nothing serves these tables yet.

Table privileges

policyengine_api/data/v2/privileges.py declares two new managed roles, and the Public API's runtime role's reads:

Tables	policyengine_v2_runtime (the Public API)	policyengine_v2_simulation_service (the simulation API on Cloud Run)	policyengine_v2_modal_worker (its Modal workers)
simulations, reports, budget_window_runs	SELECT	SELECT, INSERT	SELECT
simulation_runs, report_runs, household_jobs	SELECT	SELECT, INSERT, and UPDATE of the call-start columns	SELECT, and UPDATE of the run-state and result columns
The eleven result tables, including the two labour supply tables	SELECT	—	SELECT, INSERT
policies, dynamics, households, tax_benefit_models, tax_benefit_model_versions, regions, datasets, parameters, parameter_values	unchanged	SELECT	SELECT
Every other table (including aggregates, change_aggregates)	unchanged	—	—

UPDATE is granted per column. The privilege declarations can now name a privilege on some of a table's columns (update_columns(...)), beside whole-table privileges, and PostgreSQL refuses a statement that sets any other column:

• The Modal worker may set a run's status, modal_call_id, error_message, started_at, completed_at and updated_at, and its results: a simulation run's output_uri, output_sha256 and household_result; a report run's four cliff columns, spm_provenance and data_version; a household job's result.
• The simulation service may set status, modal_call_id, error_message, completed_at and updated_at of the runs and household jobs whose Modal calls it starts: it records a call's ID once Modal accepts the call, and fails the run when Modal refuses it.

So neither role can change what a run is (its simulation or report, versions, dataset, region, year, simulation runs or budget-window run). The database doesn't limit a worker to its own run; the workers' code does. reconcile_v2_privileges.py apply reads column grants from pg_attribute, grants each declared column privilege a role lacks (one GRANT UPDATE (…) per role and table) and reports column grants the declarations don't name. verify checks each declared column with has_column_privilege, and fails when a role holds UPDATE on a whole run table, which would let it set every column. The catalog writer and the Stage 12 role keep exactly their privileges.

Declaring a role before it exists. reconcile_v2_privileges.py apply grants nothing, and verify fails, when a declared role is missing, so declaring these roles would otherwise fail every deploy until an operator creates them. Both roles are listed in a new ROLES_AWAITING_PROVISIONING: while a listed role is missing, apply and verify skip it and name it in their output and job summary ("Declared roles not created yet, skipped until an operator creates them"), and verify reports "Every other declared v2 table privilege is held". Once the role exists, the next deploy grants its privileges and verify checks them. A missing role that is not listed still blocks every grant and fails verification. A follow-up PR removes the two roles from the list once they exist in both environments.

The rules registry's api.v2.deployed-privileges-match now describes column grants, the whole-table check and the roles awaiting creation; api.v2.runtime-privileges-declared requires each column privilege to name columns of its table and never sit beside the same privilege on the whole table; api.v2.runtime-privileges-tested-as-runtime-role names the two simulation API roles. docs/generated/rules.md is re-rendered.

tests/integration/test_v2_simulation_role_privileges.py runs the simulation API's writes as each role: the simulation service creates the rows and records or fails a call but can't set a run's start, output or definition; a worker updates its run's state and results and inserts results but can't create a simulation, change a result or change what a run is; the Public API reads but can't write. Another test checks, for both simulation API roles, every column of the three run tables and budget_window_runs: an UPDATE of it succeeds exactly when the declarations name it for that role. CI's v2 integration job runs it, and the v2 Alembic job runs tests/integration/test_alembic_v2_simulation_runs.py.

Revisions

One revision, 1356f658347d (parent c614b6bcccfc, the current master head), autogenerated with uv run alembic -c alembic-v2.ini revision --autogenerate against a disposable PostgreSQL 17 database at its parent. It replaces the five revisions this PR had before; they were deleted in new commits, so the branch moves forward without a force-push.

Its three post-generation corrections are of one kind, each commented in the revision and covered by a test: columns of new tables reuse enum types the baseline created. The generated create_table would create each type again and fail (DuplicateObject), so each column uses postgresql.ENUM(..., create_type=False):

1. simulation_runs.status reuses v2_simulation_status, which the baseline created for simulations.status;
2. budget_window_runs.status reuses v2_report_run_status;
3. budget_window_runs.trigger reuses v2_report_run_trigger.

V2_ALEMBIC_HEAD_REVISION is now 1356f658347d. The Alembic guide documents the revision and the new roles. NULLS NOT DISTINCT needs PostgreSQL 15 or later.

Data

The upgrade adds required columns without server defaults to household_jobs, report_runs and the result tables, so it needs those tables empty. They are while no managed role holds a write privilege on them (on master, the runtime, catalog writer and Stage 12 roles hold none on any table this PR changes). On a table with rows the upgrade fails inside its transaction and the release stops before deploying; nothing is migrated or deleted. The downgrade restores required columns of simulations and the result tables, so it works only while simulations is empty, which leaves no run or result row either; it drops household_jobs' new columns with their values.

Compatibility with the previous Public API revision

The deploy upgrades the v2 schema before the new release starts, so the previous release runs against this revision for a while, and after a rollback. It can't read or write any table the revision changes: its managed roles hold no privilege on them, except the catalog writer, which counts the rows of simulations, reports and report_runs (a count reads no column). tests/integration/test_alembic_v2_simulation_runs.py checks that the revision adds its four tables and changes only the simulation-side tables, and downgrades it and upgrades it again. The runtime role's new reads only add privileges.

Environment variables

This PR reads no new environment variable. The CI changes only add test files to existing steps of alembic-v2-check.yml (through .github/scripts/test_alembic_v2_lifecycle.sh), whose job env: sets the database URL and V2_ALEMBIC_DISPOSABLE_TEST as literals, and v2-integration-check.yml, whose job env: also sets DEPLOYMENT_ENVIRONMENT. The existing v2 seed workflow (.github/workflows/seed-v2-database.yml, called from push.yml with secrets: inherit and deployment_environment staging or production) applies the revision and the privileges. Its job runs in that GitHub environment and reads:

• V2_MIGRATION_DATABASE_URL: GitHub environment secret, present in staging and production; passed as step env:.
• V2_SUPABASE_PROJECT_REF, V2_SUPABASE_ENVIRONMENT: GitHub environment variables, present in both; job env:.
• PRODUCTION_V2_SUPABASE_PROJECT_REF: repository variable, present; job env:.
• DEPLOYMENT_ENVIRONMENT: the workflow input. GITHUB_STEP_SUMMARY: set by GitHub Actions.

The two new roles' credentials are created and given to the simulation API outside this repository (below); until an operator confirms those steps, they are not verified.

What this waits for before merging

• Review, including a core developer's review of the rules registry change.
• PostgreSQL 15 or later in the staging and production v2 databases, for NULLS NOT DISTINCT: met. Supabase's project details give PostgreSQL 17 for both (staging 17.6.1.166, production 17.6.1.155; checked on 2026-11-06).
• Other open PRs that add v2 revisions, including the foreign keys from saved reports and simulations to reports and simulations: whichever merges second points its first revision's down_revision at the other's head.
• No operator step blocks the merge. The two new roles may be created before or after it. The required order:
  1. This PR merges. Each environment's deploy upgrades the schema, grants the runtime role's new reads, and names the two new roles as skipped.
  2. In staging, an operator creates the login roles policyengine_v2_simulation_service and policyengine_v2_modal_worker in the v2 database (no superuser, database, role-creation, replication or RLS-bypass attributes; no role membership; no table grants by hand), generates each password without printing it, and stores each URL in Secret Manager in the simulation API's project (simulation-service-staging-database-url, modal-worker-staging-database-url). Then the "Seed v2 database" workflow runs for staging (by hand, or with the next release), which grants and verifies the two roles' privileges.
  3. The same in production (…-production-database-url, workflow environment production).
  4. A follow-up PR removes the two roles from ROLES_AWAITING_PROVISIONING, so that a missing role fails the deploy again.
  5. Only after step 3 in an environment may the simulation API start connecting there with these roles: its Cloud Run service with SIMULATION_SERVICE_DATABASE_URL (from the first secret, Connect the Simulation Entrypoint to the Public API's database with its own role policyengine-sim-api#898; the GitHub environment variable naming the secret is SIMULATION_SERVICE_DATABASE_URL_SECRET_NAME) and its Modal workers with the Modal secret policyengine-modal-worker-database (key MODAL_WORKER_DATABASE_URL, from the second).
• How to check each step without printing a secret: pg_roles and pg_auth_members show the roles' attributes and no memberships; the seed workflow's summary shows the roles' grants and Every declared v2 table privilege is held; has_table_privilege and has_column_privilege spot checks (for example the Modal worker can update simulation_runs.status but not simulation_runs.simulation_id, and holds no UPDATE on the whole table); gcloud secrets versions list shows each secret's enabled version.

Testing

Each commit was written red-green: the tests first, seen failing for the expected reason, then the code. Only the tests each commit adds or modifies were run, each by node ID, against a disposable postgres:17 database named policyengine_v2_alembic_test (set up as alembic-v2-check.yml's service container is), with alembic check, ruff, ruff format and mypy before each commit; the full suite runs in CI.

Commits since d674032a2 (after merging master at 90b212b51, which added no v2 revision):

• aa2875f40 (one revision). Seen failing: the revision chain test and the new revision source test. Passing: those two and the two integration revision tests.
• 437560940 (foreign keys stay). Seen failing: the six new key, index and refusal tests. Passing: 16, including the integration tests whose helpers now store the catalog rows and household a row names.
• d6693ad36 (report runs name two simulation runs). Seen failing: 16 tests, and the two report-run unit modules at import. Passing: 38.
• 41b22d179 (typed results). Seen failing: 9 tests, and the model test module at import. Passing: 18, and the 18 report-run unit tests on SQLite with the new check.
• c4742f225 (budget windows). Seen failing: 16 tests. Passing: 47.
• c6bb73a87 (column-level UPDATE). Seen failing: three test modules at import and the two role tests. Passing: 17.
• b990f6cb7 (one argument for the database's grants). Passing: the 15 changed planning tests and the four changed integration tests.
• 357719fad (stale tests). Seen failing: the decile key test and the two window-run key tests. Passing: 5.
• 56212cc00 (module split). Seen failing: api.quality.file-length (two files over 800 lines). Passing: the check, and all 79 tests this PR adds or changes.
• 7642bd430 (every run column). Seen failing: the new per-column role test, with an undeclared column grant added to the disposable database; passing once it was revoked.
• 772ca6b14 (the downgrade's stated precondition). Text only; the revision source test and alembic check pass.

On 772ca6b14, and again on a trial merge with master at f932adf71 (which adds no v2 revision or model, and merges without conflicts): alembic check reports no changes; lint-imports keeps all five contracts; python -m scripts.rules validate, python scripts/run_quality_guards.py and uv lock --check pass; ruff and ruff format are clean; mypy's configured scope (make typecheck-v2) is clean. policyengine_api/data/v2/ is outside that scope. Run explicitly on the 13 changed modules there and the revision, mypy reports 48 errors, each of a kind that master's versions of the same files already have (29 errors there): SQLModel's Field overloads don't accept sa_column, and autogenerated revisions reference sqlmodel.sql, as master's own revisions do.
Three review rounds ran on the rework, plus a search for existing tests the schema change would break:

• Round 1 and the search found three failures, fixed in 357719fad and 56212cc00: the file-length limit, an existing test requiring every foreign key to name a delete rule (the budget-window key is now CASCADE), and a stale decile key assertion. Round 1 also noted that no role may update budget_window_runs yet, left for the work that schedules a window's later years.
• Round 2 found no bugs; its quality findings (test coverage of every granted column, a rule's scope, two texts) are fixed in 7642bd430.
• Round 3 found no bugs; its two text findings are fixed in 772ca6b14 and in this description.
  Each round regenerated the revision from c614b6bcccfc against the models: its operations match, apart from order and the three commented corrections.

Moved code and inline suppressions

• The result models moved from policyengine_api/data/v2/models/reports.py to the new run_results.py, and the privilege planning and reporting code from privilege_reconciliation.py to the new privilege_plans.py, under api.quality.file-length. The code is unchanged. privilege_reconciliation re-exports the moved names, and the models package still exports every model. Helpers now used from a second module were renamed: _safe to safe_identifier, _skipped_roles_line to skipped_roles_line, _verification_report to verification_report.
• One new inline suppression: # noqa: E402 on the models package's import of run_results, with its reason. Like the package's other imports, it must follow the naming convention the package sets before any table is declared, so it can't move to the top of the file.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.03340% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.19%. Comparing base (90b212b) to head (772ca6b).
⚠️ Report is 35 commits behind head on master.

Files with missing lines	Patch %	Lines
policyengine_api/data/v2/privilege_plans.py	90.07%	3 Missing and 11 partials ⚠️
...licyengine_api/data/v2/privilege_reconciliation.py	88.63%	2 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4162      +/-   ##
==========================================
+ Coverage   92.14%   92.19%   +0.04%     
==========================================
  Files         297      298       +1     
  Lines       16673    16917     +244     
  Branches     2719     2754      +35     
==========================================
+ Hits        15364    15596     +232     
- Misses        792      798       +6     
- Partials      517      523       +6     

Flag	Coverage Δ
integration	52.98% <91.64%> (+0.71%)	⬆️
unit	90.10% <88.10%> (-0.19%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.