Add simulation and report definitions to the simulation schema by anth-volk · Pull Request #900 · PolicyEngine/policyengine-sim-api

anth-volk · 2026-10-22T14:29:44Z

Fixes #899

Same-Repository Draft PR Check

This PR is a draft.
This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Changelog

This PR adds a changelog fragment, changelog.d/<name>.<type>.md, whose type is breaking, added, changed, fixed or removed: changelog.d/899.added.md.

Summary

Second of four stacked PRs; it builds on #898. Revision 0002 adds the two definition tables of the simulation schema. After it deploys, both exist in staging and production with no rows; nothing writes them until stage 2.

simulation.simulations: kind (economy or household), country, policy (null for current law), dynamic, and either a household or a region code (state/ca) with an optional dataset name (null means the region's default dataset in each catalog version). No column refers to a catalog version, so the same simulation can run again on a newer one; each run resolves the region and dataset in its own version (next PR). A check requires the population to match the kind.
simulation.reports: baseline and reform simulations of the report's own country and kind (composite foreign keys on simulation, country and kind), the year (1900 to 2200) and options (a JSON object). An economy report needs a reform; a household report may have only a baseline.
Foreign keys into the Public API's tables in the same database: (policy_id, country) to policies (id, country_id), (household_id, country) to households (id, country_id) and dynamic_id to dynamics, all ON DELETE RESTRICT, each with an index. Creating them needs REFERENCES on those tables for the migrator role; a test shows the migration fails and applies nothing without it. The runtime role needs no privilege on them.
v1 IDs: unique legacy_simulation_id and legacy_report_id (plain columns, since the Public API's v1 database is separate). SimulationStore.create_migrated and ReportStore.create_migrated return the existing row when the same v1 ID is migrated again.
Definitions never change: the runtime role may read and insert them but not update or delete them. NewSimulation and NewReport check every field before a write (a kind given as text becomes its enum), and the database checks the same fields again for any other writer; a test inserts directly to reach each database check.

Environment variables

This PR reads no new environment variable. The migration job from #898 applies revision 0002 with SIMULATION_MIGRATION_DATABASE_URL as described there.

Waits for

Connect the Simulation Entrypoint to the Public API's database with its own role #898, and through it the operator step that creates sim's database roles and grants the migrator REFERENCES on policies, households and dynamics (and the catalog tables used by the next PR) in staging and production. Without the grant, the migration job fails and the deploy stops.
Migrations in the simulation chain merge one at a time.

Testing

Rebased again onto #898 on 2026-10-30, after #898 moved onto main at 4b09f76, with no conflicts. The last commit adds changelog.d/899.added.md, which main now requires. After this rebase, these passed locally: the generated rules document at every commit, the six quality guards (including the uv pin and the 800-line limit), the changelog fragment check, Ruff at the CI version, uv lock --check, and Pyright for the persistence and contract libraries, the entrypoint and the executor (at the top of the stack). No tests were run locally for this rebase; full CI runs once this is the lowest open pull request in the stack.

Earlier:

Rebased with #898 onto main at 2a9212f on 2026-10-28, without conflicts in this branch. Every commit was checked again after the rebase: persistence tests on PostgreSQL 17 with database tests required (100 passed at the tip), the contract library's tests, the entrypoint suite (797 passed), the rules guard and quality guards, Ruff, uv lock --check and Pyright. One review round found no bugs; its quality findings (the README's roles table, untested database checks, a kind given as text) are fixed here. A later review of the PR above this one found that revision 0002 gave its check constraints doubled names (ck_simulations_ck_simulations_kind), which alembic check does not compare; the last commit fixes the names and adds a test comparing every constraint name in the migrated database with the table definitions. The checks above were run locally; this push's GitHub Actions runs report their own results.

🤖 Generated with Claude Code

Add libs/policyengine-simulation-persistence, the package that will hold the Simulation Entrypoint's own Postgres schema, "simulation". This commit holds only the names and the connection: - the schema name, the runtime role name, the naming convention and the metadata every table of the schema registers in; - the runtime engine, built from the canonical postgresql:// URL the deployment stores, with the psycopg 3 driver, a small pool, pre-ping, a connect timeout and no server-side prepared statements (so the same URL works through Supabase's poolers); - a reader for the applied Alembic revision that needs no Alembic. A malformed URL raises ValueError without repeating the URL or the parser's message, since the URL holds a password. The package is unit-tested in the PR test matrix and by make test, type-checked by the Pyright job under sim.quality.pyright-libraries-and-entry, and scanned by the database access guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Replace sim.stage12.no-migration-chain, which forbade any Alembic chain in this repository, with ownership rules for the Simulation Entrypoint's own "simulation" schema: - sim.persistence.one-migration-chain: one Alembic chain at migrations/, configured by migrations/alembic.ini, keeps its version table at simulation.alembic_version, and no other Alembic configuration, environment or revision directory exists; - sim.persistence.executor-independent: the Modal executor and gateway never declare, lock or import the persistence library; - sim.persistence.no-modal-database-credentials: no Modal secret or deployment receives SIMULATION_DATABASE_URL or SIMULATION_MIGRATION_DATABASE_URL, so the comparison path's own database secret stays the only database credential Modal holds; - sim.persistence.migration-cycle: on a fresh PostgreSQL, the chain upgrades, passes alembic check, downgrades and upgrades again as a non-superuser migrator, and the runtime role reads the revision but cannot change the schema. The environment creates the schema only when it is missing, because PostgreSQL checks CREATE on the database even for CREATE SCHEMA IF NOT EXISTS; an operator may therefore create the schema in advance, owned by the migrator. The schema creation and every pending revision run in one transaction, so a failed migration leaves nothing applied. Revision 0001 grants the runtime role schema usage, revision reads and default DML privileges on the tables later revisions create. Migrations run only online and only with SIMULATION_MIGRATION_DATABASE_URL. A new "Simulation database" PR job runs the database tests on postgres:17 and is part of "CI complete". The database access guard now also scans the chain, which may import alembic and change the schema but still connects only through SQLAlchemy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… check The Simulation Entrypoint reads SIMULATION_DATABASE_URL, the runtime role's URL for its own schema. When it is set, the app builds a pooled engine (no connection opens until first use), disposes of it at shutdown, and GET /ready reports the database's applied migration revision next to HEAD_REVISION, the revision the image expects: {"status": "ready", "simulation_database": {"reachable": true, "revision": "0001", "expected_revision": "0001"}} The database does not change the readiness status, because no old-gateway or Stage 12 route uses it yet: an unreachable or slow database (bounded at three seconds) is reported as unreachable, logged with its error type only, and /ready keeps following the old gateway. Without the setting, /ready keeps its old shape. A URL that does not use the canonical postgresql:// scheme fails validation without repeating the URL. The entrypoint depends on the persistence library, so its image's build context now includes it; a new test checks that every local library the entrypoint installs is in the build context. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Each deployment (the main deploy and the Stage 12 deploy both use the reusable workflow) now runs a migrate_simulation_database job after prepare. It installs the persistence library's locked environment with the migrations extra and runs .github/scripts/migrate-simulation-database.sh, which applies the chain with alembic upgrade head and then requires the database to be at the head. The job is the only one that receives SIMULATION_MIGRATION_DATABASE_URL (a GitHub environment secret in beta and prod), in its migration step's env alone. deploy_entrypoint needs the job and runs only when it succeeded. The deploy script now requires SIMULATION_DATABASE_URL_SECRET_NAME (a GitHub environment variable in beta and prod) and mounts that Secret Manager secret's newest enabled version as SIMULATION_DATABASE_URL on every deploy, whatever the Stage 12 setting. The candidate smoke check sets REQUIRE_CURRENT_SIMULATION_DATABASE_REVISION=true and then requires /ready to report the database reachable and at the image's revision; the stable and rollback checks do not, because an older image may face a newer schema. The wiring is registered as sim.persistence.migrate-before-entrypoint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The deploy script built its Cloud Run secret list in array assignments with one command substitution per secret. Bash gives such an assignment the exit status of its last substitution only, so with SIMULATION_DATABASE_URL added last, a failure to resolve the old gateway's client secret (an invalid name, a failed version listing, or no enabled version) no longer stopped the script, and it deployed with an empty secret reference. The Stage 12 block already hid failures for the two Modal token secrets the same way. Each secret is now resolved in its own assignment, through add_runtime_secret, so set -e stops the script at the first failure, before gcloud run deploy runs. A test covers every runtime secret; three of them deployed before this change. The test is registered under sim.deploy.workflow-shell-safety. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

sim.persistence.no-modal-database-credentials checked only the two simulation URL names, and only in a Modal step's own env and run text. A step could still give Modal any other database URL, such as one named REPORTS_DATABASE_URL or a written-out postgresql:// URL, and a value set in the job's or the workflow's env reached the step without being checked. The check now finds any variable named like *DATABASE_URL* and any written-out PostgreSQL URL in the workflow, job and step environments of a step that creates a Modal secret or deploys a Modal application, in the scripts such a step runs, and in the executor's and gateway's sources. Only the comparison path's STAGE12_DATABASE_URL, and the name of the secret that holds it, may reach Modal until the comparison path is removed. The rule's statement and the library README say so, and the new tests are registered under the rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The Modal credential check required at least one character before DATABASE_URL, so a Modal step or a Modal application source using a variable named exactly DATABASE_URL passed it, although the rule allows no database URL in Modal except the comparison path's. The pattern now matches the bare name, and the tests cover it in a workflow step and in a source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…e head The Stage 12 deploy workflow can be dispatched from any branch, and it uses the same reusable workflow, so it would have applied that branch's unmerged revisions to the beta or prod database; every later deploy from main would then fail, because the applied revision is not in main's chain. The migration step now sets SIMULATION_MIGRATION_MODE from the ref: upgrade on main (alembic upgrade head, then current --check-heads), verify elsewhere (only current --check-heads). The script refuses a missing or unknown mode. Its unused SIMULATION_ALEMBIC_CONFIG override is gone. alembic current --check-heads first appeared in Alembic 1.17.1, so the persistence library's floor rises from 1.16 to 1.17.1 (the lock already pins 1.20.0). A new PostgreSQL test runs the script itself against a fresh database: verify refuses it, upgrade migrates it to the head, and verify then passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The fix that resolves each runtime secret in its own assignment introduced add_runtime_secret and used it for the main path's secrets and for the Stage 12 comparison path's Modal token and database secrets alike. The comparison path keeps its own mechanisms, so it can be removed without touching the main path: the Stage 12 block now uses add_stage12_runtime_secret, its own copy with the same set -e behavior. A test checks that each block uses only its own helper; it is registered under sim.deploy.workflow-shell-safety. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

/ready waits at most three seconds for the database, while a new connection may take up to five, and the candidate smoke read /ready once. A cold candidate whose first connection was slow therefore failed the whole deployment. When the candidate check requires the database revision and /ready reports the database unreachable, the smoke now asks again, up to SIMULATION_DATABASE_READY_MAX_ATTEMPTS times (six by default), with the usual retry delay. A database that is reachable but at another revision still fails at once. The fake curl in the deployment tests can now serve a sequence of /ready bodies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…bad examples Both rules were registered without proof, so the rules page listed them as unproven. - sim.persistence.migration-cycle: migration_cycle_failures runs a chain's upgrade, optional alembic check, downgrade and upgrade, and reports the first step that fails. The chain test uses it with the check, and the proof runs two fixture chains on PostgreSQL: one whose downgrade undoes its upgrade passes, and one whose downgrade leaves its table behind fails on the second upgrade. - sim.persistence.migrate-before-entrypoint: migration_wiring_violations checks a deploy workflow's jobs: the entrypoint waits for a successful migration job, the migrator URL reaches only the migration step, and that step applies revisions only from main. The deploy workflow passes it; the proof's bad fixture fails all three checks and the good fixture passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Each migration runs in one transaction, and adding a foreign key to a Public API table (the next revisions do) takes a lock on that table. A migration waiting behind a long Public API transaction would hold its place in the lock queue, and every Public API write to that table would queue behind it until the migration got its lock. The migration environment now sets lock_timeout to ten seconds for its transaction, so such a migration fails, applies nothing, and stops the deploy; the next deploy tries again. A PostgreSQL test holds a lock the migration needs and checks that the upgrade fails with a lock timeout instead of waiting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch tightens both rules, which had no proof: - sim.deploy.workflow-shell-safety now also requires every runtime secret to be resolved in its own assignment. combined_secret_resolutions finds a Bash array assignment that resolves more than one secret, where an earlier failure would pass unnoticed. The deploy script passes it (a new test, registered under the rule); main's script before this branch fails it, and so does the proof's bad fixture. - sim.deploy.container-context-minimal now also requires every local library the entrypoint installs to be in its build context. libraries_missing_from_context compares a project's path sources with its .dockerignore; the existing test uses it, and the proof's bad fixture leaves one library out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… failed - The Modal credential check counted only `modal secret create` and `modal deploy` as reaching Modal. A step that starts a temporary app with `modal run` (the precompute and smoke scripts do) was not examined; it is now. The current workflows still pass. - The migrator-URL checks read job and step env but not workflow-level env, which reaches every job. secret_carriers now counts it, and the deployment test uses the same scan instead of its own copy. - migration_cycle_failures reported only the exception class, so a CI failure showed neither the database error nor the detected differences. It now adds the error's first line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- The persistence README's test commands and the Alembic configuration's example now use --frozen, as every install and test run must. - DEPLOYMENT.md said the candidate smoke asks again "up to six times"; the default of six is six requests in all. - The deploy script's comment on the Stage 12 wrapper now says what is separate (the wrapper) and what both blocks still use (the secret lookups that predate the split). - The migration-cycle and migration-script tests read the applied revision through one helper in migration_support instead of two copies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A simulation is one policy applied to one population: a household, or an economy named by a region code (state/ca) with an optional dataset name, where a null dataset name means the region's default dataset in whichever catalog version a run uses. A null policy means current law. The row holds no catalog version, so the same simulation can run again on a newer one; each run resolves the region and dataset in its own version (item 1.06). Revision 0002 creates the table with: - the population checked against the kind (a household simulation has a household and no region or dataset; an economy simulation has a region and no household); - foreign keys into the Public API's tables in the same database, all ON DELETE RESTRICT: (policy_id, country) to policies (id, country_id), (household_id, country) to households (id, country_id), and dynamic_id to dynamics, with an index on each referencing column; - a unique legacy_simulation_id for simulations migrated from the Public API's v1 database, and a unique (id, country, kind) for the reports' foreign keys; - no UPDATE privilege for the runtime role, since definitions never change. Creating the foreign keys needs the operator's REFERENCES grant on the Public API tables; a test shows the chain fails without it and leaves nothing applied. SimulationStore creates, reads and finds simulations by v1 ID, and create_migrated is idempotent by legacy_simulation_id for the one-time migration (item 2.05). Importing the package now registers every table, so the migration environment always compares the whole schema. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A report is a definition: its baseline simulation, its reform simulation (required for an economy report; a household report may have only a baseline), the year (1900 to 2200) and its options as a JSON object. Composite foreign keys (simulation id, country, kind) into simulation.simulations make both simulations share the report's country and kind; they are ON DELETE RESTRICT, so a simulation a report uses cannot be deleted. A report migrated from the Public API's v1 database keeps its v1 report ID in the unique legacy_report_id. Revision 0002 now creates the reports table too, and revokes UPDATE on both definition tables from the runtime role. ReportStore creates, reads and finds reports by v1 ID; create_migrated is idempotent by legacy_report_id, for the one-time migration (item 2.05). NewReport checks the country, year, options and v1 ID before any write; the database checks them again for any other writer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

NewSimulation and NewReport typed kind as an enum but neither checked nor converted it, so a kind given as plain text (as a mapping of v1 records easily does) passed their checks and then failed with AttributeError when the store read kind.value. Both now convert the kind to its enum member when they are created, and reject an unknown kind with a ValueError that names the field. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The database repeats each check NewSimulation and NewReport make, for any other writer, but only the report year and options checks had a test, so a mistake in another check's expression would have gone unnoticed. New tests insert rows directly as the runtime role and expect each of the other checks to refuse them: the kinds, countries, region code, dataset name, both v1 IDs, and an economy report without a reform. The README's roles table still gave the runtime role UPDATE on every table, and did not say the migrator needs REFERENCES on the Public API tables before revision 0002 deploys; both are corrected. The unused REFERENCED_PUBLIC_API_TABLES constant is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Revision 0002 passed full check constraint names, such as ck_simulations_kind, which the naming convention prefixed a second time, so the database held ck_simulations_ck_simulations_kind while the table definitions name it ck_simulations_kind. alembic check does not compare check constraints, so nothing noticed; a later revision dropping one by its defined name would have failed. The revision now marks the names as final (op.f), and a test compares every constraint name in the migrated database with the names the table definitions give. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anth-volk added this pull request to stack #901 October 22, 2026 14:29

anth-volk force-pushed the v2plan/1b-105-simulations-and-reports branch from e9c7d2e to 9bf0d80 Compare October 26, 2026 12:21

anth-volk mentioned this pull request Oct 27, 2026

Add the one-time migration's provisioning command for migrated reports #944

Closed

2 tasks done

anth-volk force-pushed the v2plan/1b-105-simulations-and-reports branch from 9bf0d80 to 6282776 Compare October 27, 2026 14:38

anth-volk force-pushed the v2plan/1b-105-simulations-and-reports branch from 6282776 to c7404a1 Compare October 27, 2026 21:16

This was referenced Oct 28, 2026

Run and store v2 economy reports in the Simulation Entrypoint #974

Draft

Calculate budget windows on the v2 economy path #982

Draft

anth-volk and others added 17 commits October 30, 2026 00:38

Add a changelog fragment for the simulation schema

874e15d

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anth-volk and others added 5 commits October 30, 2026 00:55

Add a changelog fragment for the simulation and report tables

c747212

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anth-volk force-pushed the v2plan/1b-105-simulations-and-reports branch from c7404a1 to c747212 Compare October 30, 2026 12:50

anth-volk closed this Nov 5, 2026

Navigation Menu

Repository navigation

Add simulation and report definitions to the simulation schema - #900

Add simulation and report definitions to the simulation schema#900
anth-volk wants to merge 22 commits into
v2plan/1b-104-persistence-foundationfrom
v2plan/1b-105-simulations-and-reports

anth-volk commented Oct 22, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

anth-volk commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Same-Repository Draft PR Check

Changelog

Summary

Environment variables

Waits for

Testing

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

anth-volk commented Oct 22, 2026 •

edited

Loading