feat(promotion): gate promotion out of an org on a passing check

[scott-lowe-vapi]

Value

V.A.L.U.E. tier: project — PR 10 of 10 for inline simulation PR checks (TEST-141), the "check before deploy" step in promotion.

Review only the last commit (849c59f). The plan cuts this from main after #63 and #65 merge, and neither has yet. So it's stacked on #64 with #65 merged in (f3f3209 resolves #65 against #59's promote-cmd.ts refactor). Once both land, rebase onto main and retarget.

• Problem: promotion copies staging's reviewed files into production, but nothing checks that staging's agents still behave before they move on. Teams promoting dev → staging → prod need a behaviour gate between orgs, without new infrastructure.
• Who it affects: multi-org gitops users (the promotion pipeline), who get a "check before deploy" step with one line of promotion.yml. Single-org users are unaffected.
• What changes:
  • promotion.yml accepts orgs.<slug>.check: <name> (a slug), naming a vapi-checks.yml check.
  • New src/promotion-gate.ts:
    • Validation before any transition: the check must exist, and its org and runOrg must be the gated org, otherwise the run errors. vapi-checks.yml is required once any org is gated.
    • Plan line: what the gate would run, built offline.
    • Live gate: the check runs live and reduces to the worst target result.
  • src/promote-cmd.ts: in each transition, after the plan is built:
    • no changes skips the gate;
    • plan-only prints check would run <name> in <org> (<n> simulations × <t> targets);
    • --apply runs the check (after the bindings refresh, before promotionPlanApply writes anything). Any non-pass throws Promotion out of <org> blocked: check <name> <outcome> (<run url>).
    • A pass is cached per source org and dropped once a transition applies into that org.
    • promotionCommandRun(args, overrides) now takes Partial<PromotionDeps> (childRun, checkRun).
  • .github/workflows/promotion.yml: timeout-minutes: 90 on the "Reconcile configured promotions" step, not the job, so the if: always() commit step (fixed in fix(promotion): commit the files of transitions that applied when a later one fails #65) still runs after a blocked or slow gate.
  • Docs: promotion.example.yml (a commented check:), a README "Check before promoting" section, and a pointer from "PR Checks".

Evidence of value

The real gate, run live in the owner's test org on the TEST-141 parity squad.

• Setup: a scratch repo whose promotion.yml gates parity on check core, with pipeline parity → parity-prod.
• The run: promote --pipeline release --from parity --to parity-prod --apply.
• The fake: the child runner was faked, so bindings pulls were no-ops and the downstream apply.ts was recorded but not run. No second org was needed or touched.

Variant	Gate run	Result	Downstream apply	resources/parity-prod/
Degraded scheduler prompt	7ed19587: 2 of 3 failed	Promotion out of parity blocked: check core failed (https://dashboard.vapi.ai/simulations/run/7ed19587-…)	none	empty (nothing written)
Fixture as-is	95470670: 3 of 3 passed	promoted	["parity-prod"]	written; 20 applied paths recorded

The test org's resource counts were identical before and after both gate runs.

Tests: npm test goes from 477 (the merge with #65) to 484 passing.

Testing plan

• tests/promotion-gate.test.ts (6 tests, real git fixture, injected childRun / checkRun):
  • a pass applies;
  • failed and incomplete both block with the exact message, with no apply and the target untouched;
  • plan-only prints the line and runs nothing;
  • no changes skips the gate;
  • the three config errors (no vapi-checks.yml, unknown check, check in another org) stop before anything applies;
  • the pass cache: reused for two pipelines out of one org, and re-run after a transition applies into the gated org.
• tests/promotion.test.ts: orgs.<slug>.check is parsed, and a non-slug is rejected.
• Not tested:
  • A real two-org promotion: only one test org was available. The downstream apply was faked, so the blocked case shows nothing written, and the pass case shows the apply was called.
  • A GitHub Actions promotion run with a gate, including the step timeout firing.
• Found while testing (pre-existing, out of scope): promotion's dependency check rejects simulations that reference a stock personality by UUID, with "Referenced managed dependency is missing from source: personalities/a0000000-…". So a gated org's tests need local personality files until that's fixed.

Stacked on #64, with #65 merged in.

Refs TEST-141

🤖 Generated with Claude Code

[scott-lowe-vapi]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat(promotion): gate promotion out of an org on a passing check #66 👈 (View in Graphite)
• feat(check): add the opt-in Vapi checks PR workflow and user docs #64
• feat(check): run checks live and report Vapi Evals statuses #63
• feat(check): fail closed on tools and servers in inline check payloads #62
• feat(check): build inline simulation-run payloads and add a dry-run CLI #61
• feat(check): parse vapi-checks.yml check configuration #60
• refactor: move resource parsing and org connection helpers into config-free modules #59
• fix(sim): report failed, canceled and incomplete runs instead of a false pass #58
• ci: type-check tests/ and send a User-Agent on sim API requests #57
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.