Repository navigation
benchmark(lifecycle): the scripted person signs its approvals with a fixture-only key - #1565
Merged
Merged
Conversation
…fixture-only key Since #1486 a person's approval counts only when a key on the signed approver roster signs it. The soak's scripted person still wrote unsigned answers claiming a cloud sign-in, so the final soak's one approval was refused (auth_method_unverified) and the gate stopped before saving. Each run now gets its own approver: a throwaway key generated under the run directory, a one-entry roster it roots, and a deployment config that pins the root (approvers.root_public_key) and key directory, read only when its bytes match their SHA-256. Approval stores take their pin only from that config, never from OPENADAPT_APPROVER_ROOT. The ledger records the public root and the config digest; the key and passphrase stay in the run directory. A refused answer is logged with its code before it stops the gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr
enabled auto-merge
October 25, 2026 15:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes the lifecycle soak's scripted approvals count the way a person's do: each one is signed by a fixture-only approver key that belongs to the run. It follows OpenAdaptAI/OpenAdapt#1513, which has merged.
Why
Since #1486, a person's approval counts only when a key on the signed approver roster signs it. The harness still wrote unsigned answers that claimed a
cloud_aal2sign-in. In the final soak (#1513,production-2026-10-25-final), the one approval request, on item 26 after the relabel repair, was refused (auth_method_unverified). The gate stopped before saving (gate_error), and the item was entered by hand.What changes
benchmark/lifecycle_proof/approver.py):approver-keys/) with a random passphrase kept beside it, owner-only;deployment.yamlthat pinsapprovers.root_public_keyand the key directory. It is read only after its bytes match the recorded SHA-256 (pin_deployment_bytes, the same check as--config-sha256). The run gate uses the same config.config_only=True), never fromOPENADAPT_APPROVER_ROOT.ScriptedPerson(the proof) andOperatorPerson(the soak) sign every answer with that key. If an answer is refused, the ledger recordsperson_answer_refusedwith its code before the gate stops. Before, the item just ended as a baregate_error.fixture_approverrow with the public root and the config digest only. The key and passphrase never leave the run directory, which is never committed.No real person's key is used, and no product code changes.
Tests
tests/test_lifecycle_fixture_approver.py:auth_method_unverified), and so is an unsigned answer with no sign-in claim;answer_refusalagainst the config-pinned store, and the store reports it approved;ScriptedPersonsigns; with no approver, it logsperson_answer_refusedand the request stays waiting;deployment.yamlis refused;Locally: those tests,
test_lifecycle_benchmark.pyandtest_human_approval_verified.py(31 passed); ruff, ruff format and mypy are clean.Not covered
SelfRepairopens its own repair-approval store without a pin (openadapt_flow/repair/self_repair.py). A self-repair that needs a person's approval would still be refused, failing closed. The final soak never reached that path. Fixing it meansSelfRepairtaking a pinned store, which is product code and out of scope here.🤖 Generated with Claude Code