Skip to content

Teach authorization signed with approver keys, bound to every teach path - #1436

Merged
abrichr merged 9 commits into
mainfrom
claude/teach-authorization
Nov 4, 2026
Merged

abrichr merged 9 commits into
mainfrom
claude/teach-authorization

Conversation

@abrichr

@abrichr abrichr commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

A person signs what an agent may do when it teaches or repairs a program: a teach authorization, openadapt.teach-authorization/v1. The signature uses the person's approver key on the signed approver roster from #1500, not a GitHub SSH key. Every teach path is either bound to it or refused.

What it does

  • Statement (packages/flow/openadapt_flow/teach_authorization.py). It names:
    • the task, purposes (teach, repair), origins and inputs (spelled out, or as a SHA-256);
    • the one write: record kind, identity, fields and row count;
    • forbidden actions and controls;
    • the authorizer (a roster principal) and their approver public key;
    • an expiry, 24 hours by default.
  • Signature. A HumanAttestation over a separate domain plus the canonical statement. It is checked with the approver verifier itself (human_attestation.refusal → verified_approvers); this PR adds no second signature scheme.
  • Refusals (each fails closed):
    • a key not on the roster (teach_auth_unknown_signer);
    • a key removed since signing, which refuses at the next teach, admission or repair;
    • the wrong key for the named principal;
    • a changed statement (teach_auth_bad_signature);
    • an unpinned or different root, or a roster rolled back (teach_auth_identity_unverified);
    • another task or purpose;
    • a statement signed by the recording agent.
  • Admission gate (jit/lifecycle.teach_authorization_gate):
    • Every JIT draft needs a teach authorization, in all three admission modes.
    • The gate reads the signed recorder block, not editable candidate fields.
    • The draft's kept statement must match the recorded digest and verify against the roster as it is now.
    • The capture must have run under the scope the statement binds, with a verified read-back.
    • The statement is copied into the admitted bundle.
  • CLI. openadapt-flow teach-authorization create|sign|verify, plus --teach-authorization on compile-agent and on openadapt-agent serve --teach, and --approver-root on jit admit.

Entry points

Entry point State
JIT capture (compile_agent(teach_authorization=...)) Bound: the scope comes from the statement, and a teach_scope may only narrow it (the signed write adapter is kept)
compile-agent / FirstRunCompiler / compile_first_run / SkillCache Bound: run 1's scope is the statement's alone
MCP teach (serve --teach, teach_act) Bound: the task id is the authorized one, and the agent's inputs must match the statement. The server takes the roster pin only from the deployment config (--config)
Self-repair of an agent-taught program (admit_repair) Bound: the prior admission's authorization must cover repair and verify now
Agent-demonstrated fix (teach_repair) Bound: the desk verifies a statement for repair of that workflow before the click. RepairStore.approve checks it again and refuses without one
Airbag handoff Refused at admission (teach_auth_not_recorded); the guarded run can't carry a signature yet
Browser-teach writes (serve --teach-browser with a contract) Refused: no write tool is taught, and existing write tools don't run. The read-only lane is unchanged
teach_run (#1542) Not on main, so not handled here

Registering a key without a Secure Enclave

A person uses an encrypted software key file: openadapt-flow jit keygen, then jit approver add (or jit approver init to start a roster). They sign with teach-authorization sign --approver-key KEY.pem at an interactive terminal. On Windows and Linux, openadapt-desktop approver-key setup also creates such a key. Desktop's Touch ID key answers item approvals only and doesn't sign teach authorizations. See packages/flow/docs/TEACH_AUTHORIZATION.md.

Limits and breaking changes

  • Breaking for drafts and tools made before this change:
    • JIT drafts waiting for admission are refused.
    • Admitted JIT programs can't self-repair until they are taught again with an authorization.
    • Browser-teach write tools don't run.
    • The benchmarks and examples/jit/run_mockmed.py now sign a throwaway authorization per run (demo_authority) in place of the person.
    • Upgrade notes are in the doc and the changelog.
  • Unsigned provenance. The agent-demonstrated fix is recognized by its provenance, which the repair store doesn't sign. A process that can write the store can strip it.
  • Root pin. For commands a person runs, the root comes from --approver-root or the environment. Self-repair's carry and repair approval use the environment only. A process controlling both the environment and the key directory can pin its own roster, the same limit the approval docs state. For serve --teach, also pin approvers.head in the config.
  • Statement reuse. A statement can be used for any number of teach runs of its task until it expires; its nonce isn't spent.
  • Forbidden controls. They are matched by accessible label on web pages, and a click whose label probe fails is refused. Keyboard activation isn't checked.
  • Tests.
    • Unit tests use real roster keys, real signatures, real recorder signatures and real admission.
    • The hand-built drafts in the admission tests write the bound teach_scope block themselves.
    • The MockMed browser end-to-end tests were updated but not run locally; the host load was over 35. They run on main and nightly.

Review

An independent adversarial review is recorded on this PR. Round 1 requested changes; they were addressed in d56fd00. Round 2 approved with nits on condition that the benchmarks be fixed; that and two of the nits were addressed in 3f0db00. The rest are documented as limits.

abrichr and others added 4 commits October 20, 2026 12:40
Adds openadapt.teach-authorization/v1 (person-signed scope: task, inputs,
origins, permitted writes, forbidden actions), binds it on the JIT capture
path, records its digest in the signed recorder block and admission
manifest, and requires it at admission for agent-taught drafts. Other
authoring paths, the live-model harness and full test runs are not done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fuse every teach path

The signed teach authorization (openadapt.teach-authorization/v1) now uses the
approver keys and the approver verifier instead of GitHub SSH keys. Admission of
every JIT draft requires one, keyed on the signed recorder block. compile-agent,
MCP teach, the skill cache, browser-teach writes, agent-demonstrated fixes and
self-repair of agent-taught programs are bound; the airbag handoff's drafts are
refused. Adds openadapt-flow teach-authorization create/sign/verify.
@abrichr

abrichr commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Independent adversarial review (round 1), head 4f7ed30

A fresh reviewer got only the diff and the goal. It read the code and didn't run tests.

Verdict: request changes.

The core verifier holds up. teach_authorization.verify goes through human_attestation.refusal, which checks an Ed25519 signature over DOMAIN + canonical(statement). It refuses an unknown key, a removed key, a principal whose key doesn't match, and a root that isn't pinned. Duplicate JSON keys are refused, and the JIT admission gate reads the signed recorder block. The reviewer found no way for an agent without write access to the key directory to get a JIT draft admitted without a person's signature.

Findings:

  1. Major: existing flows break. Benchmarks (silent_saves, scoreboard, compile_agent, code_as_action, skill_cache) and examples/jit/run_mockmed.py don't pass an authorization, so their drafts are refused. Write tools taught before this change stop running. JIT programs admitted before this change can't self-repair. There is no changelog or migration note.
  2. Major: the browser-teach write lane doesn't bind the signed scope. It checks only the signature, the origin and the record kind. Identity, fields, inputs, row count, forbidden actions and the task are not enforced, and program.json is unsigned.
  3. Major: repair approval keys on unsigned provenance (repaired_by) and doesn't check the task. Any kept repair statement satisfies approval for any workflow.
  4. Major: root pinning ignores the deployment config and the config-only rule for agent-started processes, and passes no at_least heads. jit admit has no --approver-root.
  5. Major (docs vs code): Desktop can't sign a teach authorization. Its stdio signer accepts only item-approval answers, and the CLI's stdio branch skips the summary and the confirmation.
  6. Minor: narrowed() can replace the signed write adapter with a looser one passed by the caller.
  7. Minor: the forbidden-control check fails open when the label probe returns nothing. It is an exact label match, keyboard activation isn't checked, and the tests use a fake page.
  8. Minor: the attestation method isn't signed, so the claim that the portal key can't sign is overstated.
  9. Minor: a statement can be replayed, since nothing checks or consumes the nonce, and by default it never expires.
  10. Minor: RepairStore checks against environment trust, not the desk's --key-dir and --approver-root.
  11. Nit: the admission task check compares against an unsigned candidate field.
  12. Nit: the admission tests hand-write the bound teach_scope. The fixture ships in the product package. SkillCache needs the operator to sign a hashed task id.

The reviewer checked and found sound: hashed vs signed bytes, swapping the statement after verification, revocation and rollback at admission, the agent signing its own statement, and the airbag handoff refusal.

Refuse browser-teach writes instead of half-binding them. Pin the roster from
the deployment config for servers an agent runs; pass recorded heads; add
--approver-root to jit admit. Check the task at repair approval and at
admission from the signed recorder block. Keep the signed write adapter in
narrowed scopes. Refuse a click whose label probe fails when controls are
forbidden. Drop the Desktop stdio signing path. Default expiry 24 hours.
Changelog with the breaking changes and upgrade notes.
@abrichr abrichr changed the title WIP: signed teach authorization bound to the teach scope Teach authorization signed with approver keys, bound to every teach path Nov 3, 2026
Benchmarks and the JIT example sign a throwaway authorization per run
(demo_authority) so their drafts still admit. admission_status uses the same
trust as admit. create() defaults to a 24-hour expiry, at most 90 days.
Document the remaining trust limits.
@abrichr

abrichr commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Independent adversarial re-review (round 2), commit d56fd00

The round-1 reviewer read the new commit and the full diff. It didn't run tests.

Verdict: approve with nits. Item 1 below must be fixed before merge.

Round-1 findings, as the reviewer checked them:

New findings:

  1. Major: the benchmarks and examples/jit/run_mockmed.py still produce drafts that admission refuses.
  2. Minor: the self-repair carry and repair approval still take their trust from the environment only.
  3. Minor: the forbidden-control probe runs in the page's world. A missing position is allowed, and keyboard activation isn't checked.
  4. Minor: admission_status doesn't use the same trust as admit.
  5. Nit: dead write-lane code remains, and the schema still advertises contract.
  6. Nit: ta.create() allows a statement with no expiry.
  7. Nit: serve --teach falls back to the environment's key directory when the config has no approvers.key_dir.

Addressed in 3f0db00:

  • 1: each benchmark and the example sign a throwaway authorization per run (teach_authorization_fixture.demo_authority).
  • 4: admission_status takes trust.
  • 6: create() defaults to a 24-hour expiry and caps it at 90 days.

Documented as limits in docs/TEACH_AUTHORIZATION.md instead of fixed: 2, 3 and 7. Not done: 5.

@abrichr
abrichr marked this pull request as ready for review November 3, 2026 19:48
teach_run's scripts act through the same authorized run 1 as teach_act, so
the scope bound from the statement checks every action and the save. Tests
with a real browser, roster and admission gate: a teach_run draft without a
statement is refused at admission, one under a statement passes the gate, and
a script that types a value the statement doesn't allow saves nothing.
@abrichr

abrichr commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Focused review: merging #1542 (teach_run) into this branch, head 6dae6c0

A fresh reviewer looked at only this change: merge c34c982 and commit 6dae6c0. It read the code and did not run tests.

Verdict: approve with nits. The reviewer found no way around the gate.

Merge. #1542 changes no lifecycle, admission or authority code. teach_authorization_gate is still called from _evidence_gates, and every admission mode goes through that.

teach_run is bound to the authorization.

  • It shares FirstRunCompiler and DrivenRun with teach_act, and it gets the statement and its task id. When a statement is given, it replaces the agent-declared scope.
  • Every code-action command (click, fill, select, press, scroll) passes through harness.act and then CaptureComputer.execute. That path checks guard.check_ops before the action and guard.after_action after it.
  • press refuses modifier chords, so paste isn't available.
  • There is no goto, eval or raw-request command, and sh is sandboxed with no network.
  • A violation fails closed: ScopeViolation is re-raised, and Recording._finish refuses any run with violations and still does the read-back.
  • The held-save approval can't widen the scope.
  • The new default code agent in compile-agent uses the same harness. Without an authorization, admission refuses its draft (teach_auth_not_recorded).

Tests. They use real components: a real browser on MockMed, a real signed roster, the real TeachBridge and the real gate.

Nits, left for a follow-up because this PR is frozen:

  1. The negative test accepts refused or error. It doesn't assert typed_value_not_declared or that no draft is made.
  2. No teach_run test reaches the save under a statement: neither a wrong-record save nor the held save approved by a person.
  3. The test name says "admissible", but it calls only teach_authorization_gate.
  4. This was already true before this change, and applies to teach_act too:
    • Forbidden controls are checked on clicks only, so Tab then Enter gets past the check. Network, write-scope and read-back checks still apply.
    • A native <select> is filled by typing the option's text, so under a statement it needs that text declared.

abrichr added a commit that referenced this pull request Nov 3, 2026
…test nits; merge the #1436 head

Merge claude/teach-authorization (teach_run binding) into this branch, and:
- run 1's adapter check pins the record's key wherever the statement lists it;
- the program-signing key directory must come from the config's approvers
  section, the key is never created on demand, the program carries a signature
  domain, and running tools needs only the config's roster pin;
- the verified copy of the statement is the one kept, and teaching again
  replaces it;
- coverage maps literal matches and refuses effects it can't map, the row
  count must match, expiry is required (at most 90 days) at teach and run time,
  and run 1 counts its writes and watches the forbidden actions;
- press in run 1 takes no character keys;
- forbidden controls are read from Chromium's accessibility tree, and every
  key is checked against the control that has focus when it arrives;
- the teach_run tests assert the refusal code, reach the save under a
  statement (wrong record, and a person-approved save), and no longer say
  admissible.
- test_agent_guard_e2e: the handoff's draft is refused at admission by an
  AdmissionError, not a returned detail; assert the raise.
- docs/phi_at_rest.md lists the new teach-authorization/* bundle entry
  (test_sealed_at_rest checks every inventory entry is documented).
- The two browser-teach write e2e files assert the refusal against the real
  example app instead of skipping: flow-critical fails on any skipped test.
abrichr added a commit that referenced this pull request Nov 4, 2026
The two browser-write e2e files keep this branch's full versions, which teach
and run writes under a signed statement.
@abrichr
abrichr merged commit 7291dc2 into main Nov 4, 2026
47 checks passed
@abrichr
abrichr deleted the claude/teach-authorization branch November 4, 2026 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant