Shadow mode: check an agent's writes against the system of record, never blocking one

CAPABILITIES.md

@@ -275,6 +275,7 @@ improvising or retrying when something is off.
 | Teach over MCP: an agent does a new task once through OpenAdapt; the run is read back, compiled and admitted (a person confirms the record check; the agent admits only under a person-signed policy), and becomes a tool it calls with no model calls. Web, and Windows, macOS, Linux, RDP and Citrix with a person approving each save | On main ([#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)) | Synthetic (MockMed over MCP, and through the real RDP and Citrix backends) | `openadapt-agent serve --teach`, [packages/agent](packages/agent/README.md#teach-a-new-workflow-from-your-agent) |
 | Turn a read-only browser task into a tool for the coding agent you already use: `openadapt setup` registers the MCP server and a skill in Claude Code, Cursor, Codex and Gemini CLI (idempotent, `--dry-run`, `--undo`); the agent does the task once through `teach_start`/`teach_act`/`teach_finish`, the task must end in a download, the file is checked (columns, a column between two inputs or equal to one, fewest rows), the compiled steps are replayed once with no model, and the agent gets a `taught_<name>` tool that downloads a checked file with new inputs. The browser sends no POST, PUT, PATCH or DELETE and stays on one site; a compiled run stops when a control is missing, matches more than one element, or is relabelled | On main ([#1502](https://github.com/OpenAdaptAI/OpenAdapt/pull/1502)) | Real headless Chromium on the local example portal (scripted MCP client, in CI); a real Claude Code agent taught the export from the guide's prompt and called the tool with new dates (2026-10-24, one run) | `openadapt setup`, `openadapt-agent serve --teach-browser`, [FIRST_TOOL.md](packages/flow/docs/FIRST_TOOL.md) |
 | Verify any agent's own writes: `openadapt verify` reads the system of record before and after a write, against a contract the operator owns (REST, FHIR, SQL, GraphQL), and returns a verified receipt (`openadapt.effect-receipt/v1`) or a refusal (`REFUTED`, `UNCERTAIN`), never an unverified success. Python SDK (`openadapt.verify`, which is `openadapt_flow.verifier`: `begin_write`, `verify_write`, `verified_write`) wraps a write in any agent framework; the MCP server (`openadapt verify serve`) offers `verify_begin`, `verify_check` and `verify_recheck` to any MCP client | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589)) | Synthetic (MockMed's fault server over real HTTP: correct write, wrong record, not saved, saved twice, unreadable store) | `openadapt verify`, [VERIFY_WRITES.md](packages/flow/docs/VERIFY_WRITES.md) |
+| See what your own agent gets wrong, with no risk: `openadapt shadow` is a loopback pass-through proxy beside the agent. It checks each write against the system of record (FHIR through `ensure_record`; SQL through the verifier's whole-database frame), checks each `CONFIRMED` again later, and writes `report.html` and `report.json`. It never blocks, changes or resends a write; the only delay is the read before a write (at most `hold_ms`) | [#1678](https://github.com/OpenAdaptAI/OpenAdapt/pull/1678) | Real app (Medplum 5.2.8, 37 cases through a fault proxy: scripted agent 18 of 18 harms caught, 0 correct writes refuted; Claude Sonnet 5.5 9 of 9 caught, 4 correct but corrected writes refuted; 0 writes by the shadow; 15 to 25 ms median added per write; setup 76.5 s machine time, 3 decisions); real app, SQL mode (Dolibarr 21 on MariaDB 11.4, 13 cases: 8 of 8 harms caught, 5 of 5 correct writes confirmed, 292 ms median added); synthetic (SQLite) | `openadapt shadow`, [SHADOW_MODE.md](packages/flow/docs/SHADOW_MODE.md), [SHADOW_RESULTS.md](packages/flow/benchmark/fhir_medplum/public/SHADOW_RESULTS.md) |
 | Skill cache for framework agents: an agent in any framework does a browser task through compile-agent's harness as plain tools (`BrowserTools`); when the system of record shows exactly the task's record and the operator confirms the checks, the run is canaried, admitted and emitted as an MCP server. A repeat of the same task (instruction, record, app, contract, environment) calls that tool instead of the agent: no model calls, the program's checks and this task's record check on every run, eviction after a failed run | On main | Synthetic (MockMed, LangGraph on `claude-sonnet-5-5`, 3 trials, repeats on the same record: run 1 6 model calls and $0.022; runs 2 to 10 27/27 VERIFIED, 0 model calls, 7.8 s median); Trac 1.6: run 1 verified, not admitted | `openadapt_flow.integrations.agents.skills`, [benchmark/skill_cache](packages/flow/benchmark/skill_cache/README.md) |
 | Verified write tools for agent frameworks: `verified(tool, contract=...)` wraps a Claude Agent SDK, OpenAI Agents SDK, LangGraph or CrewAI write tool in one line. The contract and any values the operator pins come from the operator, never the agent's arguments; the store is read before and after; the agent gets a receipt or a refusal, and a refused write is held instead of retried | On main | Synthetic (each framework's own agent loop with a scripted model, writing through the example app's order form over real HTTP: verified save, a save the app drops, a save it keeps twice); one real-model run per framework, opt-in | `openadapt_flow.integrations.agents`, extras `claude-agent-sdk`, `openai-agents`, `langgraph` (CrewAI: install `crewai` beside it); [AGENT_FRAMEWORKS.md](packages/flow/docs/AGENT_FRAMEWORKS.md) |
 | Teach a browser write as a verified tool: with write contracts (`serve --teach-browser --contracts DIR`), the agent teaches a save once; run 1 is verified in the system of record before the `taught_<name>` tool exists, and every call makes the write and returns a verified receipt or a refusal. Sign-in values come from the server's environment and never reach the agent or the program. Controls in named iframes are found through their frame | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589)) | Synthetic (the example app's sign-in and order form in headless Chromium over MCP: verified save, a save the app drops, a save it keeps twice) | `openadapt-agent serve --teach-browser --contracts`, [VERIFY_WRITES.md](packages/flow/docs/VERIFY_WRITES.md#teach-a-write-as-a-tool) |
@@ -423,6 +424,7 @@ A test fails when a command has no file or this list is out of date.
 **Guard an agent you already run:**
 
 - `guard`: Attaches to the agent's own Chromium over CDP, holds each save until the right record is on screen, reads it back, and hands a verified run to the teach dir.
+- `shadow`: Shadow a FHIR or HTTP agent: check each write against the system of record without blocking it, and report.
 
 <!-- END generated by scripts/build_command_map.py -->
 

llms-full.txt

@@ -672,6 +672,7 @@ improvising or retrying when something is off.
 | Teach over MCP: an agent does a new task once through OpenAdapt; the run is read back, compiled and admitted (a person confirms the record check; the agent admits only under a person-signed policy), and becomes a tool it calls with no model calls. Web, and Windows, macOS, Linux, RDP and Citrix with a person approving each save | On main ([#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)) | Synthetic (MockMed over MCP, and through the real RDP and Citrix backends) | `openadapt-agent serve --teach`, [packages/agent](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/agent/README.md#teach-a-new-workflow-from-your-agent) |
 | Turn a read-only browser task into a tool for the coding agent you already use: `openadapt setup` registers the MCP server and a skill in Claude Code, Cursor, Codex and Gemini CLI (idempotent, `--dry-run`, `--undo`); the agent does the task once through `teach_start`/`teach_act`/`teach_finish`, the task must end in a download, the file is checked (columns, a column between two inputs or equal to one, fewest rows), the compiled steps are replayed once with no model, and the agent gets a `taught_<name>` tool that downloads a checked file with new inputs. The browser sends no POST, PUT, PATCH or DELETE and stays on one site; a compiled run stops when a control is missing, matches more than one element, or is relabelled | On main ([#1502](https://github.com/OpenAdaptAI/OpenAdapt/pull/1502)) | Real headless Chromium on the local example portal (scripted MCP client, in CI); a real Claude Code agent taught the export from the guide's prompt and called the tool with new dates (2026-10-24, one run) | `openadapt setup`, `openadapt-agent serve --teach-browser`, [FIRST_TOOL.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/FIRST_TOOL.md) |
 | Verify any agent's own writes: `openadapt verify` reads the system of record before and after a write, against a contract the operator owns (REST, FHIR, SQL, GraphQL), and returns a verified receipt (`openadapt.effect-receipt/v1`) or a refusal (`REFUTED`, `UNCERTAIN`), never an unverified success. Python SDK (`openadapt.verify`, which is `openadapt_flow.verifier`: `begin_write`, `verify_write`, `verified_write`) wraps a write in any agent framework; the MCP server (`openadapt verify serve`) offers `verify_begin`, `verify_check` and `verify_recheck` to any MCP client | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589)) | Synthetic (MockMed's fault server over real HTTP: correct write, wrong record, not saved, saved twice, unreadable store) | `openadapt verify`, [VERIFY_WRITES.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/VERIFY_WRITES.md) |
+| See what your own agent gets wrong, with no risk: `openadapt shadow` is a loopback pass-through proxy beside the agent. It checks each write against the system of record (FHIR through `ensure_record`; SQL through the verifier's whole-database frame), checks each `CONFIRMED` again later, and writes `report.html` and `report.json`. It never blocks, changes or resends a write; the only delay is the read before a write (at most `hold_ms`) | [#1678](https://github.com/OpenAdaptAI/OpenAdapt/pull/1678) | Real app (Medplum 5.2.8, 37 cases through a fault proxy: scripted agent 18 of 18 harms caught, 0 correct writes refuted; Claude Sonnet 5.5 9 of 9 caught, 4 correct but corrected writes refuted; 0 writes by the shadow; 15 to 25 ms median added per write; setup 76.5 s machine time, 3 decisions); real app, SQL mode (Dolibarr 21 on MariaDB 11.4, 13 cases: 8 of 8 harms caught, 5 of 5 correct writes confirmed, 292 ms median added); synthetic (SQLite) | `openadapt shadow`, [SHADOW_MODE.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/SHADOW_MODE.md), [SHADOW_RESULTS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/benchmark/fhir_medplum/public/SHADOW_RESULTS.md) |
 | Skill cache for framework agents: an agent in any framework does a browser task through compile-agent's harness as plain tools (`BrowserTools`); when the system of record shows exactly the task's record and the operator confirms the checks, the run is canaried, admitted and emitted as an MCP server. A repeat of the same task (instruction, record, app, contract, environment) calls that tool instead of the agent: no model calls, the program's checks and this task's record check on every run, eviction after a failed run | On main | Synthetic (MockMed, LangGraph on `claude-sonnet-5-5`, 3 trials, repeats on the same record: run 1 6 model calls and $0.022; runs 2 to 10 27/27 VERIFIED, 0 model calls, 7.8 s median); Trac 1.6: run 1 verified, not admitted | `openadapt_flow.integrations.agents.skills`, [benchmark/skill_cache](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/benchmark/skill_cache/README.md) |
 | Verified write tools for agent frameworks: `verified(tool, contract=...)` wraps a Claude Agent SDK, OpenAI Agents SDK, LangGraph or CrewAI write tool in one line. The contract and any values the operator pins come from the operator, never the agent's arguments; the store is read before and after; the agent gets a receipt or a refusal, and a refused write is held instead of retried | On main | Synthetic (each framework's own agent loop with a scripted model, writing through the example app's order form over real HTTP: verified save, a save the app drops, a save it keeps twice); one real-model run per framework, opt-in | `openadapt_flow.integrations.agents`, extras `claude-agent-sdk`, `openai-agents`, `langgraph` (CrewAI: install `crewai` beside it); [AGENT_FRAMEWORKS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/AGENT_FRAMEWORKS.md) |
 | Teach a browser write as a verified tool: with write contracts (`serve --teach-browser --contracts DIR`), the agent teaches a save once; run 1 is verified in the system of record before the `taught_<name>` tool exists, and every call makes the write and returns a verified receipt or a refusal. Sign-in values come from the server's environment and never reach the agent or the program. Controls in named iframes are found through their frame | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589)) | Synthetic (the example app's sign-in and order form in headless Chromium over MCP: verified save, a save the app drops, a save it keeps twice) | `openadapt-agent serve --teach-browser --contracts`, [VERIFY_WRITES.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/VERIFY_WRITES.md#teach-a-write-as-a-tool) |
@@ -820,6 +821,7 @@ A test fails when a command has no file or this list is out of date.
 **Guard an agent you already run:**
 
 - `guard`: Attaches to the agent's own Chromium over CDP, holds each save until the right record is on screen, reads it back, and hands a verified run to the teach dir.
+- `shadow`: Shadow a FHIR or HTTP agent: check each write against the system of record without blocking it, and report.
 
 <!-- END generated by scripts/build_command_map.py -->
 

openadapt/cli.py

@@ -1435,6 +1435,27 @@ def verify_ledger_release(
     click.echo(f"{key}  released by {by} (found {found})")
 
 
+@main.command(
+    "shadow",
+    context_settings={"ignore_unknown_options": True, "allow_extra_args": True},
+)
+@click.pass_context
+def shadow(command_ctx: click.Context) -> None:
+    """Check each write your agent makes against the system of record.
+
+    Never blocks or changes a write; writes report.html and report.json.
+    Every argument passes to `openadapt flow shadow`; its options are listed
+    by `openadapt flow shadow --help`.
+    """
+    if "shadow" not in _flow_verbs():
+        click.echo(
+            "Error: the installed openadapt-flow has no shadow command.", err=True
+        )
+        click.echo("Upgrade with: pip install --upgrade openadapt", err=True)
+        sys.exit(2)
+    _run_flow(["shadow", *command_ctx.args])
+
+
 @main.command(
     "agent",
     context_settings={

packages/flow/CHANGELOG.md

@@ -19,6 +19,8 @@
 
 ### Features
 
+- Shadow mode: `openadapt shadow` runs beside an agent you already have, as a loopback pass-through proxy, and checks each write it makes against the system of record (`CONFIRMED`, `REFUTED`, `UNCERTAIN`). It never blocks, changes or resends a write; the only delay is the read before a write, at most `hold_ms`, measured per write. FHIR writes are checked by `ensure_record.verify_ensure` and SQL writes by `verifier.begin_write`/`verify_write` with the whole-database frame. Each `CONFIRMED` is checked again later (`recheck_s`, or the contract's `recheck_after_s`), and a late `REFUTED` gets its own line in `report.html` and `report.json`. `verify_ensure` takes `declared`: resources of other writes the caller saw, judged on their own ([#1678](https://github.com/OpenAdaptAI/OpenAdapt/pull/1678)).
+
 - The whole-database frame (`frame.database`) and its recheck now work on PostgreSQL (`dialect: postgresql`). The snapshot is one read-only repeatable-read transaction over every schema but the system ones. Every column type is read as text by a declared rule, in a session pinned to UTC and exact float output: `numeric` at full precision (with its stored scale when it has no declared one), floats exactly, `bytea` as hex, `json` as stored and `jsonb` canonical, arrays, ranges, enums, domains, `uuid`, network, time, `interval`, `money`, bit, `xml`, text-search and geometric types. A table without a primary key is keyed by the whole row, with duplicates counted. A column type without a rule, a table the account can't read in full (no `SELECT`, row-level security), a foreign table and an unreadable sequence are refused unless the allow-list skips them with a reason. A sequence a column owns is that table's counter, and a counter set back counts as a change; any other sequence is compared as a one-row table. Large objects are read by the SHA-256 of their content. `frame.database.schemas` means the same as on MySQL (each must be visible, and every visible schema is read); a schema is left out only by an `allow_tables` entry `<schema>.*` with `mode: skip` and a reason.
 
 - A lost update now stops the run. `frame.versions` names a read of the target's versions (HAPI's `hfj_res_ver`, a version column, or an app's action-history table) and, where the app records one, the author of each version. The verifier reads them before and after the write; a version of a pre-existing target by anyone other than the agent, or more than one version step when no author is read, refutes the frame. A versions read that fails, a target with no version, or a version with no author leaves the frame not checked. The receipt holds counts and a problem code (`evidence.frame.versions`). Contracts without `frame.versions` keep their digests.