[abrichr]

Draft. The live Medplum run is pending: it waits for the lane-2 fixture lease.

openadapt shadow runs beside a vendor's own agent, against their own system of record. It is a loopback pass-through HTTP proxy. It checks each write the agent makes (CONFIRMED / REFUTED / UNCERTAIN) and writes report.html and report.json. It never blocks, edits or resends a write. The only delay is the read before the write, which is bounded by hold_ms and measured per write.

It has no verifier of its own:

• --mode fhir calls ensure_record.verify_ensure. The values come from the request body, read at each contract field's path.
• --mode sql calls verifier.begin_write/verify_write, with the whole-database frame and no ledger.

Design and limits: packages/flow/docs/SHADOW_MODE.md.

The verifier change: verify_ensure(..., declared=...) leaves out the resources of other writes the caller saw the agent make and judges each of them on its own. Every other change in the window still counts.

Tests

• tests/test_shadow.py: SQL mode end to end on SQLite behind an HTTP app. It covers a correct write, a wrong value, a duplicate, a side write, an unreadable store, a slow baseline, no route, a route that doesn't fit, a shadow failure, and the bytes passing through unchanged.
• tests/test_shadow_fhir.py: grouping writes into episodes, what each check is told, refused writes, writes made during a check, and a non-ASCII target.

Review

Adversarial review, round 1: one blocker (a malformed request could be dropped without a trace) and nine should-fix items, all fixed in 4b-round-1. The summary will be posted as a comment.

[abrichr]

Adversarial review, recorded for the verifier path (each round by an independent reviewer that was given the code, not my conclusions).

Round 1 (proxy, episodes, verdict mapping). Blocker: a malformed request (a raw non-ASCII target, a bad chunk size) raised outside any try. The write was dropped and never logged, and its episode showed as PENDING, which the report didn't render. Should-fix items:

• a refused write was reported CONFIRMED;
• declared excluded refs of episodes already checked;
• a provisional SQL CONFIRMED was dropped;
• repeated headers were merged and the reason phrase was replaced;
• upstream failures became a made-up 502;
• latency was counted twice;
• found_existing was reported REFUTED;
• a check started while another write was in flight;
• a route key missing from the body gave REFUTED.

All fixed, with tests (test_shadow_fhir.py was added).

Round 2 (the recheck, report counts, CLI, evidence). No code blockers, and every number in SHADOW_RESULTS.md matched the JSON. One finding would have been a blocker while the doc still claimed it: the FHIR recheck didn't compare the confirmed record itself, so a record moved to another patient after the check came back CONFIRMED. The other should-fix items:

• the SQL recheck refuted on the agent's own later writes when they were unrouted or under another contract;
• writes with unknown records (lost replies) inside a production recheck window;
• a write during a recheck left a REFUTED standing;
• a late UNCERTAIN or a pending recheck was invisible;
• the docs left out the recheck settings each run was measured with;
• the kind labels mislabelled some reasons.

All fixed, with tests. The window semantics are now stated in SHADOW_MODE.md.

Open after round 2, documented as limits:

• an unexplained write by the agent's credential refutes every CONFIRMED whose recheck window holds it;
• no run measured the 300 s default across several tasks;
• each check sleeps 1.1 s (FHIR) before reading the server's clock.

[abrichr]

Round 3 (an independent check of round 2's fixes) found two blockers in the FHIR recheck:

• A value added to a closed contract's record after the check came back CONFIRMED.
• A record moved to another patient was downgraded to UNCERTAIN by an unrelated later lost reply.

It also found these smaller items:

• an unreadable change feed or a deleted record (404/410) outranked a refutation from reading the record itself;
• unknown later writes were counted per episode rather than per write;
• pending rechecks were shown as plain CONFIRMED;
• a duplicate label blamed the agent;
• an SQL recheck that couldn't be scheduled left no trace.

All are fixed in c2bc10b, with a test for each. The reviewer's own probes now pass.

Round 4 (only c2bc10b): approve, no blockers. One non-blocking finding is left for a follow-up, since only blockers get fixed after approval. A later 2xx write whose record the shadow can't name no longer downgrades a recheck REFUTED to UNCERTAIN: a Prefer: return=minimal create with no Location, a transaction Bundle, or a conditional PUT. It can only produce a false REFUTED, never a false CONFIRMED.