[abrichr]

Checks an agent's ensure_patient_registered write on Medplum through the FHIR read path only (searches, AuditEvent, _history), never the database. The verifier, openadapt_flow.ensure_record, gives one of three outcomes: found_existing with the id, created_verified with the id, or needs_review with the reasons. The agent's own report doesn't count.

What's here

• openadapt_flow/ensure_record.py and 43 tests: the openadapt.ensure-record/v1 contract, a reader that fails closed on any incomplete read, and the judgment.
  • It closes the change window at a server instant, and reads the audit trail until two reads agree, because Medplum writes AuditEvents up to 0.1 s after the save.
  • A created record must hold only what the task gave, plus allowed extras such as active=true.
  • A found record counts only when the verifier's own read of it shows up in the audit trail, since an empty AuditEvent search can also mean the server isn't logging.
• benchmark/fhir_medplum/readpath/ensure_patient.yaml: the contract. generate-parts: the app-specific parts of a workflow, generated and verified #1520's generate-parts (generator 13e2cbdb5, run from its branch) wrote 14 of its 30 lines from three demonstrations of Medplum's new-patient form; generated/ holds its output. A person wrote or corrected 16 lines, each marked HAND.
• probe.py and probe.json: what Medplum 5.2.8's read path shows, measured live.
• RESULTS.md, MINUTES.md, and docs/ENSURE_RECORD.md (the coverage limits).
• Fixture fixes:
  • users and clients now live in a project of their own, not Medplum's seeded Super Admin project;
  • sign-in rate limits are handled;
  • restrict_agent makes the audit trail read-only to the agent;
  • Redis is pinned to 7.2 (BSD; 7.4 and later are RSALv2/SSPL).

Results

Separate agents wrote each fault suite from one brief, without seeing the verifier, and froze it by sha256. The suites and per-case results are private.

Round	Suite	Verifier	Right	Fault passed as verified
1, blind	5c1742e8 (18 cases)	705900415	17	1
2, blind	803b34c6 (20 cases)	2efc74000	17	0
3, blind	153d7899 (20 cases)	5ce6e6b26	19	0

Not blind, for reference: rounds 1 and 2 rerun at 5ce6e6b26 scored 18 of 18 and 20 of 20.

• Round 1's miss: a state stored as "Massachusetts" passed, because the Massachusetts-to-MA map was also applied to the stored value. Fixed.

• Round 2's three misses: clean records carrying active: true were sent to review.

• Round 3's miss: a clean record carrying optional use codes was sent to review. The contract was left as it was, so the number stays blind.

Across rounds 2 and 3, no fault was passed as verified: 0 of 31. Of the 9 clean runs, 5 were judged right; the other 4 were sent to review for an element the task didn't give.

Coverage: name the read path

This read path sees the creates, updates and deletes made through the FHIR API that the server logs as AuditEvents, with who made each. On Medplum 5.2.8 that includes batches, transactions and GraphQL mutations (probed). It doesn't see:

• writes that bypass FHIR;
• data FHIR doesn't expose;
• app bookkeeping, which the OpenEMR whole-database frame reads.

So a coverage claim reads "no other write through the FHIR API, as logged by AuditEvent". The verifier's evidence also depends on how the deployment is set up. Medplum lets a plain member create, change and delete AuditEvents without logging it, so the agent needs an access policy that makes AuditEvent read-only (probed: then refused with 403). It also needs saveAuditEvents on, and must not be an administrator. The suite runner checks the policy and the administrator flags before the first case.

Minutes, against OpenEMR

Both columns are agent wall-clock, not a person's hours: an agent did the work both times, and OpenEMR has no hours ledger.

	OpenEMR (whole database)	Medplum (FHIR read path)
App-specific lines written by hand	about 234 to 350	16 contract lines and one access policy
App-specific contract work	about 6.7 h, plus 3 days of review fixes	about 43 min (8 on the contract, 35 on the read path)
Blind rounds	4, the last all right	3, none all right; no fault passed in the last two

Adversarial review

A separate agent reviewed the branch at cb849fc56 and demonstrated six fail-open defects, among them:

• a found record passed even when a duplicate now existed;
• a save landing after the window closed was never seen;
• identity read only the first name and the first email;
• the heartbeat was off by default.

All are fixed in 42779eca4, each with a regression test. REVIEW.md lists every finding and what changed. These fixes came after the blind rounds, so the table above describes the verifier before them. Rerun live on the fixed verifier (not blind): 18 of 18, 20 of 20 and 19 of 20, no fault passed as verified; round 3's miss is the same use-code case. The first rerun also found a harness fault, fixed in the runner: its pause after setup was shorter than the window's 2 s lead, so the first case read the runner's own setup write (RESULTS.md).

#1520 doesn't need to land first. Its generator ran from its own branch, and this PR commits the generated output and the generator's sha.