Skip to content

benchmark(commercial_ehr): Epic and Oracle Health sandbox terms and write support #1650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
abrichr merged 1 commit into from
Oct 29, 2026
Merged

benchmark(commercial_ehr): Epic and Oracle Health sandbox terms and write support #1650

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
211 changes: 211 additions & 0 deletions packages/flow/benchmark/commercial_ehr/FINDINGS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
# Qualifying the verifier against a commercial EHR sandbox

Research date: 2026-10-29. Sources are the vendors' own terms pages and the
live sandbox CapabilityStatements, fetched without signing in. Hashes of
what was read are at the end.

## Verdict

| Vendor | Writes in the sandbox (of the four asked) | Change channel | Terms allow publishing and naming | Verdict |
| --- | --- | --- | --- | --- |
| Epic (open.epic / fhir.epic.com sandbox, R4) | Patient create; Observation create and update; AllergyIntolerance create. Appointment is read-only | No AuditEvent search, no `_history`, no `vread`. Provenance and AuditEvent only through `_revInclude` on a search; `_lastUpdated` search on Observation only | Yes, with limits: no benchmarking or confidentiality clause; factual use of the name "Epic" is fair use; no logos and no implied endorsement | **Go** for Phase 2, with the read-path gap below |
| Oracle Health (code console; secure sandbox `fhir-ehr-code.cerner.com`, R4) | All four: Patient create and patch; Observation, AllergyIntolerance create and update; Appointment create and patch. The open sandbox is read-only | No AuditEvent, no `_history`. Provenance read, search and create | **No.** Benchmarking needs Oracle's prior written approval, test results belong to Oracle, and any public announcement about using the APIs is a material breach | **No-go** without Oracle's written approval |
| athenahealth | not assessed | not assessed | not assessed: the developer portal and docs return 403 to automated reads | **Not assessed**; read the terms after signing in, if wanted |

## Epic

### (a) Writes

From `GET https://fhir.epic.com/interconnect-fhir-oauth/api/FHIR/R4/metadata`
(software "Epic", version "August 2026"):

| Resource | Interactions |
| --- | --- |
| Patient | read, search-type, **create**; operations `$match`, `$summary` |
| Observation | read, search-type, **create, update** |
| AllergyIntolerance | read, search-type, **create** (no update) |
| Appointment | read, search-type (no write in R4 or STU3) |
| Provenance | read only |

Other writable types: BodyStructure, Communication, Condition,
DocumentReference, Procedure, QuestionnaireResponse, ServiceRequest (create),
DiagnosticReport, Task (update).

The CapabilityStatement is the server's claim. Each write still needs a probe:
Epic splits APIs by category (for example Observation create for vitals),
and the app must be granted each API on its client ID.

### (b) Change channel

- No `AuditEvent` resource endpoint. Epic's search-parameter documentation
says `_revInclude=AuditEvent:entity` returns audit events "for a specific
resource instance" and only "for a subset of FHIR version R4 resources".
Whether the sandbox populates it is unknown until probed.
- `_revInclude=Provenance:target` is documented for USCDI resources since
February 2020. Provenance has no search of its own.
- No `_history` and no `vread` on any type (`readHistory: false`).
- `_lastUpdated` is a search parameter on Observation but not on Patient,
AllergyIntolerance or Appointment, so a sweep can't cover patients.

What this means for `openadapt_flow.ensure_record`: the change feed
(AuditEvent by date, `_history`, the heartbeat) has no counterpart here.
As written, the verifier can't confirm "the agent wrote nothing else", so
every run that needs the feed ends `needs_review`. The record-level checks
still work through search and read: the record exists once, holds every
value, has no extra fields, and no duplicate exists.

Phase 2 therefore has to qualify a narrower read path and name it:
"the record and its duplicates, read through Epic's FHIR search; other writes
through the agent's own client are seen only through `_revInclude` audit or
provenance, where the sandbox returns it". That needs a contract
`change_feed` mode the verifier doesn't have yet (per-resource `_revInclude`
instead of a by-date AuditEvent search). Probe first, then decide whether to
build it.

The sandbox is shared with every Epic developer and Epic may wipe it at any
time, so a run must use names and identifiers unique to the run, and a sweep
would see other developers' writes.

### (c) Terms

From the open.epic Terms of Use (fhir.epic.com/Resources/Terms, "Last
updated: October 31, 2025"):

- There is no clause on benchmarking, publishing results, or keeping
results confidential.
- Names: "If you want to use open.epic or Epic's name or other trademarks,
you may do so only in accordance with our Trademark Usage Guidelines."
Those guidelines allow "limited fair use of Epic Marks in referring to our
company, software, or services without Epic's permission", forbid logos
without written permission, and forbid uses that could make anyone think
the product is endorsed, certified, or sponsored by Epic.
- The sandbox "is for testing purposes only" and takes only sample or
synthetic data.
- The website and Materials "should not be used for performance,
scalability, or security testing". A small functional suite of our own
verifier's write checks is none of these, but it must stay small: no
timing claims about Epic, no load, and no looping over all sandbox records
(Epic's "Responsible Sandbox Testing" asks for limited windows).
- Epic may collect and report data about connected apps (45 CFR 170.407).
- The open.epic Marketing Guidelines suggest wording such as "[Product] has
successfully integrated with Epic", ask developers not to speak for Epic,
and invite drafts to open@epic.com for review (up to two weeks).

So a public line such as "checked against Epic's public FHIR sandbox
(open.epic), through FHIR search and read" is allowed, with no logo and no
"certified" or "validated by Epic". Sending the exact copy to open@epic.com
first is optional but cheap.

### (d) Registration (the founder does this; no agent creates accounts)

1. Go to https://fhir.epic.com, choose **Sign Up**, and create an Epic on
FHIR account with a work email. Confirm the email. Accept the Terms of
Use when asked.
2. Create a key pair for the app on your Mac (the private key stays outside
the repository):

```bash
mkdir -p ~/.config/openadapt/epic && openssl req -x509 -newkey rsa:2048 -nodes -keyout ~/.config/openadapt/epic/private.pem -out ~/.config/openadapt/epic/public_cert.pem -days 365 -subj "/CN=openadapt-epic-sandbox"
```

3. **Build Apps → Create**. Fill in:
- Application name: `OpenAdapt verifier sandbox check`.
- Application audience: **Backend Systems**.
- Incoming APIs (R4): Patient Read, Search, Create; Observation Read,
Search, Create, Update (pick the vital-signs or labs variant offered);
AllergyIntolerance Read, Search, Create; Appointment Read, Search;
Provenance Read. Add AuditEvent if the list offers it.
- Default FHIR version: R4.
- Non-production JWT signing: upload `public_cert.pem` (or a JWK Set URL
if the form requires one; tell me and I'll set one up).
- Accept the terms checkbox, **Save**, then **Mark Ready for Sandbox**
if that button is offered.
4. Copy the **Non-Production Client ID** from the app's page.
5. Wait up to an hour: Epic syncs new apps to the sandbox within an hour and
new APIs within 30 minutes. Calls before that return 403.
6. Add to `/Users/abrichr/oa/src/OpenAdapt/.env`:

```
EPIC_SANDBOX_CLIENT_ID=<Non-Production Client ID>
EPIC_SANDBOX_PRIVATE_KEY=/Users/abrichr/.config/openadapt/epic/private.pem
```

Then tell this session. Phase 2 starts with a read-only probe
(`probe.py` like Medplum's), then the write probe, then the contract.

## Oracle Health

### (a) Writes

From the secure sandbox CapabilityStatement
(`fhir-ehr-code.cerner.com/r4/ec2458f2-1e24-41c8-b71b-0e701af7583d/metadata`):
Patient create and patch; Observation create and update; AllergyIntolerance
create and update; Appointment create and patch; also Condition, Coverage
(with delete), DocumentReference, Encounter, Immunization,
MedicationRequest, Provenance (create) and others. The open sandbox
(`fhir-open.cerner.com`) is read-only.

### (b) Change channel

No AuditEvent, no `_history`. Provenance supports read, search and create,
which is the nearest thing to a change record.

### (c) Terms

"Oracle Terms and Conditions for Oracle Health APIs", effective 3/1/24
(code.cerner.com/terms-of-use redirects to the PDF):

- Section 1(c): you may not "perform or disclose any benchmarking,
performance or availability testing" of the APIs or products without
Oracle's prior written approval.
- Section 1(d): "Any test results ... relating to the Oracle Health APIs"
are Feedback owned by Oracle, and "You shall have no rights to disclose
such Feedback to any third party."
- Section 8: no use of Oracle Brand Features (trade names included) without
written permission.
- A later section: a public announcement relating to the use of the APIs, or
a claim that an app "has been validated by Oracle", without written
permission is a material breach.

This is the same position as the Citrix trial licence, and stricter: even
running a benchmark needs approval. A write-fault suite of our own verifier
may not count as benchmarking Oracle, but the results couldn't be shown to
anyone, so running it gains nothing for the coverage claim.

### (d) Registration, if Oracle approves

Request written approval first, through Oracle Health's developer contact
(the code console's support link or an Oracle PartnerNetwork contact),
naming the use: a functional test of a third-party verifier's FHIR write
checks in the secure sandbox, with permission to state the result and name
Oracle Health. Only then: sign up at the code console
(code-console.cerner.com), register a **system** (backend) app for the
secure sandbox with the R4 Patient, Observation, AllergyIntolerance,
Appointment and Provenance read and write scopes, and put the client ID and
secret in `.env` as `ORACLE_SANDBOX_CLIENT_ID` and
`ORACLE_SANDBOX_CLIENT_SECRET`.

## athenahealth

Not assessed. `mydata.athenahealth.com` and `docs.athenahealth.com` return
403 to scripted fetches and an empty page in a headless browser, so the
terms couldn't be read without signing in. If the founder wants a third
vendor, they can open the developer-portal terms after signing up and paste
the benchmarking, confidentiality and naming clauses here.

## Evidence

Fetched 2026-10-29, not committed (vendor copyright); the hashes identify
what was read:

| Source | SHA-256 |
| --- | --- |
| Epic R4 CapabilityStatement | `1896b11a32545f5cf568715dadfb2ad5b9fa0ac2b77ec27684637d99d699282a` |
| Epic Terms of Use page (HTML) | `be695a88a84a6fe07299cbd9fecbe1509e3294742b6196dae92514e6133f2b9b` |
| Oracle secure sandbox CapabilityStatement | `ff951024714f2e3247561b969f99668814eb1a8e9b6555e8f5206a5cb560145f` |
| Oracle open sandbox CapabilityStatement | `58c08533a4d0859c15202ff594bb5e031fee4945253bd6b0661ffa13f420e5a8` |
| Oracle Health API terms (PDF, v030124) | `cf9a822bb55311ccb437bc2f0937d281f827b4ff8ebe5f8fb2d4d295e978707f` |

The CapabilityStatements are dated at fetch time; the servers regenerate
them on each request, so the hashes will not reproduce, but the interaction
lists above will until the vendor changes them.
Loading