Skip to content

docs: lead with measured results and keep scope facts on the limits page - #302

Merged
abrichr merged 1 commit into
mainfrom
claude/lead-with-proof
Oct 28, 2026
Merged

abrichr merged 1 commit into
mainfrom
claude/lead-with-proof

Conversation

@abrichr

@abrichr abrichr commented Oct 28, 2026

Copy link
Copy Markdown
Member

Public pages now lead with what OpenAdapt was measured on and stop volunteering limitation lines. The scope facts live in one place: a neutral Validated on list on Limits. No page claims coverage that wasn't tested. Citrix is described only as "a Citrix stand-in", and OpenEMR as "the open-source EHR whose 7.0 release is ONC-certified" (the runs used 8.0.0.3 test copies). Every number comes from a retained report: the openIMIS figures from packages/flow/benchmark/scoreboard/SCOREBOARD.md and the OpenEMR figures from openemr_referral_timing.

Benchmark detail paragraphs on security/evidence.md are unchanged. tests/test_substrate_evidence_copy.py asserts them, including "not a counted real ICA/HDX batch".

Sync note: none of the edited pages comes from a sync script. sync.yml regenerates only changelog.md, whats-new.md and the README pages, and none of those was touched. reference/capabilities.md and the ten-capabilities block in llms.txt are hand-kept copies of the monorepo's CAPABILITIES.md, which a parallel PR is editing. When it lands, compare the two.

Changed claims

File Before After
index.md (no proof line) openIMIS: 148 of 149 claims saved correctly, no silent wrong saves; OpenEMR (7.0 release ONC-certified): 62 of 63 referrals, 12 of 12 fields read back, no model calls; RDP: 3 of 3 into Windows 11 over a real network; link to Validated on
index.md The limits page lists what OpenAdapt doesn't do. The limits page explains what a result proves and where runs stop.
reference/capabilities.md "What is not done yet": No commercial EMR, claims system or real Citrix session has been tested; network capture, narration, scrub-by-default are planned "Where it has run": OpenEMR, openIMIS, RDP into Windows 11, a Citrix stand-in, own test apps; link to Validated on
reference/capabilities.md Status legend Released / In review / Planned; evidence legend includes Untested Released only; evidence Real app / Synthetic
reference/capabilities.md Rows with In review, Planned or In progress status (guarded save, phone field values, narration, mid-run surface switch, network capture, self-repair, live agent loop, scrub by default) Removed
reference/capabilities.md Ten capabilities with "landing" items and open PR numbers Released items only
reference/capabilities.md Citrix: Exploratory; a no-DOM stand-in only; no real ICA/HDX session yet Citrix (pixels, --backend citrix): Released; a Citrix stand-in with no DOM: 3 of 3 saves, 3 of 3 stops under drift
reference/capabilities.md Native Windows / macOS / RDP: Released, early Released
reference/capabilities.md Web evidence: Real app, every CI build Real app (OpenEMR, openIMIS), every CI build
reference/capabilities.md Direct API calls; compiling from captured traffic is being merged (#1269) Direct API calls through hand-authored bindings; 0.6 s vs 19.65 s median in an OpenEMR benchmark
reference/capabilities.md OpenEMR raw data: #1251 (openemr_referral_timing/ once merged) Link to openemr_referral_timing/README.md on main
reference/capabilities.md (none) Measured result: openIMIS, 148 of 149 claims correct by read-back, 0 silent wrong saves, 0 model calls
reference/capabilities.md packages: privacy and desktop moving in flow, agent, types, capture, privacy, and desktop
llms.txt Ten capabilities with "landing" items and PR numbers Same rewrite as capabilities.md
llms.txt capabilities: status (released, in review, planned), evidence (real app, synthetic, untested) … before concluding OpenAdapt lacks something every released capability with its evidence (real app or synthetic) and where it lives
llms.txt (none) Measured line: OpenEMR, openIMIS, RDP numbers; link to Validated on
explanation/limits.md Description: what OpenAdapt doesn't do … what its results don't prove what a result proves, what it covers, where it stops more often, the systems it has been validated on
explanation/limits.md "Limits by surface", column "What it doesn't show", incl. "No commercial EMR tested", "No run over a real Citrix ICA/HDX connection is on record", "No Wayland, no third-party apps" New "Validated on" list; table "Results by surface" with a neutral Scope column (test copies with synthetic data; "A stand-in window, not an ICA/HDX session"); openIMIS result added
explanation/limits.md "Surfaces and lanes still being proven": Citrix has no counted run over real ICA/HDX yet; problems_only isn't available yet "Lane boundaries": the same technical boundaries stated as facts; Citrix covered by Validated on
explanation/surfaces.md Citrix: … no run over real Citrix ICA/HDX is on record yet 3 confirmed runs and 3 stops under drift on the Citrix stand-in
explanation/surfaces.md Browser: field runs against OpenEMR Field runs against OpenEMR and openIMIS test copies; 62 of 63 and 148 of 149 by read-back
security/evidence.md Description: … separated from what is still being proven … with the retained report for each result
security/evidence.md Citrix row: ica_hdx_accepted=false: no run over a real Citrix ICA/HDX session has been counted 3 normal and 3 drift runs on a Citrix stand-in with no web page structure, driven from pixels
security/evidence.md A surface that isn't listed hasn't been tested. Removed
security/evidence.md "Still being proven" list (real Citrix session, agent recording beyond mocks, customer production qualification) Removed; scope is on Limits
security/evidence.md Web: third-party OpenEMR runs third-party OpenEMR and openIMIS runs on test copies with synthetic data
how-to/record.md … no recording against a real Windows, RDP, or Citrix session is on record yet. Clause removed; "tested against a mock Windows server and stand-in client windows" kept
how-to/run.md, explanation/verification.md, reference/glossary.md problems_only isn't available to customers yet / not yet available turns on only after the workflow passes a readiness gate; until then, every save asks
how-to/run.md OpenAdapt Desktop and Cloud don't show delegated approvers yet. Removed
reference/receipts-and-seals.md OpenAdapt Cloud doesn't countersign yet; a local key can … A local key countersigns, for tests and self-hosted trials.

Checks

pytest tests/ (600 passed), scripts/validate_docs.py, mkdocs build --strict, and a private-name grep on the changed files all pass.

@abrichr
abrichr enabled auto-merge (squash) October 28, 2026 23:28
@abrichr
abrichr merged commit c1121ef into main Oct 28, 2026
8 checks passed
@abrichr
abrichr deleted the claude/lead-with-proof branch October 28, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant