Skip to content

frame: key audit entries by a record field, and declare side records - #1570

Merged
abrichr merged 7 commits into
mainfrom
fix/frame-links
Oct 29, 2026
Merged

abrichr merged 7 commits into
mainfrom
fix/frame-links

Conversation

@abrichr

@abrichr abrichr commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Two frame gaps from the independent verification suite on OpenEMR (the Phase 0 measurement):

  • attribution.record_key. OpenEMR's audit log names a patient (pid), but pnotes, lists and history_data rows are told apart by their own id. With record_key: pid, an audit entry names a row through that field, so another user's edit to a patient's note is concurrent, not unattributed collateral. Without it, the entry can't name the row.
  • side_records. OpenEMR's New Patient path adds one history_data row beside each new chart, linked by pid. A frame.side_records entry (surface, link field, the declared record's surface and field, count) declares such rows part of the save. A missing or doubled side row refutes the frame (SPEC P4); a row linked to another chart is still collateral.

Tests: tests/test_frame_links.py covers both. test_frame_conditions.py is unchanged and passes.

Draft until the live OpenEMR Phase 0 run confirms the contract that uses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr
abrichr enabled auto-merge (squash) October 28, 2026 23:21
@abrichr
abrichr disabled auto-merge October 28, 2026 23:31
Adversarial review of #1570 found three silent passes and one more held:
- a row whose record_key changed (a note refiled to another patient) was
  matched by its new value only; it now names neither and is collateral;
- a bot entry was set aside when its ref equalled a moved row's id; moved
  rows are now compared by their record_key value;
- a declared record with no record_key value added "None" to the declared
  set, excusing every bot entry without a patient; the frame is now
  not_checked;
- a side record linked to an existing record's id was part of the save; a
  link already in the baseline is no longer a new record's.
Nothing changes when record_key and side_records are unset.
@abrichr

abrichr commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

AI review (adversarial)

Head d9e631ef3. Verdict: request changes. Three silent passes from the new record_key path, plus one weak spot in side records. The probes call judge_frame directly with in-memory fixtures, as the PR's tests do. The declared write is note id=10 of patient pid=1; attribution is audit_read with record_field=patient_id, record_key=pid, self bot.

Blocking

  1. A note moved to another patient ends held. ref_of read record_key from after only. Before {id:11,pid:1}, after {id:11,pid:2}, with a nurse entry for patient 2: held.
  2. Row ids and patient ids were mixed in moved. Row {id:5,pid:3} changes, a nurse entry names patient 3, and a bot entry names patient 5, which matches nothing in the read: held (refuted without the row-5 change).
  3. A declared record without record_key added "None" to declared_keys. Any bot entry with no patient_id was then excused: held.

Should fix

  1. False hold: any other user's entry for the target patient refutes a correct save, because other_on_target now matches by patient. On a busy chart this will flag correct saves. Limit it to entries that name the declared table or row, or document the limit.
  2. A side record can be excused for an existing patient. If a new chart reuses an existing pid, a new history row for that pid is taken as part of the save: held.

Held up

A float, empty or missing patient_id, a stray row with no pid, a missing, doubled or misdirected side row, a re-pointed history row, a new chart with no pid, and an unread side surface all fail closed.


Fix pushed (45c8f5e82). This changes behaviour. Findings 1, 2, 3 and 5 are fixed, each with a regression test in tests/test_frame_links.py:

  • a row whose record_key value changed names neither value, so it is unattributed collateral;
  • moved compares record_key values when record_key is set;
  • a declared record with no record_key value (or one that changed) makes the frame not_checked;
  • a side-record link that is already in the baseline is no longer a new record's link.

Nothing changes when record_key and side_records are unset. Finding 4 is still open as a known limit. This PR needs a fresh adversarial review before it is armed.

abrichr added a commit that referenced this pull request Oct 28, 2026
…base

#1582 already reads record_key from the record before the change and
treats a moved key as unattributed (#1570 finding 1). Ported onto its
ref_of: moved records' entries compare by their ref (a record_key, its
baseline MAC, or the row key), a declared record with no record_key value
is not_checked, and a side-record link already in the baseline is not a
new record's (findings 2, 3 and 5).
Re-review of #1570: with link_field other than record_key, a side row
linked to the new chart but filed under another patient was part of the
save, and its record_key value excused that patient's bot log entries.
It is now an ordinary unexplained addition.
@abrichr

abrichr commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

AI re-review, round 2 (adversarial)

Head 45c8f5e82. Verdict: request changes. Round-1 findings 1, 2, 3 and 5 are fixed. One new silent pass remains in side records, but only when link_field is not the record_key field (this PR's OpenEMR setup uses pid for both).

# Round-1 finding Now
1 Note refiled to another patient refuted
2 Row ids and patient ids mixed in moved refuted
3 Declared record without record_key not_checked
4 Another user's entry for the target patient refutes a correct save still open (known limit)
5 New chart reusing an existing pid refuted

New

  • Blocking (non-default config): with link_field=patient_row and of_field=id, a history row linked to the new chart but filed under patient 7 was part of the save. Its pid went into declared_keys, so the bot's log entry for patient 7 was excused: held.
  • Should fix: a side row is checked by link only, so with that setup a row filed under an existing patient passes even without attribution.
  • Limit: record_key is patient-level. A second bot log entry for the declared patient (say, a write to a table the frame doesn't read) is excused. A declared refile (a target's pid changes) is now not_checked.
  • Held up: refs doesn't collide across surfaces. With record_key and side_records unset, a 4000-case random comparison with origin/main's judge_frame gave identical results.

Fix pushed (dc4cc3d6e, bf591b94d). This changes behaviour. The side-link pre-pass now stores each link's parent record_key value. A side row with a different value is not part of the save: it is an ordinary unexplained addition, and the missing side row refutes too. A side row with no record_key value at all (OpenEMR's contact row) is still its parent's. Regression test: test_a_side_row_filed_under_another_record_key_is_collateral. The same change is ported onto #1582.

Re-review round 3 of #1570 and of its port to #1582:
- the parent-ref rule let an extra side row filed under another patient
  fall through to attribution, where a colleague's entry excused it. With
  record_key set, a side record must now link by record_key on both sides
  (validated), so a side row's link and its record_key are the same value;
  the parent-ref rule is gone.
- moved records compared by record_key let another user's concurrent change
  to a patient excuse the bot's own entry for that patient (a write the
  frame doesn't read). Only collateral records excuse entries now.
@abrichr

abrichr commented Oct 29, 2026

Copy link
Copy Markdown
Member Author

AI re-review, round 3 (adversarial)

Head bf591b94d. Verdict: request changes. The round-2 blocker is fixed, but the new parent rule brought back one silent pass, again only when link_field isn't the record_key field.

  1. Blocking: an extra side row filed under another patient ends held when a colleague touched that patient. The parent rule rejects the row, it falls through to attribute(), and the nurse's entry makes it "other". It was refuted at 45c8f5e82 (as a doubled side row).
  2. Should fix: with row-id links, a new chart that reuses an existing pid is excused (taken holds of_field values, not pid).

Held up: side rows with no pid, the default pid/pid link with int and string values, history plus contact on one surface, and a colleague's change not excusing the bot's own entry.


Fix pushed (95510abac). This changes behaviour.

  • With record_key set, a side record must link by it on both sides (link_field and of_field both equal record_key). This is validated, so a side row's link and its record key are the same value, and a link already in the baseline is a taken record key. The parent-ref rule is gone, which closes both findings: such configs are refused.
  • A separate reviewer found (in the frame: diff the whole database, with a reviewed allow-list #1582 port) that comparing moved records by record_key let a colleague's concurrent change excuse the bot's own entry for that patient. Only collateral records excuse entries now.

Regression tests: test_with_a_record_key_a_side_record_links_by_it and test_a_colleagues_change_never_excuses_the_bots_entry_for_that_patient.

@abrichr

abrichr commented Oct 29, 2026

Copy link
Copy Markdown
Member Author

AI re-review, round 4 (adversarial)

Head 95510abac. Verdict: approve. The round-3 blocker is closed, and I found no silent pass in a config the validator accepts.

Confirmed fixes

  • record_key: pid with link_field: patient_row or of_field: id is refused, in every combination.
  • With the allowed pid/pid link, an extra history row for pid 7 is refuted, whether or not the bot logs pid 7. A doubled side row is refuted.
  • A nurse edits pid 7, and the bot logs pid 7 for a table the frame doesn't read: refuted.
  • "Only collateral excuses" adds no false holds. These all stay held: a nurse's edit to pid 7, a clean new chart, int against string pid, and a clean edit save. A new chart reusing pid 1 is refuted.

Should fix (docs)

  1. An extra history row for pid 7 ends held when the bot logs only pid 2 and a nurse has any entry for pid 7. This PR has no event kinds, so a view is enough.
  2. A colleague's entry for the patient being edited refutes the save, even when the colleague wrote elsewhere.
  3. A bot write for the new chart to a table the frame doesn't read is excused (the read-set limit, which frame.database closes).

Docs added (c02399937) on record_key for items 1–3. Arming once CI is green.

@abrichr
abrichr enabled auto-merge (squash) October 29, 2026 00:46
abrichr added a commit that referenced this pull request Oct 29, 2026
…are theirs

Re-review of the #1570 port, round 2: a side row linked to a colleague's new
chart was theirs with no count, so the bot's extra contact or history row
for that chart ended held. Rows beyond the side record's count are now
unattributed collateral.
@abrichr
abrichr merged commit f34353e into main Oct 29, 2026
40 checks passed
@abrichr
abrichr deleted the fix/frame-links branch October 29, 2026 01:21
abrichr added a commit that referenced this pull request Oct 29, 2026
* frame: key audit entries by a record field, and declare side records

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: diff the whole database, with a reviewed allow-list

frame.database snapshots every table and column before and after the save
(three statements, whatever the table count) and judges each table as a
slice. A row added, removed or changed is collateral unless a declared
effect, a side record or an attributed concurrent writer explains it. A new
declared or side row that sets a column nothing declares is collateral too.
Tables over max_rows are fingerprinted. allow_tables, reviewed per app,
replaces the read-list; an unreadable snapshot leaves the frame not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: read rows wider than one JSON call, keeping null columns

SQLite caps a function at 127 arguments and OpenEMR's chart has more
columns, so the snapshot failed and the frame was not checked. Each row is
now read as JSON objects of at most 50 columns, merged in Python (a JSON
merge patch would drop null values).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a default the database computes at insert is a default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: audit attribution can name a table's record through another field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: whole-database diff without values, in one consistent read

From the adversarial review of #1582:
- PHI: the baseline keeps keyed MACs (BLAKE2b with the run salt) per cell
  and per row key, and keyed ids of audit entries; plain values are read only
  for rows the save added, in memory.
- Consistency: one connection, one read-only transaction (START TRANSACTION
  WITH CONSISTENT SNAPSHOT, READ ONLY; BEGIN on SQLite), with the audit log
  read inside it when it shares the connection.
- Large tables: salted SHA-256 per bucket of 1024 keys up to the baseline's
  largest key, added rows read in full, so a clean save on a table over
  max_rows holds. A table keeps its baseline representation after the save.
- Silent passes closed: a value is a default only if it equals the declared
  one; a clock default must fall between the snapshots' database clocks;
  ignore_fields apply to surfaces and database.ignore_columns to tables;
  sqlite_sequence and AUTO_INCREMENT catch a write then delete; table
  definitions and triggers, views, routines and events are digested; only a
  single generated key may hold any value; with frame.database, only a write
  event on the same table attributes a change (event_field, write_events,
  insert_events, table_field).
- Allow-listed tables are append-only unless skipped with a reason; every
  visible schema is read; settle_s delays the read for late writes; the
  receipt counts tables compared, append-only and skipped and binds the
  allow-list digest.
- Another actor's later write to the bot's new row, in a column the save
  doesn't declare, is concurrent when the bot only inserted it; a bot write
  to that column still fails. A surface that is a view of a snapshotted table
  is judged through the table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: excuse another actor's change to a declared record only through a write to its table, and never on a record the bot also wrote

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a DEFAULT NULL column (shown as the text NULL) has no default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a side row of another actor's new record is theirs; report their additions by surface id

OpenEMR logs a contact row with no patient, so no write event attributes it;
its link to a chart another user created in the window does, the same way
side records attach to the bot's chart. FrameVerdict.added_by_others names
such records as their surface does, for checks that read the surface.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the MySQL snapshot sends no statement that can fail

A driver may end the session on an error, so the snapshot checks for MySQL
8's information_schema_stats_expiry before setting it, instead of trying it
and catching MariaDB's error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: rows by key in every table, attribution by kind and count, MAC key outside the baseline

From the second review of #1582:
- Every table keeps, per row, a 128-bit MAC of its key and of its contents
  (without ignore_columns); cell MACs for all columns up to max_rows (now
  2000), and for the reference columns above it. A changed row in a large
  table goes through attribution by its key; ignore_columns hold; an added
  row is found by key for any primary key. Buckets are gone.
- Attribution: the record is named as it was before the change; an added
  row needs an insert entry, a removed one a delete entry (delete_events), a
  changed one an update; each entry explains at most one row; a row whose
  record key changed is never explained. The log's own limits are documented.
- MACs: BLAKE2b, 128 bits, bound to table and column, keyed by the salt and a
  host secret (OPENADAPT_FRAME_SECRET, or an owner-only key file) that is
  never in the baseline. Audit-entry ids use the same key.
- MySQL: AUTO_INCREMENT is read just before the consistent snapshot starts,
  on the column the database numbers whether or not it is the key (the live
  run's false hold on patient_data); no GROUP_CONCAT; the frame is not
  checked when the read account can't see triggers (it needs TRIGGER); a
  required schema the account can't see is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: tests use their own host secret; document that a deployment keeps it from the agent

The test suite created the real ~/.config/openadapt/frame.key; an autouse
fixture now sets OPENADAPT_FRAME_SECRET per test. The default key file only
keeps the secret out of the baseline, since a same-user agent can read it:
a deployment sets the secret from a store the agent can't read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the host secret is created whole and race-free on first use

Two processes starting together could read a half-written key file and
MAC the baseline and the read after it with different keys. The secret is
now written to a private temp file and linked into place; the loser reads
the winner's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: narrow types for mypy (renames only, no behaviour change)

* frame: record_key refs never excuse what they don't name

Adversarial review of #1570 found three silent passes and one more held:
- a row whose record_key changed (a note refiled to another patient) was
  matched by its new value only; it now names neither and is collateral;
- a bot entry was set aside when its ref equalled a moved row's id; moved
  rows are now compared by their record_key value;
- a declared record with no record_key value added "None" to the declared
  set, excusing every bot entry without a patient; the frame is now
  not_checked;
- a side record linked to an existing record's id was part of the save; a
  link already in the baseline is no longer a new record's.
Nothing changes when record_key and side_records are unset.

* frame: a side row must be filed under its parent's record_key

Re-review of #1570: with link_field other than record_key, a side row
linked to the new chart but filed under another patient was part of the
save, and its record_key value excused that patient's bot log entries.
It is now an ordinary unexplained addition.

* frame: a side row with no record_key value can still be its parent's

* frame: with record_key, side records link by it; only collateral excuses

Re-review round 3 of #1570 and of its port to #1582:
- the parent-ref rule let an extra side row filed under another patient
  fall through to attribution, where a colleague's entry excused it. With
  record_key set, a side record must now link by record_key on both sides
  (validated), so a side row's link and its record_key are the same value;
  the parent-ref rule is gone.
- moved records compared by record_key let another user's concurrent change
  to a patient excuse the bot's own entry for that patient (a write the
  frame doesn't read). Only collateral records excuse entries now.

* frame: at most count side rows per link of another actor's new chart are theirs

Re-review of the #1570 port, round 2: a side row linked to a colleague's new
chart was theirs with no count, so the bot's extra contact or history row
for that chart ended held. Rows beyond the side record's count are now
unattributed collateral.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 29, 2026
…, rollback and RL (#1604)

* frame: key audit entries by a record field, and declare side records

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: diff the whole database, with a reviewed allow-list

frame.database snapshots every table and column before and after the save
(three statements, whatever the table count) and judges each table as a
slice. A row added, removed or changed is collateral unless a declared
effect, a side record or an attributed concurrent writer explains it. A new
declared or side row that sets a column nothing declares is collateral too.
Tables over max_rows are fingerprinted. allow_tables, reviewed per app,
replaces the read-list; an unreadable snapshot leaves the frame not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: read rows wider than one JSON call, keeping null columns

SQLite caps a function at 127 arguments and OpenEMR's chart has more
columns, so the snapshot failed and the frame was not checked. Each row is
now read as JSON objects of at most 50 columns, merged in Python (a JSON
merge patch would drop null values).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a default the database computes at insert is a default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: audit attribution can name a table's record through another field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: whole-database diff without values, in one consistent read

From the adversarial review of #1582:
- PHI: the baseline keeps keyed MACs (BLAKE2b with the run salt) per cell
  and per row key, and keyed ids of audit entries; plain values are read only
  for rows the save added, in memory.
- Consistency: one connection, one read-only transaction (START TRANSACTION
  WITH CONSISTENT SNAPSHOT, READ ONLY; BEGIN on SQLite), with the audit log
  read inside it when it shares the connection.
- Large tables: salted SHA-256 per bucket of 1024 keys up to the baseline's
  largest key, added rows read in full, so a clean save on a table over
  max_rows holds. A table keeps its baseline representation after the save.
- Silent passes closed: a value is a default only if it equals the declared
  one; a clock default must fall between the snapshots' database clocks;
  ignore_fields apply to surfaces and database.ignore_columns to tables;
  sqlite_sequence and AUTO_INCREMENT catch a write then delete; table
  definitions and triggers, views, routines and events are digested; only a
  single generated key may hold any value; with frame.database, only a write
  event on the same table attributes a change (event_field, write_events,
  insert_events, table_field).
- Allow-listed tables are append-only unless skipped with a reason; every
  visible schema is read; settle_s delays the read for late writes; the
  receipt counts tables compared, append-only and skipped and binds the
  allow-list digest.
- Another actor's later write to the bot's new row, in a column the save
  doesn't declare, is concurrent when the bot only inserted it; a bot write
  to that column still fails. A surface that is a view of a snapshotted table
  is judged through the table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: excuse another actor's change to a declared record only through a write to its table, and never on a record the bot also wrote

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a DEFAULT NULL column (shown as the text NULL) has no default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a side row of another actor's new record is theirs; report their additions by surface id

OpenEMR logs a contact row with no patient, so no write event attributes it;
its link to a chart another user created in the window does, the same way
side records attach to the bot's chart. FrameVerdict.added_by_others names
such records as their surface does, for checks that read the surface.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the MySQL snapshot sends no statement that can fail

A driver may end the session on an error, so the snapshot checks for MySQL
8's information_schema_stats_expiry before setting it, instead of trying it
and catching MariaDB's error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: rows by key in every table, attribution by kind and count, MAC key outside the baseline

From the second review of #1582:
- Every table keeps, per row, a 128-bit MAC of its key and of its contents
  (without ignore_columns); cell MACs for all columns up to max_rows (now
  2000), and for the reference columns above it. A changed row in a large
  table goes through attribution by its key; ignore_columns hold; an added
  row is found by key for any primary key. Buckets are gone.
- Attribution: the record is named as it was before the change; an added
  row needs an insert entry, a removed one a delete entry (delete_events), a
  changed one an update; each entry explains at most one row; a row whose
  record key changed is never explained. The log's own limits are documented.
- MACs: BLAKE2b, 128 bits, bound to table and column, keyed by the salt and a
  host secret (OPENADAPT_FRAME_SECRET, or an owner-only key file) that is
  never in the baseline. Audit-entry ids use the same key.
- MySQL: AUTO_INCREMENT is read just before the consistent snapshot starts,
  on the column the database numbers whether or not it is the key (the live
  run's false hold on patient_data); no GROUP_CONCAT; the frame is not
  checked when the read account can't see triggers (it needs TRIGGER); a
  required schema the account can't see is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: one versioned OpenEMR bookkeeping profile for the frame, rollback and RL

openadapt_flow.app_profiles.load_profile('openemr'): the tables and columns
OpenEMR writes on its own (logs and id maps append-only; per-session state
skipped with a reason; uuid backfill and the users ON UPDATE stamp as column
rules), from the RL environment's calibration (2026-10-28.1) and the frame's
live witness. No rule may name a patient-data table except for a system
column. frame.database.profile applies it; the receipt names the version,
its digest and the rules that excused something in the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: contact is append-only bookkeeping (openemr-frame-profile/2026-10-28.2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: tests use their own host secret; document that a deployment keeps it from the agent

The test suite created the real ~/.config/openadapt/frame.key; an autouse
fixture now sets OPENADAPT_FRAME_SECRET per test. The default key file only
keeps the secret out of the baseline, since a same-user agent can read it:
a deployment sets the secret from a store the agent can't read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: compare credential tables; backfills only; an allowlist; a pinned digest

From the third review of #1604:
- No table is skipped any more. Credentials and access are compared:
  users_secure and login_mfa_registrations excuse only sign-in counters and
  timestamps on the acting user's own row; keys are append-only;
  background_services excuses only next_run and running; user_settings and
  recent_patients only the acting user's own rows (owner by user id, looked
  up in the snapshot).
- uuid rules excuse a backfill only, from NULL to a value; users.last_updated
  only beside one.
- The patient-data denylist is now an allowlist: a rule may name only a
  reviewed bookkeeping table (lower-cased, unqualified); '*' only a system
  column backfill.
- Profiles load only from the package, by name; frame.database.profile_sha256
  pins the digest (bound into frame_sha256), and a mismatch is refused.
- rules_used lists only the rules that excused something in the run.
- A table without a primary key keys its rows without the columns a rule
  lets move. The profile is registered in public-artifacts.json.

Profile version openemr-frame-profile/2026-10-29.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the host secret is created whole and race-free on first use

Two processes starting together could read a half-written key file and
MAC the baseline and the read after it with different keys. The secret is
now written to a private temp file and linked into place; the loser reads
the winner's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: a colleague who wrote in the window owns their per-user UI state

own_rows rules (user_settings, recent_patients) also excuse the rows of
another user with a write entry in the window: their own work. Credential
column rules stay on the acting user's row. The users table's ids and names
are read in the snapshot, in memory only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: keys and MFA registrations are compared in full (2026-10-29.2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: narrow types for mypy (renames only, no behaviour change)

* frame: record_key refs never excuse what they don't name

Adversarial review of #1570 found three silent passes and one more held:
- a row whose record_key changed (a note refiled to another patient) was
  matched by its new value only; it now names neither and is collateral;
- a bot entry was set aside when its ref equalled a moved row's id; moved
  rows are now compared by their record_key value;
- a declared record with no record_key value added "None" to the declared
  set, excusing every bot entry without a patient; the frame is now
  not_checked;
- a side record linked to an existing record's id was part of the save; a
  link already in the baseline is no longer a new record's.
Nothing changes when record_key and side_records are unset.

* frame: a side row must be filed under its parent's record_key

Re-review of #1570: with link_field other than record_key, a side row
linked to the new chart but filed under another patient was part of the
save, and its record_key value excused that patient's bot log entries.
It is now an ordinary unexplained addition.

* frame: a side row with no record_key value can still be its parent's

* frame: with record_key, side records link by it; only collateral excuses

Re-review round 3 of #1570 and of its port to #1582:
- the parent-ref rule let an extra side row filed under another patient
  fall through to attribution, where a colleague's entry excused it. With
  record_key set, a side record must now link by record_key on both sides
  (validated), so a side row's link and its record_key are the same value;
  the parent-ref rule is gone.
- moved records compared by record_key let another user's concurrent change
  to a patient excuse the bot's own entry for that patient (a write the
  frame doesn't read). Only collateral records excuse entries now.

* frame: at most count side rows per link of another actor's new chart are theirs

Re-review of the #1570 port, round 2: a side row linked to a colleague's new
chart was theirs with no count, so the bot's extra contact or history row
for that chart ended held. Rows beyond the side record's count are now
unattributed collateral.

* Restore main's changes after #1582 (#1618) dropped by the merge

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 29, 2026
…#1599)

* frame: key audit entries by a record field, and declare side records

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: diff the whole database, with a reviewed allow-list

frame.database snapshots every table and column before and after the save
(three statements, whatever the table count) and judges each table as a
slice. A row added, removed or changed is collateral unless a declared
effect, a side record or an attributed concurrent writer explains it. A new
declared or side row that sets a column nothing declares is collateral too.
Tables over max_rows are fingerprinted. allow_tables, reviewed per app,
replaces the read-list; an unreadable snapshot leaves the frame not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: read rows wider than one JSON call, keeping null columns

SQLite caps a function at 127 arguments and OpenEMR's chart has more
columns, so the snapshot failed and the frame was not checked. Each row is
now read as JSON objects of at most 50 columns, merged in Python (a JSON
merge patch would drop null values).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a default the database computes at insert is a default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: audit attribution can name a table's record through another field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: whole-database diff without values, in one consistent read

From the adversarial review of #1582:
- PHI: the baseline keeps keyed MACs (BLAKE2b with the run salt) per cell
  and per row key, and keyed ids of audit entries; plain values are read only
  for rows the save added, in memory.
- Consistency: one connection, one read-only transaction (START TRANSACTION
  WITH CONSISTENT SNAPSHOT, READ ONLY; BEGIN on SQLite), with the audit log
  read inside it when it shares the connection.
- Large tables: salted SHA-256 per bucket of 1024 keys up to the baseline's
  largest key, added rows read in full, so a clean save on a table over
  max_rows holds. A table keeps its baseline representation after the save.
- Silent passes closed: a value is a default only if it equals the declared
  one; a clock default must fall between the snapshots' database clocks;
  ignore_fields apply to surfaces and database.ignore_columns to tables;
  sqlite_sequence and AUTO_INCREMENT catch a write then delete; table
  definitions and triggers, views, routines and events are digested; only a
  single generated key may hold any value; with frame.database, only a write
  event on the same table attributes a change (event_field, write_events,
  insert_events, table_field).
- Allow-listed tables are append-only unless skipped with a reason; every
  visible schema is read; settle_s delays the read for late writes; the
  receipt counts tables compared, append-only and skipped and binds the
  allow-list digest.
- Another actor's later write to the bot's new row, in a column the save
  doesn't declare, is concurrent when the bot only inserted it; a bot write
  to that column still fails. A surface that is a view of a snapshotted table
  is judged through the table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: excuse another actor's change to a declared record only through a write to its table, and never on a record the bot also wrote

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a DEFAULT NULL column (shown as the text NULL) has no default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a side row of another actor's new record is theirs; report their additions by surface id

OpenEMR logs a contact row with no patient, so no write event attributes it;
its link to a chart another user created in the window does, the same way
side records attach to the bot's chart. FrameVerdict.added_by_others names
such records as their surface does, for checks that read the surface.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verify-run: the baseline holds the read set as keyed MACs, not values

The baseline kept the contract's read set (a REST patient list, say) as
plain rows: a copy of the records on disk. It now keeps, per record, a keyed
MAC of its identity and of each field (BLAKE2b keyed with the run's salt, as
the database frame does). At judging time one read rebuilds the records:
unchanged ones as read, a moved field as a marker no value equals, a gone
record as a removed placeholder. Every check downstream is unchanged. A
baseline from before this still loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the MySQL snapshot sends no statement that can fail

A driver may end the session on an error, so the snapshot checks for MySQL
8's information_schema_stats_expiry before setting it, instead of trying it
and catching MariaDB's error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: rows by key in every table, attribution by kind and count, MAC key outside the baseline

From the second review of #1582:
- Every table keeps, per row, a 128-bit MAC of its key and of its contents
  (without ignore_columns); cell MACs for all columns up to max_rows (now
  2000), and for the reference columns above it. A changed row in a large
  table goes through attribution by its key; ignore_columns hold; an added
  row is found by key for any primary key. Buckets are gone.
- Attribution: the record is named as it was before the change; an added
  row needs an insert entry, a removed one a delete entry (delete_events), a
  changed one an update; each entry explains at most one row; a row whose
  record key changed is never explained. The log's own limits are documented.
- MACs: BLAKE2b, 128 bits, bound to table and column, keyed by the salt and a
  host secret (OPENADAPT_FRAME_SECRET, or an owner-only key file) that is
  never in the baseline. Audit-entry ids use the same key.
- MySQL: AUTO_INCREMENT is read just before the consistent snapshot starts,
  on the column the database numbers whether or not it is the key (the live
  run's false hold on patient_data); no GROUP_CONCAT; the frame is not
  checked when the read account can't see triggers (it needs TRIGGER); a
  required schema the account can't see is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Merge #1582's round 2 into the sealed baseline; seal with the derived key, bound to surface and field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: tests use their own host secret; document that a deployment keeps it from the agent

The test suite created the real ~/.config/openadapt/frame.key; an autouse
fixture now sets OPENADAPT_FRAME_SECRET per test. The default key file only
keeps the secret out of the baseline, since a same-user agent can read it:
a deployment sets the secret from a store the agent can't read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the host secret is created whole and race-free on first use

Two processes starting together could read a half-written key file and
MAC the baseline and the read after it with different keys. The secret is
now written to a private temp file and linked into place; the loser reads
the winner's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: narrow types for mypy (renames only, no behaviour change)

* sealed: rename the looked-up record for mypy (no behaviour change)

* frame: record_key refs never excuse what they don't name

Adversarial review of #1570 found three silent passes and one more held:
- a row whose record_key changed (a note refiled to another patient) was
  matched by its new value only; it now names neither and is collateral;
- a bot entry was set aside when its ref equalled a moved row's id; moved
  rows are now compared by their record_key value;
- a declared record with no record_key value added "None" to the declared
  set, excusing every bot entry without a patient; the frame is now
  not_checked;
- a side record linked to an existing record's id was part of the save; a
  link already in the baseline is no longer a new record's.
Nothing changes when record_key and side_records are unset.

* sealed: type-tagged cells, a random marker, unknown attribution paths

Adversarial review of #1599 found three silent passes the raw baseline
caught:
- canonical() drops the type, so True -> "true" or an object -> its JSON
  text sealed alike; a cell's MAC now covers its type (numbers share one
  class, so 1 and "1" now differ: stricter, never looser);
- CHANGED and REMOVED were fixed strings an app can store; they are now
  random per process;
- a record_field attribution path through a rebuilt (changed) field read
  as None, so the same-modifier guard never fired; such a change is now
  unattributed.

* frame: a side row must be filed under its parent's record_key

Re-review of #1570: with link_field other than record_key, a side row
linked to the new chart but filed under another patient was part of the
save, and its record_key value excused that patient's bot log entries.
It is now an ordinary unexplained addition.

* frame: a side row with no record_key value can still be its parent's

* sealed: bytes and other types carry their own tag

Re-review of #1599 (N2): every type outside null, bool, number and JSON
sealed as its str(), so b"x" and the text "b'x'" sealed alike. Bytes
seal as hex, other types with their type name.

* frame: with record_key, side records link by it; only collateral excuses

Re-review round 3 of #1570 and of its port to #1582:
- the parent-ref rule let an extra side row filed under another patient
  fall through to attribution, where a colleague's entry excused it. With
  record_key set, a side record must now link by record_key on both sides
  (validated), so a side row's link and its record_key are the same value;
  the parent-ref rule is gone.
- moved records compared by record_key let another user's concurrent change
  to a patient excuse the bot's own entry for that patient (a write the
  frame doesn't read). Only collateral records excuse entries now.

* frame: at most count side rows per link of another actor's new chart are theirs

Re-review of the #1570 port, round 2: a side row linked to a colleague's new
chart was theirs with no count, so the bot's extra contact or history row
for that chart ended held. Rows beyond the side record's count are now
unattributed collateral.

* Restore main's composite duplicate-key check (#1618) dropped by the merge

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 29, 2026
* frame: key audit entries by a record field, and declare side records

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: diff the whole database, with a reviewed allow-list

frame.database snapshots every table and column before and after the save
(three statements, whatever the table count) and judges each table as a
slice. A row added, removed or changed is collateral unless a declared
effect, a side record or an attributed concurrent writer explains it. A new
declared or side row that sets a column nothing declares is collateral too.
Tables over max_rows are fingerprinted. allow_tables, reviewed per app,
replaces the read-list; an unreadable snapshot leaves the frame not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: read rows wider than one JSON call, keeping null columns

SQLite caps a function at 127 arguments and OpenEMR's chart has more
columns, so the snapshot failed and the frame was not checked. Each row is
now read as JSON objects of at most 50 columns, merged in Python (a JSON
merge patch would drop null values).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a default the database computes at insert is a default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: audit attribution can name a table's record through another field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: a stable Python API for checking one write

openadapt_flow.verifier: capture() reads the system of record before the
write, verify_write() judges it against the same contract and returns
verified, the outcome, which parts were checked, plain reasons, the receipt
and its evidence. It wraps verify_run and checks exactly what it checks; a
snapshot verifies one write. A test pins the public names and signatures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: actor kinds, refuted vs unconfirmed, receipt hash, database diff

From the MCP skills, RL reward and undo sessions:
- Actor(kind, id, build): the receipt records who wrote (program, agent,
  script, person);
- WriteResult.status: refuted (the store shows a different, extra or
  duplicate write) vs unconfirmed (it doesn't show the write, or couldn't
  be read: reconcile before retrying), and receipt_sha256;
- capture() raises BaselineUnreadable when the store can't be read, so a
  caller can stop before writing; write_attempted passes through;
- capture_database() and diff_database(): a whole-database snapshot and a
  per-table diff (rows added, removed, changed, column names, no values),
  over the database's own driver.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: whole-database diff without values, in one consistent read

From the adversarial review of #1582:
- PHI: the baseline keeps keyed MACs (BLAKE2b with the run salt) per cell
  and per row key, and keyed ids of audit entries; plain values are read only
  for rows the save added, in memory.
- Consistency: one connection, one read-only transaction (START TRANSACTION
  WITH CONSISTENT SNAPSHOT, READ ONLY; BEGIN on SQLite), with the audit log
  read inside it when it shares the connection.
- Large tables: salted SHA-256 per bucket of 1024 keys up to the baseline's
  largest key, added rows read in full, so a clean save on a table over
  max_rows holds. A table keeps its baseline representation after the save.
- Silent passes closed: a value is a default only if it equals the declared
  one; a clock default must fall between the snapshots' database clocks;
  ignore_fields apply to surfaces and database.ignore_columns to tables;
  sqlite_sequence and AUTO_INCREMENT catch a write then delete; table
  definitions and triggers, views, routines and events are digested; only a
  single generated key may hold any value; with frame.database, only a write
  event on the same table attributes a change (event_field, write_events,
  insert_events, table_field).
- Allow-listed tables are append-only unless skipped with a reason; every
  visible schema is read; settle_s delays the read for late writes; the
  receipt counts tables compared, append-only and skipped and binds the
  allow-list digest.
- Another actor's later write to the bot's new row, in a column the save
  doesn't declare, is concurrent when the bot only inserted it; a bot write
  to that column still fails. A surface that is a view of a snapshotted table
  is judged through the table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: excuse another actor's change to a declared record only through a write to its table, and never on a record the bot also wrote

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a DEFAULT NULL column (shown as the text NULL) has no default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a side row of another actor's new record is theirs; report their additions by surface id

OpenEMR logs a contact row with no patient, so no write event attributes it;
its link to a chart another user created in the window does, the same way
side records attach to the bot's chart. FrameVerdict.added_by_others names
such records as their surface does, for checks that read the surface.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the MySQL snapshot sends no statement that can fail

A driver may end the session on an error, so the snapshot checks for MySQL
8's information_schema_stats_expiry before setting it, instead of trying it
and catching MariaDB's error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: rows by key in every table, attribution by kind and count, MAC key outside the baseline

From the second review of #1582:
- Every table keeps, per row, a 128-bit MAC of its key and of its contents
  (without ignore_columns); cell MACs for all columns up to max_rows (now
  2000), and for the reference columns above it. A changed row in a large
  table goes through attribution by its key; ignore_columns hold; an added
  row is found by key for any primary key. Buckets are gone.
- Attribution: the record is named as it was before the change; an added
  row needs an insert entry, a removed one a delete entry (delete_events), a
  changed one an update; each entry explains at most one row; a row whose
  record key changed is never explained. The log's own limits are documented.
- MACs: BLAKE2b, 128 bits, bound to table and column, keyed by the salt and a
  host secret (OPENADAPT_FRAME_SECRET, or an owner-only key file) that is
  never in the baseline. Audit-entry ids use the same key.
- MySQL: AUTO_INCREMENT is read just before the consistent snapshot starts,
  on the column the database numbers whether or not it is the key (the live
  run's false hold on patient_data); no GROUP_CONCAT; the frame is not
  checked when the read account can't see triggers (it needs TRIGGER); a
  required schema the account can't see is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: the database snapshot's max_rows default follows the frame (2000)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: one versioned OpenEMR bookkeeping profile for the frame, rollback and RL

openadapt_flow.app_profiles.load_profile('openemr'): the tables and columns
OpenEMR writes on its own (logs and id maps append-only; per-session state
skipped with a reason; uuid backfill and the users ON UPDATE stamp as column
rules), from the RL environment's calibration (2026-10-28.1) and the frame's
live witness. No rule may name a patient-data table except for a system
column. frame.database.profile applies it; the receipt names the version,
its digest and the rules that excused something in the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: capture_database takes the app profile the frame uses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: contact is append-only bookkeeping (openemr-frame-profile/2026-10-28.2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: tests use their own host secret; document that a deployment keeps it from the agent

The test suite created the real ~/.config/openadapt/frame.key; an autouse
fixture now sets OPENADAPT_FRAME_SECRET per test. The default key file only
keeps the secret out of the baseline, since a same-user agent can read it:
a deployment sets the secret from a store the agent can't read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: compare credential tables; backfills only; an allowlist; a pinned digest

From the third review of #1604:
- No table is skipped any more. Credentials and access are compared:
  users_secure and login_mfa_registrations excuse only sign-in counters and
  timestamps on the acting user's own row; keys are append-only;
  background_services excuses only next_run and running; user_settings and
  recent_patients only the acting user's own rows (owner by user id, looked
  up in the snapshot).
- uuid rules excuse a backfill only, from NULL to a value; users.last_updated
  only beside one.
- The patient-data denylist is now an allowlist: a rule may name only a
  reviewed bookkeeping table (lower-cased, unqualified); '*' only a system
  column backfill.
- Profiles load only from the package, by name; frame.database.profile_sha256
  pins the digest (bound into frame_sha256), and a mismatch is refused.
- rules_used lists only the rules that excused something in the run.
- A table without a primary key keys its rows without the columns a rule
  lets move. The profile is registered in public-artifacts.json.

Profile version openemr-frame-profile/2026-10-29.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: capture_database takes before= and actor_names; diff_database applies the profile

Rollback needs the plain values of rows that changed after a run began
(before=, in memory only), and the same judgment the frame uses: with
profile= and the acting user's names, diff_database leaves out a row change
the profile calls the app's own (the user's sign-in counters, a uuid
backfill), so a clean undo isn't reported dirty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the host secret is created whole and race-free on first use

Two processes starting together could read a half-written key file and
MAC the baseline and the read after it with different keys. The secret is
now written to a private temp file and linked into place; the loser reads
the winner's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: a colleague who wrote in the window owns their per-user UI state

own_rows rules (user_settings, recent_patients) also excuse the rows of
another user with a write entry in the window: their own work. Credential
column rules stay on the acting user's row. The users table's ids and names
are read in the snapshot, in memory only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: keys and MFA registrations are compared in full (2026-10-29.2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: narrow types for mypy (renames only, no behaviour change)

* docs: refresh llms-full.txt for the verifier API row

* frame: record_key refs never excuse what they don't name

Adversarial review of #1570 found three silent passes and one more held:
- a row whose record_key changed (a note refiled to another patient) was
  matched by its new value only; it now names neither and is collateral;
- a bot entry was set aside when its ref equalled a moved row's id; moved
  rows are now compared by their record_key value;
- a declared record with no record_key value added "None" to the declared
  set, excusing every bot entry without a patient; the frame is now
  not_checked;
- a side record linked to an existing record's id was part of the save; a
  link already in the baseline is no longer a new record's.
Nothing changes when record_key and side_records are unset.

* verifier: a snapshot is single use across reloads and copies; errors are VerifierError

Adversarial review of #1588: verify_write copied each snapshot to a fresh
temp file, so Snapshot.load twice, a payload copy, or a baseline the CLI had
already used each verified a second write (an RL replay hole). A process-wide
set of used salts now refuses copies, and a loaded snapshot verifies through
its own file, so verify-run's .used marker applies. VerifyRunError (unbound
params, another contract's snapshot) now surfaces as VerifierError, the
docstring says a refused call uses up the snapshot, and repr hides the
payload.

* test: plain setup for the other-contract snapshot case

* frame: a side row must be filed under its parent's record_key

Re-review of #1570: with link_field other than record_key, a side row
linked to the new chart but filed under another patient was part of the
save, and its record_key value excused that patient's bot log entries.
It is now an ordinary unexplained addition.

* frame: a side row with no record_key value can still be its parent's

* verifier: single use survives copies and other processes

Re-review of #1588: the used-salt set lived in one process, so a copied
file, a load-save-load to a new path, or a snapshot pickled to a worker
verified a second write in a new process. A use is now also recorded as an
O_EXCL file named by the salt's digest in a private per-user directory,
checked on every call. A missing salt, a non-dict payload and an OSError
raise VerifierError with their own messages; the docstring says an
unconfirmed result also uses up the snapshot. Snapshot's fields and
load/save signatures are pinned.

* frame: with record_key, side records link by it; only collateral excuses

Re-review round 3 of #1570 and of its port to #1582:
- the parent-ref rule let an extra side row filed under another patient
  fall through to attribution, where a colleague's entry excused it. With
  record_key set, a side record must now link by record_key on both sides
  (validated), so a side row's link and its record_key are the same value;
  the parent-ref rule is gone.
- moved records compared by record_key let another user's concurrent change
  to a patient excuse the bot's own entry for that patient (a write the
  frame doesn't read). Only collateral records excuse entries now.

* frame: at most count side rows per link of another actor's new chart are theirs

Re-review of the #1570 port, round 2: a side row linked to a colleague's new
chart was theirs with no count, so the bot's extra contact or history row
for that chart ended held. Rows beyond the side record's count are now
unattributed collateral.

* verifier: capture issues a record, verify removes it

Re-review round 3 of #1588: the used-salt marker lived under the caller's
HOME, so a worker with another HOME (or none, another user, container or
host) replayed a used snapshot. capture() now records each snapshot as
issued (an O_EXCL file named by the digest of its salt bytes) and
verify_write() removes it; only one caller can, and a snapshot with no
record is refused, so another state directory fails closed instead of
replaying. Markers no longer pile up. OPENADAPT_VERIFIER_STATE_DIR gives
every worker one directory; it must be a private directory owned by the
user. The salt is keyed by its bytes, not its hex case, and the in-memory
payload whose salt was used is what's judged.

* verifier: the state root is private too; document capture-once and the state dir

Re-review round 4 of #1588: check the state root as well as issued/ (T6),
and say in the module docstring that the judged writer must not reach the
state directory (T5), that a snapshot verifies only under the state dir it
was captured under, that the caller captures once per write (T4), and that
an unverified capture leaves its record (T7).

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 29, 2026
* frame: key audit entries by a record field, and declare side records

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: diff the whole database, with a reviewed allow-list

frame.database snapshots every table and column before and after the save
(three statements, whatever the table count) and judges each table as a
slice. A row added, removed or changed is collateral unless a declared
effect, a side record or an attributed concurrent writer explains it. A new
declared or side row that sets a column nothing declares is collateral too.
Tables over max_rows are fingerprinted. allow_tables, reviewed per app,
replaces the read-list; an unreadable snapshot leaves the frame not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: read rows wider than one JSON call, keeping null columns

SQLite caps a function at 127 arguments and OpenEMR's chart has more
columns, so the snapshot failed and the frame was not checked. Each row is
now read as JSON objects of at most 50 columns, merged in Python (a JSON
merge patch would drop null values).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a default the database computes at insert is a default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: audit attribution can name a table's record through another field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: whole-database diff without values, in one consistent read

From the adversarial review of #1582:
- PHI: the baseline keeps keyed MACs (BLAKE2b with the run salt) per cell
  and per row key, and keyed ids of audit entries; plain values are read only
  for rows the save added, in memory.
- Consistency: one connection, one read-only transaction (START TRANSACTION
  WITH CONSISTENT SNAPSHOT, READ ONLY; BEGIN on SQLite), with the audit log
  read inside it when it shares the connection.
- Large tables: salted SHA-256 per bucket of 1024 keys up to the baseline's
  largest key, added rows read in full, so a clean save on a table over
  max_rows holds. A table keeps its baseline representation after the save.
- Silent passes closed: a value is a default only if it equals the declared
  one; a clock default must fall between the snapshots' database clocks;
  ignore_fields apply to surfaces and database.ignore_columns to tables;
  sqlite_sequence and AUTO_INCREMENT catch a write then delete; table
  definitions and triggers, views, routines and events are digested; only a
  single generated key may hold any value; with frame.database, only a write
  event on the same table attributes a change (event_field, write_events,
  insert_events, table_field).
- Allow-listed tables are append-only unless skipped with a reason; every
  visible schema is read; settle_s delays the read for late writes; the
  receipt counts tables compared, append-only and skipped and binds the
  allow-list digest.
- Another actor's later write to the bot's new row, in a column the save
  doesn't declare, is concurrent when the bot only inserted it; a bot write
  to that column still fails. A surface that is a view of a snapshotted table
  is judged through the table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: excuse another actor's change to a declared record only through a write to its table, and never on a record the bot also wrote

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a DEFAULT NULL column (shown as the text NULL) has no default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a side row of another actor's new record is theirs; report their additions by surface id

OpenEMR logs a contact row with no patient, so no write event attributes it;
its link to a chart another user created in the window does, the same way
side records attach to the bot's chart. FrameVerdict.added_by_others names
such records as their surface does, for checks that read the surface.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the MySQL snapshot sends no statement that can fail

A driver may end the session on an error, so the snapshot checks for MySQL
8's information_schema_stats_expiry before setting it, instead of trying it
and catching MariaDB's error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: two reads after the save must match

The database is read twice after the save, settle_s apart (default 1 s). If
the reads differ, the app is still writing (a background job, a queued
write) and what the save caused can't be told yet: the frame is not
checked, so the run is not verified.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: rows by key in every table, attribution by kind and count, MAC key outside the baseline

From the second review of #1582:
- Every table keeps, per row, a 128-bit MAC of its key and of its contents
  (without ignore_columns); cell MACs for all columns up to max_rows (now
  2000), and for the reference columns above it. A changed row in a large
  table goes through attribution by its key; ignore_columns hold; an added
  row is found by key for any primary key. Buckets are gone.
- Attribution: the record is named as it was before the change; an added
  row needs an insert entry, a removed one a delete entry (delete_events), a
  changed one an update; each entry explains at most one row; a row whose
  record key changed is never explained. The log's own limits are documented.
- MACs: BLAKE2b, 128 bits, bound to table and column, keyed by the salt and a
  host secret (OPENADAPT_FRAME_SECRET, or an owner-only key file) that is
  never in the baseline. Audit-entry ids use the same key.
- MySQL: AUTO_INCREMENT is read just before the consistent snapshot starts,
  on the column the database numbers whether or not it is the key (the live
  run's false hold on patient_data); no GROUP_CONCAT; the frame is not
  checked when the read account can't see triggers (it needs TRIGGER); a
  required schema the account can't see is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: two reads after the save must match (on #1582's round-2 snapshot)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: tests use their own host secret; document that a deployment keeps it from the agent

The test suite created the real ~/.config/openadapt/frame.key; an autouse
fixture now sets OPENADAPT_FRAME_SECRET per test. The default key file only
keeps the secret out of the baseline, since a same-user agent can read it:
a deployment sets the secret from a store the agent can't read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the host secret is created whole and race-free on first use

Two processes starting together could read a half-written key file and
MAC the baseline and the read after it with different keys. The secret is
now written to a private temp file and linked into place; the loser reads
the winner's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: narrow types for mypy (renames only, no behaviour change)

* frame: record_key refs never excuse what they don't name

Adversarial review of #1570 found three silent passes and one more held:
- a row whose record_key changed (a note refiled to another patient) was
  matched by its new value only; it now names neither and is collateral;
- a bot entry was set aside when its ref equalled a moved row's id; moved
  rows are now compared by their record_key value;
- a declared record with no record_key value added "None" to the declared
  set, excusing every bot entry without a patient; the frame is now
  not_checked;
- a side record linked to an existing record's id was part of the save; a
  link already in the baseline is no longer a new record's.
Nothing changes when record_key and side_records are unset.

* frame: a side row must be filed under its parent's record_key

Re-review of #1570: with link_field other than record_key, a side row
linked to the new chart but filed under another patient was part of the
save, and its record_key value excused that patient's bot log entries.
It is now an ordinary unexplained addition.

* frame: a side row with no record_key value can still be its parent's

* frame: with record_key, side records link by it; only collateral excuses

Re-review round 3 of #1570 and of its port to #1582:
- the parent-ref rule let an extra side row filed under another patient
  fall through to attribution, where a colleague's entry excused it. With
  record_key set, a side record must now link by record_key on both sides
  (validated), so a side row's link and its record_key are the same value;
  the parent-ref rule is gone.
- moved records compared by record_key let another user's concurrent change
  to a patient excuse the bot's own entry for that patient (a write the
  frame doesn't read). Only collateral records excuse entries now.

* frame: at most count side rows per link of another actor's new chart are theirs

Re-review of the #1570 port, round 2: a side row linked to a colleague's new
chart was theirs with no count, so the bot's extra contact or history row
for that chart ended held. Rows beyond the side record's count are now
unattributed collateral.

* Restore main's changes after #1582 (#1618) dropped by the merge

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 29, 2026
They merged after the 2.4.1 bump and before 2.4.1 published, so they
ship in it. Regenerates llms-full.txt.
abrichr added a commit that referenced this pull request Oct 29, 2026
…ble uncertain-write ledger (#1628)

* frame: key audit entries by a record field, and declare side records

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: diff the whole database, with a reviewed allow-list

frame.database snapshots every table and column before and after the save
(three statements, whatever the table count) and judges each table as a
slice. A row added, removed or changed is collateral unless a declared
effect, a side record or an attributed concurrent writer explains it. A new
declared or side row that sets a column nothing declares is collateral too.
Tables over max_rows are fingerprinted. allow_tables, reviewed per app,
replaces the read-list; an unreadable snapshot leaves the frame not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: read rows wider than one JSON call, keeping null columns

SQLite caps a function at 127 arguments and OpenEMR's chart has more
columns, so the snapshot failed and the frame was not checked. Each row is
now read as JSON objects of at most 50 columns, merged in Python (a JSON
merge patch would drop null values).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a default the database computes at insert is a default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: audit attribution can name a table's record through another field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: a stable Python API for checking one write

openadapt_flow.verifier: capture() reads the system of record before the
write, verify_write() judges it against the same contract and returns
verified, the outcome, which parts were checked, plain reasons, the receipt
and its evidence. It wraps verify_run and checks exactly what it checks; a
snapshot verifies one write. A test pins the public names and signatures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: verified writes for any agent, and taught browser writes with receipts

openadapt verify: a write gets a verified receipt or a refusal, never an
unverified success.
- openadapt_flow.verify (also openadapt.verify): WriteContract,
  begin_write, verify_write and the verified_write context manager, a thin
  public face over verify_run's capture_baseline and verify_run. Any
  executor wraps its own write: a compiled program, an agent in any
  framework, a script or a person.
- `openadapt verify serve --contracts DIR` (and `openadapt-agent serve
  --verify DIR`): MCP tools verify_contracts, verify_begin and
  verify_check. The operator owns the contracts; the agent names one and
  the values it will write.

Taught writes: `serve --teach-browser --contracts DIR [--sign-in NAME=ENV]`.
A task that names a write contract may save to the taught site. Run 1 is
verified in the system of record before taught_<name> exists; every call
makes the write and returns done with a receipt, or needs_review with
REFUTED or RECONCILIATION_REQUIRED. Sign-in values come from the server's
environment and never reach the agent, a result or the program. Controls
in named iframes are listed and found through their frame, and a goto
action stays on the taught site.

The example app gains a sign-in, a New Order form and /api/orders to read
orders back. Quickstart: packages/flow/docs/VERIFY_WRITES.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: link #1589 in CAPABILITIES.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: actor kinds, refuted vs unconfirmed, receipt hash, database diff

From the MCP skills, RL reward and undo sessions:
- Actor(kind, id, build): the receipt records who wrote (program, agent,
  script, person);
- WriteResult.status: refuted (the store shows a different, extra or
  duplicate write) vs unconfirmed (it doesn't show the write, or couldn't
  be read: reconcile before retrying), and receipt_sha256;
- capture() raises BaselineUnreadable when the store can't be read, so a
  caller can stop before writing; write_attempted passes through;
- capture_database() and diff_database(): a whole-database snapshot and a
  per-table diff (rows added, removed, changed, column names, no values),
  over the database's own driver.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: whole-database diff without values, in one consistent read

From the adversarial review of #1582:
- PHI: the baseline keeps keyed MACs (BLAKE2b with the run salt) per cell
  and per row key, and keyed ids of audit entries; plain values are read only
  for rows the save added, in memory.
- Consistency: one connection, one read-only transaction (START TRANSACTION
  WITH CONSISTENT SNAPSHOT, READ ONLY; BEGIN on SQLite), with the audit log
  read inside it when it shares the connection.
- Large tables: salted SHA-256 per bucket of 1024 keys up to the baseline's
  largest key, added rows read in full, so a clean save on a table over
  max_rows holds. A table keeps its baseline representation after the save.
- Silent passes closed: a value is a default only if it equals the declared
  one; a clock default must fall between the snapshots' database clocks;
  ignore_fields apply to surfaces and database.ignore_columns to tables;
  sqlite_sequence and AUTO_INCREMENT catch a write then delete; table
  definitions and triggers, views, routines and events are digested; only a
  single generated key may hold any value; with frame.database, only a write
  event on the same table attributes a change (event_field, write_events,
  insert_events, table_field).
- Allow-listed tables are append-only unless skipped with a reason; every
  visible schema is read; settle_s delays the read for late writes; the
  receipt counts tables compared, append-only and skipped and binds the
  allow-list digest.
- Another actor's later write to the bot's new row, in a column the save
  doesn't declare, is concurrent when the bot only inserted it; a bot write
  to that column still fails. A surface that is a view of a snapshotted table
  is judged through the table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: excuse another actor's change to a declared record only through a write to its table, and never on a record the bot also wrote

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a DEFAULT NULL column (shown as the text NULL) has no default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: a side row of another actor's new record is theirs; report their additions by surface id

OpenEMR logs a contact row with no patient, so no write event attributes it;
its link to a chart another user created in the window does, the same way
side records attach to the bot's chart. FrameVerdict.added_by_others names
such records as their surface does, for checks that read the surface.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: taught writes go through the shared wire check; sealed receipts

From the #1589 review:
- A taught write's requests are pinned (method, path and the digest of its
  field names). Any other write is aborted before it is sent, while
  teaching and on every call, and the call ends in needs_review.
- Sign-in values never travel in an address and never reach the agent in
  any encoding (raw, percent, form, JSON, HTML, base64). They go only into
  a password field or a field of a POST sign-in form.
- Verified receipts are sealed with this computer's Seal key and say the
  values were declared by the agent and whether an operator's
  authorization was bound.

browser_teach now uses openadapt_flow.wire_check (from #1581) instead of
its own copy of the route guard, masking and body shapes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the MySQL snapshot sends no statement that can fail

A driver may end the session on an error, so the snapshot checks for MySQL
8's information_schema_stats_expiry before setting it, instead of trying it
and catching MariaDB's error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: rows by key in every table, attribution by kind and count, MAC key outside the baseline

From the second review of #1582:
- Every table keeps, per row, a 128-bit MAC of its key and of its contents
  (without ignore_columns); cell MACs for all columns up to max_rows (now
  2000), and for the reference columns above it. A changed row in a large
  table goes through attribution by its key; ignore_columns hold; an added
  row is found by key for any primary key. Buckets are gone.
- Attribution: the record is named as it was before the change; an added
  row needs an insert entry, a removed one a delete entry (delete_events), a
  changed one an update; each entry explains at most one row; a row whose
  record key changed is never explained. The log's own limits are documented.
- MACs: BLAKE2b, 128 bits, bound to table and column, keyed by the salt and a
  host secret (OPENADAPT_FRAME_SECRET, or an owner-only key file) that is
  never in the baseline. Audit-entry ids use the same key.
- MySQL: AUTO_INCREMENT is read just before the consistent snapshot starts,
  on the column the database numbers whether or not it is the key (the live
  run's false hold on patient_data); no GROUP_CONCAT; the frame is not
  checked when the read account can't see triggers (it needs TRIGGER); a
  required schema the account can't see is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: the database snapshot's max_rows default follows the frame (2000)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: one versioned OpenEMR bookkeeping profile for the frame, rollback and RL

openadapt_flow.app_profiles.load_profile('openemr'): the tables and columns
OpenEMR writes on its own (logs and id maps append-only; per-session state
skipped with a reason; uuid backfill and the users ON UPDATE stamp as column
rules), from the RL environment's calibration (2026-10-28.1) and the frame's
live witness. No rule may name a patient-data table except for a system
column. frame.database.profile applies it; the receipt names the version,
its digest and the rules that excused something in the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: capture_database takes the app profile the frame uses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: contact is append-only bookkeeping (openemr-frame-profile/2026-10-28.2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: tests use their own host secret; document that a deployment keeps it from the agent

The test suite created the real ~/.config/openadapt/frame.key; an autouse
fixture now sets OPENADAPT_FRAME_SECRET per test. The default key file only
keeps the secret out of the baseline, since a same-user agent can read it:
a deployment sets the secret from a store the agent can't read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: compare credential tables; backfills only; an allowlist; a pinned digest

From the third review of #1604:
- No table is skipped any more. Credentials and access are compared:
  users_secure and login_mfa_registrations excuse only sign-in counters and
  timestamps on the acting user's own row; keys are append-only;
  background_services excuses only next_run and running; user_settings and
  recent_patients only the acting user's own rows (owner by user id, looked
  up in the snapshot).
- uuid rules excuse a backfill only, from NULL to a value; users.last_updated
  only beside one.
- The patient-data denylist is now an allowlist: a rule may name only a
  reviewed bookkeeping table (lower-cased, unqualified); '*' only a system
  column backfill.
- Profiles load only from the package, by name; frame.database.profile_sha256
  pins the digest (bound into frame_sha256), and a mismatch is refused.
- rules_used lists only the rules that excused something in the run.
- A table without a primary key keys its rows without the columns a rule
  lets move. The profile is registered in public-artifacts.json.

Profile version openemr-frame-profile/2026-10-29.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* verifier: capture_database takes before= and actor_names; diff_database applies the profile

Rollback needs the plain values of rows that changed after a run began
(before=, in memory only), and the same judgment the frame uses: with
profile= and the acting user's names, diff_database leaves out a row change
the profile calls the app's own (the user's sign-in counters, a uuid
backfill), so a clean undo isn't reported dirty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frame: the host secret is created whole and race-free on first use

Two processes starting together could read a half-written key file and
MAC the baseline and the read after it with different keys. The secret is
now written to a private temp file and linked into place; the loser reads
the winner's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: a colleague who wrote in the window owns their per-user UI state

own_rows rules (user_settings, recent_patients) also excuse the rows of
another user with a write entry in the window: their own work. Credential
column rules stay on the acting user's row. The users table's ids and names
are read in the snapshot, in memory only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* app profiles: keys and MFA registrations are compared in full (2026-10-29.2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: say when a receipt is unsealed; write-lane button label

From the #1589 re-review: a verified result now carries seal (sealed, off,
or unsealed with why the key couldn't be used) instead of a silent null
seal_path, and the write lane lists POST buttons as sending only the
contract's writes rather than refused.

* fix: the teach browser blocks service workers

Their requests skip context routes, so the wire check couldn't see them.

* frame: narrow types for mypy (renames only, no behaviour change)

* docs: refresh llms-full.txt for the verifier API row

* test: emit's no-mcp-import check starts without mcp loaded

Another test in the same process (the verify MCP tools) imports mcp, so
the check now clears it first and sees only what emission imports.

* frame: record_key refs never excuse what they don't name

Adversarial review of #1570 found three silent passes and one more held:
- a row whose record_key changed (a note refiled to another patient) was
  matched by its new value only; it now names neither and is collateral;
- a bot entry was set aside when its ref equalled a moved row's id; moved
  rows are now compared by their record_key value;
- a declared record with no record_key value added "None" to the declared
  set, excusing every bot entry without a patient; the frame is now
  not_checked;
- a side record linked to an existing record's id was part of the save; a
  link already in the baseline is no longer a new record's.
Nothing changes when record_key and side_records are unset.

* verifier: a snapshot is single use across reloads and copies; errors are VerifierError

Adversarial review of #1588: verify_write copied each snapshot to a fresh
temp file, so Snapshot.load twice, a payload copy, or a baseline the CLI had
already used each verified a second write (an RL replay hole). A process-wide
set of used salts now refuses copies, and a loaded snapshot verifies through
its own file, so verify-run's .used marker applies. VerifyRunError (unbound
params, another contract's snapshot) now surfaces as VerifierError, the
docstring says a refused call uses up the snapshot, and repr hides the
payload.

* test: plain setup for the other-contract snapshot case

* frame: a side row must be filed under its parent's record_key

Re-review of #1570: with link_field other than record_key, a side row
linked to the new chart but filed under another patient was part of the
save, and its record_key value excused that patient's bot log entries.
It is now an ordinary unexplained addition.

* frame: a side row with no record_key value can still be its parent's

* verifier: single use survives copies and other processes

Re-review of #1588: the used-salt set lived in one process, so a copied
file, a load-save-load to a new path, or a snapshot pickled to a worker
verified a second write in a new process. A use is now also recorded as an
O_EXCL file named by the salt's digest in a private per-user directory,
checked on every call. A missing salt, a non-dict payload and an OSError
raise VerifierError with their own messages; the docstring says an
unconfirmed result also uses up the snapshot. Snapshot's fields and
load/save signatures are pinned.

* frame: with record_key, side records link by it; only collateral excuses

Re-review round 3 of #1570 and of its port to #1582:
- the parent-ref rule let an extra side row filed under another patient
  fall through to attribution, where a colleague's entry excused it. With
  record_key set, a side record must now link by record_key on both sides
  (validated), so a side row's link and its record_key are the same value;
  the parent-ref rule is gone.
- moved records compared by record_key let another user's concurrent change
  to a patient excuse the bot's own entry for that patient (a write the
  frame doesn't read). Only collateral records excuse entries now.

* frame: at most count side rows per link of another actor's new chart are theirs

Re-review of the #1570 port, round 2: a side row linked to a colleague's new
chart was theirs with no count, so the bot's extra contact or history row
for that chart ended held. Rows beyond the side record's count are now
unattributed collateral.

* verifier: capture issues a record, verify removes it

Re-review round 3 of #1588: the used-salt marker lived under the caller's
HOME, so a worker with another HOME (or none, another user, container or
host) replayed a used snapshot. capture() now records each snapshot as
issued (an O_EXCL file named by the digest of its salt bytes) and
verify_write() removes it; only one caller can, and a snapshot with no
record is refused, so another state directory fails closed instead of
replaying. Markers no longer pile up. OPENADAPT_VERIFIER_STATE_DIR gives
every worker one directory; it must be a private directory owned by the
user. The salt is keyed by its bytes, not its hex case, and the in-memory
payload whose salt was used is what's judged.

* verifier: the state root is private too; document capture-once and the state dir

Re-review round 4 of #1588: check the state root as well as issued/ (T6),
and say in the module docstring that the judged writer must not reach the
state directory (T5), that a snapshot verifies only under the state dir it
was captured under, that the caller captures once per write (T4), and that
an unverified capture leaves its record (T7).

* docs(flow): one verifier API and the uncertain-write ledger (design)

* feat(flow): one verifier API with three outcomes, and a durable uncertain-write ledger

openadapt_flow.verifier is the one public API for checking a write:
begin_write(contract, params, actor=...) then verify_write(pending). Every
result has an outcome (CONFIRMED, REFUTED or UNCERTAIN) and a scope: the
read path, the stores read, the time window, and what it doesn't cover.
openadapt_flow.verify (#1589) is a deprecated wrapper with the same
signatures; #1588's capture/Snapshot is replaced.

begin_write consults an append-only ledger keyed by the write's identity
(an HMAC over the contract, its bound values and an optional write key)
before anything is sent. A flock lease shows whether another worker holds
the write; a free lease on an open attempt means its owner died between
the send and the read-back, and reconciliation judges it against its own
baseline before any resend. REFUTED and UNCERTAIN attempts stay held until
a read-back resolves them or a person releases them
(openadapt verify ledger list, reconcile, release).

* verifier: delivery is the caller's call, per-value declared_by, isinstance kept in the wrapper

* verifier: fix the adversarial review's findings

- B1: key a write on Contract.identity (effects, unbound, no read settings),
  so a contract edit between a crash and a restart can't drop the hold.
- B2: a landed write without a write key is handed back to an identical call
  for repeat_s (10 min); verify_begin takes an optional write_key.
- B3: reconciliation never confirms when another attempt with the same values
  began after this one.
- B4: a frame breach that other ledger attempts in the window explain gives
  UNCERTAIN with landed=true (not held, never resent), not REFUTED.
- S1: an attempt's values and baseline are removed once it resolves for good.
- S2: ledger failures raise WriteHeld; S3: a released pending write doesn't
  record; S4: browser_teach checks for sends after closing; S5: values are
  normalized before keying.

* verifier: second review round

- N1: Contract.identity includes the store (verifier kind and location,
  environment_id, no credentials, loopback normalized).
- N2/N3: verify_write and reconcile never confirm when another attempt with
  the same values was open during the window (sub-second ledger times; the
  same key's predecessors are excluded, the lease orders them).
- N4: only attempts on the same store that landed in the window, or hold a
  live lease, excuse added records; the result is landed + review and stays
  listed for a person.

* verifier: third review round

- R1: a released attempt is free, also when it was still open.
- R2: released attempts are closed for the overlap checks; an open attempt
  counts while its owner is alive or for the contract's window.
- R3: a landed write under review is handed back until a person releases it,
  so a later attempt never hides it from the list.
- R4: store addresses are normalized (scheme, host case, default port,
  trailing slash); the same effects and values open under another address
  hold the write.
- *_env names stay in the store identity (tenants by credential); the CLI
  list shows the repeat-window state and review.

* verifier: fourth review round

- R5: the same write under another address for the store is held whatever
  its state (in flight, unresolved or made), never resent.
- R6: that hold applies only within one verifier kind, environment_id and
  host, so another environment is never held.
- An attempt released before any outcome never counts as overlapping.
- release takes --found absent|landed; landed hands the write back for good.

* verifier: fifth review round

- Contract.shape includes SQL server host, port and database names, and
  *_env credential names, so two servers or tenants never hold each other.
- Ledger.release requires found=absent|landed.
- Document the host-alias limit and per-store environment_id.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 30, 2026
…1634)

* Compensating rollback after a stop, proven by a whole-database diff

Squashed from #1590 (claude/compensating-rollback) onto main now that the
frame stack (#1570, #1582, #1599, #1600, #1604) and the verifier API (#1588)
it was built on have landed. Only the rollback's own changes are carried;
the stack code it had merged is main's.

rollback, rollback_state, rollback_qualify, cli_rollback; the deployment's
rollback: section; RunReport.rollback; UndoStep kinds; MockMed and OpenEMR
harnesses and results; ROLLBACK.md, LIMITS, CAPABILITIES.

* rollback OpenEMR harness: record cross_checked (profile_rows is gone)

* rollback: address the adversarial review of #1634

Blocking:
- A row added or removed is set aside only on another actor's insert or
  delete in the audit log; another actor's change to any row the run could
  have written is a conflict, in any table. The verifier cross-check always
  runs and may see changes only where a row was set aside or kept.
- Without attribution nothing is compensated, except on a declared test
  copy (only the run writes there).
- nothing_landed and rolled_back need the change ledger to agree: a
  confirmed write the read can't see, or a write left uncompensated, ends
  RECONCILIATION_REQUIRED. Only a whole-database nothing_landed overrides
  the run's own uncertainty.

Should fix:
- diff.json is written owner-only even over a leftover staging file
  (O_EXCL, O_NOFOLLOW); the rollback directory is 0700.
- Audit attribution reads write events only, per table and by kind, as the
  frame does.
- Rollback on a desktop, RDP or pool run is refused, not silently off.
- An unreadable first read refuses the run.
- Compensations run with the run's item context and without the
  undo-or-person check (a compensation is the approved undo).
- A failed verifier cross-check proves nothing; a kept void row must have
  changed since the stop.

MockMed live rerun through the CLI: after-save ROLLED_BACK, before-save
HALTED_BEFORE_EFFECT, unapproved refused before the run.

* rollback: address the round-2 adversarial review of #1634

Blocking:
- Rows set aside as another user's must be exactly as the stop left them
  at the final read; another user's write naming a run row's record (the
  same patient, any table) is a conflict before compensating.
- Audit attribution without record_key, event_field, write_events,
  insert_events and table_field is refused (it named no row: silently off).
- After the compensations, another user's write to a run row in the audit
  window means RECONCILIATION_REQUIRED.
- A last-writer field (record_field) never sets a row aside: any other
  user's write is a conflict.
- A rolled-back run is never resumed (its rollback directory marks it).

Should fix: removed rows are never set aside; a row whose record column
moved is never another user's; the verifier cross-check compares row
counts per table; exclusive needs rollback.exclusive on a declared test
copy; the rollback directory is 0700 for every rollback and never a
symlink; clearer conflict wording.

* rollback: address the round-3 adversarial review of #1634

Silence is never "no one else":
- An audit log that can't be read before the run refuses it; one that
  can't be read after the stop or the rollback compensates nothing.
- A run row attribution can't speak for (no record value, no last-writer
  value) is a conflict unless the store is exclusive.
- A rolled-back run is refused inside runtime.durable.resume.resume() and
  at the attended console's continue, reconcile and skip, once a
  compensation has started (rollback/compensated); a pause where nothing
  was compensated stays resumable.

Should fix: the store is read again before each compensation; a set-aside
row may change again only by that same other user, compared by row
fingerprint (profile-ignored columns aside); the final pass applies the
moved-record-column guard; a last-writer ROLLED_BACK says what it can't
see; docs name the per-user bookkeeping misfire.

* rollback: compensate only while the store holds the run's own changes

Round 4 of the adversarial review found that every way of setting other
users' rows aside opened a new hole. Remove set-aside entirely:

- A compensation runs only while every row changed since the first read
  is positively the run's own (Attributor.is_own: the audit log holds the
  run's write for the row's record and table and no other user's; or the
  last-writer field names the run's user). Checked at the stop, on a full
  read before and after each compensation, and in the audit window at the
  end. Anything else ends RECONCILIATION_REQUIRED with the diff.
- A compensation that changes a row the run didn't write stops the
  rollback right after it, with the diff.
- Durable (pausable) runs with rollback on are refused; attribution whose
  own user can't be resolved refuses the run at arm.
- RollbackRecord.other_actor_rows is gone.

Fail-closed on a busy store, documented in ROLLBACK.md and LIMITS.md.

* rollback: address the round-5 adversarial review of #1634

- Any stop in the compensation loop (a run row moved, a change that isn't
  the run's, a compensation that changed a row the run didn't write) ends
  not_rolled_back, with the read that caught it as the diff.
- A last-writer field vouches only for a run with one write: with two,
  another user's edit between them would be hidden.
- An audit log's logged inserts must account for the run's new rows per
  record and table.
- A database id counter (AUTO_INCREMENT, AUTOINCREMENT) may move by the
  rows the run's compensations deleted, and no more: a delete can now
  roll back on such tables.
- Durable refusal is a tested helper and names the Standard and Regulated
  profiles. Docs: the audit-log assumption, the bot account used by the run
  alone, and what a rollback can't see during a compensation.

* rollback: audit log only; counters at every read; the person gets the store as it is

Round 6 of the adversarial review:
- A last-writer field (record_field) is refused for rollback (deployment
  validation, RollbackPlan, and roll_back itself): it can't show another
  user's edit between two of the run's writes, including a reversible step
  that writes without a ledger entry. Compensations need the app's audit
  log, or a declared exclusive test copy.
- An id counter moved only past the run's own deleted rows is excused at
  every read, not just the final one: two deletes on AUTO_INCREMENT tables
  now roll back.
- A stopped rollback hands over the final read plus any rows a
  compensation took that no read shows, never an older read.

* rollback: a step with no declared effect is judged by the whole-database reads

OpenEMR live (stop before the save, shared app profile): the store showed
nothing changed, but three 'press Enter' ledger entries stayed unconfirmed
with no effect and no compensation, so nothing_landed was refused, and
after a real save the rollback would stop at them. Under a whole-database
read such a step is judged by the reads: unchanged at the stop means it
wrote nothing; after compensations, anything it wrote is residue. Writes
only to a reviewed allow-listed table or a profile-excused column stay
unseen (documented). Confirmed writes and steps with declared effects keep
the strict rule.

* rollback: address the round-7 review of #1634 (approved; should-fix items)

- A late write found in the audit window hands over the final read plus
  the rows that user wrote, as the stop left them, not the after-stop read.
- Without a fresh session (an attached browser) only an API-only
  compensation counts as an undo: the undo-or-person rule and the gate
  send a save with a screen compensation to a person.
- The ROLLBACK.md example uses OpenEMR's real audit query.
- Nits: the unreachable last-writer clause is gone; the counter bound no
  longer counts the unread pseudo-row.

* rollback: address the round-8 review of #1634

- An effect-less, unconfirmed ledger step is judged by the reads only with
  the audit log's agreement: every write the run's user logged must be a
  row the read shows changed (insert entries naming no record in the
  profile's bookkeeping tables aside, as the frame does). A write to a
  table the read keeps append-only (a disclosure log) is caught by its log
  entry. Without an audit log the old rule holds.
- 'Declares no effect' checks the ledger entry, the step and its API
  binding.
- Rows another user wrote while the rollback ran go to rollback/late.json
  (late_diff_path); remaining_diff stays the final read.
- The ROLLBACK.md example no longer allow-lists a table OpenEMR may lack.
- The OpenEMR harness reads OpenEMR's own audit log for the profile case.

* rollback: address the round-9 review of #1634 (approved; should-fix items)

- The audit check mirrors the frame: an insert naming no record (none or
  patient 0) in a bookkeeping table, and the acting user's own-row tables,
  are the app's own.
- The check applies to the whole run: with the audit log, a write by the
  run's user the reads don't account for (any step, any table) never ends
  nothing_landed or ROLLED_BACK.
- A not_rolled_back rollback is always RECONCILIATION_REQUIRED, whatever
  the run's own evidence says.
- The example and the OpenEMR harness list replace among write events.

* rollback: address the round-10 review of #1634 (approved; should-fix items)

- The hidden-write check runs on an exclusive store too: exclusive is
  about other users, not about writes the reads can't see.
- An own-rows table is excused only where the read compares it row by
  row (never an allow-listed one), matched by exact table name.
- A not_rolled_back rollback keeps the run's own outcome only when every
  changed row is positively another user's (colleague_only); otherwise a
  person reconciles.
- Clearer reason for a logged write no changed row accounts for.
- End-to-end tests for both hidden-write checks, the colleague-only
  outcome and the allow-listed own-rows table.

* rollback: address the round-11 review of #1634

- Drop colleague_only: a colleague's logged write to a record can't rule
  out an unlogged write by the run to the same record, so every
  not_rolled_back rollback is RECONCILIATION_REQUIRED.
- With audit attribution, nothing_landed also needs a readable audit log
  showing no hidden write by the run's user.
- Tests for the masking case, the unreadable log at the stop, and the
  final check on an exclusive store (each fails if its guard is removed).
- VERIFY_RUN.md's OpenEMR example lists replace among write events.

* rollback: say when the audit log couldn't be read (after the stop, or after the rollback)

* rollback: the audit check counts only writes to tables the read doesn't compare

OpenEMR live (stop before the save, shared profile, OpenEMR's own audit
log): the store showed nothing changed, but the bot's logged sign-in
bookkeeping (a users uuid backfill the profile excuses) counted as a hidden
write, so nothing_landed was refused. A write to a table the read compares
row by row can't hide: the diff shows it, or it touched only what the
reviewed profile calls bookkeeping. The check now counts only entries for
tables kept append-only, allow-listed, skipped or not read, plus the
frame's no-record bookkeeping inserts exception. The own-rows special case
is subsumed.

* rollback OpenEMR harness: record the logged tables the reads couldn't judge

* rollback: address the round-13 review of #1634

The 'compared tables can't hide a write' rule was false (a write landing
after the read, a uuid backfill over max_rows, a same-named table in
another schema). Replace it:

- A logged write by the run's user is accounted for only by a row the
  reads show in exactly the table it names: one that moved, or one the
  app profile excused (now recorded in TableDiff.excused), by record; an
  entry naming no record by any excused row there. As in the frame, a
  no-record insert in a bookkeeping table is the app's own, and so is a
  write to a bookkeeping table the profile compares only in part.
- Over max_rows (cell MACs for some columns only), the profile excuses
  nothing.
- Table names resolve exactly: an unqualified entry never matches another
  schema's table.
- The harness records the logged tables nothing accounts for.

* Revert "rollback: address the round-13 review of #1634"

This reverts commit c7f73a2.

* Revert "rollback OpenEMR harness: record the logged tables the reads couldn't judge"

This reverts commit baaab9e.

* Revert "rollback: the audit check counts only writes to tables the read doesn't compare"

This reverts commit fa0f80f.

* rollback: record the OpenEMR stop-before-save rerun on the approved head

Both rows end RECONCILIATION_REQUIRED: without attribution nothing is
compensated; with the openemr profile the read shows nothing the run did, but
the effect-less ledger entries aren't judged because OpenEMR logs its own
bookkeeping under the run's user. Fail-closed; no code change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant