Skip to content

generate-parts: the app-specific parts of a workflow, generated and verified - #1520

Closed
abrichr wants to merge 4 commits into
mainfrom
claude/generate-parts
Closed

abrichr wants to merge 4 commits into
mainfrom
claude/generate-parts

Conversation

@abrichr

@abrichr abrichr commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Draft, parked. Before review it needs two things: the live OpenEMR back-test (queued for the shared fixture slot), and approve signing with a registered approver key on the #1486 path instead of a typed name. The pagination engine fix moved to #1538; this branch drops it when #1538 merges.

openadapt-flow generate-parts builds the four parts of a workflow that are specific to one application at setup time. Each is admitted only by a check that does not depend on the generator. Healthy runs still make no model calls, and nothing in this PR calls a model. Design: packages/flow/docs/GENERATED_PARTS.md.

Generated, and how each is admitted

Part Generated from Admitted by
Read channel (generate/channel.py) OpenAPI document, FHIR CapabilityStatement, or teach-time traffic Channel test. It shows each demonstration's record exactly once and reads every page. Through a fault proxy it must read as unreadable, never as an empty or partial set: no auth, 401, 500, truncated body, not JSON, a failed next page
Record contract (generate/contract.py) Before/after difference across 2-3 varied demonstrations: bound fields, constants, volatile fields, match keys, frame The fault gate (#1445) on a test copy; each surviving fault class adds the check that kills it, until all are killed or reported
Identity (generate/identity_template.py) A reviewed template (patient-identity/1: #1516's record check, never generated) plus a generated mapping of this system's fields and the task's parameters onto it The demonstrations (each saved record reads back as typed) and a perturbation suite from the site's own records, with one case per slot only that slot can catch; adopted only if it does no worse than the site's hand-written adapter
Input types (generate/inputs.py) Sample source documents: labels, dates and the source's date order, code tables (Massachusetts -> MA) Round trip of every sample; an ambiguous date or an unknown code halts
  • The floor (generate/baseline.py): every typed parameter is checked, the frame is on, nothing ignores or drops a written field, no collateral rule is off, and there is no attribution. Generation can add checks, never remove one.
  • Never generated: the safety policy (profile, approvals, switch, consequential classification, gate order, attribution).
  • Review: parts.review.md states every check in plain words, including what it can't catch. generate-parts approve needs a person at a terminal. It refuses an agent's name and any part that changed after generation, because the approval is bound to the manifest digest.
  • Hostile content is data, with a test for each:
    • an "instruction" in an OpenAPI description;
    • untouched records filled with instructions and changed every round, which still can't hide the email check;
    • an instruction line in a referral document.
  • Verifier fixes found by the channel test:
    • The FHIR read used to read only the first page. It now follows next links on the same origin, and a failed page, another origin, more than 50 pages, or a total larger than what was read all read as unreadable.
    • The REST read can follow records_next_path (new, optional; digests unchanged when unset).

Credit

FHIR/REST pagination also proposed by @yunusarda-sudo in #1511 (superseded by this PR).

Changelog entry (for the next openadapt-flow release)

  • openadapt-flow generate-parts generates the app-specific parts of a workflow (read channel, record contract, identity mapping onto a reviewed template, input types) from two or three demonstrations, admits each by a check that doesn't depend on the generator, and records a person's approval of the plain-language review (#1520). FHIR reads follow every next page on the same origin, REST reads follow records_next_path, and a broken, cross-origin or short page reads as unreadable. FHIR/REST pagination also proposed by @yunusarda-sudo in #1511 (superseded by this PR).

Measured

Identity: a generated mapping onto the reviewed template, against #1516's hand-written adapters. #1516's people and its 676-item stream (223 true duplicates, 440 new patients) are rendered in each site shape. The mapping is generated from the site's rows and three demonstrations (benchmark/generated_parts/identity_mapping_backtest.py).

Site shape Suite misses, of 580 (reference) Adopted Stream misses, of 223 (reference) Stream false stops (reference)
#1516's rows 5 (5) generated 24 (24) 2 (2)
OpenEMR REST rows 5 (5) generated 24 (24) 2 (2)
FHIR Patient 5 (5) generated 24 (24) 2 (2)
FHIR read by first list item only 30 (5) reference 30 (24) 2 (2)

The generated mapping matches the hand-written adapters exactly on all three shapes. The fourth row loses the email and the prior name. The suite catches that where the demonstrations can't, and the adoption rule keeps the reference.

A rule calibrated on one site's records (the approach the template replaces): at every false-stop budget it missed 52-54 of 223 against the hand-built 24, with 0 false stops against 2. With 30 records, calibration can't learn which near-matches are one person. The frontier, and both adoption rules (--reference-identity, --reference-duplicate-check), are in the README.

Not in this PR

The live OpenEMR back-test is not in this PR; it waits for the fixture lease (autogen-backtest is in the queue). It will generate all four parts from three demonstrations on the fixture and run the fault gate on the generated and the hand-built contract with the same copy. It will report coverage per class, false stops, cost and time, review effort, and setup effort.

Tests

  • New: test_generate_{contract,channel,inputs,identity,identity_mapping,parts_pipeline}.py (37 tests).
  • 283 passed locally across those plus the verifier, FHIR, REST, deployment, verify-run, fault-gate, frame, provenance, intake and command-map suites.
  • ruff and mypy clean on the changed modules; source-boundary check OK.

🤖 Generated with Claude Code

abrichr and others added 4 commits October 24, 2026 20:52
…erified

openadapt-flow generate-parts builds the four parts of a workflow that are
specific to one application at setup time, and admits each only by a check
that does not depend on the generator. Healthy runs still make no model
calls; nothing here calls a model.

- Read channel (generate/channel.py): candidates from an OpenAPI document,
  a FHIR CapabilityStatement or teach-time traffic. The channel test reads
  each demonstration's record, every page, and must read as unreadable
  through a fault proxy (no auth, 401, 500, truncated, not JSON, a failed
  next page).
- Record contract (generate/contract.py): from the before/after difference
  across 2-3 varied demonstrations (bound fields, constants, volatile
  fields, match keys, frame), refined on a test copy by the fault gate until
  every fault class is killed or reported.
- Duplicate rule (generate/identity.py): roles from the schema, clauses
  calibrated on the system's own records with a generic error model and
  per-comparison false-match rates; a person labels the pairs that block a
  clause; unknown goes to a person.
- Input types (generate/inputs.py): labels, dates and the source's date
  order, code tables, admitted by a round trip; ambiguity halts.
- A floor generated parts can't go below (generate/baseline.py); a plain-
  language review, a digest manifest and a person's approval bound to it
  (generate/build.py, generate/cli.py).
- Verifiers: the FHIR read follows next links on the same origin and
  refuses a short read; the REST read follows records_next_path.
- benchmark/generated_parts: the generated duplicate rule on #1516's stream
  (676 items, 223 duplicates): 54 missed, 0 false stops, against the
  hand-built check's 24 and 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A missed duplicate costs far more than a false stop, so the duplicate
rule is now chosen for the fewest calibration misses within an estimated
false-stop budget (a knapsack: gain-first and gain-per-cost greedy, the
better kept). New signals: a generic, model-made table of given-name
variants (common names only; no record is sent), the medical record
number as an exact match, and typos in both names in the error model.

Adoption rule: a generated rule replaces a system's hand-built duplicate
check only if it misses no more on the same calibration set (identity.adopt;
generate-parts build --reference-duplicate-check). On #1516's stream it
does not (it misses 45 or more of 604 calibration duplicates where the
hand-built check misses 3), so the hand-built check is kept. The frontier,
the reasons and every arm are in benchmark/generated_parts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibrating a matching rule on one site's few dozen records can't learn
which near-matches are one person, so identity is now a reviewed template
(patient-identity/1: #1516's record check, never generated) fed by a
generated per-site mapping: which record fields feed each slot (given,
family, prior last name, date of birth and its format, MRN, phone, email,
street) and which task parameter feeds each input. Fields are chosen by
grounding on the demonstrations, the slot's name and its value shape; FHIR
coded lists are read by code (telecom[system=email].value).

The mapping is verified against the demonstrations and a perturbation
suite built from the site's own records, with one case per slot that only
that slot can catch, and adopted only if it does no worse than the site's
hand-written adapter (adopt_mapping; build --reference-identity).

Back-test on #1516's people and stream, in three site shapes (#1516's rows,
OpenEMR REST, FHIR): the generated mapping matched the hand-written
adapters on the suite and the stream (24 of 223 missed, 2 false stops). A
FHIR mapping read by first list item only (no email, no prior name) misses
30 suite duplicates against 5 and is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- baseline: a contract without a frame, or a frame without an identity
  field, is below the floor.
- adoption: a generated rule or mapping must catch every duplicate the
  reference catches (a superset), not just miss fewer; the README says
  the yardsticks are the generator's own synthetic sets and lean toward
  generation.
- a REST next link to another origin reads as unreadable (test); the
  FHIR one had a test.
- CI: import order in rest.py; generate-parts channels' URL flags in
  BACKEND_PARITY.md; the approve hint uses the CLI spelling helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr

abrichr commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Closing: its parts (read channel, record contract, identity-field mapping, input types, per-part admission checks) are folded into the zero-touch onboarding design in #1753, which validates generated setup against planted faults in a sandbox.

@abrichr abrichr closed this Nov 3, 2026
abrichr added a commit that referenced this pull request Nov 4, 2026
…n answers (#1753)

* Zero-touch setup: literature review, design, sandbox validator prototype (WIP)

* Zero-touch: fault_gate class names, #1520 decisions, now_checks docs, capability row

* Zero-touch gate: fail closed on unplanted faults, fault every column, require core classes, strict pins, production equals validated

* Zero-touch: type-made plants for typed columns; strict_pins pins a one-demonstration constant; docs

* Zero-touch: two-phase sandbox restore; a duplicate the store's unique key refuses is not applicable (local, for review)

* Zero-touch: a duplicate is not applicable only when a declared unique index covering unchanged columns refuses it; recorded per kind

* Zero-touch round 2: failed reads never count as stopped; found keeps only constants of strict roles; validated field sets only; key and unrelated-edit faults on creates

* Zero-touch: plants avoid declared unique keys; failing cases log their reason

* Zero-touch design: failed reads, forbidden duplicates, validated field sets, strict pins

* Zero-touch round 3: fresh app-made unique values in duplicates; transactional restore stops the setup; structured read evidence; plain unique indexes only

* Zero-touch round 4: task-given columns come from the generator's bindings, not value matches; stopped setup keeps old contracts aside and exits non-zero

* Zero-touch: typing-only fixes for mypy (no Optional reassignment, one type per variable)
abrichr added a commit that referenced this pull request Nov 4, 2026
* Engine generality audit; move the benchmark harnesses out of the wheel

Audit every place the engine (packages/flow/openadapt_flow) names a
specific application, bucketed as example, data, harness in the package,
or real special-casing: docs/GENERALITY_AUDIT.md.

Move openadapt_flow/benchmark and openadapt_flow/instrument to
packages/flow/benchmark/harness (import benchmark.harness). The wheel
packages only openadapt_flow, so it no longer ships them. The benchmark
command loads the harness from a source checkout and says so from a
wheel; instrument already failed to import from a wheel.

Reword two comments that named an app on a generic branch.

* Generality audit: link CONTRIBUTING without the guard PR's anchor

* Engine generality audit; move the benchmark harnesses out of the wheel

Audit every place the engine (packages/flow/openadapt_flow) names a
specific application, bucketed as example, data, harness in the package,
or real special-casing: docs/GENERALITY_AUDIT.md.

Move openadapt_flow/benchmark and openadapt_flow/instrument to
packages/flow/benchmark/harness (import benchmark.harness). The wheel
packages only openadapt_flow, so it no longer ships them. The benchmark
command loads the harness from a source checkout and says so from a
wheel; instrument already failed to import from a wheel.

Reword two comments that named an app on a generic branch.

* Generality audit: link CONTRIBUTING without the guard PR's anchor

* Move test fixtures and demo harnesses out of the openadapt-flow wheel

The wheel ships only the engine (see packages/flow/docs/GENERALITY_AUDIT.md).
Moved out of openadapt_flow:

- integrations/harness2 -> benchmark/harness/harness2 (the harness2 extra
  is dropped; install [browser] beside Harness² from a checkout)
- mockloan/ and demo_driver.record_disbursement_demo ->
  benchmark/harness/mockloan/ (recorder in demo.py)
- guard_demo.py -> benchmark/harness/guard_demo.py
- agent/ledger_demo.py -> packages/agent/tests/ledger_demo.py
- mockmed/teach_scope.py -> packages/flow/tests/_teach_scope.py
- learning/synth_stream.py: StructuralDiffInducer -> learning/induce.py;
  the MockMed drift stream -> benchmark/harness/synth_stream.py

reward/seed.py is split: the bundle helpers (write_bundle, calibrate_bundle,
self_signed_certificate) are now reward/bundle.py. The MockMed seed stays,
because serve-reward --seed-mockmed needs it from a wheel.

* Point the new teach-authorization test at tests._teach_scope

* Restore #1741's packaging excludes, lazy-import guard and sys.path.append lost in the main merge

* fix(diff): never apply a column rule to a row whose columns have no cell MACs (#1793 item 1) (#1801)

Over max_rows a table keeps cell MACs only for its reference columns, but each
row's contents MAC covers every column. A uuid backfill beside a clinical
edit in the same row then showed only the uuid, and the profile's
uuid-backfill rule excused the whole row in the rollback diff, the verifier's
diff_database and the verify-run frame. Each of the three now applies a
profile rule only on a table where every column has a cell MAC
(db_snapshot.every_cell_hashed). Adds no excuse; only removes one.

* API span: re-read the checks after step writes; keep an earlier halt reason (#1788)

* API span: read the checks again after step writes; keep an earlier halt's reason

The checks before saving ran in prepare(), before the span's step writes
went out in send(). The GUI checks its empty-result guard after those
writes, right before the save. When the span has step writes, send() now
reads the checks again after the last write and halts without sending the
save if one no longer passes.

The end-of-run read of a still-armed wire check replaced the last step's
error and category. It now adds the late refusal to an existing error and
keeps an existing failure_category.

* Test B: assert the span's own message, and pin category retention on a governed halt

* fix(rollback): an audit entry names exactly one snapshot table (#1793 item 2) (#1803)

rollback_state._same_table matched an unqualified log entry to any
schema.table with that name. A rollback snapshot keys the read's own schema's
tables unqualified and every other schema's as schema.table, so that let a
default-schema entry credit (attribute, others_touched, _actors) or account
for (unaccounted_writes) another schema's row. It now matches exactly, case
aside. Every use only gets stricter: an entry that names no compared table
attributes nothing, and the rollback goes to a person. The verify-run frame
keeps its own matcher: its config names tables schema-qualified.

* ci: skip the required reporters on a green run; balance Flow's unit shards (#1800)

* ci: skip the required reporters on a green run; balance Flow's unit shards

On 2026-11-03 a ci run had five serial stages, and each waited 15 to 30
minutes for a runner. ci and the two legacy required names did no work but
added two of those stages. They now need packages, launcher and smoke
directly and run only to fail the check (on a failure, a cancel or a draft).
On a green run they are skipped, which GitHub counts as passing. The inner
packages-gate still runs.

flow-unit dealt files round-robin, so one run's shards took 4.9 to 9.4
minutes, and -n auto started 2 xdist workers on the 4-vCPU runner.
scripts/ci/flow_shards.py packs files by measured time, longest first. Pull
requests run 3 shards instead of 4, and flow-unit and flow-critical run one
worker per vCPU. Every file still runs exactly once.

CONTRIBUTING asks for one push per batch of fixes, a merge of main only on
conflict, and gh run rerun --failed instead of empty commits.

* ci: flow_shards review fixes

Parse gh run view --log lines and colour codes, refuse to write empty
durations, print nothing for an empty shard, and require every other ci.yml
job in the reporters' needs.

* Verifier baselines: OpenAdapt's verifier against a UV-class judge and API read-back (#1794)

* Verifier baselines: pre-register the protocol and pin the inputs, before any scored run

* Verifier baselines: UV-class judge port, effect_e2e arms, scorer, and results so far

* Verifier baselines: report, final scores, and per-run data moved to the private corpus

* Verifier baselines: score our verifier from the corrected Medplum replay (debdfe3)

* Rollback: check desktop start screens through the step resolver (#1699) (#1799)

* rollback(desktop): check start screens through the step resolver

The desktop fresh session checked the run's start screen with a guard
predicate, which resolves anchors through the visual ladder only. After a run
the start screen's fields hold data, so the recorded crop no longer matched
(measured on a 2880x1800 Windows lab: the start check refused with nothing
covering the app). The start checks now use the same resolve() ladder a step
uses: on UI Automation or AT-SPI its structural rung finds the same control
and refuses one that is covered or off screen; pixel-only backends keep the
visual ladder. Guard predicates in general are unchanged (their receipts are
re-verified offline from frames). Qualified cleanup steps still run first.

* lab(rollback): host profiles, operator review of selection keys, UIA lookup timing

The Windows lab harness reads its guest paths, Python and RDP credentials from
the lab's host profile (winrm_lab, #1789), so one script runs on both labs.
It records each UIA lookup's timing, and applies the operator's risk review
(qualification.set_action_classification) only to text-selection keys with no
effect, recording each reviewed step and the rule.

* rollback(desktop): a structural start hit counts only when nothing covers the control

Review fixes for #1799. UI Automation and AT-SPI find a control behind a
dialog or another app, so a structural hit alone doesn't show the start
screen. It now counts only when a screen-wide hit test at its point names the
same control (WindowsBackend: ControlFromPoint, hit_test_sees_every_window),
and on any other backend only when the pixels show it too. A start state that
isn't an anchor keeps the halt cleanup's guard. Each check keeps its frame and
verdict under rollback/.fresh-session-N.

* Linux AT-SPI element identity and CI qualification; macOS qualified on calibre (#1781)

* WIP: Linux AT-SPI element identity, macOS Qt AX fixes, real-app qualification harnesses

* Linux: AT-SPI read-back, physical input under a window manager; macOS: Qt cells, row identity; harnesses

* Review fixes: AX splice race and UTF-16, macOS read-back keeps cell text, census reads rows, no alert dialog role; docs

* Linux AT-SPI qualification results (fixture; GTK3 chart: record, compile, governed run, faults, repair)

* Docs and results: macOS calibre qualification, Linux CI qualification, capabilities

* fix(flow): an ISO date typed into a native date input lands as that date (#1806)

* fix(flow): an ISO date typed into a native date input lands as that date

Chromium splits <input type=date> into locale-ordered segments, so a typed
"1984-03-07" doesn't land: "-" doesn't move to the next segment. type_text and
type_text_guarded now set an ISO date into a native date input with fill,
which takes the ISO value whatever the locale and fires input and change.
The guarded path fills the leased element itself, and a failed fill raises
(ActionDeliveryUncertain) instead of falling back to typing. Other text and
other fields are typed as before.

Salvaged from #1548 (second installation, Phase A).

* fix(flow): date fill only for a real ASCII ISO date into an editable date input, through frames and shadow roots

Review findings on #1806: fill clears a date input before rejecting a
malformed value, so 2001-02-30 and non-ASCII digits are typed as before and
never reach fill; readonly and disabled date inputs are typed as before; the
guarded check and fill run on one element handle with a 1 s timeout; the
unguarded path finds the focused date input inside frames and open shadow
roots, and checks and fills that same handle.

* Empty-result guard: mine a dialog's statement in any language from agreeing demonstrations (#1770)

* Empty-result guard: mine a dialog's statement in any language from agreeing demonstrations

A dialog line outside the English vocabulary is kept as a candidate by the
compile and mined by induction when every demonstration, each with a
different search, shows the same line. Induction no longer copies trace 0's
guard when the demonstrations disagree; it drops it and notes why. The
opener and its Confirm click are no longer read as a two-row loop.
Qualification needs a business-rule case for a guard mined this way.

* Empty-result guard evidence: refuse on disagreement, scope the checks (review round 1)

Induction refuses (ambiguous_guard) when the demonstrations disagree on a
mined dialog check, on the straight path and in optional, divergent or loop
columns. The search must differ in the value typed last before the dialog.
A question is never a candidate line. Qualification finds the checks on
induced program graphs and only on a confirmation dialog's Confirm click.
The induce CLI passes bundle compile notes; recording notes are not
duplicated.

* Empty-result guard evidence: keep induction's notes, compare openers too (review round 2)

admit writes induction's dialog-check decisions into the induced bundle's
compile_notes.json. The off-path agreement check also compares the opener's
exact postcondition. The docs say plainly that the qualification campaign
cannot run on an induced bundle yet.

* Empty-result guard evidence: every typed value must differ, not just the last (review round 3)

A later field that differs (a comment) no longer passes for a different
search: every value typed since the last save and before the opener must be
a parameter that differs across the demonstrations. Regression test on real
recordings with the same name and different comments. Notes use step ids,
and a confirmation pair off the straight path with no check is noted.

* Empty-result guard evidence: count every typed value, one off-path note (review round 4)

Drop the last-save boundary: an in-form Next or Apply button is inferred
irreversible and let a later field stand in for the search. Every typed or
selected value before the opener must differ. The off-path note is emitted
once per column with the copied step's ids, and is tested. A record chosen by
a click is documented as not counted.

* Search-dialog fixtures: record headless, explicitly

* Verifier specificity: operation semantics as contract data, and the leaderboard re-scored (#1795)

* Leaderboard re-score: the verifier's reads served from the frozen Medplum history

The write leaderboard's verifier lost its read token after an hour, so 163 of
222 scored runs were judged on an HTTP 401. The two stacks' databases kept
every version and audit event. This replays ensure_record unchanged against
them, as the server stood at each run's baseline and check, and reproduces
all 59 verdicts the live verifier did reach (outcome, reasons, evidence).

* ensure_record: operation semantics as contract data, and a per-app semantics spec

An operations: section says what one write may also do; every default keeps
the old rule. correction.between: consistent accepts a record saved in steps
when every version held only the task's values; cardinality.own_transient
accepts the agent's own duplicate deleted again, read by no one else;
bookkeeping excuses exact element paths on other records, with an optional
now_checks temporal rule. Nothing on the written record or on any record that
is the person is excused. openadapt.app-semantics/v1 holds these per app and
version, pinned by each contract; Medplum's is the sample.

The Medplum harness renews the verifier's read token before each read: set
once, it expired after an hour and the leaderboard's verifier judged 163 of
222 runs on HTTP 401.

* Leaderboard re-score: pre-registration, before any unjudged run is re-scored

* Leaderboard re-score: results, three logged deviations, and correction mode either

The verifier, able to read, on the same 222 frozen runs: specificity 46/72
and 57/68 with the contracts on main, 55/72 and 42/68 with the
pre-registered operations rule, 64/72 and 57/68 with the exploratory
between: either (never stricter than #1655); sensitivity 25/28 and 30/33,
every miss the post-check write A10. The replay now stops at the harness's
next step (deviation 1) and reads a retried run's last seed (deviation 2);
the as-registered results are kept beside the corrected ones.

* Review fixes: a referenced duplicate refuses; bookkeeping names only counters, times and one extension

From the independent review of #1795: a cleaned-up duplicate that any
other write in the window refers to refuses, and the transient rule needs
the audit heartbeat. Bookkeeping may name only meta.versionId,
meta.lastUpdated and extension[url=...], never the contract's own type,
and only for the agent's own updates; now_checks compares per path and
checks dates. The semantics pin names the spec's sha256 and a relative
path. consistent refuses extra elements on open contracts. Every re-score
verdict is unchanged (deviation 4).

* Register the re-score's public artifacts in the inventory

* docs: add AGENTS.md with rules for automated coding agents (#1817)

* docs: add AGENTS.md with rules for automated coding agents

* docs(AGENTS.md): one claim rule for every agent

* docs(AGENTS.md): claim through the refs API, which refuses an existing branch

* Serve the fault benches as OpenEnv and Prime Intellect RL environments (#1780)

* feat(flow): serve the fault benches as OpenEnv and Prime Intellect RL environments

One generic browser environment (openadapt_flow.rl_env) serves every app over
the OpenEnv protocol. The reward is verify_run's verdict on the app's record,
read through a channel the agent's browser cannot reach; each episode also
returns an outcome vector (completion, harm, false refusal, steps).

- MockMed: 11-task public sample with hidden faults. MockLoan and the held-out
  MockMed faults ship no public tasks; they load from a pack kept outside the
  package (OPENADAPT_ENV_EXTRA_TASKS), and every pack is hashed.
- OpenEMR (GPL-3.0), Frappe Lending (GPL-3.0) and openIMIS (AGPL-3.0): manifests
  name upstream images by digest; no app code is copied. OpenEMR has a task
  pack awaiting a live run; Frappe has its oracle; openIMIS has no reset yet.
- A failure report on an impossible task pays only after the browser sent the
  task's write, so not trying never earns reward.
- environments/prime/openadapt_web_envs: verifiers v1 taskset over the same
  servers.

* fix(flow): close the review's reward leaks in the RL environments

- Public generator holds public cases only; held-out packs are built from a
  private cases file. The held-out instances in the previous commit are retired
  and re-authored privately.
- Task ids are numbers and the observation carries no task id, so nothing the
  agent sees names the fault.
- External apps (one shared database) serve one session at a time.
- A failed reset leaves no live task or judge; a route-handler error can no
  longer hang a request; write detection compares trimmed values and numbers
  by value; the write_requests rule also covers extra packs; goto takes only
  paths; missing environment files raise TaskError.
- run_eval wraps verifiers' eval: it never uploads traces without
  --allow-upload, and a test fails if our tooling calls eval directly.

* docs(flow): validation report for the RL environments; refuse impossible wrong_record cases

MockMed's public sample passes over OpenEnv and the Prime path with zero model
calls; 8 parallel sessions all reach the maximum reward; reset leaves no state
behind. The generator now refuses a wrong_record task on the patient that fault
writes to, which validation caught in a held-out pack.

* fix(flow): register the RL environment files; keep the openenv extra on Python 3.11+

The public artifact inventory now lists environments/openenv. On Python 3.10
the OpenEnv stack resolved to dnspython 2.8.0 (GHSA-45c3-73f7-pv4m); the extra
now applies on 3.11+ only, which verifiers already requires.

* fix(flow): end an RL episode whose step or reset gets stuck

A step longer than 120 s (reset 180 s) now returns a final observation with
reward 0 and verdict INDETERMINATE, drops the stuck browser, and lets the next
reset start afresh, instead of leaving the client waiting. Seen once on a WSL
host during held-out validation.

* fix(flow): a stuck RL step that wakes later cannot touch the next episode

On abandonment the watchdog now kills the stuck browser's process tree (the
Playwright driver and the Chromium under it) and replaces an in-process app
copy, so the next reset starts on a fresh store. A test lets the stuck step
wake after the reset and try its write: the new episode's store stays empty
and its scripted run still earns 1.0.

* fix(flow): mypy in rl_env environment (judge close, elements annotation)

* Rollback on durable runs, and after a crash (#1699) (#1772)

* feat(rollback): roll back desktop and remote-display runs (#1699)

A desktop or RDP run with rollback on no longer refuses. A GUI compensation
starts from the run's start screen: the halt cleanup (only when qualified)
and its start check on the run's own backend. A screen that doesn't come back
refuses the compensation, and the run ends RECONCILIATION_REQUIRED. The
whole-database reads before and after each compensation prove the undo, as
on the web.

Adds the Windows lab harness (native and RDP) for Patient Notes.

* test(rollback): a compensation the app reports done but never wrote; lab undo restores the kept value

* feat(rollback): roll back durable runs and continue after a crash (#1699)

A durable run with rollback on no longer refuses. The read from before the
run, its row-digest key, the audit log as it was and the run's parameters are
sealed (AES-256-GCM, the deployment's key) in <run>/rollback. A durable pause
waits for a person: resume it, or roll it back with resume --roll-back. The
first attempt seals its read after the stop and journals each compensation it
starts; a later attempt judges the run's changes by that read, stops if
anything else changed since, and re-reads before treating any started
compensation as done. Without a key a durable run with rollback is refused.

* lab(rollback): drop screen checks that quote the case-dependent note; search without Enter

* docs(rollback): durable runs and a rollback a crash interrupted

* rollback(desktop): check the compensation's own start screen; a screen undo counts for undo-or-person only with a qualified cleanup; docs

Review fixes for #1771: before a GUI compensation on a desktop or remote
display, check the compensation's first screen as well as the run's (a
persistent menu can satisfy the run's check while the wrong record is up).
A GUI undo there counts as an undo for the undo-or-person rule only when the
bundle's halt cleanup has steps and is qualified; otherwise each save goes to
a person first. Fresh-session dirs can't collide with undo ids. ROLLBACK.md,
CAPABILITIES.md and llms-full.txt describe the desktop path.

* rollback(durable): bind sealed state to its run; never run a compensation twice; --roll-back keeps resume's guards

Review fixes for #1772:
- Every sealed file carries the arming's id; a file from another run or a
  missing journal is refused. Arming afresh clears an earlier stop read and
  journal. A durable pause binds the state to the manifest's run id.
- The journal is sealed. A compensation it marks finished never runs again;
  one only started is skipped when a fresh read shows the change gone, and
  otherwise goes to a person. A rollback that ended keeps its verdict, drops
  its reads, and refuses --roll-back.
- --roll-back refuses a changed bundle version, changed rollback settings
  (attribution, exclusive, allow_delete, scope, profile), a managed dispatch
  or qualification authority, and a second concurrent rollback (lock).
- Only durable runs seal state. A resumed run that stops again prints the
  --roll-back command.

* fix(rollback): CI on #1771: fresh_session takes the compensation (mypy); check screen undo only with rollback on

* rollback(durable): bind the sealed state before an in-process rollback, so a crash there can be continued; platform-guard the Windows lock (mypy)

* fix(rollback): roll-back hints use the spelling the person typed (CI on #1772)

* Zero-touch setup: generated contracts admitted by a sandbox, no person answers (#1753)

* Zero-touch setup: literature review, design, sandbox validator prototype (WIP)

* Zero-touch: fault_gate class names, #1520 decisions, now_checks docs, capability row

* Zero-touch gate: fail closed on unplanted faults, fault every column, require core classes, strict pins, production equals validated

* Zero-touch: type-made plants for typed columns; strict_pins pins a one-demonstration constant; docs

* Zero-touch: two-phase sandbox restore; a duplicate the store's unique key refuses is not applicable (local, for review)

* Zero-touch: a duplicate is not applicable only when a declared unique index covering unchanged columns refuses it; recorded per kind

* Zero-touch round 2: failed reads never count as stopped; found keeps only constants of strict roles; validated field sets only; key and unrelated-edit faults on creates

* Zero-touch: plants avoid declared unique keys; failing cases log their reason

* Zero-touch design: failed reads, forbidden duplicates, validated field sets, strict pins

* Zero-touch round 3: fresh app-made unique values in duplicates; transactional restore stops the setup; structured read evidence; plain unique indexes only

* Zero-touch round 4: task-given columns come from the generator's bindings, not value matches; stopped setup keeps old contracts aside and exits non-zero

* Zero-touch: typing-only fixes for mypy (no Optional reassignment, one type per variable)

* fix(flow): refuse volatile FHIR metadata before run 1 (#1819)

* fix(flow): refuse volatile FHIR metadata before run 1

Signed-off-by: abrichr <richard.abrich@mldsai.com>

* ci: refresh checks after marking PR ready

Signed-off-by: abrichr <richard.abrich@mldsai.com>

---------

Signed-off-by: abrichr <richard.abrich@mldsai.com>

* desktop: prepare review card input before paint (#1825)

* fix(desktop): prepare review card input before paint

Signed-off-by: abrichr <richard.abrich@mldsai.com>

* chore(desktop): prepare 0.25.1 patch release metadata

Signed-off-by: abrichr <richard.abrich@mldsai.com>

---------

Signed-off-by: abrichr <richard.abrich@mldsai.com>

---------

Signed-off-by: abrichr <richard.abrich@mldsai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant