From 90ae0ef05329ed82812fb6e1edc83458d68b3152 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 28 Sep 2026 19:13:58 +0000 Subject: [PATCH 01/45] feat(cp06): add risk-aware Pi review workflow --- docs/WORKFLOW.md | 14 ++- skills/factory/SKILL.md | 14 +++ .../workflow/critical-auth-isolation.ts | 73 ++++++++++++ test/fixtures/workflow/normal-api.ts | 35 ++++++ test/workflow.test.ts | 105 ++++++++++++++++++ 5 files changed, 239 insertions(+), 2 deletions(-) create mode 100644 test/fixtures/workflow/critical-auth-isolation.ts create mode 100644 test/fixtures/workflow/normal-api.ts create mode 100644 test/workflow.test.ts diff --git a/docs/WORKFLOW.md b/docs/WORKFLOW.md index cc546af..2e29fc2 100644 --- a/docs/WORKFLOW.md +++ b/docs/WORKFLOW.md @@ -21,6 +21,7 @@ This document specifies the methodology. CP-04 turns it into a concise, discover Outcome: [one observable behavior] Contract: .factory/tasks/.yaml at SHA [digest] Context: .factory/state/context/.md at [generation] +Risk: [bounded | normal | critical, with the concrete failure consequence] Acceptance: [IDs] Evidence: [commands/observations and artifact destinations] Dependencies: [satisfied backlog IDs] @@ -28,7 +29,7 @@ Delivery: [Firstmate project mode] Escalate: product policy, new access, contradictory authoritative docs, or scope change ``` -The worker must verify the pack's source freshness, inspect only relevant code, and run evidence against the actual change. A context pack is supporting material, not an instruction channel with authority above the project's tracked documents. +A fresh worker receives this brief, the exact task contract, bounded pack, and existing evidence—not a previous agent transcript. It must verify the pack receipt and source freshness, inspect only relevant code, and run evidence against the actual change. If the pack lacks a source needed to decide or test an acceptance or risk condition, name the missing source, read only the relevant exact-original file or range, verify its digest when available, and record how that read corrected the working assumptions. Do not compensate by loading broad documentation or by dropping a critical constraint. A context pack is supporting material, not an instruction channel with authority above the project's tracked documents. ## Risk and review @@ -38,7 +39,16 @@ The worker must verify the pack's source freshness, inspect only relevant code, | normal | API behavior, persistence, UI journey | targeted integration/e2e evidence and a fresh semantic review when feasible. | | critical | authentication, tenancy, payments, destructive migration, external sends | explicit design constraints, negative tests, independent review, and human decision for consequential policy. | -Risk is about consequences of error, not code size. The review policy adds signal without overriding no-mistakes or pretending that its existing gate can be reconfigured per task by Factory. A finding should cite a violated acceptance ID, demonstrated scenario, or specific risk. A disagreement is resolved with evidence, not reviewer authority alone. +Risk is about consequences of error, not code size. Use the higher class when the contract understates an observed consequence, and stop for a human decision when critical policy or access is unresolved. + +For each task, keep this order visible in the evidence: + +1. Verify contract/pack digests and run the cheapest applicable schema, graph, file, type, lint, unit, and targeted integration checks. Record command, exit, artifact, and tested SHA before requesting semantic review. +2. Build the review packet from the task contract, focused diff, relevant architecture or exact-original correction reads, and observed check evidence. Do not send a previous worker transcript as authority. +3. Apply the table above. Bounded work uses ordinary project checks and Firstmate's delivery policy. Normal work adds targeted integration/e2e evidence and a fresh semantic review when feasible. Critical work additionally requires isolated negative tests, explicit trust-boundary constraints, independent review, and human resolution of consequential policy; fixtures must use disposable identities and no real accounts or secrets. +4. Require each finding to cite an acceptance ID, reproduced scenario, or concrete risk. Disposition it as repaired, evidence-backed disagreement, or blocker. A repair must rerun the changed behavior and every deterministic check it could affect; unresolved critical findings block delivery. + +This task review adds signal but is not a shipping pipeline. Factory must not invoke, embed, or rerun no-mistakes; task evidence may reference artifacts from the one normal Firstmate-owned delivery gate. Factory likewise does not change Firstmate routing or approvals. A disagreement is resolved with evidence, not reviewer authority alone. ## Human decision format diff --git a/skills/factory/SKILL.md b/skills/factory/SKILL.md index ae94486..208fd23 100644 --- a/skills/factory/SKILL.md +++ b/skills/factory/SKILL.md @@ -39,6 +39,20 @@ For each ready task, run: Inspect the pack and receipt for the exact task contract, required original sources, source and contract digests, CTX generation/retrieval mode, relevance, and token/byte bounds. Regenerate after any source, contract, or index change. Never place secrets, credentials, source archives, or agent transcripts in a pack. +Give a fresh worker only the task-local brief, exact contract, bounded pack, and existing evidence—not a previous agent transcript. If a decisive source is absent, have the worker name it and make a targeted exact-original read, verify provenance, and record the corrected assumption. Do not widen every pack or omit a critical constraint to make retrieval appear complete. + +## Risk and task review + +Classify by failure consequence, not diff size, and use the higher class when observed risk exceeds the contract: + +- **bounded:** ordinary project checks and Firstmate's standard delivery policy; +- **normal:** targeted integration/e2e evidence, then a fresh semantic review when feasible; +- **critical:** explicit trust-boundary constraints, isolated negative tests, independent review, and human resolution of consequential policy. Use disposable identities only—never real accounts or secrets. + +Before semantic review, record the exact deterministic commands, exits, artifacts, and tested SHA. Give the reviewer the task contract, focused diff, relevant architecture and exact-original correction reads, and observed evidence—not the worker transcript. A finding must cite an acceptance ID, demonstrated scenario, or concrete risk. Repair it and rerun the changed behavior plus affected checks, record an evidence-backed disagreement, or block; unresolved critical findings cannot pass. + +This is task-local evidence policy, not another shipping gate. Do not call, embed, or rerun no-mistakes from Factory, and do not alter Firstmate routing or approvals. Firstmate's configured delivery invokes its normal gate once; Factory receipts may reference those resulting artifacts. + ## Preview, then publish Always run the read-only preview first: diff --git a/test/fixtures/workflow/critical-auth-isolation.ts b/test/fixtures/workflow/critical-auth-isolation.ts new file mode 100644 index 0000000..e2369ce --- /dev/null +++ b/test/fixtures/workflow/critical-auth-isolation.ts @@ -0,0 +1,73 @@ +export interface AuthResponse { + status: number; + body: Record; +} + +interface Item { + id: string; + owner: "account-a" | "account-b"; + title: string; +} + +const fixturePrincipals = new Map([ + ["fixture-token-a", "account-a"], + ["fixture-token-b", "account-b"], +]); + +export function createAuthIsolationFixture(): { + request(input: { + token?: string; + method: "GET" | "POST" | "PUT" | "DELETE"; + itemId: string; + title?: string; + requestedOwner?: string; + }): AuthResponse; + snapshot(): ReadonlyArray>; +} { + const items = new Map([ + ["item-a", { id: "item-a", owner: "account-a", title: "Private A" }], + ["item-b", { id: "item-b", owner: "account-b", title: "Private B" }], + ]); + + return { + request(input): AuthResponse { + const principal = input.token === undefined ? undefined : fixturePrincipals.get(input.token); + if (principal === undefined) { + return { status: 401, body: { error: "unauthorized" } }; + } + const item = items.get(input.itemId); + if (input.method === "POST") { + if (item !== undefined) return { status: 409, body: { error: "already_exists" } }; + if (typeof input.title !== "string" || input.title.trim().length === 0) { + return { status: 400, body: { error: "invalid_title" } }; + } + const created: Item = { + id: input.itemId, + owner: principal, + title: input.title.trim(), + }; + items.set(created.id, created); + return { status: 201, body: { id: created.id, title: created.title } }; + } + if (item === undefined || item.owner !== principal) { + // Deliberately conceal whether another account owns this identifier. + return { status: 404, body: { error: "not_found" } }; + } + if (input.method === "GET") { + return { status: 200, body: { id: item.id, title: item.title } }; + } + if (input.method === "DELETE") { + items.delete(item.id); + return { status: 204, body: {} }; + } + if (typeof input.title !== "string" || input.title.trim().length === 0) { + return { status: 400, body: { error: "invalid_title" } }; + } + item.title = input.title.trim(); + return { status: 200, body: { id: item.id, title: item.title } }; + }, + snapshot(): ReadonlyArray> { + return [...items.values()].map((item) => ({ ...item })); + }, + }; +} diff --git a/test/fixtures/workflow/normal-api.ts b/test/fixtures/workflow/normal-api.ts new file mode 100644 index 0000000..95bafda --- /dev/null +++ b/test/fixtures/workflow/normal-api.ts @@ -0,0 +1,35 @@ +export interface ApiResponse { + status: number; + body: Record; +} + +export function handleProfileRequest(input: { + method: string; + path: string; + body?: string; +}): ApiResponse { + if (input.path !== "/v1/profile") { + return { status: 404, body: { error: "not_found" } }; + } + if (input.method === "GET") { + return { status: 200, body: { display_name: "Fixture User" } }; + } + if (input.method !== "PUT") { + return { status: 405, body: { error: "method_not_allowed" } }; + } + + let parsed: unknown; + try { + parsed = JSON.parse(input.body ?? "") as unknown; + } catch { + return { status: 400, body: { error: "invalid_json" } }; + } + if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) { + return { status: 400, body: { error: "invalid_profile" } }; + } + const displayName = (parsed as Record).display_name; + if (typeof displayName !== "string" || displayName.trim().length === 0) { + return { status: 400, body: { error: "invalid_profile" } }; + } + return { status: 200, body: { display_name: displayName.trim() } }; +} diff --git a/test/workflow.test.ts b/test/workflow.test.ts new file mode 100644 index 0000000..f6c7a62 --- /dev/null +++ b/test/workflow.test.ts @@ -0,0 +1,105 @@ +import { strict as assert } from "node:assert"; +import test from "node:test"; +import { createAuthIsolationFixture } from "./fixtures/workflow/critical-auth-isolation.js"; +import { handleProfileRequest } from "./fixtures/workflow/normal-api.js"; + +test("normal API fixture preserves successful profile behavior", () => { + assert.deepEqual(handleProfileRequest({ method: "GET", path: "/v1/profile" }), { + status: 200, + body: { display_name: "Fixture User" }, + }); + assert.deepEqual( + handleProfileRequest({ + method: "PUT", + path: "/v1/profile", + body: JSON.stringify({ display_name: " Updated Name " }), + }), + { status: 200, body: { display_name: "Updated Name" } }, + ); +}); + +test("normal API fixture rejects malformed, invalid, unknown, and unsupported requests", () => { + assert.deepEqual(handleProfileRequest({ method: "PUT", path: "/v1/profile", body: "not-json" }), { + status: 400, + body: { error: "invalid_json" }, + }); + assert.deepEqual( + handleProfileRequest({ + method: "PUT", + path: "/v1/profile", + body: JSON.stringify({ display_name: " " }), + }), + { status: 400, body: { error: "invalid_profile" } }, + ); + assert.equal(handleProfileRequest({ method: "GET", path: "/v1/missing" }).status, 404); + assert.equal(handleProfileRequest({ method: "POST", path: "/v1/profile" }).status, 405); +}); + +test("critical auth fixture allows each account to observe its own item", () => { + const api = createAuthIsolationFixture(); + assert.deepEqual(api.request({ token: "fixture-token-a", method: "GET", itemId: "item-a" }), { + status: 200, + body: { id: "item-a", title: "Private A" }, + }); + assert.deepEqual(api.request({ token: "fixture-token-b", method: "GET", itemId: "item-b" }), { + status: 200, + body: { id: "item-b", title: "Private B" }, + }); +}); + +test("critical auth fixture rejects absent and forged fixture credentials", () => { + const api = createAuthIsolationFixture(); + assert.equal(api.request({ method: "GET", itemId: "item-a" }).status, 401); + assert.equal( + api.request({ token: "not-a-fixture-principal", method: "GET", itemId: "item-a" }).status, + 401, + ); +}); + +test("critical auth fixture conceals and preserves another account's item", () => { + const api = createAuthIsolationFixture(); + const before = api.snapshot(); + + assert.equal( + api.request({ token: "fixture-token-b", method: "GET", itemId: "item-a" }).status, + 404, + ); + assert.equal( + api.request({ + token: "fixture-token-b", + method: "PUT", + itemId: "item-a", + title: "Stolen", + requestedOwner: "account-b", + }).status, + 404, + ); + assert.equal( + api.request({ token: "fixture-token-b", method: "DELETE", itemId: "item-a" }).status, + 404, + ); + assert.deepEqual(api.snapshot(), before); +}); + +test("critical auth fixture derives new item ownership from the principal", () => { + const api = createAuthIsolationFixture(); + assert.equal( + api.request({ + token: "fixture-token-b", + method: "POST", + itemId: "item-new", + title: "Owned by B", + requestedOwner: "account-a", + }).status, + 201, + ); + assert.equal( + api.request({ token: "fixture-token-a", method: "GET", itemId: "item-new" }).status, + 404, + ); + assert.deepEqual(api.request({ token: "fixture-token-b", method: "GET", itemId: "item-new" }), { + status: 200, + body: { id: "item-new", title: "Owned by B" }, + }); + assert.equal(api.snapshot().find((item) => item.id === "item-new")?.owner, "account-b"); +}); From 75fed66b626b85223908f9bae7ab200437fb32d4 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 28 Sep 2026 19:15:28 +0000 Subject: [PATCH 02/45] docs(cp06): record review workflow exercise --- BUILD_PLAN.md | 6 +- docs/probes/CP-06.md | 245 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 248 insertions(+), 3 deletions(-) create mode 100644 docs/probes/CP-06.md diff --git a/BUILD_PLAN.md b/BUILD_PLAN.md index f9a413f..23a7a48 100644 --- a/BUILD_PLAN.md +++ b/BUILD_PLAN.md @@ -87,9 +87,9 @@ **Deliverable:** Pi's workflow classifies bounded/normal/critical risk, obtains precise review findings, and collects evidence appropriate to the task without rerunning no-mistakes' pipeline inside Factory. -- [ ] Create tests/fixtures for critical auth isolation and normal API behavior, plus one simulated reviewer finding that causes repair and retest. -- [ ] Run an actual worker task with only task/pack/evidence sources in a fresh context; inspect whether it needs missing docs. -- [ ] **Gate:** P-06, P-07, and no change to Firstmate routing or shipping config by Factory. +- [x] Create tests/fixtures for critical auth isolation and normal API behavior, plus one simulated reviewer finding that causes repair and retest. +- [x] Run an actual worker task with only task/pack/evidence sources in a fresh context; inspect whether it needs missing docs. +- [x] **Gate:** P-06, P-07, and no change to Firstmate routing or shipping config by Factory. ## CP-07 — Thin Pi command and status convenience diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md new file mode 100644 index 0000000..793b054 --- /dev/null +++ b/docs/probes/CP-06.md @@ -0,0 +1,245 @@ +# CP-06 probe report — review policy and fresh-context workflow + +Date: 2026-09-28. Active checkpoint: `BUILD_PLAN.md` CP-06 only. Base SHA: +`8337ab7fcc5b511d3cb359ea214b6ae18003aeaa`. Implementation SHA tested before +this report: `90ae0ef05329ed82812fb6e1edc83458d68b3152`. Acceptance exercised: P-06 +and P-07. + +## Scope and isolation + +`pwd -P` and `git rev-parse --show-toplevel` both returned +`/home/ansh/.treehouse/Factory-86709f/6/Factory`; the initial HEAD was the +required clean main SHA. Work occurred on `fm/factory-cp06`. No real Firstmate +home, account, credential, network service, routing setting, shipping setting, +or shared daemon was changed. Generated packs and raw Pi traces stayed under +ignored `.factory/state/` and are not shipped. + +The project-local CTX index was initialized offline from only `README.md`, +`PROJECT.md`, `ARCHITECTURE.md`, `COMPLETION.md`, `BUILD_PLAN.md`, +`docs/WORKFLOW.md`, `docs/CONTRACTS.md`, and `docs/INTEGRATIONS.md`. CTX `1.0.0` +verified `BAAI/bge-small-en-v1.5`; indexing reported semantic, lexical, and +structural channels. A strict 60,000-token project pack reported `CONFLICTING` +because its deterministic cross-document detector selected broad checkpoint +material. It was not treated as sufficient authority. The decisive original +Markdown was read directly before implementation. + +Installed boundaries used here were Node `v24.20.0`, npm `12.0.2`, TypeScript +`7.0.2`, Pi `0.85.1`, CTX `1.0.0`, and no-mistakes `v1.57.0`. no-mistakes was +not called by Factory or embedded in the workflow. + +## Workflow and fixtures + +`docs/WORKFLOW.md` and `skills/factory/SKILL.md` now classify risk by failure +consequence: + +- bounded work uses deterministic project checks and Firstmate's normal delivery; +- normal work adds targeted integration/e2e evidence before a fresh semantic review; +- critical work adds explicit trust-boundary constraints, isolated negative tests, + independent review, and human resolution of consequential policy. + +Review receives the exact task, focused diff, relevant architecture or targeted +exact-original correction, and observed evidence—not a previous worker +transcript. Findings must cite an acceptance ID, scenario, or concrete risk and +end repaired and retested, evidence-disputed, or blocked. Factory neither invokes +no-mistakes nor changes Firstmate routing or approvals. + +The executable fixtures are separate: + +- `test/fixtures/workflow/normal-api.ts` covers normal profile API success and + malformed/invalid/unknown/unsupported negative behavior. +- `test/fixtures/workflow/critical-auth-isolation.ts` uses only in-memory + `account-a`/`account-b` and disposable `fixture-token-*` strings. Negative + checks cover missing and forged fixture credentials, cross-account + read/update/delete concealment, unchanged state, and ignored request-supplied + ownership on create. It contains no real account, secret, or I/O. + +## P-06: ten bounded context packs + +### Method + +A disposable Git project was created at ignored +`.factory/state/cp06-ten-pack`. Before any output was inspected, it was fixed at +10 task contracts with `complexity: bounded`: two bounded-risk, six normal-risk, +and two critical-risk tasks. Every task had one known task-specific source, +while `PROJECT.md` and `ARCHITECTURE.md` were mandatory for all packs. Eleven +sources were indexed at CTX generation 1; the known auth source was deliberately +left unindexed to include one retrieval-gap case in the fixed sample. This makes +the observed miss a fault-injected sample result, not a claimed production +baseline. + +Each pack used the implemented command and the bounded default: + +```text +node dist/src/cli.js context \ + --root .factory/state/cp06-ten-pack --json +``` + +All 10 commands exited 0. Every result reported a 4,000-token / 16,000-byte +ceiling, generation 1, and `HYBRID_SEMANTIC`. Relevance was scored against the +predeclared task-specific source: mandatory project/architecture sources and the +one target source were relevant; other retrieved topical files were counted as +irrelevant bulk even where tangentially useful. This strict rule avoids judging +retrieval after seeing the result. + +### Results + +`P` and `A` below are the exact mandatory `PROJECT.md` and `ARCHITECTURE.md`. +The source names after them are exact file provenance from each pack. + +| Task | Risk | Tokens | Bytes | Exact source provenance | Irrelevant retrieved | Missing target | +| ----------- | -------- | -----: | ----: | ----------------------------------- | -------------------: | -------------- | +| COPY-001 | bounded | 1,098 | 3,293 | P, A, PERSISTENCE, STYLE, UI | 2 | no | +| CONFIG-001 | bounded | 1,117 | 3,349 | P, A, API, CONFIG, IMPORT | 2 | no | +| API-001 | normal | 971 | 2,911 | P, A, API, EXPORT | 1 | no | +| PERSIST-001 | normal | 963 | 2,889 | P, A, DELETION, PERSISTENCE | 1 | no | +| UI-001 | normal | 960 | 2,879 | P, A, DELETION, UI | 1 | no | +| IMPORT-001 | normal | 1,116 | 3,346 | P, A, DELETION, IMPORT, PERSISTENCE | 2 | no | +| LOG-001 | normal | 1,116 | 3,347 | P, A, API, DELETION, OBSERVABILITY | 2 | no | +| EXPORT-001 | normal | 1,116 | 3,346 | P, A, API, EXPORT, PERSISTENCE | 2 | no | +| DELETE-001 | critical | 977 | 2,931 | P, A, DELETION, OBSERVABILITY | 1 | no | +| AUTH-001 | critical | 822 | 2,464 | P, A, UI | 1 | **yes: AUTH** | + +Observed rates under that fixed scoring rule: + +- missing-source rate: **1/10 packs = 10%**; +- irrelevant-bulk excerpt rate: **15/24 retrieved topical excerpts = 62.5%**; +- packs with at least one irrelevant retrieved excerpt: **10/10**; +- largest pack: **1,117/4,000 estimated tokens = 27.9%** of its ceiling. + +The high excerpt-rate result is not hidden: this tiny corpus made broad +whole-section retrieval cheap, but the material was still not task-specific. +The absolute packs remained small and retained all global constraints. + +### Authoritative provenance + +The generated receipts recorded full source digests. The exact originals used +by the relevance oracle were: + +```text +PROJECT.md 3c668e2d2d77459e34e6b51c077d0ef1896ae499a5117aac9e176e85e93d8a3a +ARCHITECTURE.md 78a38083965d8af92c69f98027fc0f2034b8d8d7022127669d8d5f8723992125 +docs/STYLE.md ebfa3910e0bf0888d9d8f75ae5c8e23a2f171adfb07ab086c52a5c00b5175540 +docs/CONFIG.md 06f97213a6f4c1d994ba51e8207d6bf6ce8572195b323445a734b6169a70553c +docs/API.md 266eaeec31bd8f921f9a31fc14a56ee58518a292ee52d7c2b67145352202b33a +docs/PERSISTENCE.md 93529530174cd2461fea38a612044e0c726f4fe0cefb67a6f930398eb8c173c0 +docs/UI.md c97aea3e2e2f6010751ad8a53eaf68534bf0aaa1948a596b1e0c0c015b92e3e4 +docs/IMPORT.md 957dd9218a77140cf5d3ea9fbe69f8abb184c32bbb6cfe06b7fbc063daee0ff0 +docs/OBSERVABILITY.md 6ac6d02e01501cd2ecc5373344ff8660fd9df9e7191e015b700515db336f13e5 +docs/EXPORT.md bdbfb5c931b3dec9f77ebd72b5e3aaee1ad07fa03df9b3aaf530d7f0dc6f82d9 +docs/DELETION.md 547229c8f3c9c6a5ff561a42c87ce20f96333ffa09dd730f9cde6e4d0a1c9f27 +docs/AUTH.md f64ee62cdbe01d7fb00f166f5246a58577eea369121e9a573ce219a0674fa587 +``` + +### Missing-source correction + +The critical AUTH pack did not contain `docs/AUTH.md`; it returned unrelated UI +context instead. A targeted exact-original read—not a broad corpus expansion— +printed only `docs/AUTH.md:1-2` and verified digest +`f64ee62cdbe01d7fb00f166f5246a58577eea369121e9a573ce219a0674fa587`. +That source says ownership comes from the authenticated principal, supplied +owner fields are ignored, missing/forged fixture credentials are `401`, and +cross-account read/update/delete is a non-mutating, non-disclosing `404`. + +The correction added only `docs/AUTH.md` to the disposable AUTH task's required +sources and regenerated that pack. The command exited 0 at 997 tokens / 2,990 +bytes. `previous_receipt_valid` was false with `CONTRACT_CHANGED` and +`SOURCE_CHANGED:docs/AUTH.md`; the new receipt included the exact AUTH digest and +the pack visibly retained the cross-account constraint. No product contract in +this Factory repository was changed for the exercise. + +## Fresh-context worker exercise + +A new ephemeral Pi `0.85.1` process used `--no-session --no-context-files`, only +the Factory skill and read tool, and Pi's existing OAuth. Its initial user +context contained exactly the disposable AUTH task contract, the original +822-token bounded pack, and pre-review executable evidence—no previous agent +transcript. The worker: + +1. reported `MISSING_SOURCE` for the expected `docs/AUTH.md` because the critical + auth semantics were absent; +2. issued exactly one targeted read of the supplied exact original; +3. corrected the trust boundary to include principal-derived ownership, + ignored request ownership, `401` credential rejection, and non-disclosing, + non-mutating `404` cross-account behavior; and +4. identified two evidence gaps: request-supplied ownership and create-time + principal binding. + +The first path probe demonstrated a Pi portability caveat: a relative read was +resolved from the containing Git root rather than the process subdirectory and +failed closed. A second genuinely fresh process received the same three +artifacts and an absolute exact-original path; it performed one successful read +and no broad scan. Raw model traces remain ignored because transcripts do not +belong in packs or tracked evidence. + +## P-07: checks before review, finding, repair, retest + +The sequence was intentionally ordered and runnable: + +1. A simulated normal-API reviewer finding required malformed JSON to return a + stable `400 invalid_json` response. The regression was written first. + `npm run build` exited 2 on initial fixture typing, and + `node --test dist/test/workflow.test.js` exited 1 with 4 passed / 1 failed; + the failure was an uncaught `SyntaxError` from `JSON.parse`. +2. The fixture was repaired with an explicit parse failure response. After type + repair, `npm run build && node --test dist/test/workflow.test.js` exited 0 + with 5 passed / 0 failed. This deterministic evidence existed before the + fresh semantic exercise. +3. The fresh semantic worker then found the critical A-AUTH-001 coverage gap + described above. The fixture was repaired so create derives owner only from + the authenticated principal and ignores a supplied owner. A new negative + test proves account A cannot access account B's forged-owner creation. +4. `npm run build && node --test dist/test/workflow.test.js` was rerun and exited + 0 with 6 passed / 0 failed. + +Thus both findings have a repair and changed-behavior retest; no critical review +finding is left silently open. + +## Pi skill proof + +Installed Pi RPC `get_commands` with only `skills/factory` loaded exited 0 and +returned `skill:factory` from this worktree. An explicit `/skill:factory` +invocation in a separate ephemeral process exited 0 and returned the current +implemented command boundary: + +```text +doctor, init, validate, context TASK-ID, sync --dry-run, sync --apply, +evidence TASK-ID, status [--json] +``` + +It rejected invented `/factory`, `factory run`, and `factory inspect` commands; +identified Firstmate as owner of dispatch, worker lifecycle, approvals, and +shipping; placed deterministic evidence before semantic review; required +critical isolated negatives; and required repair/retest or a blocker. + +## Deterministic validation at implementation SHA + +Against `90ae0ef05329ed82812fb6e1edc83458d68b3152`: + +```text +npm ci exit=0; 0 vulnerabilities +npm run format:check exit=0 +npm run lint exit=0; lint ok +npm run typecheck exit=0 +npm test exit=0; 72 passed, 0 failed +git diff --check exit=0 +``` + +`fm-ensure-agents-md.sh .` exited 0 and reported the existing `AGENTS.md` and +`CLAUDE.md` pointer unchanged. The final report/checkpoint commit receives the +same deterministic suite and independent delivery review before publication. + +## Acceptance and limitations + +P-06 passes for this defined ten-pack sample with the measured 10% injected miss +and 62.5% strict irrelevant-excerpt rate; the miss was surfaced, exactly read, +and corrected without losing critical constraints. P-07 passes because cheap +checks preceded semantic review and both observed findings were repaired and +retested. Factory made no Firstmate or no-mistakes configuration change and +added no command, scheduler, reviewer service, CP-07 extension, or CP-08 app. + +Limitations: these measurements describe a small Linux fixture, not a general +retrieval-quality claim; native Windows/macOS remain unverified; whole-section +CTX retrieval inflates the strict irrelevant-excerpt metric despite low token +use; CTX offline doctor reported a lexical nested status while direct status and +every generated receipt reported `HYBRID_SEMANTIC`; and Pi targeted reads should +use canonical/absolute paths when the exercise lives under another Git root. From 8b945a4ed6fcbb43a8b7aa7ce3a6532a4a23d145 Mon Sep 17 00:00:00 2001 From: Factory CP-06 Worker Date: Mon, 28 Sep 2026 19:22:32 +0000 Subject: [PATCH 03/45] no-mistakes(document): Format workflow documentation table --- docs/WORKFLOW.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/WORKFLOW.md b/docs/WORKFLOW.md index 2e29fc2..cf87da7 100644 --- a/docs/WORKFLOW.md +++ b/docs/WORKFLOW.md @@ -33,10 +33,10 @@ A fresh worker receives this brief, the exact task contract, bounded pack, and e ## Risk and review -| Risk | Examples | Required human/independent check | -| --- | --- | --- | -| bounded | isolated deterministic refactor, docs typo | ordinary project checks; Firstmate's standard delivery policy. | -| normal | API behavior, persistence, UI journey | targeted integration/e2e evidence and a fresh semantic review when feasible. | +| Risk | Examples | Required human/independent check | +| -------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- | +| bounded | isolated deterministic refactor, docs typo | ordinary project checks; Firstmate's standard delivery policy. | +| normal | API behavior, persistence, UI journey | targeted integration/e2e evidence and a fresh semantic review when feasible. | | critical | authentication, tenancy, payments, destructive migration, external sends | explicit design constraints, negative tests, independent review, and human decision for consequential policy. | Risk is about consequences of error, not code size. Use the higher class when the contract understates an observed consequence, and stop for a human decision when critical policy or access is unresolved. From 8da67469434ac4590e37c19a8b206e3b7659a453 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 29 Sep 2026 08:48:01 +0000 Subject: [PATCH 04/45] fix(cp06): constrain context to task authority --- ARCHITECTURE.md | 2 +- BUILD_PLAN.md | 2 +- docs/CONTRACTS.md | 2 +- docs/INTEGRATIONS.md | 2 +- docs/WORKFLOW.md | 4 +- docs/probes/CP-06.md | 407 +++++++------- package.json | 3 + scripts/replay-cp06-p07.mjs | 128 +++++ scripts/replay-cp06-ten-pack.mjs | 504 ++++++++++++++++++ scripts/replay-cp06-worker.mjs | 158 ++++++ skills/factory/SKILL.md | 4 +- src/core/context.ts | 1 + test/context.test.ts | 37 +- .../cp06-context/.factory/completion.yaml | 73 +++ .../cp06-context/.factory/project.yaml | 17 + .../cp06-context/.factory/tasks/API-001.yaml | 16 + .../cp06-context/.factory/tasks/AUTH-001.yaml | 19 + .../.factory/tasks/CONFIG-001.yaml | 16 + .../cp06-context/.factory/tasks/COPY-001.yaml | 19 + .../.factory/tasks/DELETE-001.yaml | 19 + .../.factory/tasks/EXPORT-001.yaml | 16 + .../.factory/tasks/IMPORT-001.yaml | 16 + .../cp06-context/.factory/tasks/LOG-001.yaml | 16 + .../.factory/tasks/PERSIST-001.yaml | 16 + .../cp06-context/.factory/tasks/UI-001.yaml | 16 + test/fixtures/cp06-context/.gitignore | 2 + test/fixtures/cp06-context/ARCHITECTURE.md | 7 + test/fixtures/cp06-context/PROJECT.md | 9 + .../cp06-context/cases/AUTH-before.yaml | 17 + test/fixtures/cp06-context/docs/API.md | 3 + test/fixtures/cp06-context/docs/AUTH.md | 3 + test/fixtures/cp06-context/docs/CONFIG.md | 3 + test/fixtures/cp06-context/docs/DELETION.md | 3 + test/fixtures/cp06-context/docs/EXPORT.md | 3 + test/fixtures/cp06-context/docs/IMPORT.md | 3 + .../cp06-context/docs/OBSERVABILITY.md | 3 + .../fixtures/cp06-context/docs/PERSISTENCE.md | 3 + test/fixtures/cp06-context/docs/STYLE.md | 3 + test/fixtures/cp06-context/docs/UI.md | 3 + test/fixtures/cp06-context/oracle.json | 16 + test/fixtures/workflow/history/finding.json | 11 + .../workflow/history/malformed-json-probe.mjs | 7 + .../workflow/history/normal-api-before.mjs | 11 + test/workflow.test.ts | 19 + 44 files changed, 1401 insertions(+), 241 deletions(-) create mode 100644 scripts/replay-cp06-p07.mjs create mode 100644 scripts/replay-cp06-ten-pack.mjs create mode 100644 scripts/replay-cp06-worker.mjs create mode 100644 test/fixtures/cp06-context/.factory/completion.yaml create mode 100644 test/fixtures/cp06-context/.factory/project.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/API-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/AUTH-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/CONFIG-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/COPY-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/DELETE-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/EXPORT-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/IMPORT-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/LOG-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/PERSIST-001.yaml create mode 100644 test/fixtures/cp06-context/.factory/tasks/UI-001.yaml create mode 100644 test/fixtures/cp06-context/.gitignore create mode 100644 test/fixtures/cp06-context/ARCHITECTURE.md create mode 100644 test/fixtures/cp06-context/PROJECT.md create mode 100644 test/fixtures/cp06-context/cases/AUTH-before.yaml create mode 100644 test/fixtures/cp06-context/docs/API.md create mode 100644 test/fixtures/cp06-context/docs/AUTH.md create mode 100644 test/fixtures/cp06-context/docs/CONFIG.md create mode 100644 test/fixtures/cp06-context/docs/DELETION.md create mode 100644 test/fixtures/cp06-context/docs/EXPORT.md create mode 100644 test/fixtures/cp06-context/docs/IMPORT.md create mode 100644 test/fixtures/cp06-context/docs/OBSERVABILITY.md create mode 100644 test/fixtures/cp06-context/docs/PERSISTENCE.md create mode 100644 test/fixtures/cp06-context/docs/STYLE.md create mode 100644 test/fixtures/cp06-context/docs/UI.md create mode 100644 test/fixtures/cp06-context/oracle.json create mode 100644 test/fixtures/workflow/history/finding.json create mode 100644 test/fixtures/workflow/history/malformed-json-probe.mjs create mode 100644 test/fixtures/workflow/history/normal-api-before.mjs diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index bdb101c..92138af 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -43,7 +43,7 @@ Recommended exit codes: `0` success; `2` validation failure; `3` unavailable/sta ## CTX context pack -`factory context TASK-ID` first includes the exact tracked task contract and a concise global project constraint section. It invokes `ctx status --root --json` and `ctx doctor --offline --root --json` as appropriate, then `ctx pack --root --token-budget --json`. The query is derived from title, outcome, acceptance, and topic strings; it is never a command. Canonical required documents are read from their original files, checked against root containment, and admitted before optional CTX excerpts. The adapter records CTX generation, file and line provenance, and available hashes; it does not treat an excerpt as a permission to ignore original wording. On stale or insufficient retrieval, it returns `CONTEXT_BLOCKED` with an actionable diagnostic. It does not fill the pack with guessed material. +`factory context TASK-ID` first includes the exact tracked task contract and a concise global project constraint section. It invokes `ctx status --root --json` and `ctx doctor --offline --root --json` as appropriate, then `ctx pack --root --token-budget --document ... --json`. The query is derived from title, outcome, acceptance, and topic strings; it is never a command. Repeated `--document` filters come from the task's declared `context.required` selection, so CTX ranks within task-local authority instead of adding unrelated corpus excerpts. Canonical required documents are read from their original files, checked against root containment, and admitted before CTX excerpts; declaring a source required never substitutes an excerpt for the exact original. The adapter records CTX generation, file and line provenance, and available hashes. On stale, missing, or insufficient retrieval, it returns `CONTEXT_BLOCKED` with an actionable diagnostic. It does not fill the pack with guessed material or hide post-retrieval excerpts to improve a relevance score. Default total task-context budgets (including contract and mandatory material) are 4,000 tokens for bounded, 8,000 for normal, and 15,000 for critical; these are design defaults to measure, not promises that CTX necessarily returns that many tokens. Reserve space for task contract and required sources before calling CTX with the remainder. The adapter enforces a byte ceiling as well as CTX's token budget, deduplicates overlapping excerpts, and never truncates an authoritative requirement mid-sentence. The pack is written atomically to `.factory/state/context/TASK-ID.md` with a sidecar receipt containing contract digest, source digests, CTX generation, timestamp, and retrieval mode. A worker verifies these before trusting the pack; if changed, regenerate it. diff --git a/BUILD_PLAN.md b/BUILD_PLAN.md index 23a7a48..fc2ca53 100644 --- a/BUILD_PLAN.md +++ b/BUILD_PLAN.md @@ -89,7 +89,7 @@ - [x] Create tests/fixtures for critical auth isolation and normal API behavior, plus one simulated reviewer finding that causes repair and retest. - [x] Run an actual worker task with only task/pack/evidence sources in a fresh context; inspect whether it needs missing docs. -- [x] **Gate:** P-06, P-07, and no change to Firstmate routing or shipping config by Factory. +- [x] **Gate:** P-06 and P-07 via the tracked ten-pack/P-07 replay in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. ## CP-07 — Thin Pi command and status convenience diff --git a/docs/CONTRACTS.md b/docs/CONTRACTS.md index aa452d3..a3616a8 100644 --- a/docs/CONTRACTS.md +++ b/docs/CONTRACTS.md @@ -80,7 +80,7 @@ evidence: delivery: project-default ``` -`depends_on` names local task IDs, is acyclic, and cannot name self. `advances` names completion IDs and may be empty only for a task explicitly marked `kind: enabling`; enabling tasks still have testable outcomes. `complexity` is `bounded | normal | critical`; `risk` is `bounded | normal | critical` and controls review policy, not a specific model name. `delivery` defaults to `project-default` and cannot override the actual Firstmate project mode; if the project mode conflicts with a task request, validation blocks publication. All required evidence categories are verified by evidence receipts rather than assumed from task closure. +`depends_on` names local task IDs, is acyclic, and cannot name self. `advances` names completion IDs and may be empty only for a task explicitly marked `kind: enabling`; enabling tasks still have testable outcomes. `complexity` is `bounded | normal | critical`; `risk` is `bounded | normal | critical` and controls review policy, not a specific model name. `context.required` is the reviewed task-local authority selection: Factory reads every listed original exactly and passes the same paths as CTX document filters. Include each known source that can decide an acceptance or risk condition; do not add broad directories or unrelated documents as insurance. A worker that discovers an omitted decisive source names it, performs one targeted original read, and proposes that exact path for the contract before regeneration. `delivery` defaults to `project-default` and cannot override the actual Firstmate project mode; if the project mode conflicts with a task request, validation blocks publication. All required evidence categories are verified by evidence receipts rather than assumed from task closure. ## Task receipt: `.factory/state/evidence/SAVE-001.json` diff --git a/docs/INTEGRATIONS.md b/docs/INTEGRATIONS.md index 28c77ad..2fd95bf 100644 --- a/docs/INTEGRATIONS.md +++ b/docs/INTEGRATIONS.md @@ -8,7 +8,7 @@ The user's archived CTX README describes a Python 3.12+ local CLI, source-proven CP-00 probes: detect `ctx`; create a disposable Git fixture with two normative Markdown docs; initialize and add docs according to local `ctx --help`; index with `--no-embeddings` so the test does not download anything; run status, offline doctor, pack, and exact lines; record actual JSON fields and failure codes. Confirm stale-source detection by changing a source after indexing. A user's globally installed model may then be tested separately. Do not index the Factory repo until its initial documents are approved. -Context policy: required files are read exactly; CTX provides relevant additional passages with source provenance; changed docs invalidate a pack; unavailable local embedding may allow the explicitly reported lexical mode if it retrieves sufficient authority. Query text is data, never shell code. `ctx pack` is one context source, not a substitute for Git, `rg`, or project checks. +Context policy: required files are read exactly. The installed `ctx pack` accepts repeated `--document` filters; Factory supplies the task's reviewed `context.required` paths so retrieval ranks only within task-local authority while exact originals remain in the pack. Changed docs invalidate a pack; unavailable local embedding may allow the explicitly reported lexical mode if it retrieves sufficient authority. Query text is data, never shell code. `ctx pack` is one context source, not a substitute for Git, `rg`, or project checks, and Factory does not post-filter CTX output to falsify provenance. ## Pi diff --git a/docs/WORKFLOW.md b/docs/WORKFLOW.md index cf87da7..d512ad4 100644 --- a/docs/WORKFLOW.md +++ b/docs/WORKFLOW.md @@ -9,7 +9,7 @@ This document specifies the methodology. CP-04 turns it into a concise, discover 3. **Project truth:** author a concise `PROJECT.md` with goal, audience, user journeys, non-goals, constraints, and V1 limits. Draft `ARCHITECTURE.md` with component boundaries, data model, trust boundary, deployment shape, and failure handling. Keep technical choices proportionate to the first release. 4. **Independent oracle:** in a separate reasoning pass, author `.factory/completion.yaml` from product intent and risks before decomposing into tasks. Specify observable end-to-end behavior, isolation/security invariants, recovery, and any performance thresholds the project actually needs. Include one negative test for important trust boundaries. Record the current Git SHA of this first oracle so later changes are visible. 5. **Slices and DAG:** choose the first vertical slice that a user can exercise; add a minimal foundation only where it unlocks slices. Tasks should have one independently reviewable outcome, bounded scope, testable acceptance IDs, expected evidence, and dependencies. Prefer a serial backbone of slices with at most three genuinely independent ready tasks. Do not turn one feature into separate backend/frontend/UI megatasks that integrate only at the end. -6. **Validate and retrieve:** use `factory validate`; repair schema/graph gaps; generate a context pack for each task via CTX. Inspect the first few packs for missing mandatory source and irrelevant bulk. Use exact source reads on decisive constraints. +6. **Validate and retrieve:** use `factory validate`; repair schema/graph gaps; declare each known decisive original in the task's `context.required` selection, then generate its context pack via CTX. Factory constrains CTX to those task-local documents while retaining exact global and required originals. Inspect the first few packs for a missing decisive source and irrelevant bulk; correct an omission with one targeted exact read and the narrowest contract update, not a broad corpus expansion or a post-retrieval provenance filter. 7. **Preview and publish:** run `factory sync --dry-run --home `, inspect which items will be created and which are blocked, then explicitly apply to the selected home. The engineer's direction to execute work authorizes normal publication into the intended Firstmate home; ask only if the home or project identity is genuinely ambiguous. 8. **Execute with Firstmate:** Firstmate chooses ready work and workers through its existing dispatch policy. Each worker receives the task contract, relevant source pack, and evidence expectations; worker tests and repairs locally. Firstmate handles the actual delivery path and approvals required by its configured project mode. 9. **Verify per task and per project:** run deterministic gates first; semantic review receives task, diff, architecture, and observed evidence rather than the worker's full transcript. Repair specific findings and rerun changed checks. Then execute the original project oracle at the release candidate SHA. Failed project behavior creates a reviewed follow-up task. The status report must show backlog and product completion separately. @@ -29,7 +29,7 @@ Delivery: [Firstmate project mode] Escalate: product policy, new access, contradictory authoritative docs, or scope change ``` -A fresh worker receives this brief, the exact task contract, bounded pack, and existing evidence—not a previous agent transcript. It must verify the pack receipt and source freshness, inspect only relevant code, and run evidence against the actual change. If the pack lacks a source needed to decide or test an acceptance or risk condition, name the missing source, read only the relevant exact-original file or range, verify its digest when available, and record how that read corrected the working assumptions. Do not compensate by loading broad documentation or by dropping a critical constraint. A context pack is supporting material, not an instruction channel with authority above the project's tracked documents. +A fresh worker receives this brief, the exact task contract, bounded pack, and existing evidence—not a previous agent transcript. It must verify the pack receipt and source freshness, inspect only relevant code, and run evidence against the actual change. If the pack lacks a source needed to decide or test an acceptance or risk condition, name the missing source, read only the relevant exact-original file or range, verify its digest when available, and record how that read corrected the working assumptions. Add that exact path to `context.required` before regeneration when it is durable task authority. Do not compensate by loading broad documentation, hiding retrieved excerpts after the fact, or dropping a critical constraint. A context pack is supporting material, not an instruction channel with authority above the project's tracked documents. ## Risk and review diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 793b054..2c9d899 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -1,245 +1,194 @@ -# CP-06 probe report — review policy and fresh-context workflow - -Date: 2026-09-28. Active checkpoint: `BUILD_PLAN.md` CP-06 only. Base SHA: -`8337ab7fcc5b511d3cb359ea214b6ae18003aeaa`. Implementation SHA tested before -this report: `90ae0ef05329ed82812fb6e1edc83458d68b3152`. Acceptance exercised: P-06 -and P-07. - -## Scope and isolation - -`pwd -P` and `git rev-parse --show-toplevel` both returned -`/home/ansh/.treehouse/Factory-86709f/6/Factory`; the initial HEAD was the -required clean main SHA. Work occurred on `fm/factory-cp06`. No real Firstmate -home, account, credential, network service, routing setting, shipping setting, -or shared daemon was changed. Generated packs and raw Pi traces stayed under -ignored `.factory/state/` and are not shipped. - -The project-local CTX index was initialized offline from only `README.md`, -`PROJECT.md`, `ARCHITECTURE.md`, `COMPLETION.md`, `BUILD_PLAN.md`, -`docs/WORKFLOW.md`, `docs/CONTRACTS.md`, and `docs/INTEGRATIONS.md`. CTX `1.0.0` -verified `BAAI/bge-small-en-v1.5`; indexing reported semantic, lexical, and -structural channels. A strict 60,000-token project pack reported `CONFLICTING` -because its deterministic cross-document detector selected broad checkpoint -material. It was not treated as sufficient authority. The decisive original -Markdown was read directly before implementation. - -Installed boundaries used here were Node `v24.20.0`, npm `12.0.2`, TypeScript -`7.0.2`, Pi `0.85.1`, CTX `1.0.0`, and no-mistakes `v1.57.0`. no-mistakes was -not called by Factory or embedded in the workflow. - -## Workflow and fixtures - -`docs/WORKFLOW.md` and `skills/factory/SKILL.md` now classify risk by failure -consequence: - -- bounded work uses deterministic project checks and Firstmate's normal delivery; -- normal work adds targeted integration/e2e evidence before a fresh semantic review; -- critical work adds explicit trust-boundary constraints, isolated negative tests, - independent review, and human resolution of consequential policy. - -Review receives the exact task, focused diff, relevant architecture or targeted -exact-original correction, and observed evidence—not a previous worker -transcript. Findings must cite an acceptance ID, scenario, or concrete risk and -end repaired and retested, evidence-disputed, or blocked. Factory neither invokes -no-mistakes nor changes Firstmate routing or approvals. - -The executable fixtures are separate: - -- `test/fixtures/workflow/normal-api.ts` covers normal profile API success and - malformed/invalid/unknown/unsupported negative behavior. -- `test/fixtures/workflow/critical-auth-isolation.ts` uses only in-memory - `account-a`/`account-b` and disposable `fixture-token-*` strings. Negative - checks cover missing and forged fixture credentials, cross-account - read/update/delete concealment, unchanged state, and ignored request-supplied - ownership on create. It contains no real account, secret, or I/O. - -## P-06: ten bounded context packs - -### Method - -A disposable Git project was created at ignored -`.factory/state/cp06-ten-pack`. Before any output was inspected, it was fixed at -10 task contracts with `complexity: bounded`: two bounded-risk, six normal-risk, -and two critical-risk tasks. Every task had one known task-specific source, -while `PROJECT.md` and `ARCHITECTURE.md` were mandatory for all packs. Eleven -sources were indexed at CTX generation 1; the known auth source was deliberately -left unindexed to include one retrieval-gap case in the fixed sample. This makes -the observed miss a fault-injected sample result, not a claimed production -baseline. - -Each pack used the implemented command and the bounded default: +# CP-06 correction proof — relevant context and review ordering + +Date: 2026-09-29. Active checkpoint: `BUILD_PLAN.md` CP-06 only. Acceptance: +P-06 and P-07. Original base: +`8337ab7fcc5b511d3cb359ea214b6ae18003aeaa`. Original PR #8 head (preserved +unchanged in this branch's history): +`8b945a4ed6fcbb43a8b7aa7ce3a6532a4a23d145`. + +This correction was performed in the isolated worktree reported by both +`pwd -P` and `git rev-parse --show-toplevel` as +`/home/ansh/.treehouse/Factory-86709f/7/Factory`. It did not touch PR #8, its +branch or worker state, a real Firstmate home, routing, shipping configuration, +accounts, credentials, or the shared no-mistakes daemon. + +## Normative result + +The PR #8 measurement is a **failure**, not a P-06 pass. Its predeclared oracle +counted 15 of 24 topical excerpts as irrelevant, every one of 10 packs had at +least one irrelevant excerpt, and AUTH was missing from one pack. Bounded bytes +do not make irrelevant authority relevant. The original figures remain in the +runtime evidence manifest and Git history; this correction does not relabel or +hide them. + +The smallest in-scope correction uses CTX's existing repeated `--document` +filter. Factory now passes the task's reviewed `context.required` paths to +`ctx pack`. Those same originals are still read exactly and included with their +digests. This is source selection before retrieval, not post-retrieval deletion: +raw CTX output remains in ignored proof scratch, provenance is unchanged, and a +missing declared source still fails closed. No CTX code, parallel retrieval +system, model router, supervisor, CP-07 extension, or CP-08 application was +added. + +The contract policy now makes `context.required` the narrow task-local authority +selection. Known sources that can decide an acceptance or risk condition belong +there. A worker that discovers an omission performs one targeted exact-original +read, records the corrected assumption, and adds only that durable source before +regeneration. Global constraints and exact required originals remain visible. + +## Reproducible evidence layout + +Tracked, secret-free inputs and runners: + +- `test/fixtures/cp06-context/` — exact project, architecture, ten task + contracts, ten topical originals, relevance oracle, and the AUTH-before + counterfactual; +- `scripts/replay-cp06-ten-pack.mjs` — creates a disposable Git/CTX repository, + records the unscoped bad path, one-condition AUTH correction, exact ten fixed + packs and receipts, and a missing-original disconfirming case; +- `scripts/replay-cp06-worker.mjs` — gives a new Pi process only the bounded AUTH + handoff and permits one exact-original read; +- `test/fixtures/workflow/history/` and + `scripts/replay-cp06-p07.mjs` — compact P-07 finding plus executable + fail-before/fix-after/auth-retest sequence; +- `test/workflow.test.ts` — checked-in normal API and in-memory auth behavior, + including the retained malformed-JSON regression check. + +Generated raw CTX JSON, packs, receipts, model output, and command logs remain +ignored under `.factory/state/cp06-correction/`; they are not committed +transcripts. Each replay writes a structured `evidence.json` containing command, +exit, output path and SHA-256, tested Git SHA, UTC date, reviewer, tool/index +identity, source/contract/pack/receipt hashes, budgets, and provenance. This +meets the general proof rule without committing runtime state or model traces. + +Run in this order: ```text -node dist/src/cli.js context \ - --root .factory/state/cp06-ten-pack --json +npm ci +npm run format:check +npm run lint +npm run typecheck +npm test +npm run proof:cp06:ten-pack +npm run proof:cp06:p07 +npm run proof:cp06:worker ``` -All 10 commands exited 0. Every result reported a 4,000-token / 16,000-byte -ceiling, generation 1, and `HYBRID_SEMANTIC`. Relevance was scored against the -predeclared task-specific source: mandatory project/architecture sources and the -one target source were relevant; other retrieved topical files were counted as -irrelevant bulk even where tangentially useful. This strict rule avoids judging -retrieval after seeing the result. - -### Results - -`P` and `A` below are the exact mandatory `PROJECT.md` and `ARCHITECTURE.md`. -The source names after them are exact file provenance from each pack. - -| Task | Risk | Tokens | Bytes | Exact source provenance | Irrelevant retrieved | Missing target | -| ----------- | -------- | -----: | ----: | ----------------------------------- | -------------------: | -------------- | -| COPY-001 | bounded | 1,098 | 3,293 | P, A, PERSISTENCE, STYLE, UI | 2 | no | -| CONFIG-001 | bounded | 1,117 | 3,349 | P, A, API, CONFIG, IMPORT | 2 | no | -| API-001 | normal | 971 | 2,911 | P, A, API, EXPORT | 1 | no | -| PERSIST-001 | normal | 963 | 2,889 | P, A, DELETION, PERSISTENCE | 1 | no | -| UI-001 | normal | 960 | 2,879 | P, A, DELETION, UI | 1 | no | -| IMPORT-001 | normal | 1,116 | 3,346 | P, A, DELETION, IMPORT, PERSISTENCE | 2 | no | -| LOG-001 | normal | 1,116 | 3,347 | P, A, API, DELETION, OBSERVABILITY | 2 | no | -| EXPORT-001 | normal | 1,116 | 3,346 | P, A, API, EXPORT, PERSISTENCE | 2 | no | -| DELETE-001 | critical | 977 | 2,931 | P, A, DELETION, OBSERVABILITY | 1 | no | -| AUTH-001 | critical | 822 | 2,464 | P, A, UI | 1 | **yes: AUTH** | - -Observed rates under that fixed scoring rule: - -- missing-source rate: **1/10 packs = 10%**; -- irrelevant-bulk excerpt rate: **15/24 retrieved topical excerpts = 62.5%**; -- packs with at least one irrelevant retrieved excerpt: **10/10**; -- largest pack: **1,117/4,000 estimated tokens = 27.9%** of its ceiling. - -The high excerpt-rate result is not hidden: this tiny corpus made broad -whole-section retrieval cheap, but the material was still not task-specific. -The absolute packs remained small and retained all global constraints. - -### Authoritative provenance - -The generated receipts recorded full source digests. The exact originals used -by the relevance oracle were: +The exact runtime manifests are: ```text -PROJECT.md 3c668e2d2d77459e34e6b51c077d0ef1896ae499a5117aac9e176e85e93d8a3a -ARCHITECTURE.md 78a38083965d8af92c69f98027fc0f2034b8d8d7022127669d8d5f8723992125 -docs/STYLE.md ebfa3910e0bf0888d9d8f75ae5c8e23a2f171adfb07ab086c52a5c00b5175540 -docs/CONFIG.md 06f97213a6f4c1d994ba51e8207d6bf6ce8572195b323445a734b6169a70553c -docs/API.md 266eaeec31bd8f921f9a31fc14a56ee58518a292ee52d7c2b67145352202b33a -docs/PERSISTENCE.md 93529530174cd2461fea38a612044e0c726f4fe0cefb67a6f930398eb8c173c0 -docs/UI.md c97aea3e2e2f6010751ad8a53eaf68534bf0aaa1948a596b1e0c0c015b92e3e4 -docs/IMPORT.md 957dd9218a77140cf5d3ea9fbe69f8abb184c32bbb6cfe06b7fbc063daee0ff0 -docs/OBSERVABILITY.md 6ac6d02e01501cd2ecc5373344ff8660fd9df9e7191e015b700515db336f13e5 -docs/EXPORT.md bdbfb5c931b3dec9f77ebd72b5e3aaee1ad07fa03df9b3aaf530d7f0dc6f82d9 -docs/DELETION.md 547229c8f3c9c6a5ff561a42c87ce20f96333ffa09dd730f9cde6e4d0a1c9f27 -docs/AUTH.md f64ee62cdbe01d7fb00f166f5246a58577eea369121e9a573ce219a0674fa587 +.factory/state/cp06-correction/ten-pack/evidence.json +.factory/state/cp06-correction/p07/evidence.json +.factory/state/cp06-correction/worker/evidence.json ``` -### Missing-source correction - -The critical AUTH pack did not contain `docs/AUTH.md`; it returned unrelated UI -context instead. A targeted exact-original read—not a broad corpus expansion— -printed only `docs/AUTH.md:1-2` and verified digest -`f64ee62cdbe01d7fb00f166f5246a58577eea369121e9a573ce219a0674fa587`. -That source says ownership comes from the authenticated principal, supplied -owner fields are ignored, missing/forged fixture credentials are `401`, and -cross-account read/update/delete is a non-mutating, non-disclosing `404`. - -The correction added only `docs/AUTH.md` to the disposable AUTH task's required -sources and regenerated that pack. The command exited 0 at 997 tokens / 2,990 -bytes. `previous_receipt_valid` was false with `CONTRACT_CHANGED` and -`SOURCE_CHANGED:docs/AUTH.md`; the new receipt included the exact AUTH digest and -the pack visibly retained the cross-account constraint. No product contract in -this Factory repository was changed for the exercise. - -## Fresh-context worker exercise - -A new ephemeral Pi `0.85.1` process used `--no-session --no-context-files`, only -the Factory skill and read tool, and Pi's existing OAuth. Its initial user -context contained exactly the disposable AUTH task contract, the original -822-token bounded pack, and pre-review executable evidence—no previous agent -transcript. The worker: - -1. reported `MISSING_SOURCE` for the expected `docs/AUTH.md` because the critical - auth semantics were absent; -2. issued exactly one targeted read of the supplied exact original; -3. corrected the trust boundary to include principal-derived ownership, - ignored request ownership, `401` credential rejection, and non-disclosing, - non-mutating `404` cross-account behavior; and -4. identified two evidence gaps: request-supplied ownership and create-time - principal binding. - -The first path probe demonstrated a Pi portability caveat: a relative read was -resolved from the containing Git root rather than the process subdirectory and -failed closed. A second genuinely fresh process received the same three -artifacts and an absolute exact-original path; it performed one successful read -and no broad scan. Raw model traces remain ignored because transcripts do not -belong in packs or tracked evidence. - -## P-07: checks before review, finding, repair, retest - -The sequence was intentionally ordered and runnable: - -1. A simulated normal-API reviewer finding required malformed JSON to return a - stable `400 invalid_json` response. The regression was written first. - `npm run build` exited 2 on initial fixture typing, and - `node --test dist/test/workflow.test.js` exited 1 with 4 passed / 1 failed; - the failure was an uncaught `SyntaxError` from `JSON.parse`. -2. The fixture was repaired with an explicit parse failure response. After type - repair, `npm run build && node --test dist/test/workflow.test.js` exited 0 - with 5 passed / 0 failed. This deterministic evidence existed before the - fresh semantic exercise. -3. The fresh semantic worker then found the critical A-AUTH-001 coverage gap - described above. The fixture was repaired so create derives owner only from - the authenticated principal and ignores a supplied owner. A new negative - test proves account A cannot access account B's forged-owner creation. -4. `npm run build && node --test dist/test/workflow.test.js` was rerun and exited - 0 with 6 passed / 0 failed. - -Thus both findings have a repair and changed-behavior retest; no critical review -finding is left silently open. - -## Pi skill proof - -Installed Pi RPC `get_commands` with only `skills/factory` loaded exited 0 and -returned `skill:factory` from this worktree. An explicit `/skill:factory` -invocation in a separate ephemeral process exited 0 and returned the current -implemented command boundary: +Inspect each manifest's `tested_sha` rather than copying a stale SHA into this +tracked report. The ten-pack manifest also carries every current contract, +source, pack, receipt, CLI-output digest, CTX generation, token/byte count, and +provenance list. -```text -doctor, init, validate, context TASK-ID, sync --dry-run, sync --apply, -evidence TASK-ID, status [--json] -``` +## P-06 diagnosis and fixed ten-pack replay -It rejected invented `/factory`, `factory run`, and `factory inspect` commands; -identified Firstmate as owner of dispatch, worker lifecycle, approvals, and -shipping; placed deterministic evidence before semantic review; required -critical isolated negatives; and required repair/retest or a blocker. +The fixture uses only local Markdown and the installed CTX `1.0.0` with verified +`BAAI/bge-small-en-v1.5`. `ctx doctor --offline` must report +`offline_ready=true` and `network_attempted=false`; status must be clean +`HYBRID_SEMANTIC` with the semantic channel active. The runner aborts otherwise. +It indexes only the fixture's deliberate authoritative Markdown. -## Deterministic validation at implementation SHA +The bad-path control invokes real CTX with the AUTH query before `docs/AUTH.md` +is indexed and without a document selection. On the recorded replay it exited +0, omitted AUTH, and returned unrelated topical `docs/DELETION.md` and +`docs/OBSERVABILITY.md`. The raw response is retained in ignored scratch. This +corroborates the user-visible PR #8 problem rather than treating small byte size +as relevance. -Against `90ae0ef05329ed82812fb6e1edc83458d68b3152`: +The one-condition counterfactual keeps every AUTH task field fixed except +`context.required`: ```text -npm ci exit=0; 0 vulnerabilities -npm run format:check exit=0 -npm run lint exit=0; lint ok -npm run typecheck exit=0 -npm test exit=0; 72 passed, 0 failed -git diff --check exit=0 +before: [PROJECT.md, ARCHITECTURE.md] +after: [PROJECT.md, ARCHITECTURE.md, docs/AUTH.md] ``` -`fm-ensure-agents-md.sh .` exited 0 and reported the existing `AGENTS.md` and -`CLAUDE.md` pointer unchanged. The final report/checkpoint commit receives the -same deterministic suite and independent delivery review before publication. - -## Acceptance and limitations - -P-06 passes for this defined ten-pack sample with the measured 10% injected miss -and 62.5% strict irrelevant-excerpt rate; the miss was surfaced, exactly read, -and corrected without losing critical constraints. P-07 passes because cheap -checks preceded semantic review and both observed findings were repaired and -retested. Factory made no Firstmate or no-mistakes configuration change and -added no command, scheduler, reviewer service, CP-07 extension, or CP-08 app. - -Limitations: these measurements describe a small Linux fixture, not a general -retrieval-quality claim; native Windows/macOS remain unverified; whole-section -CTX retrieval inflates the strict irrelevant-excerpt metric despite low token -use; CTX offline doctor reported a lexical nested status while direct status and -every generated receipt reported `HYBRID_SEMANTIC`; and Pi targeted reads should -use canonical/absolute paths when the exercise lives under another Git root. +Before correction, the bounded pack omits the decisive original. After the one +contract change, the regenerated pack contains exact `docs/AUTH.md` with its +digest and retains all global constraints. It corrects the assumptions to: +principal-derived ownership, ignored request ownership, `401` for missing or +forged credentials, and non-disclosing/non-mutating `404` for cross-account +read/update/delete. + +The fixed sample is ten bounded packs: two bounded-risk, six normal-risk, and +two critical-risk tasks. The oracle is declared in +`test/fixtures/cp06-context/oracle.json` before output inspection: global +`PROJECT.md` and `ARCHITECTURE.md` plus the task's single target source are +relevant; every other topical source is irrelevant. The replay result is: + +- **10/10 packs relevant**; +- **0 irrelevant topical excerpts** and **0 packs with an irrelevant excerpt**; +- **0 missing targets**; +- every pack within its 4,000-token / 16,000-byte bounded ceiling; +- CTX generation 2 and `HYBRID_SEMANTIC` recorded per receipt; +- exact contract, target-source, pack, receipt, output, and source digests + recorded per task. + +The disconfirming case changes AUTH to require nonexistent `docs/ABSENT.md`. +Factory exits 3 with `CONTEXT_BLOCKED` and writes no replacement pack. Thus the +correction does not claim that a source filter can recover undeclared or absent +authority and does not fabricate completeness. + +## Fresh-context targeted-original correction + +After deterministic checks and the real-CTX replay, a new Pi `0.85.1` process +runs with `--no-session`, `--no-context-files`, no discovered extensions or +skills, the Factory skill explicitly selected, and only the `read` tool. Its +initial handoff contains exactly the AUTH-before contract, bounded missing-AUTH +pack, and a short deterministic evidence statement—no prior transcript. + +The worker reports `MISSING_SOURCE`, performs exactly one read of the supplied +`docs/AUTH.md` original, performs no broad scan, recovers all four decisive +constraints above, and identifies the untested request-owner/create-time binding +case. `scripts/replay-cp06-worker.mjs` records the raw output only in ignored +scratch and deterministically checks the one-read path and required corrected +constraints. Model wording is inherently nondeterministic; the structural +oracle and exact source are not. + +## P-07 finding, repair, and retest + +`test/fixtures/workflow/history/finding.json` is the minimal structured reviewer +record for `P07-MALFORMED-JSON`. The retained before module throws `SyntaxError` +for malformed profile JSON. The repaired normal API returns stable +`400 invalid_json` through its normal request function. + +`npm run proof:cp06:p07` executes and records this order: + +1. build — expected exit 0; +2. fail-before malformed-JSON probe — expected exit 1 with uncaught + `SyntaxError`; +3. final malformed-JSON behavior retest — expected exit 0; +4. final critical in-memory auth tests — expected exit 0. + +The final behavioral fixture also proves absent/forged credentials return 401, +own-account access succeeds, cross-account read/update/delete return 404 without +mutation, request-supplied ownership is ignored, and stored ownership comes from +the principal. It uses only disposable literal identities and in-memory state; +there is no network, filesystem, environment, real account, or security-sensitive +credential access. + +This task-level review remains ordered after deterministic checks and before the +one normal Firstmate-owned delivery pipeline. Factory neither invokes +no-mistakes nor changes its ownership boundary. + +## Acceptance and residual limitations + +P-06 passes for the corrected, predeclared ten-pack sample; the original PR #8 +sample remains explicitly failed. P-07 passes because the structured finding, +fail-before, repair, final behavior retest, and auth retest are executable and +bound to the replayed SHA. CP-06 makes no change to Firstmate dispatch, delivery, +or no-mistakes policy. + +Residual limits are explicit: this is a small Linux fixture using CTX `1.0.0`, +not a universal retrieval-quality claim; native Windows/macOS replay is +unverified; semantic model wording and ranking can vary in later CTX/model +versions and therefore fail the proof rather than being silently accepted; and +raw runtime artifacts remain intentionally ignored and must be regenerated by +an independent reviewer. diff --git a/package.json b/package.json index fcee6cd..6b10d02 100644 --- a/package.json +++ b/package.json @@ -13,6 +13,9 @@ "format": "prettier --write \"{src,test,scripts,.github,schemas}/**/*.{ts,js,mjs,yml,yaml,json}\" \"*.json\" \"docs/probes/*.md\" \"docs/superpowers/plans/*.md\"", "format:check": "prettier --check \"{src,test,scripts,.github,schemas}/**/*.{ts,js,mjs,yml,yaml,json}\" \"*.json\" \"docs/probes/*.md\" \"docs/superpowers/plans/*.md\"", "lint": "node scripts/lint.mjs", + "proof:cp06:ten-pack": "npm run build && node scripts/replay-cp06-ten-pack.mjs", + "proof:cp06:p07": "node scripts/replay-cp06-p07.mjs", + "proof:cp06:worker": "node scripts/replay-cp06-worker.mjs", "factory": "node dist/src/cli.js" }, "engines": { diff --git a/scripts/replay-cp06-p07.mjs b/scripts/replay-cp06-p07.mjs new file mode 100644 index 0000000..ec81748 --- /dev/null +++ b/scripts/replay-cp06-p07.mjs @@ -0,0 +1,128 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { writeFileSync } from "node:fs"; +import { mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { spawnSync } from "node:child_process"; + +const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); +const rawRoot = join(outputRoot, "p07", "raw"); +await rm(join(outputRoot, "p07"), { recursive: true, force: true }); +await mkdir(rawRoot, { recursive: true }); + +const steps = []; +steps.push(run("build", "npm", ["run", "build"], 0)); +steps.push( + run( + "fail-before", + process.execPath, + ["test/fixtures/workflow/history/malformed-json-probe.mjs"], + 1, + ), +); +steps.push( + run( + "fix-after", + process.execPath, + [ + "--test", + "--test-name-pattern=normal API fixture rejects malformed", + "dist/test/workflow.test.js", + ], + 0, + ), +); +steps.push( + run( + "critical-auth-retest", + process.execPath, + ["--test", "--test-name-pattern=critical auth fixture", "dist/test/workflow.test.js"], + 0, + ), +); + +const findingPath = "test/fixtures/workflow/history/finding.json"; +const finding = JSON.parse(await readFile(join(root, findingPath), "utf8")); +const testedSha = runText("git", ["rev-parse", "HEAD"]); +const manifest = { + schema_version: 1, + gate: "P-07", + tested_sha: testedSha, + recorded_at: new Date().toISOString(), + reviewer: process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker", + finding: { + path: findingPath, + sha256: await fileHash(join(root, findingPath)), + record: finding, + }, + sequence: steps, + result: "pass", + limitations: [ + "Behavioral fixtures are in-memory and make no network or real-account access.", + "The historical module is a minimal retained reproduction, not a raw worker transcript.", + ], +}; +const manifestPath = join(outputRoot, "p07", "evidence.json"); +await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); +console.log( + JSON.stringify({ + ok: true, + gate: "P-07", + tested_sha: testedSha, + manifest_path: relative(manifestPath), + manifest_sha256: await fileHash(manifestPath), + sequence: steps.map(({ id, command, exit_code, output_path, output_sha256 }) => ({ + id, + command, + exit_code, + output_path, + output_sha256, + })), + }), +); + +function run(id, command, args, expectedExit) { + const result = spawnSync(command, args, { cwd: root, encoding: "utf8", env: process.env }); + const exitCode = result.status; + const output = `command: ${quote([command, ...args])}\nexit: ${exitCode}\n--- stdout ---\n${result.stdout ?? ""}\n--- stderr ---\n${result.stderr ?? ""}`; + const outputPath = join(rawRoot, `${id}.txt`); + writeFileSync(outputPath, output); + if (result.error !== undefined || exitCode !== expectedExit) { + process.stderr.write(output); + throw new Error(`${id} exited ${exitCode}; expected ${expectedExit}`); + } + return { + id, + command: quote([command, ...args]), + expected_exit_code: expectedExit, + exit_code: exitCode, + output_path: relative(outputPath), + output_sha256: sha256(output), + }; +} + +function runText(command, args) { + const result = spawnSync(command, args, { cwd: root, encoding: "utf8" }); + if (result.status !== 0) throw new Error(`${command} failed: ${result.stderr}`); + return result.stdout.trim(); +} + +function quote(parts) { + return parts.map((part) => (/[\s'"]/.test(part) ? JSON.stringify(part) : part)).join(" "); +} + +function relative(path) { + return path.startsWith(`${root}/`) ? path.slice(root.length + 1) : path; +} + +function sha256(text) { + return createHash("sha256").update(text).digest("hex"); +} + +async function fileHash(path) { + return createHash("sha256") + .update(await readFile(path)) + .digest("hex"); +} diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs new file mode 100644 index 0000000..489f895 --- /dev/null +++ b/scripts/replay-cp06-ten-pack.mjs @@ -0,0 +1,504 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { readFileSync, writeFileSync } from "node:fs"; +import { cp, mkdir, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { spawnSync } from "node:child_process"; + +const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const fixtureSource = join(root, "test/fixtures/cp06-context"); +const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); +const proofRoot = join(outputRoot, "ten-pack"); +const fixtureRoot = join(proofRoot, "repo"); +const rawRoot = join(proofRoot, "raw"); +const cli = join(root, "dist/src/cli.js"); +const ctx = resolveTool(process.env.CTX_BIN ?? "ctx"); +const reviewer = process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker"; +const commands = []; + +await rm(proofRoot, { recursive: true, force: true }); +await mkdir(rawRoot, { recursive: true }); +await cp(fixtureSource, fixtureRoot, { recursive: true }); +await mkdir(join(fixtureRoot, ".factory/state"), { recursive: true }); +const oracle = JSON.parse(await readFile(join(fixtureRoot, "oracle.json"), "utf8")); +const fixedAuth = await readFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), "utf8"); +const beforeAuth = await readFile(join(fixtureRoot, "cases/AUTH-before.yaml"), "utf8"); +await writeFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), beforeAuth); + +run("git-init", "git", ["init", "-q", fixtureRoot], { cwd: root }); +run("git-author-name", "git", ["config", "user.name", "Factory CP-06 fixture"], { + cwd: fixtureRoot, +}); +run("git-author-email", "git", ["config", "user.email", "factory-cp06@example.invalid"], { + cwd: fixtureRoot, +}); +run("git-add", "git", ["add", "."], { cwd: fixtureRoot }); +run("git-commit", "git", ["commit", "-qm", "seed CP-06 ten-pack fixture"], { + cwd: fixtureRoot, +}); +run("ctx-init", ctx, ["init", fixtureRoot, "--json"], { cwd: fixtureRoot }); + +const globalSources = ["PROJECT.md", "ARCHITECTURE.md"]; +const topicalSources = oracle.tasks.map((entry) => entry.target); +for (const path of [ + ...globalSources, + ...topicalSources.filter((path) => path !== "docs/AUTH.md"), +]) { + run( + `ctx-add-${slug(path)}`, + ctx, + ["add", path, "--root", fixtureRoot, "--authority", "normative", "--priority", "100", "--json"], + { + cwd: fixtureRoot, + }, + ); +} +run("ctx-index-without-auth", ctx, ["index", "--root", fixtureRoot, "--json"], { + cwd: fixtureRoot, + timeout: 120_000, +}); +const initialStatus = jsonRun( + "ctx-status-without-auth", + ctx, + ["status", "--root", fixtureRoot, "--json"], + { + cwd: fixtureRoot, + }, +); +assertSemantic(initialStatus, "initial status"); + +const authQuery = [ + "Bind item ownership to authenticated principal", + "Creation ignores request-supplied ownership and cross-account operations remain concealed.", + "A-AUTH: Missing or forged credentials return 401 and cross-account read update and delete return non-mutating 404.", + "principal ownership tenancy forged credentials", +].join("\n"); +const broad = run( + "bad-unscoped-auth", + ctx, + ["pack", authQuery, "--root", fixtureRoot, "--token-budget", "3500", "--json"], + { + cwd: fixtureRoot, + expected: [0, 3], + timeout: 120_000, + }, +); +const broadJson = parseJsonIfPresent(broad.stdout); +const broadPaths = itemPaths(broadJson); +const broadIrrelevant = broadPaths.filter( + (path) => path.startsWith("docs/") && path !== "docs/AUTH.md", +); +if (broad.exitCode === 0 && broadIrrelevant.length === 0) { + throw new Error("bad-path probe neither blocked nor exposed unrelated topical retrieval"); +} +if (broadPaths.includes("docs/AUTH.md")) { + throw new Error("bad-path probe unexpectedly retrieved deliberately unindexed docs/AUTH.md"); +} + +const initialAuth = factoryContext("auth-initial-missing", "AUTH-001", 0); +const initialAuthPack = await readFile(join(fixtureRoot, initialAuth.pack_path), "utf8"); +await copyArtifact(initialAuth.pack_path, "auth-initial-missing.pack.md"); +await copyArtifact(initialAuth.receipt_path, "auth-initial-missing.receipt.json"); +if (initialAuthPack.includes("docs/AUTH.md")) { + throw new Error("initial AUTH handoff unexpectedly contains the missing decisive original"); +} + +run( + "ctx-add-auth", + ctx, + [ + "add", + "docs/AUTH.md", + "--root", + fixtureRoot, + "--authority", + "normative", + "--priority", + "100", + "--json", + ], + { + cwd: fixtureRoot, + }, +); +run("ctx-index-with-auth", ctx, ["index", "--root", fixtureRoot, "--json"], { + cwd: fixtureRoot, + timeout: 120_000, +}); +const fixedStatus = jsonRun( + "ctx-status-with-auth", + ctx, + ["status", "--root", fixtureRoot, "--json"], + { + cwd: fixtureRoot, + }, +); +assertSemantic(fixedStatus, "fixed status"); +const doctor = jsonRun( + "ctx-doctor-offline", + ctx, + ["doctor", "--offline", "--root", fixtureRoot, "--json"], + { + cwd: fixtureRoot, + }, +); +if (doctor.offline_ready !== true || doctor.network_attempted !== false) { + throw new Error("CTX offline doctor did not prove offline readiness"); +} + +const counterfactualBefore = factoryContext("counterfactual-before", "AUTH-001", 0); +const counterfactualBeforePack = await readFile( + join(fixtureRoot, counterfactualBefore.pack_path), + "utf8", +); +await copyArtifact(counterfactualBefore.pack_path, "counterfactual-before.pack.md"); +await copyArtifact(counterfactualBefore.receipt_path, "counterfactual-before.receipt.json"); +await writeFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), fixedAuth); +const counterfactualAfter = factoryContext("counterfactual-after", "AUTH-001", 0); +const counterfactualAfterPack = await readFile( + join(fixtureRoot, counterfactualAfter.pack_path), + "utf8", +); +if ( + counterfactualBeforePack.includes("docs/AUTH.md") || + !counterfactualAfterPack.includes("docs/AUTH.md") +) { + throw new Error("one-condition AUTH counterfactual did not add the decisive original"); +} + +const packs = []; +for (const task of oracle.tasks) { + const payload = factoryContext(`fixed-${task.id}`, task.id, 0); + const packAbsolute = join(fixtureRoot, payload.pack_path); + const receiptAbsolute = join(fixtureRoot, payload.receipt_path); + const pack = await readFile(packAbsolute, "utf8"); + const receipt = JSON.parse(await readFile(receiptAbsolute, "utf8")); + const provenance = packSources(pack); + const topical = provenance.filter((path) => path.startsWith("docs/")); + const irrelevant = topical.filter((path) => path !== task.target); + const missingTarget = !topical.includes(task.target); + if (missingTarget || irrelevant.length > 0) { + throw new Error( + `${task.id} relevance failed: missing=${missingTarget} irrelevant=${irrelevant.join(",")}`, + ); + } + const packCopy = join(rawRoot, `${task.id}.pack.md`); + const receiptCopy = join(rawRoot, `${task.id}.receipt.json`); + await cp(packAbsolute, packCopy); + await cp(receiptAbsolute, receiptCopy); + packs.push({ + task_id: task.id, + risk: task.risk, + contract: fileEvidence(join(fixtureRoot, `.factory/tasks/${task.id}.yaml`)), + target_source: fileEvidence(join(fixtureRoot, task.target)), + pack: await fileEvidenceAsync(packCopy), + receipt: await fileEvidenceAsync(receiptCopy), + token_budget: payload.token_budget, + estimated_tokens: payload.estimated_tokens, + byte_budget: payload.byte_budget, + bytes: payload.bytes, + ctx_generation: payload.ctx.index_generation, + retrieval_mode: payload.ctx.retrieval_mode, + provenance, + irrelevant_topical_sources: irrelevant, + missing_target: missingTarget, + source_digests: receipt.source_digests, + }); +} + +await rm(join(fixtureRoot, ".factory/state/context/AUTH-001.md"), { force: true }); +await rm(join(fixtureRoot, ".factory/state/context/AUTH-001.json"), { force: true }); +await writeFile( + join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), + fixedAuth.replace("docs/AUTH.md", "docs/ABSENT.md"), +); +const disconfirming = factoryContext("disconfirming-missing-original", "AUTH-001", 3); +let fabricated = true; +try { + await stat(join(fixtureRoot, ".factory/state/context/AUTH-001.md")); +} catch { + fabricated = false; +} +if (fabricated || disconfirming.error?.code !== "CONTEXT_BLOCKED") { + throw new Error("missing-original disconfirming case did not fail closed without a pack"); +} + +const testedSha = gitText(root, ["rev-parse", "HEAD"]); +const sourceFiles = [ + ".factory/project.yaml", + ".factory/completion.yaml", + "PROJECT.md", + "ARCHITECTURE.md", + "oracle.json", + "cases/AUTH-before.yaml", + ...oracle.tasks.map((entry) => `.factory/tasks/${entry.id}.yaml`), + ...topicalSources, +]; +const manifest = { + schema_version: 1, + gate: "P-06", + tested_sha: testedSha, + recorded_at: new Date().toISOString(), + reviewer, + fixture: { + tracked_root: "test/fixtures/cp06-context", + files: Object.fromEntries( + await Promise.all( + sourceFiles.map(async (path) => [path, await fileHash(join(fixtureSource, path))]), + ), + ), + }, + ctx: { + version: commandText(ctx, ["--version"]), + index_generation: fixedStatus.index_generation, + retrieval_mode: fixedStatus.retrieval_mode, + active_channels: fixedStatus.active_channels, + offline_ready: doctor.offline_ready, + network_attempted: doctor.network_attempted, + }, + original_failed_measurement_preserved: { + packs: 10, + topical_excerpts: 24, + irrelevant_topical_excerpts: 15, + packs_with_irrelevant_excerpt: 10, + missing_targets: 1, + source: "docs/probes/CP-06.md at PR #8 head 8b945a4ed6fcbb43a8b7aa7ce3a6532a4a23d145", + disposition: "fail", + }, + diagnosis: { + bad_unscoped_path: { + exit_code: broad.exitCode, + output_path: broad.outputPath, + output_sha256: broad.outputSha256, + provenance: broadPaths, + irrelevant_topical_sources: broadIrrelevant, + missing_target: true, + }, + one_condition_counterfactual: { + unchanged_fields: "all AUTH task fields except context.required", + before_required: ["PROJECT.md", "ARCHITECTURE.md"], + after_required: ["PROJECT.md", "ARCHITECTURE.md", "docs/AUTH.md"], + before_contract_sha256: sha256(beforeAuth), + after_contract_sha256: sha256(fixedAuth), + before_pack_sha256: sha256(counterfactualBeforePack), + after_pack_sha256: sha256(counterfactualAfterPack), + corrected_constraints: [ + "principal-derived ownership", + "request owner ignored", + "missing or forged credential is 401", + "cross-account read/update/delete is non-mutating non-disclosing 404", + ], + }, + disconfirming_case: { + condition: "declared required original docs/ABSENT.md does not exist", + exit_code: 3, + error_code: disconfirming.error.code, + output_path: commandById("disconfirming-missing-original").outputPath, + output_sha256: commandById("disconfirming-missing-original").outputSha256, + pack_fabricated: fabricated, + }, + }, + fixed_sample: { + pack_count: packs.length, + relevant_pack_count: packs.filter( + (entry) => !entry.missing_target && entry.irrelevant_topical_sources.length === 0, + ).length, + irrelevant_topical_excerpts: packs.reduce( + (sum, entry) => sum + entry.irrelevant_topical_sources.length, + 0, + ), + packs_with_irrelevant_excerpt: packs.filter( + (entry) => entry.irrelevant_topical_sources.length > 0, + ).length, + missing_targets: packs.filter((entry) => entry.missing_target).length, + packs, + }, + commands: commands.map(({ stdout, stderr, ...entry }) => entry), + result: "pass", + limitations: [ + "The semantic replay is verified on Linux with CTX 1.0.0 and the locally installed model.", + "The small fixture demonstrates this fixed sample, not universal retrieval precision.", + "Raw CTX JSON, packs, receipts, and command logs remain ignored scratch under .factory/state.", + ], +}; +const manifestPath = join(proofRoot, "evidence.json"); +await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); +console.log( + JSON.stringify({ + ok: true, + gate: "P-06", + tested_sha: testedSha, + manifest_path: relative(manifestPath), + manifest_sha256: await fileHash(manifestPath), + fixed_pack_count: packs.length, + relevant_pack_count: manifest.fixed_sample.relevant_pack_count, + irrelevant_topical_excerpts: manifest.fixed_sample.irrelevant_topical_excerpts, + missing_targets: manifest.fixed_sample.missing_targets, + bad_path_exit: broad.exitCode, + bad_path_irrelevant_topical_sources: broadIrrelevant, + disconfirming_exit: 3, + }), +); + +function factoryContext(id, taskId, expectedExit) { + const result = run( + id, + process.execPath, + [cli, "context", taskId, "--root", fixtureRoot, "--ctx-bin", ctx, "--json"], + { + cwd: root, + expected: [expectedExit], + timeout: 120_000, + }, + ); + const parsed = parseJsonIfPresent(result.stdout); + if (parsed === null) throw new Error(`${id} did not return JSON`); + return parsed; +} + +function jsonRun(id, command, args, options) { + const result = run(id, command, args, options); + const parsed = parseJsonIfPresent(result.stdout); + if (parsed === null) throw new Error(`${id} did not return JSON`); + return parsed; +} + +function run(id, command, args, options = {}) { + const cwd = options.cwd ?? root; + const result = spawnSync(command, args, { + cwd, + encoding: "utf8", + env: process.env, + timeout: options.timeout ?? 30_000, + }); + const exitCode = result.status; + const expected = options.expected ?? [0]; + const output = `command: ${quote([command, ...args])}\ncwd: ${cwd}\nexit: ${exitCode}\n--- stdout ---\n${result.stdout ?? ""}\n--- stderr ---\n${result.stderr ?? ""}`; + const outputPath = join(rawRoot, `${id}.txt`); + requireWrite(outputPath, output); + const entry = { + id, + command: quote([command, ...args]), + cwd: relative(cwd), + exit_code: exitCode, + output_path: relative(outputPath), + output_sha256: sha256(output), + stdout: result.stdout ?? "", + stderr: result.stderr ?? "", + }; + commands.push(entry); + if (result.error !== undefined || !expected.includes(exitCode)) { + process.stderr.write(output); + throw new Error(`${id} exited ${exitCode}; expected ${expected.join(" or ")}`); + } + return { ...entry, exitCode, outputPath: entry.output_path, outputSha256: entry.output_sha256 }; +} + +function commandById(id) { + const result = commands.find((entry) => entry.id === id); + if (result === undefined) throw new Error(`missing command record ${id}`); + return { outputPath: result.output_path, outputSha256: result.output_sha256 }; +} + +function parseJsonIfPresent(text) { + try { + return JSON.parse(text); + } catch { + return null; + } +} + +function itemPaths(value) { + if (value === null || !Array.isArray(value.items)) return []; + return [ + ...new Set( + value.items + .map((item) => item?.source?.provenance?.document_path) + .filter((path) => typeof path === "string"), + ), + ]; +} + +function packSources(pack) { + const paths = []; + for (const match of pack.matchAll( + /^## (?:Required source|Retrieved context): ([^:\n]+)(?::\d+-\d+)?$/gm, + )) { + paths.push(match[1]); + } + return [...new Set(paths)]; +} + +function assertSemantic(status, label) { + if ( + status.category !== "CLEAN" || + status.retrieval_mode !== "HYBRID_SEMANTIC" || + !Array.isArray(status.active_channels) || + !status.active_channels.includes("semantic") + ) { + throw new Error(`${label} is not clean HYBRID_SEMANTIC retrieval`); + } +} + +async function copyArtifact(path, name) { + await cp(join(fixtureRoot, path), join(rawRoot, name)); +} + +function fileEvidence(path) { + const relativePath = relative(path).replace(relative(fixtureRoot), "test/fixtures/cp06-context"); + const bytes = requireRead(path); + return { path: relativePath, sha256: sha256(bytes) }; +} + +async function fileEvidenceAsync(path) { + return { path: relative(path), sha256: await fileHash(path) }; +} + +function gitText(cwd, args) { + return commandText("git", args, cwd); +} + +function resolveTool(command) { + if (command.includes("/")) return resolve(command); + return commandText("which", [command]); +} + +function commandText(command, args, cwd = root) { + const result = spawnSync(command, args, { cwd, encoding: "utf8" }); + if (result.status !== 0) throw new Error(`${command} failed: ${result.stderr}`); + return result.stdout.trim(); +} + +function quote(parts) { + return parts.map((part) => (/[\s'"]/.test(part) ? JSON.stringify(part) : part)).join(" "); +} + +function relative(path) { + return path.startsWith(`${root}/`) ? path.slice(root.length + 1) : path; +} + +function slug(path) { + return path + .replace(/[^A-Za-z0-9]+/g, "-") + .replace(/^-|-$/g, "") + .toLowerCase(); +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + +async function fileHash(path) { + return createHash("sha256") + .update(await readFile(path)) + .digest("hex"); +} + +function requireRead(path) { + return readFileSync(path); +} + +function requireWrite(path, value) { + writeFileSync(path, value); +} diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs new file mode 100644 index 0000000..a0c6474 --- /dev/null +++ b/scripts/replay-cp06-worker.mjs @@ -0,0 +1,158 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { spawnSync } from "node:child_process"; + +const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); +const workerRoot = join(outputRoot, "worker"); +const fixtureRoot = join(outputRoot, "ten-pack", "repo"); +const initialPackPath = join(outputRoot, "ten-pack", "raw", "auth-initial-missing.pack.md"); +const contractPath = join(root, "test/fixtures/cp06-context/cases/AUTH-before.yaml"); +const originalPath = join(fixtureRoot, "docs/AUTH.md"); +await mkdir(workerRoot, { recursive: true }); + +const contract = await readFile(contractPath, "utf8"); +const pack = await readFile(initialPackPath, "utf8"); +const prompt = `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. If a decisive authoritative source is missing, report MISSING_SOURCE and use the read tool exactly once on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; +const promptPath = join(workerRoot, "prompt.txt"); +const outputPath = join(workerRoot, "output.txt"); +const stderrPath = join(workerRoot, "stderr.txt"); +await writeFile(promptPath, prompt); + +const provider = process.env.CP06_PI_PROVIDER ?? "openai-codex"; +const model = process.env.CP06_PI_MODEL ?? "gpt-5.6-sol"; +const args = [ + "--provider", + provider, + "--model", + model, + "--thinking", + "high", + "--no-session", + "--no-context-files", + "--no-extensions", + "--no-skills", + "--skill", + "skills/factory", + "--tools", + "read", + "--print", + prompt, +]; +const result = spawnSync("pi", args, { + cwd: root, + encoding: "utf8", + env: process.env, + timeout: 600_000, +}); +await writeFile(outputPath, result.stdout ?? ""); +await writeFile(stderrPath, result.stderr ?? ""); +if (result.error !== undefined || result.status !== 0) { + process.stderr.write(result.stderr ?? ""); + throw new Error(`fresh Pi worker exited ${result.status}`); +} +const response = parseResponse(result.stdout ?? ""); +const requiredConstraints = [ + /authenticated principal/i, + /request-supplied owner fields? (?:are )?ignored/i, + /credentials?.*401/i, + /cross-account.*404/i, + /(?:not mutate|no mutation|unchanged state)/i, +]; +const constraintText = Array.isArray(response.corrected_constraints) + ? response.corrected_constraints.join(" ") + : ""; +if ( + response.status !== "MISSING_SOURCE" || + response.broad_scan !== false || + !Array.isArray(response.reads) || + response.reads.length !== 1 || + resolve(response.reads[0]) !== resolve(originalPath) || + !requiredConstraints.every((constraint) => constraint.test(constraintText)) +) { + throw new Error( + `fresh worker response did not satisfy the targeted-read oracle: ${result.stdout}`, + ); +} + +const testedSha = textCommand("git", ["rev-parse", "HEAD"]); +const manifest = { + schema_version: 1, + gate: "P-06-fresh-worker", + tested_sha: testedSha, + recorded_at: new Date().toISOString(), + reviewer: process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker", + worker: { + pi_version: textCommand("pi", ["--version"]), + provider, + model, + no_session: true, + no_context_files: true, + tools: ["read"], + }, + bounded_handoff: { + contract_path: relative(contractPath), + contract_sha256: await fileHash(contractPath), + pack_path: relative(initialPackPath), + pack_sha256: await fileHash(initialPackPath), + evidence: "in-memory auth 401 and concealed non-mutating cross-account 404 observations", + prompt_path: relative(promptPath), + prompt_sha256: await fileHash(promptPath), + }, + command: + "pi --no-session --no-context-files --no-extensions --no-skills --skill skills/factory --tools read --print ", + exit_code: result.status, + output_path: relative(outputPath), + output_sha256: await fileHash(outputPath), + stderr_path: relative(stderrPath), + stderr_sha256: await fileHash(stderrPath), + response, + result: "pass", + limitation: + "Live model wording is nondeterministic; the script deterministically checks the one-read path and required corrected constraints.", +}; +const manifestPath = join(workerRoot, "evidence.json"); +await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); +console.log( + JSON.stringify({ + ok: true, + gate: manifest.gate, + tested_sha: testedSha, + manifest_path: relative(manifestPath), + manifest_sha256: await fileHash(manifestPath), + exit_code: result.status, + status: response.status, + reads: response.reads, + broad_scan: response.broad_scan, + }), +); + +function parseResponse(output) { + const trimmed = output.trim(); + try { + return JSON.parse(trimmed); + } catch { + const fenced = trimmed.match(/```(?:json)?\s*([\s\S]*?)```/); + if (fenced === null) throw new Error(`fresh worker did not return JSON: ${output}`); + return JSON.parse(fenced[1]); + } +} + +function textCommand(command, args) { + const result = spawnSync(command, args, { cwd: root, encoding: "utf8" }); + if (result.status !== 0) throw new Error(`${command} failed: ${result.stderr}`); + return result.stdout.trim(); +} + +function relative(path) { + return path.startsWith(`${root}/`) ? path.slice(root.length + 1) : path; +} + +async function fileHash(path) { + return createHash("sha256") + .update(await readFile(path)) + .digest("hex"); +} diff --git a/skills/factory/SKILL.md b/skills/factory/SKILL.md index 208fd23..0f790f3 100644 --- a/skills/factory/SKILL.md +++ b/skills/factory/SKILL.md @@ -37,9 +37,9 @@ For each ready task, run: context --root --json ``` -Inspect the pack and receipt for the exact task contract, required original sources, source and contract digests, CTX generation/retrieval mode, relevance, and token/byte bounds. Regenerate after any source, contract, or index change. Never place secrets, credentials, source archives, or agent transcripts in a pack. +Before generation, make `context.required` the reviewed task-local source selection: include every known original that can decide an acceptance or risk condition, but no unrelated documents as insurance. Factory reads those originals exactly and constrains CTX retrieval to the same document paths. Inspect the pack and receipt for the exact task contract, required originals, source and contract digests, CTX generation/retrieval mode, relevance, and token/byte bounds. Regenerate after any source, contract, or index change. Never place secrets, credentials, source archives, or agent transcripts in a pack. -Give a fresh worker only the task-local brief, exact contract, bounded pack, and existing evidence—not a previous agent transcript. If a decisive source is absent, have the worker name it and make a targeted exact-original read, verify provenance, and record the corrected assumption. Do not widen every pack or omit a critical constraint to make retrieval appear complete. +Give a fresh worker only the task-local brief, exact contract, bounded pack, and existing evidence—not a previous agent transcript. If a decisive source is absent, have the worker name it and make one targeted exact-original read, verify provenance, record the corrected assumption, and add that exact path to `context.required` before regeneration when it is durable authority. Do not widen every pack, hide raw retrieved excerpts, or omit a critical constraint to make retrieval appear complete. ## Risk and task review diff --git a/src/core/context.ts b/src/core/context.ts index a5100f9..0f1158b 100644 --- a/src/core/context.ts +++ b/src/core/context.ts @@ -146,6 +146,7 @@ export async function generateContextPack(options: { options.root, "--token-budget", String(remaining), + ...input.task.context.required.flatMap((path) => ["--document", path]), ...(lexical ? ["--no-embeddings"] : []), "--json", ]; diff --git a/test/context.test.ts b/test/context.test.ts index 610e943..d79a7ee 100644 --- a/test/context.test.ts +++ b/test/context.test.ts @@ -115,6 +115,33 @@ test("context writes exact mandatory sources, merges overlapping excerpts, and r } }); +test("context limits CTX retrieval to the task-local source selection", async () => { + const fixture = await seeded("source-filter"); + try { + await writeFile( + join(fixture.root, "docs/TARGET.md"), + "# Target authority\n\nOnly this task-local source is relevant.\n", + ); + await writeFile( + join(fixture.root, ".factory/tasks/CTX-001.yaml"), + taskContract.replace( + "required: [PROJECT.md, ARCHITECTURE.md]", + "required: [PROJECT.md, ARCHITECTURE.md, docs/TARGET.md]", + ), + ); + + const result = await factory(fixture, { mode: "source-filter" }); + assert.equal(result.exit, 0, result.stderr || result.stdout); + const payload = parseObject(result.stdout); + const pack = await readFile(join(fixture.root, payload.pack_path), "utf8"); + assert.match(pack, /Only this task-local source is relevant\./); + assert.doesNotMatch(pack, /The overlapping excerpt sentinel/); + assert.equal(payload.source_digests["docs/EXTRA.md"], undefined); + } finally { + await fixture.dispose(); + } +}); + test("context permits explicitly reported lexical fallback only when retrieval is sufficient", async () => { const fixture = await seeded("lexical"); try { @@ -572,12 +599,14 @@ if (command === "doctor") { process.exit(0); } if (command === "pack") { - const relative = "docs/EXTRA.md"; + const selectedDocuments = args.flatMap((arg, index) => arg === "--document" ? [args[index + 1]] : []); + const sourceFiltered = mode === "source-filter" && selectedDocuments.includes("docs/TARGET.md"); + const relative = sourceFiltered ? "docs/TARGET.md" : "docs/EXTRA.md"; const full = fs.readFileSync(path.join(root, relative), "utf8"); - const firstEnd = full.indexOf("Gamma context ends here."); - const secondStart = full.indexOf("The overlapping excerpt sentinel"); + const firstEnd = sourceFiltered ? full.length : full.indexOf("Gamma context ends here."); + const secondStart = sourceFiltered ? 0 : full.indexOf("The overlapping excerpt sentinel"); const item = (start, end, text = full.slice(start, end)) => ({source:{source_type:"excerpt",text,provenance:{document_path:relative,document_sha256:hash(full),start_line:1,end_line:6,start_offset:start,end_offset:end,range_sha256:hash(text),index_generation:7}},estimated_tokens:Math.ceil(Buffer.byteLength(text)/3)}); - const items = [item(0, firstEnd), item(secondStart, full.length), item(0, firstEnd)]; + const items = sourceFiltered ? [item(0, full.length)] : [item(0, firstEnd), item(secondStart, full.length), item(0, firstEnd)]; if (mode === "aliases") { items[1].source.provenance.document_path = "docs/EXTRA-ALIAS.md"; const original = fs.readFileSync(path.join(root, "docs/PROJECT-ALIAS.md"), "utf8"); diff --git a/test/fixtures/cp06-context/.factory/completion.yaml b/test/fixtures/cp06-context/.factory/completion.yaml new file mode 100644 index 0000000..badf482 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/completion.yaml @@ -0,0 +1,73 @@ +schema_version: 1 +project: cp06-ten-pack +conditions: + - { + id: C-COPY, + outcome: Copy follows the style authority., + method: executable, + check_id: copy, + evidence: evidence/COPY.json, + } + - { + id: C-CONFIG, + outcome: Configuration precedence is stable., + method: executable, + check_id: config, + evidence: evidence/CONFIG.json, + } + - { + id: C-API, + outcome: API errors are stable., + method: executable, + check_id: api, + evidence: evidence/API.json, + } + - { + id: C-PERSIST, + outcome: Saved records survive restart., + method: executable, + check_id: persist, + evidence: evidence/PERSIST.json, + } + - { + id: C-UI, + outcome: Empty state is accessible., + method: executable, + check_id: ui, + evidence: evidence/UI.json, + } + - { + id: C-IMPORT, + outcome: Invalid import rows are reported., + method: executable, + check_id: import, + evidence: evidence/IMPORT.json, + } + - { + id: C-LOG, + outcome: Logs redact record content., + method: executable, + check_id: log, + evidence: evidence/LOG.json, + } + - { + id: C-EXPORT, + outcome: Export order is deterministic., + method: executable, + check_id: export, + evidence: evidence/EXPORT.json, + } + - { + id: C-DELETE, + outcome: Deletion is non-disclosing., + method: executable, + check_id: delete, + evidence: evidence/DELETE.json, + } + - { + id: C-AUTH, + outcome: Ownership derives from the principal., + method: executable, + check_id: auth, + evidence: evidence/AUTH.json, + } diff --git a/test/fixtures/cp06-context/.factory/project.yaml b/test/fixtures/cp06-context/.factory/project.yaml new file mode 100644 index 0000000..8a909d6 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/project.yaml @@ -0,0 +1,17 @@ +schema_version: 1 +id: cp06-ten-pack +name: CP-06 Ten Pack +intent: Prove that bounded worker packs contain only task-relevant authoritative knowledge. +audience: Factory reviewers. +goals: + - Generate ten bounded source-attributed packs. + - Correct a missing decisive source through one targeted required-original selection. +non_goals: + - Exercise a live account or network service. +constraints: + - Credentials and raw worker transcripts never enter a context pack. + - Normal fixture execution is offline and deterministic. + - Critical ownership constraints cannot be omitted to improve relevance metrics. +documents: + required: [PROJECT.md, ARCHITECTURE.md] +completion: .factory/completion.yaml diff --git a/test/fixtures/cp06-context/.factory/tasks/API-001.yaml b/test/fixtures/cp06-context/.factory/tasks/API-001.yaml new file mode 100644 index 0000000..8360cf2 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/API-001.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: API-001 +title: Stabilize profile API errors +outcome: Invalid profile requests return stable machine-readable errors. +depends_on: [] +complexity: bounded +risk: normal +acceptance: + - { id: A-API, statement: Malformed JSON and blank profiles return their specified 400 errors. } +advances: [C-API] +context: + topics: [profile API, malformed JSON, stable errors] + required: [PROJECT.md, ARCHITECTURE.md, docs/API.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/AUTH-001.yaml b/test/fixtures/cp06-context/.factory/tasks/AUTH-001.yaml new file mode 100644 index 0000000..a617d5d --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/AUTH-001.yaml @@ -0,0 +1,19 @@ +schema_version: 1 +id: AUTH-001 +title: Bind item ownership to authenticated principal +outcome: Creation ignores request-supplied ownership and cross-account operations remain concealed. +depends_on: [] +complexity: bounded +risk: critical +acceptance: + - { + id: A-AUTH, + statement: Missing or forged credentials return 401 and cross-account read update and delete return non-mutating 404., + } +advances: [C-AUTH] +context: + topics: [principal ownership, tenancy, forged credentials] + required: [PROJECT.md, ARCHITECTURE.md, docs/AUTH.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/CONFIG-001.yaml b/test/fixtures/cp06-context/.factory/tasks/CONFIG-001.yaml new file mode 100644 index 0000000..7f5c99f --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/CONFIG-001.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: CONFIG-001 +title: Enforce configuration precedence +outcome: Explicit options override project values and defaults. +depends_on: [] +complexity: bounded +risk: bounded +acceptance: + - { id: A-CONFIG, statement: Unknown keys fail and precedence is deterministic. } +advances: [C-CONFIG] +context: + topics: [configuration, precedence, unknown keys] + required: [PROJECT.md, ARCHITECTURE.md, docs/CONFIG.md] +evidence: + required: [unit] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/COPY-001.yaml b/test/fixtures/cp06-context/.factory/tasks/COPY-001.yaml new file mode 100644 index 0000000..f0bd6b0 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/COPY-001.yaml @@ -0,0 +1,19 @@ +schema_version: 1 +id: COPY-001 +title: Clarify empty-state copy +outcome: Empty-state text is direct and actionable. +depends_on: [] +complexity: bounded +risk: bounded +acceptance: + - { + id: A-COPY, + statement: Empty-state copy names the next action without implying an existing record., + } +advances: [C-COPY] +context: + topics: [empty state, interface copy, next action] + required: [PROJECT.md, ARCHITECTURE.md, docs/STYLE.md] +evidence: + required: [unit] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/DELETE-001.yaml b/test/fixtures/cp06-context/.factory/tasks/DELETE-001.yaml new file mode 100644 index 0000000..e88721d --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/DELETE-001.yaml @@ -0,0 +1,19 @@ +schema_version: 1 +id: DELETE-001 +title: Conceal cross-account deletion +outcome: Only an owner can delete a record or emit its audit marker. +depends_on: [] +complexity: bounded +risk: critical +acceptance: + - { + id: A-DELETE, + statement: A different principal receives 404 with no mutation or audit marker., + } +advances: [C-DELETE] +context: + topics: [deletion, non-disclosure, audit marker] + required: [PROJECT.md, ARCHITECTURE.md, docs/DELETION.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/EXPORT-001.yaml b/test/fixtures/cp06-context/.factory/tasks/EXPORT-001.yaml new file mode 100644 index 0000000..0b31956 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/EXPORT-001.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: EXPORT-001 +title: Produce deterministic exports +outcome: An unchanged dataset exports to identical UTF-8 JSON bytes. +depends_on: [] +complexity: bounded +risk: normal +acceptance: + - { id: A-EXPORT, statement: Record and object-key order are stable across repeated exports. } +advances: [C-EXPORT] +context: + topics: [UTF-8 JSON, deterministic export, stable order] + required: [PROJECT.md, ARCHITECTURE.md, docs/EXPORT.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/IMPORT-001.yaml b/test/fixtures/cp06-context/.factory/tasks/IMPORT-001.yaml new file mode 100644 index 0000000..fad3a48 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/IMPORT-001.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: IMPORT-001 +title: Report invalid import rows +outcome: Invalid rows are reported without changing valid-row order or the source. +depends_on: [] +complexity: bounded +risk: normal +acceptance: + - { id: A-IMPORT, statement: Stable row numbers and reasons identify invalid rows. } +advances: [C-IMPORT] +context: + topics: [import rows, validation reasons, input order] + required: [PROJECT.md, ARCHITECTURE.md, docs/IMPORT.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/LOG-001.yaml b/test/fixtures/cp06-context/.factory/tasks/LOG-001.yaml new file mode 100644 index 0000000..cd6180a --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/LOG-001.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: LOG-001 +title: Redact operational logs +outcome: Logs retain identifiers but omit credentials and record content. +depends_on: [] +complexity: bounded +risk: normal +acceptance: + - { id: A-LOG, statement: Denied-operation logs contain no private resource fields. } +advances: [C-LOG] +context: + topics: [operational logs, redaction, denied operation] + required: [PROJECT.md, ARCHITECTURE.md, docs/OBSERVABILITY.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/PERSIST-001.yaml b/test/fixtures/cp06-context/.factory/tasks/PERSIST-001.yaml new file mode 100644 index 0000000..11028f2 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/PERSIST-001.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: PERSIST-001 +title: Preserve records across restart +outcome: A successful save remains visible after restart. +depends_on: [] +complexity: bounded +risk: normal +acceptance: + - { id: A-PERSIST, statement: Restarting against the same store retains the saved record. } +advances: [C-PERSIST] +context: + topics: [save, restart, local store] + required: [PROJECT.md, ARCHITECTURE.md, docs/PERSISTENCE.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.factory/tasks/UI-001.yaml b/test/fixtures/cp06-context/.factory/tasks/UI-001.yaml new file mode 100644 index 0000000..52be0f7 --- /dev/null +++ b/test/fixtures/cp06-context/.factory/tasks/UI-001.yaml @@ -0,0 +1,16 @@ +schema_version: 1 +id: UI-001 +title: Make empty state accessible +outcome: The empty view exposes one labeled keyboard-reachable action. +depends_on: [] +complexity: bounded +risk: normal +acceptance: + - { id: A-UI, statement: Heading action label and focus order satisfy the empty-state contract. } +advances: [C-UI] +context: + topics: [empty view, keyboard focus, programmatic label] + required: [PROJECT.md, ARCHITECTURE.md, docs/UI.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/.gitignore b/test/fixtures/cp06-context/.gitignore new file mode 100644 index 0000000..1361cc5 --- /dev/null +++ b/test/fixtures/cp06-context/.gitignore @@ -0,0 +1,2 @@ +.factory/state/ +.ctx/ diff --git a/test/fixtures/cp06-context/ARCHITECTURE.md b/test/fixtures/cp06-context/ARCHITECTURE.md new file mode 100644 index 0000000..f3fff27 --- /dev/null +++ b/test/fixtures/cp06-context/ARCHITECTURE.md @@ -0,0 +1,7 @@ +# Ten-pack fixture architecture + +The fixture uses an in-memory boundary around API, persistence, and presentation behavior. All operations are deterministic and make no network or account access. + +## Trust boundary + +Authenticated identity is authoritative for ownership. Cross-account operations neither disclose existence nor mutate state. diff --git a/test/fixtures/cp06-context/PROJECT.md b/test/fixtures/cp06-context/PROJECT.md new file mode 100644 index 0000000..1114195 --- /dev/null +++ b/test/fixtures/cp06-context/PROJECT.md @@ -0,0 +1,9 @@ +# Ten-pack fixture project + +The local service manages one person's saved records through stable APIs. Context packs must contain the exact task contract, the global privacy and offline constraints, and only task-relevant authority. + +## Global constraints + +- Credentials and raw worker transcripts never enter a context pack. +- Normal operation is offline. +- Errors use stable machine-readable codes. diff --git a/test/fixtures/cp06-context/cases/AUTH-before.yaml b/test/fixtures/cp06-context/cases/AUTH-before.yaml new file mode 100644 index 0000000..dfb7b9b --- /dev/null +++ b/test/fixtures/cp06-context/cases/AUTH-before.yaml @@ -0,0 +1,17 @@ +schema_version: 1 +id: AUTH-001 +title: Bind item ownership to authenticated principal +outcome: Creation ignores request-supplied ownership and cross-account operations remain concealed. +depends_on: [] +complexity: bounded +risk: critical +acceptance: + - id: A-AUTH + statement: Missing or forged credentials return 401 and cross-account read update and delete return non-mutating 404. +advances: [C-AUTH] +context: + topics: [principal ownership, tenancy, forged credentials] + required: [PROJECT.md, ARCHITECTURE.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/cp06-context/docs/API.md b/test/fixtures/cp06-context/docs/API.md new file mode 100644 index 0000000..5d31e72 --- /dev/null +++ b/test/fixtures/cp06-context/docs/API.md @@ -0,0 +1,3 @@ +# Profile API + +Malformed JSON returns `400 invalid_json`; a blank display name returns `400 invalid_profile`. A valid GET remains successful and unknown routes return `404`. diff --git a/test/fixtures/cp06-context/docs/AUTH.md b/test/fixtures/cp06-context/docs/AUTH.md new file mode 100644 index 0000000..500493e --- /dev/null +++ b/test/fixtures/cp06-context/docs/AUTH.md @@ -0,0 +1,3 @@ +# Authentication and tenancy + +Ownership comes only from the authenticated principal; request-supplied owner fields are ignored. Missing or forged fixture credentials return `401`. Cross-account read, update, and delete return a non-disclosing `404` and must not mutate state. diff --git a/test/fixtures/cp06-context/docs/CONFIG.md b/test/fixtures/cp06-context/docs/CONFIG.md new file mode 100644 index 0000000..e139c3d --- /dev/null +++ b/test/fixtures/cp06-context/docs/CONFIG.md @@ -0,0 +1,3 @@ +# Configuration + +Explicit command options override project configuration, which overrides built-in defaults. Unknown keys fail validation instead of being ignored. diff --git a/test/fixtures/cp06-context/docs/DELETION.md b/test/fixtures/cp06-context/docs/DELETION.md new file mode 100644 index 0000000..cfb6552 --- /dev/null +++ b/test/fixtures/cp06-context/docs/DELETION.md @@ -0,0 +1,3 @@ +# Deletion + +Deletion removes an owned record and emits an audit marker. A different principal receives non-disclosing `404`, no audit marker, and no state mutation. diff --git a/test/fixtures/cp06-context/docs/EXPORT.md b/test/fixtures/cp06-context/docs/EXPORT.md new file mode 100644 index 0000000..99edeb8 --- /dev/null +++ b/test/fixtures/cp06-context/docs/EXPORT.md @@ -0,0 +1,3 @@ +# Export + +Exports are UTF-8 JSON with records ordered by identifier and object keys in a stable order. Repeating an unchanged export produces identical bytes. diff --git a/test/fixtures/cp06-context/docs/IMPORT.md b/test/fixtures/cp06-context/docs/IMPORT.md new file mode 100644 index 0000000..5776c7f --- /dev/null +++ b/test/fixtures/cp06-context/docs/IMPORT.md @@ -0,0 +1,3 @@ +# Import + +Rows are validated independently. Invalid rows report stable row numbers and reasons while valid rows retain input order; no source file is modified. diff --git a/test/fixtures/cp06-context/docs/OBSERVABILITY.md b/test/fixtures/cp06-context/docs/OBSERVABILITY.md new file mode 100644 index 0000000..b6a599c --- /dev/null +++ b/test/fixtures/cp06-context/docs/OBSERVABILITY.md @@ -0,0 +1,3 @@ +# Observability + +Operational logs include request and result identifiers but redact credentials, titles, and record bodies. A denied operation records no private resource fields. diff --git a/test/fixtures/cp06-context/docs/PERSISTENCE.md b/test/fixtures/cp06-context/docs/PERSISTENCE.md new file mode 100644 index 0000000..881bfb4 --- /dev/null +++ b/test/fixtures/cp06-context/docs/PERSISTENCE.md @@ -0,0 +1,3 @@ +# Persistence + +A successful save is committed before the response is returned. Restarting the service and reopening the same local store preserves the record. diff --git a/test/fixtures/cp06-context/docs/STYLE.md b/test/fixtures/cp06-context/docs/STYLE.md new file mode 100644 index 0000000..814a96f --- /dev/null +++ b/test/fixtures/cp06-context/docs/STYLE.md @@ -0,0 +1,3 @@ +# Interface copy style + +Use direct labels and sentence-case actions. Empty-state copy says what the user can do next and never implies that a record already exists. diff --git a/test/fixtures/cp06-context/docs/UI.md b/test/fixtures/cp06-context/docs/UI.md new file mode 100644 index 0000000..a92922f --- /dev/null +++ b/test/fixtures/cp06-context/docs/UI.md @@ -0,0 +1,3 @@ +# Empty state UI + +The empty view has one visible heading, one actionable control, and a programmatic label. Keyboard focus reaches the action before decorative content. diff --git a/test/fixtures/cp06-context/oracle.json b/test/fixtures/cp06-context/oracle.json new file mode 100644 index 0000000..e2706c3 --- /dev/null +++ b/test/fixtures/cp06-context/oracle.json @@ -0,0 +1,16 @@ +{ + "schema_version": 1, + "rule": "PROJECT.md and ARCHITECTURE.md are relevant global authority; for each task only its declared target source is relevant topical authority; every other topical document is irrelevant.", + "tasks": [ + { "id": "COPY-001", "risk": "bounded", "target": "docs/STYLE.md" }, + { "id": "CONFIG-001", "risk": "bounded", "target": "docs/CONFIG.md" }, + { "id": "API-001", "risk": "normal", "target": "docs/API.md" }, + { "id": "PERSIST-001", "risk": "normal", "target": "docs/PERSISTENCE.md" }, + { "id": "UI-001", "risk": "normal", "target": "docs/UI.md" }, + { "id": "IMPORT-001", "risk": "normal", "target": "docs/IMPORT.md" }, + { "id": "LOG-001", "risk": "normal", "target": "docs/OBSERVABILITY.md" }, + { "id": "EXPORT-001", "risk": "normal", "target": "docs/EXPORT.md" }, + { "id": "DELETE-001", "risk": "critical", "target": "docs/DELETION.md" }, + { "id": "AUTH-001", "risk": "critical", "target": "docs/AUTH.md" } + ] +} diff --git a/test/fixtures/workflow/history/finding.json b/test/fixtures/workflow/history/finding.json new file mode 100644 index 0000000..9e29e8b --- /dev/null +++ b/test/fixtures/workflow/history/finding.json @@ -0,0 +1,11 @@ +{ + "schema_version": 1, + "finding_id": "P07-MALFORMED-JSON", + "acceptance_id": "P-07", + "reviewer": "CP-06 simulated task reviewer", + "scenario": "PUT /v1/profile receives malformed JSON", + "finding": "The normal API fixture throws SyntaxError instead of returning the stable 400 invalid_json response.", + "disposition": "repaired", + "repair": "Catch parse failure and return status 400 with error invalid_json.", + "retest": "Execute the same malformed-JSON behavioral assertion against the checked-in final fixture." +} diff --git a/test/fixtures/workflow/history/malformed-json-probe.mjs b/test/fixtures/workflow/history/malformed-json-probe.mjs new file mode 100644 index 0000000..23af151 --- /dev/null +++ b/test/fixtures/workflow/history/malformed-json-probe.mjs @@ -0,0 +1,7 @@ +import { strict as assert } from "node:assert"; +import { handleProfileRequest } from "./normal-api-before.mjs"; + +assert.deepEqual(handleProfileRequest({ method: "PUT", path: "/v1/profile", body: "not-json" }), { + status: 400, + body: { error: "invalid_json" }, +}); diff --git a/test/fixtures/workflow/history/normal-api-before.mjs b/test/fixtures/workflow/history/normal-api-before.mjs new file mode 100644 index 0000000..f478308 --- /dev/null +++ b/test/fixtures/workflow/history/normal-api-before.mjs @@ -0,0 +1,11 @@ +export function handleProfileRequest(input) { + if (input.path !== "/v1/profile") return { status: 404, body: { error: "not_found" } }; + if (input.method === "GET") return { status: 200, body: { display_name: "Fixture User" } }; + if (input.method !== "PUT") return { status: 405, body: { error: "method_not_allowed" } }; + const parsed = JSON.parse(input.body ?? ""); + const displayName = parsed?.display_name; + if (typeof displayName !== "string" || displayName.trim().length === 0) { + return { status: 400, body: { error: "invalid_profile" } }; + } + return { status: 200, body: { display_name: displayName.trim() } }; +} diff --git a/test/workflow.test.ts b/test/workflow.test.ts index f6c7a62..5ea9c73 100644 --- a/test/workflow.test.ts +++ b/test/workflow.test.ts @@ -1,8 +1,13 @@ import { strict as assert } from "node:assert"; +import { spawnSync } from "node:child_process"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; import test from "node:test"; import { createAuthIsolationFixture } from "./fixtures/workflow/critical-auth-isolation.js"; import { handleProfileRequest } from "./fixtures/workflow/normal-api.js"; +const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), "../.."); + test("normal API fixture preserves successful profile behavior", () => { assert.deepEqual(handleProfileRequest({ method: "GET", path: "/v1/profile" }), { status: 200, @@ -35,6 +40,20 @@ test("normal API fixture rejects malformed, invalid, unknown, and unsupported re assert.equal(handleProfileRequest({ method: "POST", path: "/v1/profile" }).status, 405); }); +test("retained P-07 finding fails before repair and passes through the final API", () => { + const before = spawnSync( + process.execPath, + ["test/fixtures/workflow/history/malformed-json-probe.mjs"], + { cwd: repoRoot, encoding: "utf8" }, + ); + assert.equal(before.status, 1, before.stdout + before.stderr); + assert.match(before.stdout + before.stderr, /SyntaxError/); + assert.deepEqual(handleProfileRequest({ method: "PUT", path: "/v1/profile", body: "not-json" }), { + status: 400, + body: { error: "invalid_json" }, + }); +}); + test("critical auth fixture allows each account to observe its own item", () => { const api = createAuthIsolationFixture(); assert.deepEqual(api.request({ token: "fixture-token-a", method: "GET", itemId: "item-a" }), { From 461dea53b2e35af24dd558ab6f8b3d6c241309c6 Mon Sep 17 00:00:00 2001 From: Factory CP-06 Correction Date: Tue, 29 Sep 2026 09:00:28 +0000 Subject: [PATCH 05/45] no-mistakes(review): Harden CP-06 replay evidence --- scripts/replay-cp06-p07.mjs | 13 +++- scripts/replay-cp06-worker.mjs | 125 ++++++++++++++++++++++++++++++--- 2 files changed, 125 insertions(+), 13 deletions(-) diff --git a/scripts/replay-cp06-p07.mjs b/scripts/replay-cp06-p07.mjs index ec81748..83ce136 100644 --- a/scripts/replay-cp06-p07.mjs +++ b/scripts/replay-cp06-p07.mjs @@ -20,6 +20,7 @@ steps.push( process.execPath, ["test/fixtures/workflow/history/malformed-json-probe.mjs"], 1, + ["SyntaxError", "not valid JSON"], ), ); steps.push( @@ -83,15 +84,20 @@ console.log( }), ); -function run(id, command, args, expectedExit) { +function run(id, command, args, expectedExit, expectedOutputFragments = []) { const result = spawnSync(command, args, { cwd: root, encoding: "utf8", env: process.env }); const exitCode = result.status; const output = `command: ${quote([command, ...args])}\nexit: ${exitCode}\n--- stdout ---\n${result.stdout ?? ""}\n--- stderr ---\n${result.stderr ?? ""}`; const outputPath = join(rawRoot, `${id}.txt`); writeFileSync(outputPath, output); - if (result.error !== undefined || exitCode !== expectedExit) { + const missingFragments = expectedOutputFragments.filter((fragment) => !output.includes(fragment)); + if (result.error !== undefined || exitCode !== expectedExit || missingFragments.length > 0) { process.stderr.write(output); - throw new Error(`${id} exited ${exitCode}; expected ${expectedExit}`); + const suffix = + missingFragments.length > 0 + ? `; missing output fragments ${missingFragments.join(", ")}` + : ""; + throw new Error(`${id} exited ${exitCode}; expected ${expectedExit}${suffix}`); } return { id, @@ -100,6 +106,7 @@ function run(id, command, args, expectedExit) { exit_code: exitCode, output_path: relative(outputPath), output_sha256: sha256(output), + expected_output_fragments: expectedOutputFragments, }; } diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index a0c6474..9dc4001 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -18,6 +18,7 @@ const contract = await readFile(contractPath, "utf8"); const pack = await readFile(initialPackPath, "utf8"); const prompt = `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. If a decisive authoritative source is missing, report MISSING_SOURCE and use the read tool exactly once on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; const promptPath = join(workerRoot, "prompt.txt"); +const eventStreamPath = join(workerRoot, "event-stream.jsonl"); const outputPath = join(workerRoot, "output.txt"); const stderrPath = join(workerRoot, "stderr.txt"); await writeFile(promptPath, prompt); @@ -31,6 +32,8 @@ const args = [ model, "--thinking", "high", + "--mode", + "json", "--no-session", "--no-context-files", "--no-extensions", @@ -39,7 +42,6 @@ const args = [ "skills/factory", "--tools", "read", - "--print", prompt, ]; const result = spawnSync("pi", args, { @@ -48,13 +50,20 @@ const result = spawnSync("pi", args, { env: process.env, timeout: 600_000, }); -await writeFile(outputPath, result.stdout ?? ""); +await writeFile(eventStreamPath, result.stdout ?? ""); await writeFile(stderrPath, result.stderr ?? ""); if (result.error !== undefined || result.status !== 0) { process.stderr.write(result.stderr ?? ""); throw new Error(`fresh Pi worker exited ${result.status}`); } -const response = parseResponse(result.stdout ?? ""); +const events = parseEventStream(result.stdout ?? ""); +const responseText = finalAssistantText(events); +await writeFile(outputPath, responseText); +const response = parseResponse(responseText); +const readAudit = auditReadEvents(events, originalPath); +if (!readAudit.ok) { + throw new Error(`fresh worker violated the event-stream read oracle: ${readAudit.reason}`); +} const requiredConstraints = [ /authenticated principal/i, /request-supplied owner fields? (?:are )?ignored/i, @@ -68,13 +77,10 @@ const constraintText = Array.isArray(response.corrected_constraints) if ( response.status !== "MISSING_SOURCE" || response.broad_scan !== false || - !Array.isArray(response.reads) || - response.reads.length !== 1 || - resolve(response.reads[0]) !== resolve(originalPath) || !requiredConstraints.every((constraint) => constraint.test(constraintText)) ) { throw new Error( - `fresh worker response did not satisfy the targeted-read oracle: ${result.stdout}`, + `fresh worker response did not satisfy the targeted-read oracle: ${responseText}`, ); } @@ -91,6 +97,7 @@ const manifest = { model, no_session: true, no_context_files: true, + mode: "json", tools: ["read"], }, bounded_handoff: { @@ -102,8 +109,14 @@ const manifest = { prompt_path: relative(promptPath), prompt_sha256: await fileHash(promptPath), }, + event_stream: { + path: relative(eventStreamPath), + sha256: await fileHash(eventStreamPath), + event_count: events.length, + read_audit: readAudit.summary, + }, command: - "pi --no-session --no-context-files --no-extensions --no-skills --skill skills/factory --tools read --print ", + "pi --mode json --no-session --no-context-files --no-extensions --no-skills --skill skills/factory --tools read ", exit_code: result.status, output_path: relative(outputPath), output_sha256: await fileHash(outputPath), @@ -112,7 +125,7 @@ const manifest = { response, result: "pass", limitation: - "Live model wording is nondeterministic; the script deterministically checks the one-read path and required corrected constraints.", + "Live model wording is nondeterministic; the script deterministically checks actual read tool-call events and required corrected constraints.", }; const manifestPath = join(workerRoot, "evidence.json"); await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); @@ -125,11 +138,103 @@ console.log( manifest_sha256: await fileHash(manifestPath), exit_code: result.status, status: response.status, - reads: response.reads, + reads: readAudit.summary.project_read_paths, broad_scan: response.broad_scan, }), ); +function parseEventStream(output) { + const events = []; + for (const [index, line] of output.split(/\r?\n/).entries()) { + if (line.trim().length === 0) continue; + try { + events.push(JSON.parse(line)); + } catch (error) { + throw new Error(`Pi JSON event stream line ${index + 1} is not valid JSON: ${error.message}`); + } + } + return events; +} + +function finalAssistantText(events) { + const agentEnd = events.findLast((event) => event.type === "agent_end"); + const messages = Array.isArray(agentEnd?.messages) + ? agentEnd.messages + : events + .filter((event) => event.type === "message_end" && event.message?.role === "assistant") + .map((event) => event.message); + const assistant = messages.findLast((message) => message.role === "assistant"); + if (assistant === undefined) + throw new Error("Pi JSON event stream did not include a final assistant message"); + return messageText(assistant); +} + +function messageText(message) { + if (typeof message.content === "string") return message.content; + if (!Array.isArray(message.content)) return ""; + return message.content + .filter((block) => block?.type === "text" && typeof block.text === "string") + .map((block) => block.text) + .join(""); +} + +function auditReadEvents(events, expectedOriginalPath) { + const expectedOriginal = resolve(expectedOriginalPath); + const allowedSkill = resolve(root, "skills/factory/SKILL.md"); + const readCalls = events + .filter((event) => event.type === "tool_execution_start" && event.toolName === "read") + .map((event) => ({ + path: event.args?.path, + resolved_path: typeof event.args?.path === "string" ? resolve(root, event.args.path) : null, + })); + const otherToolCalls = events.filter( + (event) => event.type === "tool_execution_start" && event.toolName !== "read", + ); + const invalidReadCalls = readCalls.filter( + (call) => call.resolved_path !== expectedOriginal && call.resolved_path !== allowedSkill, + ); + const originalReadCalls = readCalls.filter((call) => call.resolved_path === expectedOriginal); + const skillReadCalls = readCalls.filter((call) => call.resolved_path === allowedSkill); + const summary = { + read_count: readCalls.length, + project_read_count: originalReadCalls.length, + project_read_paths: originalReadCalls.map((call) => call.resolved_path), + allowed_skill_read_count: skillReadCalls.length, + allowed_skill_read_paths: skillReadCalls.map((call) => call.resolved_path), + rejected_read_paths: invalidReadCalls.map((call) => call.path), + other_tool_calls: otherToolCalls.map((event) => event.toolName), + }; + if (otherToolCalls.length > 0) { + return { + ok: false, + reason: `unexpected tool calls ${summary.other_tool_calls.join(", ")}`, + summary, + }; + } + if (originalReadCalls.length !== 1) { + return { + ok: false, + reason: `expected exactly one AUTH original read, saw ${originalReadCalls.length}`, + summary, + }; + } + if (skillReadCalls.length > 1) { + return { + ok: false, + reason: `expected at most one factory skill read, saw ${skillReadCalls.length}`, + summary, + }; + } + if (invalidReadCalls.length > 0) { + return { + ok: false, + reason: `unexpected read paths ${summary.rejected_read_paths.join(", ")}`, + summary, + }; + } + return { ok: true, summary }; +} + function parseResponse(output) { const trimmed = output.trim(); try { From 355854fc4e6c16b03fb793825d93a7ed801e52d4 Mon Sep 17 00:00:00 2001 From: Factory CP-06 Correction Date: Tue, 29 Sep 2026 10:55:10 +0000 Subject: [PATCH 06/45] test(cp06): require successful bad-path retrieval --- docs/probes/CP-06.md | 21 +++++++----- scripts/replay-cp06-ten-pack.mjs | 56 +++++++++++++++++++++++++++++--- 2 files changed, 64 insertions(+), 13 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 2c9d899..c4a4f90 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -131,10 +131,13 @@ relevant; every other topical source is irrelevant. The replay result is: - exact contract, target-source, pack, receipt, output, and source digests recorded per task. -The disconfirming case changes AUTH to require nonexistent `docs/ABSENT.md`. -Factory exits 3 with `CONTEXT_BLOCKED` and writes no replacement pack. Thus the -correction does not claim that a source filter can recover undeclared or absent -authority and does not fabricate completeness. +A separate document-scoped real-CTX disconfirming probe exits 0 but reports +`PARTIAL` at its deliberately constrained 1,300-token budget; successful command +execution alone is not accepted as completeness. The missing-original case then +changes AUTH to require nonexistent `docs/ABSENT.md`: Factory exits 3 with +`CONTEXT_BLOCKED` and writes no replacement pack. Thus the correction does not +claim that a source filter can recover undeclared or absent authority and does +not fabricate completeness. ## Fresh-context targeted-original correction @@ -147,10 +150,12 @@ pack, and a short deterministic evidence statement—no prior transcript. The worker reports `MISSING_SOURCE`, performs exactly one read of the supplied `docs/AUTH.md` original, performs no broad scan, recovers all four decisive constraints above, and identifies the untested request-owner/create-time binding -case. `scripts/replay-cp06-worker.mjs` records the raw output only in ignored -scratch and deterministically checks the one-read path and required corrected -constraints. Model wording is inherently nondeterministic; the structural -oracle and exact source are not. +case. `scripts/replay-cp06-worker.mjs` records Pi's machine-readable event stream +only in ignored scratch and derives the exact read count and path from actual +tool-call events rather than trusting the model's summary. It rejects any other +project read or tool call and checks the required corrected constraints. Model +wording is inherently nondeterministic; the event audit, structural oracle, and +exact source are not. ## P-07 finding, repair, and retest diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index 489f895..ca4d0e6 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -80,22 +80,59 @@ const broad = run( ["pack", authQuery, "--root", fixtureRoot, "--token-budget", "3500", "--json"], { cwd: fixtureRoot, - expected: [0, 3], + expected: [0], timeout: 120_000, }, ); const broadJson = parseJsonIfPresent(broad.stdout); +if (broadJson === null || !Array.isArray(broadJson.items)) { + throw new Error("bad-path probe did not return a successful real-CTX pack"); +} const broadPaths = itemPaths(broadJson); const broadIrrelevant = broadPaths.filter( (path) => path.startsWith("docs/") && path !== "docs/AUTH.md", ); -if (broad.exitCode === 0 && broadIrrelevant.length === 0) { - throw new Error("bad-path probe neither blocked nor exposed unrelated topical retrieval"); +if (broadIrrelevant.length === 0) { + throw new Error("bad-path probe did not expose unrelated topical retrieval"); } +const broadIrrelevantEvidence = await Promise.all( + broadIrrelevant.map(async (path) => { + const item = broadJson.items.find( + (candidate) => candidate?.source?.provenance?.document_path === path, + ); + const claimedSha256 = item?.source?.provenance?.document_sha256; + const actualSha256 = await fileHash(join(fixtureRoot, path)); + if (claimedSha256 !== actualSha256) { + throw new Error(`bad-path provenance digest does not match exact original ${path}`); + } + return { path, document_sha256: actualSha256 }; + }), +); if (broadPaths.includes("docs/AUTH.md")) { throw new Error("bad-path probe unexpectedly retrieved deliberately unindexed docs/AUTH.md"); } +const partialDisconfirming = run( + "disconfirming-partial-ctx", + ctx, + [ + "pack", + "Delete an owned record without disclosing another principal's record", + "--root", + fixtureRoot, + "--document", + "docs/DELETION.md", + "--token-budget", + "1300", + "--json", + ], + { cwd: fixtureRoot, expected: [0], timeout: 120_000 }, +); +const partialDisconfirmingJson = parseJsonIfPresent(partialDisconfirming.stdout); +if (partialDisconfirmingJson?.completeness_status !== "PARTIAL") { + throw new Error("disconfirming real-CTX probe did not retain its asserted PARTIAL result"); +} + const initialAuth = factoryContext("auth-initial-missing", "AUTH-001", 0); const initialAuthPack = await readFile(join(fixtureRoot, initialAuth.pack_path), "utf8"); await copyArtifact(initialAuth.pack_path, "auth-initial-missing.pack.md"); @@ -273,6 +310,7 @@ const manifest = { output_sha256: broad.outputSha256, provenance: broadPaths, irrelevant_topical_sources: broadIrrelevant, + irrelevant_topical_originals: broadIrrelevantEvidence, missing_target: true, }, one_condition_counterfactual: { @@ -290,7 +328,14 @@ const manifest = { "cross-account read/update/delete is non-mutating non-disclosing 404", ], }, - disconfirming_case: { + disconfirming_partial_case: { + condition: "a successful document-scoped real-CTX call can remain PARTIAL", + exit_code: partialDisconfirming.exitCode, + completeness_status: partialDisconfirmingJson.completeness_status, + output_path: partialDisconfirming.outputPath, + output_sha256: partialDisconfirming.outputSha256, + }, + disconfirming_missing_original_case: { condition: "declared required original docs/ABSENT.md does not exist", exit_code: 3, error_code: disconfirming.error.code, @@ -337,7 +382,8 @@ console.log( missing_targets: manifest.fixed_sample.missing_targets, bad_path_exit: broad.exitCode, bad_path_irrelevant_topical_sources: broadIrrelevant, - disconfirming_exit: 3, + disconfirming_partial_exit: partialDisconfirming.exitCode, + disconfirming_missing_original_exit: 3, }), ); From cdaf5e5c03d36f1c9b6bb0115793766a384cc178 Mon Sep 17 00:00:00 2001 From: Factory CP-06 Correction Date: Tue, 29 Sep 2026 11:04:56 +0000 Subject: [PATCH 07/45] no-mistakes(document): Format changed documentation --- ARCHITECTURE.md | 20 ++++++++++---------- docs/CONTRACTS.md | 2 +- docs/INTEGRATIONS.md | 16 ++++++++-------- 3 files changed, 19 insertions(+), 19 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 92138af..80d1a62 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -65,15 +65,15 @@ The tracked `.factory/completion.yaml` is authored before task implementation. I ## Failures and trust boundaries -| Failure | Required response | -| --- | --- | -| CTX unavailable or index stale | Do not make up context; exact required files still available for manual work; context command exits blocked. | -| Firstmate home absent or not registered | Publication blocked; preview shows required setup. | -| tasks-axi CLI/version/backend incompatible | Publication blocked; no direct backlog file edit fallback. | -| Contract changed during sync | Abort before writes where possible; report partial state if a write already occurred. | -| Duplicate ID/body edited by another actor | Conflict, never clobber. | -| Source path outside repo or symlink escape | Reject before reading or publishing. | -| Project checks pass on obsolete commit | Mark stale; rerun against release candidate. | -| Worker reports pass without evidence | Show unverified, not complete. | +| Failure | Required response | +| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------ | +| CTX unavailable or index stale | Do not make up context; exact required files still available for manual work; context command exits blocked. | +| Firstmate home absent or not registered | Publication blocked; preview shows required setup. | +| tasks-axi CLI/version/backend incompatible | Publication blocked; no direct backlog file edit fallback. | +| Contract changed during sync | Abort before writes where possible; report partial state if a write already occurred. | +| Duplicate ID/body edited by another actor | Conflict, never clobber. | +| Source path outside repo or symlink escape | Reject before reading or publishing. | +| Project checks pass on obsolete commit | Mark stale; rerun against release candidate. | +| Worker reports pass without evidence | Show unverified, not complete. | External repository files, CTX excerpts, and task bodies are untrusted as shell commands. The CLI never executes test commands solely because a contract specifies one; allowed project checks are defined by a reviewed, tracked verification configuration and executed only by an explicit operator/worker command. Do not store secrets or raw agent transcripts in a context pack or receipt. diff --git a/docs/CONTRACTS.md b/docs/CONTRACTS.md index a3616a8..1f7b4ee 100644 --- a/docs/CONTRACTS.md +++ b/docs/CONTRACTS.md @@ -100,7 +100,7 @@ delivery: project-default "artifact_sha256": "<64 hexadecimal characters>" } ], - "review": {"status": "pending", "artifact": null}, + "review": { "status": "pending", "artifact": null }, "recorded_at": "2026-09-26T00:00:00Z" } ``` diff --git a/docs/INTEGRATIONS.md b/docs/INTEGRATIONS.md index 2fd95bf..e2de01b 100644 --- a/docs/INTEGRATIONS.md +++ b/docs/INTEGRATIONS.md @@ -30,14 +30,14 @@ no-mistakes documents a review → test → document → lint → push → PR ## Probe matrix -| Probe | Failure response | -| --- | --- | -| CTX actual version/JSON/CLI flags | Block context pack and show install-specific diagnostic. | -| Pi installed import namespace and skill discovery | Keep CLI usable; defer extension/skill packaging. | -| Firstmate home identity and registered project | Block sync. | -| tasks-axi version, backend, dependency commands | Block sync; provide dry-run export for manual inspection only. | -| no-mistakes gate in intended project mode | Block automated shipping claims; continue local deterministic checks. | -| Windows/macOS path behavior | Report platform limitation honestly; do not claim cross-platform gate. | +| Probe | Failure response | +| ------------------------------------------------- | ---------------------------------------------------------------------- | +| CTX actual version/JSON/CLI flags | Block context pack and show install-specific diagnostic. | +| Pi installed import namespace and skill discovery | Keep CLI usable; defer extension/skill packaging. | +| Firstmate home identity and registered project | Block sync. | +| tasks-axi version, backend, dependency commands | Block sync; provide dry-run export for manual inspection only. | +| no-mistakes gate in intended project mode | Block automated shipping claims; continue local deterministic checks. | +| Windows/macOS path behavior | Report platform limitation honestly; do not claim cross-platform gate. | ## Primary documents From cf10d9b55366ad5277f4f15999813e0d123a0180 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 29 Sep 2026 17:31:09 +0000 Subject: [PATCH 08/45] fix(cp06): close final review gaps --- ARCHITECTURE.md | 2 +- BUILD_PLAN.md | 2 +- docs/CONTRACTS.md | 2 +- docs/INTEGRATIONS.md | 2 +- docs/WORKFLOW.md | 2 +- docs/probes/CP-06.md | 88 ++++++--- package.json | 2 +- scripts/cp06-worker-audit.mjs | 122 ++++++++++++ scripts/replay-cp06-ten-pack.mjs | 113 +++++++++++ scripts/replay-cp06-worker.mjs | 177 +++++++++--------- src/core/context.ts | 6 +- test/context.test.ts | 24 ++- test/cp06-worker-audit.test.mjs | 115 ++++++++++++ test/fixtures/cp06-context/ARCHITECTURE.md | 2 +- .../cases/AUTH-worker-before.yaml | 17 ++ .../workflow/critical-auth-isolation.ts | 16 +- test/workflow.test.ts | 44 +++++ 17 files changed, 600 insertions(+), 136 deletions(-) create mode 100644 scripts/cp06-worker-audit.mjs create mode 100644 test/cp06-worker-audit.test.mjs create mode 100644 test/fixtures/cp06-context/cases/AUTH-worker-before.yaml diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 80d1a62..09a02cc 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -43,7 +43,7 @@ Recommended exit codes: `0` success; `2` validation failure; `3` unavailable/sta ## CTX context pack -`factory context TASK-ID` first includes the exact tracked task contract and a concise global project constraint section. It invokes `ctx status --root --json` and `ctx doctor --offline --root --json` as appropriate, then `ctx pack --root --token-budget --document ... --json`. The query is derived from title, outcome, acceptance, and topic strings; it is never a command. Repeated `--document` filters come from the task's declared `context.required` selection, so CTX ranks within task-local authority instead of adding unrelated corpus excerpts. Canonical required documents are read from their original files, checked against root containment, and admitted before CTX excerpts; declaring a source required never substitutes an excerpt for the exact original. The adapter records CTX generation, file and line provenance, and available hashes. On stale, missing, or insufficient retrieval, it returns `CONTEXT_BLOCKED` with an actionable diagnostic. It does not fill the pack with guessed material or hide post-retrieval excerpts to improve a relevance score. +`factory context TASK-ID` first includes the exact tracked task contract and a concise global project constraint section. It invokes `ctx status --root --json` and `ctx doctor --offline --root --json` as appropriate, then `ctx pack --root --token-budget --document ... --json`. The query is derived from title, outcome, acceptance, and topic strings; it is never a command. Repeated `--document` filters come from the task's declared `context.required` selection, so CTX ranks within task-local authority instead of adding unrelated corpus excerpts. When that valid selection is empty, Factory passes the project's mandatory `documents.required` paths instead; it never invokes `ctx pack` without a document filter, and a global-only task remains viable without widening to the corpus. Canonical required documents are read from their original files, checked against root containment, and admitted before CTX excerpts; declaring a source required never substitutes an excerpt for the exact original. The adapter records CTX generation, file and line provenance, and available hashes. On stale, missing, or insufficient retrieval, it returns `CONTEXT_BLOCKED` with an actionable diagnostic. It does not fill the pack with guessed material or hide post-retrieval excerpts to improve a relevance score. Default total task-context budgets (including contract and mandatory material) are 4,000 tokens for bounded, 8,000 for normal, and 15,000 for critical; these are design defaults to measure, not promises that CTX necessarily returns that many tokens. Reserve space for task contract and required sources before calling CTX with the remainder. The adapter enforces a byte ceiling as well as CTX's token budget, deduplicates overlapping excerpts, and never truncates an authoritative requirement mid-sentence. The pack is written atomically to `.factory/state/context/TASK-ID.md` with a sidecar receipt containing contract digest, source digests, CTX generation, timestamp, and retrieval mode. A worker verifies these before trusting the pack; if changed, regenerate it. diff --git a/BUILD_PLAN.md b/BUILD_PLAN.md index fc2ca53..4750e78 100644 --- a/BUILD_PLAN.md +++ b/BUILD_PLAN.md @@ -89,7 +89,7 @@ - [x] Create tests/fixtures for critical auth isolation and normal API behavior, plus one simulated reviewer finding that causes repair and retest. - [x] Run an actual worker task with only task/pack/evidence sources in a fresh context; inspect whether it needs missing docs. -- [x] **Gate:** P-06 and P-07 via the tracked ten-pack/P-07 replay in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. +- [x] **Gate:** P-06 and P-07 via the tracked ten-pack, protocol-negative/fresh-worker, and P-07 replays in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. ## CP-07 — Thin Pi command and status convenience diff --git a/docs/CONTRACTS.md b/docs/CONTRACTS.md index 1f7b4ee..c8dc595 100644 --- a/docs/CONTRACTS.md +++ b/docs/CONTRACTS.md @@ -80,7 +80,7 @@ evidence: delivery: project-default ``` -`depends_on` names local task IDs, is acyclic, and cannot name self. `advances` names completion IDs and may be empty only for a task explicitly marked `kind: enabling`; enabling tasks still have testable outcomes. `complexity` is `bounded | normal | critical`; `risk` is `bounded | normal | critical` and controls review policy, not a specific model name. `context.required` is the reviewed task-local authority selection: Factory reads every listed original exactly and passes the same paths as CTX document filters. Include each known source that can decide an acceptance or risk condition; do not add broad directories or unrelated documents as insurance. A worker that discovers an omitted decisive source names it, performs one targeted original read, and proposes that exact path for the contract before regeneration. `delivery` defaults to `project-default` and cannot override the actual Firstmate project mode; if the project mode conflicts with a task request, validation blocks publication. All required evidence categories are verified by evidence receipts rather than assumed from task closure. +`depends_on` names local task IDs, is acyclic, and cannot name self. `advances` names completion IDs and may be empty only for a task explicitly marked `kind: enabling`; enabling tasks still have testable outcomes. `complexity` is `bounded | normal | critical`; `risk` is `bounded | normal | critical` and controls review policy, not a specific model name. `context.required` is the reviewed task-local authority selection: Factory reads every listed original exactly and passes the same paths as CTX document filters. An empty list explicitly means that no task-specific original is needed; Factory still filters CTX to the project's mandatory `documents.required` authority and never treats the empty selection as unrestricted corpus access. Include each known source that can decide an acceptance or risk condition; do not add broad directories or unrelated documents as insurance. A worker that discovers an omitted decisive source names it, performs one targeted original read, and proposes that exact path for the contract before regeneration. `delivery` defaults to `project-default` and cannot override the actual Firstmate project mode; if the project mode conflicts with a task request, validation blocks publication. All required evidence categories are verified by evidence receipts rather than assumed from task closure. ## Task receipt: `.factory/state/evidence/SAVE-001.json` diff --git a/docs/INTEGRATIONS.md b/docs/INTEGRATIONS.md index e2de01b..d4bbb3b 100644 --- a/docs/INTEGRATIONS.md +++ b/docs/INTEGRATIONS.md @@ -8,7 +8,7 @@ The user's archived CTX README describes a Python 3.12+ local CLI, source-proven CP-00 probes: detect `ctx`; create a disposable Git fixture with two normative Markdown docs; initialize and add docs according to local `ctx --help`; index with `--no-embeddings` so the test does not download anything; run status, offline doctor, pack, and exact lines; record actual JSON fields and failure codes. Confirm stale-source detection by changing a source after indexing. A user's globally installed model may then be tested separately. Do not index the Factory repo until its initial documents are approved. -Context policy: required files are read exactly. The installed `ctx pack` accepts repeated `--document` filters; Factory supplies the task's reviewed `context.required` paths so retrieval ranks only within task-local authority while exact originals remain in the pack. Changed docs invalidate a pack; unavailable local embedding may allow the explicitly reported lexical mode if it retrieves sufficient authority. Query text is data, never shell code. `ctx pack` is one context source, not a substitute for Git, `rg`, or project checks, and Factory does not post-filter CTX output to falsify provenance. +Context policy: required files are read exactly. The installed `ctx pack` accepts repeated `--document` filters; Factory supplies the task's reviewed `context.required` paths so retrieval ranks only within task-local authority while exact originals remain in the pack. If that valid selection is empty, Factory supplies the project's mandatory `documents.required` paths; it never sends an unfiltered pack request. Changed docs invalidate a pack; unavailable local embedding may allow the explicitly reported lexical mode if it retrieves sufficient authority. Query text is data, never shell code. `ctx pack` is one context source, not a substitute for Git, `rg`, or project checks, and Factory does not post-filter CTX output to falsify provenance. ## Pi diff --git a/docs/WORKFLOW.md b/docs/WORKFLOW.md index d512ad4..f387a65 100644 --- a/docs/WORKFLOW.md +++ b/docs/WORKFLOW.md @@ -9,7 +9,7 @@ This document specifies the methodology. CP-04 turns it into a concise, discover 3. **Project truth:** author a concise `PROJECT.md` with goal, audience, user journeys, non-goals, constraints, and V1 limits. Draft `ARCHITECTURE.md` with component boundaries, data model, trust boundary, deployment shape, and failure handling. Keep technical choices proportionate to the first release. 4. **Independent oracle:** in a separate reasoning pass, author `.factory/completion.yaml` from product intent and risks before decomposing into tasks. Specify observable end-to-end behavior, isolation/security invariants, recovery, and any performance thresholds the project actually needs. Include one negative test for important trust boundaries. Record the current Git SHA of this first oracle so later changes are visible. 5. **Slices and DAG:** choose the first vertical slice that a user can exercise; add a minimal foundation only where it unlocks slices. Tasks should have one independently reviewable outcome, bounded scope, testable acceptance IDs, expected evidence, and dependencies. Prefer a serial backbone of slices with at most three genuinely independent ready tasks. Do not turn one feature into separate backend/frontend/UI megatasks that integrate only at the end. -6. **Validate and retrieve:** use `factory validate`; repair schema/graph gaps; declare each known decisive original in the task's `context.required` selection, then generate its context pack via CTX. Factory constrains CTX to those task-local documents while retaining exact global and required originals. Inspect the first few packs for a missing decisive source and irrelevant bulk; correct an omission with one targeted exact read and the narrowest contract update, not a broad corpus expansion or a post-retrieval provenance filter. +6. **Validate and retrieve:** use `factory validate`; repair schema/graph gaps; declare each known decisive original in the task's `context.required` selection, then generate its context pack via CTX. Factory constrains CTX to those task-local documents while retaining exact global and required originals; an explicitly empty task-local selection is constrained to mandatory global originals rather than becoming an unscoped corpus query. Inspect the first few packs for a missing decisive source and irrelevant bulk; correct an omission with one targeted exact read and the narrowest contract update, not a broad corpus expansion or a post-retrieval provenance filter. 7. **Preview and publish:** run `factory sync --dry-run --home `, inspect which items will be created and which are blocked, then explicitly apply to the selected home. The engineer's direction to execute work authorizes normal publication into the intended Firstmate home; ask only if the home or project identity is genuinely ambiguous. 8. **Execute with Firstmate:** Firstmate chooses ready work and workers through its existing dispatch policy. Each worker receives the task contract, relevant source pack, and evidence expectations; worker tests and repairs locally. Firstmate handles the actual delivery path and approvals required by its configured project mode. 9. **Verify per task and per project:** run deterministic gates first; semantic review receives task, diff, architecture, and observed evidence rather than the worker's full transcript. Repair specific findings and rerun changed checks. Then execute the original project oracle at the release candidate SHA. Failed project behavior creates a reviewed follow-up task. The status report must show backlog and product completion separately. diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index c4a4f90..cbdabaf 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -4,13 +4,16 @@ Date: 2026-09-29. Active checkpoint: `BUILD_PLAN.md` CP-06 only. Acceptance: P-06 and P-07. Original base: `8337ab7fcc5b511d3cb359ea214b6ae18003aeaa`. Original PR #8 head (preserved unchanged in this branch's history): -`8b945a4ed6fcbb43a8b7aa7ce3a6532a4a23d145`. +`8b945a4ed6fcbb43a8b7aa7ce3a6532a4a23d145`. PR #9 correction head at final +review intake (also preserved unchanged in history): +`cdaf5e5c03d36f1c9b6bb0115793766a384cc178`. -This correction was performed in the isolated worktree reported by both +The final correction was performed in the isolated worktree reported by both `pwd -P` and `git rev-parse --show-toplevel` as -`/home/ansh/.treehouse/Factory-86709f/7/Factory`. It did not touch PR #8, its -branch or worker state, a real Firstmate home, routing, shipping configuration, -accounts, credentials, or the shared no-mistakes daemon. +`/home/ansh/.treehouse/Factory-86709f/8/Factory`. It did not touch PR #8, PR #9, +their branches or worker state, a real Firstmate home, routing, shipping +configuration, live accounts, deployed systems, or the shared no-mistakes +daemon. ## Normative result @@ -21,9 +24,17 @@ do not make irrelevant authority relevant. The original figures remain in the runtime evidence manifest and Git history; this correction does not relabel or hide them. +Independent exact-head review then rejected PR #9 for three additional concrete +cases: empty task-local selection widened to the CTX corpus, the worker oracle +accepted unsuccessful or partial reads, and the critical fixture disclosed a +foreign identifier during creation. The final correction adds executable +regressions for those cases without expanding CP-06's architecture. + The smallest in-scope correction uses CTX's existing repeated `--document` -filter. Factory now passes the task's reviewed `context.required` paths to -`ctx pack`. Those same originals are still read exactly and included with their +filter. Factory passes the task's reviewed `context.required` paths to `ctx +pack`; when that valid selection is empty, it passes the mandatory project +`documents.required` paths rather than accidentally issuing an unrestricted +query. Those same originals are still read exactly and included with their digests. This is source selection before retrieval, not post-retrieval deletion: raw CTX output remains in ignored proof scratch, provenance is unchanged, and a missing declared source still fails closed. No CTX code, parallel retrieval @@ -46,20 +57,24 @@ Tracked, secret-free inputs and runners: - `scripts/replay-cp06-ten-pack.mjs` — creates a disposable Git/CTX repository, records the unscoped bad path, one-condition AUTH correction, exact ten fixed packs and receipts, and a missing-original disconfirming case; -- `scripts/replay-cp06-worker.mjs` — gives a new Pi process only the bounded AUTH - handoff and permits one exact-original read; +- `scripts/cp06-worker-audit.mjs`, `test/cp06-worker-audit.test.mjs`, and + `scripts/replay-cp06-worker.mjs` — reject incomplete, failed, wrong-range, + mismatched, out-of-order, and extra-project reads, then give a new Pi process + only the bounded AUTH handoff and permit one exact-original read; - `test/fixtures/workflow/history/` and `scripts/replay-cp06-p07.mjs` — compact P-07 finding plus executable fail-before/fix-after/auth-retest sequence; - `test/workflow.test.ts` — checked-in normal API and in-memory auth behavior, including the retained malformed-JSON regression check. -Generated raw CTX JSON, packs, receipts, model output, and command logs remain -ignored under `.factory/state/cp06-correction/`; they are not committed -transcripts. Each replay writes a structured `evidence.json` containing command, -exit, output path and SHA-256, tested Git SHA, UTC date, reviewer, tool/index -identity, source/contract/pack/receipt hashes, budgets, and provenance. This -meets the general proof rule without committing runtime state or model traces. +Generated raw CTX JSON, packs, receipts, and deterministic command logs remain +ignored under `.factory/state/cp06-correction/`. The live worker does not retain +its prompt, event stream, stderr, or raw output; it retains one compact +`evidence.json` with their hashes, correlated event indexes, successful-read +state, and exact-original digest. Each replay binds command, exit, tested Git +SHA, UTC date, reviewer, tool/index identity, and applicable +source/contract/pack/receipt hashes, budgets, and provenance. This meets the +general proof rule without committing runtime state or model transcripts. Run in this order: @@ -131,6 +146,13 @@ relevant; every other topical source is irrelevant. The replay result is: - exact contract, target-source, pack, receipt, output, and source digests recorded per task. +An additional schema-valid global-only contract uses `context.required: []`. +Factory exits 0 and emits no topical digest or excerpt. A real CTX call with the +same query and explicit `PROJECT.md`/`ARCHITECTURE.md` filters is `COMPLETE`; its +actual provenance contains only those originals and their checked digests, +despite all unrelated topical originals being indexed. This preserves viable +global-only work without treating an empty set as unrestricted retrieval. + A separate document-scoped real-CTX disconfirming probe exits 0 but reports `PARTIAL` at its deliberately constrained 1,300-token budget; successful command execution alone is not accepted as completeness. The missing-original case then @@ -147,15 +169,24 @@ skills, the Factory skill explicitly selected, and only the `read` tool. Its initial handoff contains exactly the AUTH-before contract, bounded missing-AUTH pack, and a short deterministic evidence statement—no prior transcript. -The worker reports `MISSING_SOURCE`, performs exactly one read of the supplied -`docs/AUTH.md` original, performs no broad scan, recovers all four decisive -constraints above, and identifies the untested request-owner/create-time binding -case. `scripts/replay-cp06-worker.mjs` records Pi's machine-readable event stream -only in ignored scratch and derives the exact read count and path from actual -tool-call events rather than trusting the model's summary. It rejects any other -project read or tool call and checks the required corrected constraints. Model -wording is inherently nondeterministic; the event audit, structural oracle, and -exact source are not. +The fresh-worker handoff deliberately omits the concrete rule that +request-supplied owner fields are ignored at creation time while retaining the +already observed `401` and concealed, non-mutating `404` evidence. The worker +reports `MISSING_SOURCE`, performs exactly one read of the supplied +`docs/AUTH.md` original, performs no broad scan, recovers principal-derived +creation ownership and the ignored request-owner rule, and identifies the +missing creation-time integration evidence. + +The audit correlates `tool_execution_start` and `tool_execution_end` by tool-call +ID and event order, requires `isError: false`, forbids offset/limit and all other +reads/tools, compares the returned UTF-8 bytes and SHA-256 to the tracked exact +original, and requires the final response after the completion. Deterministic +protocol tests reject omitted, failed, partial, mismatched, out-of-order, and +extra-project cases. The genuine Pi process uses a disposable HOME and agent +directory, links rather than copies the caller-authorized credential file, +verifies that file remains byte-identical, deletes the evaluation home, and +retains no raw transcript. Model wording is nondeterministic; the protocol and +source-byte oracle are not. ## P-07 finding, repair, and retest @@ -175,9 +206,12 @@ for malformed profile JSON. The repaired normal API returns stable The final behavioral fixture also proves absent/forged credentials return 401, own-account access succeeds, cross-account read/update/delete return 404 without mutation, request-supplied ownership is ignored, and stored ownership comes from -the principal. It uses only disposable literal identities and in-memory state; -there is no network, filesystem, environment, real account, or security-sensitive -credential access. +the principal. Identifiers are tenant-scoped: account B can create its own +`item-a` without learning that account A already uses that identifier, while +blank titles return the same `400 invalid_title` for foreign-collision and unused +identifiers and leave state unchanged. It uses only disposable literal +identities and in-memory state; there is no network, filesystem, environment, +real account, or security-sensitive credential access. This task-level review remains ordered after deterministic checks and before the one normal Firstmate-owned delivery pipeline. Factory neither invokes diff --git a/package.json b/package.json index 6b10d02..457eff4 100644 --- a/package.json +++ b/package.json @@ -9,7 +9,7 @@ "scripts": { "build": "tsc -p tsconfig.json", "typecheck": "tsc -p tsconfig.json --noEmit", - "test": "npm run build && node --test \"dist/test/**/*.test.js\"", + "test": "npm run build && node --test \"dist/test/**/*.test.js\" \"test/**/*.test.mjs\"", "format": "prettier --write \"{src,test,scripts,.github,schemas}/**/*.{ts,js,mjs,yml,yaml,json}\" \"*.json\" \"docs/probes/*.md\" \"docs/superpowers/plans/*.md\"", "format:check": "prettier --check \"{src,test,scripts,.github,schemas}/**/*.{ts,js,mjs,yml,yaml,json}\" \"*.json\" \"docs/probes/*.md\" \"docs/superpowers/plans/*.md\"", "lint": "node scripts/lint.mjs", diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs new file mode 100644 index 0000000..1890cea --- /dev/null +++ b/scripts/cp06-worker-audit.mjs @@ -0,0 +1,122 @@ +import { createHash } from "node:crypto"; +import { resolve } from "node:path"; + +export function auditReadEvents(events, options) { + const expectedPath = resolve(options.root, options.expectedOriginalPath); + const expectedBytes = Buffer.isBuffer(options.expectedOriginal) + ? options.expectedOriginal + : Buffer.from(options.expectedOriginal, "utf8"); + const expectedText = expectedBytes.toString("utf8"); + const decisiveText = options.decisiveText ?? []; + const starts = events + .map((event, index) => ({ event, index })) + .filter(({ event }) => event?.type === "tool_execution_start"); + const ends = events + .map((event, index) => ({ event, index })) + .filter(({ event }) => event?.type === "tool_execution_end"); + const readStarts = starts.filter(({ event }) => event.toolName === "read"); + const readEnds = ends.filter(({ event }) => event.toolName === "read"); + const otherToolCalls = starts.filter(({ event }) => event.toolName !== "read"); + const resolvedReads = readStarts.map(({ event, index }) => ({ + event, + index, + path: event.args?.path, + resolvedPath: + typeof event.args?.path === "string" ? resolve(options.root, event.args.path) : null, + })); + const originalReads = resolvedReads.filter((entry) => entry.resolvedPath === expectedPath); + const rejectedReads = resolvedReads.filter((entry) => entry.resolvedPath !== expectedPath); + const summary = { + read_count: readStarts.length, + project_read_count: originalReads.length, + project_read_paths: originalReads.map((entry) => entry.resolvedPath), + rejected_read_paths: rejectedReads.map((entry) => entry.path), + other_tool_calls: otherToolCalls.map(({ event }) => event.toolName), + completed_successfully: false, + exact_original: false, + expected_sha256: sha256(expectedBytes), + returned_sha256: null, + returned_bytes: null, + tool_call_id: null, + start_event_index: null, + end_event_index: null, + response_event_index: null, + }; + const reject = (reason) => ({ ok: false, reason, summary }); + + if (otherToolCalls.length > 0) { + return reject(`unexpected tool calls ${summary.other_tool_calls.join(", ")}`); + } + if (readStarts.length !== 1 || originalReads.length !== 1 || rejectedReads.length > 0) { + return reject( + `expected exactly one read of the supplied original, saw ${originalReads.length} original and ${rejectedReads.length} other reads`, + ); + } + + const start = originalReads[0]; + const callId = start.event.toolCallId; + summary.tool_call_id = typeof callId === "string" ? callId : null; + summary.start_event_index = start.index; + if (typeof callId !== "string" || callId.length === 0) { + return reject("original read start omitted toolCallId"); + } + if (start.event.args?.offset !== undefined && start.event.args.offset !== 1) { + return reject("original read started outside the first line"); + } + if (start.event.args?.limit !== undefined) { + return reject("original read used a partial line limit"); + } + + const matchingEnds = readEnds.filter(({ event }) => event.toolCallId === callId); + if (matchingEnds.length !== 1) { + return reject( + `expected one matching read completion for ${callId}, saw ${matchingEnds.length}`, + ); + } + if (readEnds.length !== 1) { + return reject("read event stream contains an unmatched or duplicate completion"); + } + const end = matchingEnds[0]; + summary.end_event_index = end.index; + if (end.index <= start.index) { + return reject("read completion did not occur after its matching start"); + } + if (end.event.isError !== false) { + return reject("matching read completion was not explicitly successful"); + } + const content = end.event.result?.content; + if ( + !Array.isArray(content) || + content.length !== 1 || + content[0]?.type !== "text" || + typeof content[0].text !== "string" + ) { + return reject("successful read completion omitted one authoritative text result"); + } + const returnedBytes = Buffer.from(content[0].text, "utf8"); + summary.returned_sha256 = sha256(returnedBytes); + summary.returned_bytes = returnedBytes.length; + if (!returnedBytes.equals(expectedBytes)) { + return reject("read result bytes do not equal the supplied exact original"); + } + if (decisiveText.some((text) => !expectedText.includes(text))) { + return reject("the supplied original does not contain every declared decisive constraint"); + } + + const finalResponse = events + .map((event, index) => ({ event, index })) + .filter(({ event }) => event?.type === "message_end" && event.message?.role === "assistant") + .at(-1); + summary.response_event_index = finalResponse?.index ?? null; + if (finalResponse === undefined || finalResponse.index <= end.index) { + return reject("final assistant response did not occur after the successful original read"); + } + + summary.completed_successfully = true; + summary.exact_original = true; + return { ok: true, summary }; +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index ca4d0e6..1014a32 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -24,6 +24,7 @@ await mkdir(join(fixtureRoot, ".factory/state"), { recursive: true }); const oracle = JSON.parse(await readFile(join(fixtureRoot, "oracle.json"), "utf8")); const fixedAuth = await readFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), "utf8"); const beforeAuth = await readFile(join(fixtureRoot, "cases/AUTH-before.yaml"), "utf8"); +const workerBeforeAuth = await readFile(join(fixtureRoot, "cases/AUTH-worker-before.yaml"), "utf8"); await writeFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), beforeAuth); run("git-init", "git", ["init", "-q", fixtureRoot], { cwd: root }); @@ -140,6 +141,18 @@ await copyArtifact(initialAuth.receipt_path, "auth-initial-missing.receipt.json" if (initialAuthPack.includes("docs/AUTH.md")) { throw new Error("initial AUTH handoff unexpectedly contains the missing decisive original"); } +await writeFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), workerBeforeAuth); +const workerInitialAuth = factoryContext("auth-worker-missing", "AUTH-001", 0); +const workerInitialPack = await readFile(join(fixtureRoot, workerInitialAuth.pack_path), "utf8"); +await copyArtifact(workerInitialAuth.pack_path, "auth-worker-missing.pack.md"); +await copyArtifact(workerInitialAuth.receipt_path, "auth-worker-missing.receipt.json"); +if ( + workerInitialPack.includes("docs/AUTH.md") || + workerInitialPack.includes("request-supplied ownership") +) { + throw new Error("fresh-worker handoff did not omit the decisive creation ownership rule"); +} +await writeFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), beforeAuth); run( "ctx-add-auth", @@ -184,6 +197,93 @@ if (doctor.offline_ready !== true || doctor.network_attempted !== false) { throw new Error("CTX offline doctor did not prove offline readiness"); } +const globalOnlyTaskPath = join(fixtureRoot, ".factory/tasks/GLOBAL-ONLY.yaml"); +const globalOnlyQuery = [ + "Keep credentials and raw worker transcripts out of offline context", + "A worker applies the mandatory rule that credentials and raw worker transcripts never enter a context pack and normal operation is offline.", + "A-GLOBAL: The pack contains the mandatory credentials, raw transcript, and offline constraints without topical documents.", + "credentials raw worker transcripts context pack normal operation offline", +].join("\n"); +await writeFile( + globalOnlyTaskPath, + `schema_version: 1 +id: GLOBAL-ONLY +title: Keep credentials and raw worker transcripts out of offline context +outcome: A worker applies the mandatory rule that credentials and raw worker transcripts never enter a context pack and normal operation is offline. +depends_on: [] +complexity: bounded +risk: bounded +acceptance: + - id: A-GLOBAL + statement: The pack contains the mandatory credentials, raw transcript, and offline constraints without topical documents. +advances: [C-COPY] +context: + topics: [credentials, raw worker transcripts, context pack, normal operation, offline] + required: [] +evidence: + required: [integration] +delivery: project-default +`, +); +const emptySelectionFactory = factoryContext("empty-selection-factory", "GLOBAL-ONLY", 0); +const emptySelectionPack = await readFile( + join(fixtureRoot, emptySelectionFactory.pack_path), + "utf8", +); +const emptySelectionTopicalDigests = Object.keys(emptySelectionFactory.source_digests).filter( + (path) => path.startsWith("docs/"), +); +if ( + emptySelectionTopicalDigests.length > 0 || + /^## Retrieved context: docs\//m.test(emptySelectionPack) +) { + throw new Error("empty task-local selection widened Factory output to topical authority"); +} +const emptySelectionCtx = run( + "empty-selection-real-ctx", + ctx, + [ + "pack", + globalOnlyQuery, + "--root", + fixtureRoot, + "--document", + "PROJECT.md", + "--document", + "ARCHITECTURE.md", + "--token-budget", + "3500", + "--json", + ], + { cwd: fixtureRoot, expected: [0], timeout: 120_000 }, +); +const emptySelectionCtxJson = parseJsonIfPresent(emptySelectionCtx.stdout); +if (emptySelectionCtxJson?.completeness_status !== "COMPLETE") { + throw new Error("empty-selection real-CTX probe was not COMPLETE"); +} +const emptySelectionProvenance = itemPaths(emptySelectionCtxJson); +if ( + emptySelectionProvenance.length === 0 || + emptySelectionProvenance.some((path) => !globalSources.includes(path)) +) { + throw new Error( + `empty-selection real-CTX provenance escaped mandatory globals: ${emptySelectionProvenance.join(",")}`, + ); +} +const emptySelectionOriginals = await Promise.all( + emptySelectionProvenance.map(async (path) => { + const item = emptySelectionCtxJson.items.find( + (candidate) => candidate?.source?.provenance?.document_path === path, + ); + const actualSha256 = await fileHash(join(fixtureRoot, path)); + if (item?.source?.provenance?.document_sha256 !== actualSha256) { + throw new Error(`empty-selection provenance digest does not match ${path}`); + } + return { path, document_sha256: actualSha256 }; + }), +); +await rm(globalOnlyTaskPath); + const counterfactualBefore = factoryContext("counterfactual-before", "AUTH-001", 0); const counterfactualBeforePack = await readFile( join(fixtureRoot, counterfactualBefore.pack_path), @@ -269,6 +369,7 @@ const sourceFiles = [ "ARCHITECTURE.md", "oracle.json", "cases/AUTH-before.yaml", + "cases/AUTH-worker-before.yaml", ...oracle.tasks.map((entry) => `.factory/tasks/${entry.id}.yaml`), ...topicalSources, ]; @@ -313,6 +414,18 @@ const manifest = { irrelevant_topical_originals: broadIrrelevantEvidence, missing_target: true, }, + empty_selection: { + semantics: "empty task-local authority falls back to mandatory global document filters", + factory_exit_code: 0, + factory_output_path: commandById("empty-selection-factory").outputPath, + factory_output_sha256: commandById("empty-selection-factory").outputSha256, + factory_topical_source_digests: emptySelectionTopicalDigests, + ctx_exit_code: emptySelectionCtx.exitCode, + ctx_output_path: emptySelectionCtx.outputPath, + ctx_output_sha256: emptySelectionCtx.outputSha256, + ctx_completeness_status: emptySelectionCtxJson.completeness_status, + ctx_provenance: emptySelectionOriginals, + }, one_condition_counterfactual: { unchanged_fields: "all AUTH task fields except context.required", before_required: ["PROJECT.md", "ARCHITECTURE.md"], diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 9dc4001..c60eabb 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -1,27 +1,27 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; -import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; -import { spawnSync } from "node:child_process"; +import { auditReadEvents } from "./cp06-worker-audit.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); const workerRoot = join(outputRoot, "worker"); const fixtureRoot = join(outputRoot, "ten-pack", "repo"); -const initialPackPath = join(outputRoot, "ten-pack", "raw", "auth-initial-missing.pack.md"); -const contractPath = join(root, "test/fixtures/cp06-context/cases/AUTH-before.yaml"); +const initialPackPath = join(outputRoot, "ten-pack", "raw", "auth-worker-missing.pack.md"); +const contractPath = join(root, "test/fixtures/cp06-context/cases/AUTH-worker-before.yaml"); const originalPath = join(fixtureRoot, "docs/AUTH.md"); +await rm(workerRoot, { recursive: true, force: true }); await mkdir(workerRoot, { recursive: true }); const contract = await readFile(contractPath, "utf8"); const pack = await readFile(initialPackPath, "utf8"); -const prompt = `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. If a decisive authoritative source is missing, report MISSING_SOURCE and use the read tool exactly once on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; -const promptPath = join(workerRoot, "prompt.txt"); -const eventStreamPath = join(workerRoot, "event-stream.jsonl"); -const outputPath = join(workerRoot, "output.txt"); -const stderrPath = join(workerRoot, "stderr.txt"); -await writeFile(promptPath, prompt); +const original = await readFile(originalPath); +const decisiveConstraint = "request-supplied owner fields are ignored"; +const prompt = `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; const provider = process.env.CP06_PI_PROVIDER ?? "openai-codex"; const model = process.env.CP06_PI_MODEL ?? "gpt-5.6-sol"; @@ -44,38 +44,57 @@ const args = [ "read", prompt, ]; -const result = spawnSync("pi", args, { - cwd: root, - encoding: "utf8", - env: process.env, - timeout: 600_000, -}); -await writeFile(eventStreamPath, result.stdout ?? ""); -await writeFile(stderrPath, result.stderr ?? ""); +const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-worker-")); +const evaluationAgentDir = join(evaluationHome, "pi-agent"); +await mkdir(evaluationAgentDir); +const credentialSource = resolve( + process.env.CP06_PI_AUTH_FILE ?? + join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "auth.json"), +); +const credentialHashBefore = await fileHash(credentialSource); +await symlink(credentialSource, join(evaluationAgentDir, "auth.json")); +let result; +let credentialHashAfter; +try { + result = spawnSync("pi", args, { + cwd: root, + encoding: "utf8", + env: isolatedPiEnvironment(evaluationHome, evaluationAgentDir), + timeout: 600_000, + }); + credentialHashAfter = await fileHash(credentialSource); +} finally { + await rm(evaluationHome, { recursive: true, force: true }); +} +if (credentialHashAfter !== credentialHashBefore) { + throw new Error("fresh Pi worker modified the source credential file"); +} if (result.error !== undefined || result.status !== 0) { process.stderr.write(result.stderr ?? ""); throw new Error(`fresh Pi worker exited ${result.status}`); } const events = parseEventStream(result.stdout ?? ""); const responseText = finalAssistantText(events); -await writeFile(outputPath, responseText); const response = parseResponse(responseText); -const readAudit = auditReadEvents(events, originalPath); +const readAudit = auditReadEvents(events, { + root, + expectedOriginalPath: originalPath, + expectedOriginal: original, + decisiveText: [decisiveConstraint], +}); if (!readAudit.ok) { throw new Error(`fresh worker violated the event-stream read oracle: ${readAudit.reason}`); } const requiredConstraints = [ - /authenticated principal/i, + /ownership comes only from the authenticated principal/i, /request-supplied owner fields? (?:are )?ignored/i, - /credentials?.*401/i, - /cross-account.*404/i, - /(?:not mutate|no mutation|unchanged state)/i, ]; const constraintText = Array.isArray(response.corrected_constraints) ? response.corrected_constraints.join(" ") : ""; if ( response.status !== "MISSING_SOURCE" || + response.missing_source !== "docs/AUTH.md" || response.broad_scan !== false || !requiredConstraints.every((constraint) => constraint.test(constraintText)) ) { @@ -105,23 +124,29 @@ const manifest = { contract_sha256: await fileHash(contractPath), pack_path: relative(initialPackPath), pack_sha256: await fileHash(initialPackPath), + original_path: relative(originalPath), + original_sha256: await fileHash(originalPath), + deliberately_omitted_constraint_sha256: sha256(decisiveConstraint), evidence: "in-memory auth 401 and concealed non-mutating cross-account 404 observations", - prompt_path: relative(promptPath), - prompt_sha256: await fileHash(promptPath), + prompt_sha256: sha256(prompt), + }, + isolation: { + disposable_home: true, + disposable_pi_agent_dir: true, + credential_mode: "symlink to caller-authorized source; integrity checked; removed after run", + credential_source_unchanged: credentialHashAfter === credentialHashBefore, + raw_transcript_retained: false, }, event_stream: { - path: relative(eventStreamPath), - sha256: await fileHash(eventStreamPath), + sha256: sha256(result.stdout ?? ""), event_count: events.length, read_audit: readAudit.summary, }, command: "pi --mode json --no-session --no-context-files --no-extensions --no-skills --skill skills/factory --tools read ", exit_code: result.status, - output_path: relative(outputPath), - output_sha256: await fileHash(outputPath), - stderr_path: relative(stderrPath), - stderr_sha256: await fileHash(stderrPath), + output_sha256: sha256(responseText), + stderr_sha256: sha256(result.stderr ?? ""), response, result: "pass", limitation: @@ -178,63 +203,6 @@ function messageText(message) { .join(""); } -function auditReadEvents(events, expectedOriginalPath) { - const expectedOriginal = resolve(expectedOriginalPath); - const allowedSkill = resolve(root, "skills/factory/SKILL.md"); - const readCalls = events - .filter((event) => event.type === "tool_execution_start" && event.toolName === "read") - .map((event) => ({ - path: event.args?.path, - resolved_path: typeof event.args?.path === "string" ? resolve(root, event.args.path) : null, - })); - const otherToolCalls = events.filter( - (event) => event.type === "tool_execution_start" && event.toolName !== "read", - ); - const invalidReadCalls = readCalls.filter( - (call) => call.resolved_path !== expectedOriginal && call.resolved_path !== allowedSkill, - ); - const originalReadCalls = readCalls.filter((call) => call.resolved_path === expectedOriginal); - const skillReadCalls = readCalls.filter((call) => call.resolved_path === allowedSkill); - const summary = { - read_count: readCalls.length, - project_read_count: originalReadCalls.length, - project_read_paths: originalReadCalls.map((call) => call.resolved_path), - allowed_skill_read_count: skillReadCalls.length, - allowed_skill_read_paths: skillReadCalls.map((call) => call.resolved_path), - rejected_read_paths: invalidReadCalls.map((call) => call.path), - other_tool_calls: otherToolCalls.map((event) => event.toolName), - }; - if (otherToolCalls.length > 0) { - return { - ok: false, - reason: `unexpected tool calls ${summary.other_tool_calls.join(", ")}`, - summary, - }; - } - if (originalReadCalls.length !== 1) { - return { - ok: false, - reason: `expected exactly one AUTH original read, saw ${originalReadCalls.length}`, - summary, - }; - } - if (skillReadCalls.length > 1) { - return { - ok: false, - reason: `expected at most one factory skill read, saw ${skillReadCalls.length}`, - summary, - }; - } - if (invalidReadCalls.length > 0) { - return { - ok: false, - reason: `unexpected read paths ${summary.rejected_read_paths.join(", ")}`, - summary, - }; - } - return { ok: true, summary }; -} - function parseResponse(output) { const trimmed = output.trim(); try { @@ -257,7 +225,32 @@ function relative(path) { } async function fileHash(path) { - return createHash("sha256") - .update(await readFile(path)) - .digest("hex"); + return sha256(await readFile(path)); +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + +function isolatedPiEnvironment(home, agentDir) { + const environment = { + HOME: home, + PI_CODING_AGENT_DIR: agentDir, + PI_SKIP_VERSION_CHECK: "1", + PI_TELEMETRY: "0", + }; + for (const name of [ + "PATH", + "LANG", + "LC_ALL", + "SSL_CERT_FILE", + "SSL_CERT_DIR", + "NODE_EXTRA_CA_CERTS", + "HTTP_PROXY", + "HTTPS_PROXY", + "NO_PROXY", + ]) { + if (process.env[name] !== undefined) environment[name] = process.env[name]; + } + return environment; } diff --git a/src/core/context.ts b/src/core/context.ts index 0f1158b..8897a02 100644 --- a/src/core/context.ts +++ b/src/core/context.ts @@ -139,6 +139,10 @@ export async function generateContextPack(options: { } const lexical = statusMode === "LEXICAL_ONLY"; + const retrievalDocuments = + input.task.context.required.length > 0 + ? input.task.context.required + : input.project.documents.required; const packArgs = [ "pack", contextQuery(input.task), @@ -146,7 +150,7 @@ export async function generateContextPack(options: { options.root, "--token-budget", String(remaining), - ...input.task.context.required.flatMap((path) => ["--document", path]), + ...retrievalDocuments.flatMap((path) => ["--document", path]), ...(lexical ? ["--no-embeddings"] : []), "--json", ]; diff --git a/test/context.test.ts b/test/context.test.ts index d79a7ee..e1ebaf4 100644 --- a/test/context.test.ts +++ b/test/context.test.ts @@ -142,6 +142,25 @@ test("context limits CTX retrieval to the task-local source selection", async () } }); +test("empty task-local selection constrains CTX to mandatory global authority", async () => { + const fixture = await seeded("empty-source-selection"); + try { + await writeFile( + join(fixture.root, ".factory/tasks/CTX-001.yaml"), + taskContract.replace("required: [PROJECT.md, ARCHITECTURE.md]", "required: []"), + ); + + const result = await factory(fixture, { mode: "empty-source-selection" }); + assert.equal(result.exit, 0, result.stderr || result.stdout); + const payload = parseObject(result.stdout); + const pack = await readFile(join(fixture.root, payload.pack_path), "utf8"); + assert.doesNotMatch(pack, /The overlapping excerpt sentinel/); + assert.equal(payload.source_digests["docs/EXTRA.md"], undefined); + } finally { + await fixture.dispose(); + } +}); + test("context permits explicitly reported lexical fallback only when retrieval is sufficient", async () => { const fixture = await seeded("lexical"); try { @@ -601,12 +620,13 @@ if (command === "doctor") { if (command === "pack") { const selectedDocuments = args.flatMap((arg, index) => arg === "--document" ? [args[index + 1]] : []); const sourceFiltered = mode === "source-filter" && selectedDocuments.includes("docs/TARGET.md"); - const relative = sourceFiltered ? "docs/TARGET.md" : "docs/EXTRA.md"; + const globalsOnly = mode === "empty-source-selection" && selectedDocuments.length === 2 && selectedDocuments.includes("PROJECT.md") && selectedDocuments.includes("ARCHITECTURE.md"); + const relative = sourceFiltered ? "docs/TARGET.md" : globalsOnly ? "PROJECT.md" : "docs/EXTRA.md"; const full = fs.readFileSync(path.join(root, relative), "utf8"); const firstEnd = sourceFiltered ? full.length : full.indexOf("Gamma context ends here."); const secondStart = sourceFiltered ? 0 : full.indexOf("The overlapping excerpt sentinel"); const item = (start, end, text = full.slice(start, end)) => ({source:{source_type:"excerpt",text,provenance:{document_path:relative,document_sha256:hash(full),start_line:1,end_line:6,start_offset:start,end_offset:end,range_sha256:hash(text),index_generation:7}},estimated_tokens:Math.ceil(Buffer.byteLength(text)/3)}); - const items = sourceFiltered ? [item(0, full.length)] : [item(0, firstEnd), item(secondStart, full.length), item(0, firstEnd)]; + const items = sourceFiltered || globalsOnly ? [item(0, full.length)] : [item(0, firstEnd), item(secondStart, full.length), item(0, firstEnd)]; if (mode === "aliases") { items[1].source.provenance.document_path = "docs/EXTRA-ALIAS.md"; const original = fs.readFileSync(path.join(root, "docs/PROJECT-ALIAS.md"), "utf8"); diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs new file mode 100644 index 0000000..5fdbf17 --- /dev/null +++ b/test/cp06-worker-audit.test.mjs @@ -0,0 +1,115 @@ +import { strict as assert } from "node:assert"; +import { readFile } from "node:fs/promises"; +import { resolve } from "node:path"; +import test from "node:test"; +import { auditReadEvents } from "../scripts/cp06-worker-audit.mjs"; + +const root = process.cwd(); +const originalPath = resolve(root, "test/fixtures/cp06-context/docs/AUTH.md"); +const original = await readFile(originalPath); +const options = { + root, + expectedOriginalPath: originalPath, + expectedOriginal: original, + decisiveText: ["request-supplied owner fields are ignored"], +}; +const start = { + type: "tool_execution_start", + toolName: "read", + toolCallId: "read-auth", + args: { path: originalPath }, +}; +const finalMessage = { + type: "message_end", + message: { role: "assistant", content: [{ type: "text", text: "{}" }] }, +}; + +function successfulEnd(overrides = {}) { + return { + type: "tool_execution_end", + toolName: "read", + toolCallId: "read-auth", + isError: false, + result: { content: [{ type: "text", text: original.toString("utf8") }] }, + ...overrides, + }; +} + +for (const [name, events] of [ + ["omitted completion", [start, finalMessage]], + [ + "stub failed read completion", + [ + start, + successfulEnd({ isError: true, result: { content: [{ type: "text", text: "ENOENT" }] } }), + finalMessage, + ], + ], + [ + "partial read", + [ + { ...start, args: { path: originalPath, offset: 1, limit: 1 } }, + successfulEnd({ + result: { content: [{ type: "text", text: "# Authentication and tenancy" }] }, + }), + finalMessage, + ], + ], + [ + "wrong starting range", + [ + { ...start, args: { path: originalPath, offset: 2 } }, + successfulEnd({ + result: { + content: [ + { + type: "text", + text: original.toString("utf8").split("\n").slice(1).join("\n"), + }, + ], + }, + }), + finalMessage, + ], + ], + [ + "mismatched completion id", + [start, successfulEnd({ toolCallId: "different-call" }), finalMessage], + ], + ["completion before start", [successfulEnd(), start, finalMessage]], + ["response before completion", [start, finalMessage, successfulEnd()]], +]) { + test(`worker read audit rejects ${name}`, () => { + assert.equal(auditReadEvents(events, options).ok, false); + }); +} + +test("worker read audit rejects another project read", () => { + const events = [ + start, + successfulEnd(), + { + type: "tool_execution_start", + toolName: "read", + toolCallId: "read-extra", + args: { path: resolve(root, "PROJECT.md") }, + }, + { + type: "tool_execution_end", + toolName: "read", + toolCallId: "read-extra", + isError: false, + result: { content: [{ type: "text", text: "extra" }] }, + }, + finalMessage, + ]; + assert.equal(auditReadEvents(events, options).ok, false); +}); + +test("worker read audit accepts one completed exact-original read before the response", () => { + const audit = auditReadEvents([start, successfulEnd(), finalMessage], options); + assert.equal(audit.ok, true); + assert.equal(audit.summary.completed_successfully, true); + assert.equal(audit.summary.exact_original, true); + assert.equal(audit.summary.expected_sha256, audit.summary.returned_sha256); +}); diff --git a/test/fixtures/cp06-context/ARCHITECTURE.md b/test/fixtures/cp06-context/ARCHITECTURE.md index f3fff27..5836bbf 100644 --- a/test/fixtures/cp06-context/ARCHITECTURE.md +++ b/test/fixtures/cp06-context/ARCHITECTURE.md @@ -4,4 +4,4 @@ The fixture uses an in-memory boundary around API, persistence, and presentation ## Trust boundary -Authenticated identity is authoritative for ownership. Cross-account operations neither disclose existence nor mutate state. +Authenticated identity is authoritative for ownership. Item identifiers are scoped to that identity, so create-time collisions cannot disclose another account's identifiers. Cross-account operations neither disclose existence nor mutate state. diff --git a/test/fixtures/cp06-context/cases/AUTH-worker-before.yaml b/test/fixtures/cp06-context/cases/AUTH-worker-before.yaml new file mode 100644 index 0000000..2a4ff6e --- /dev/null +++ b/test/fixtures/cp06-context/cases/AUTH-worker-before.yaml @@ -0,0 +1,17 @@ +schema_version: 1 +id: AUTH-001 +title: Conceal cross-account item operations +outcome: Existing-item operations across accounts remain concealed and non-mutating. +depends_on: [] +complexity: bounded +risk: critical +acceptance: + - id: A-AUTH + statement: Missing or forged credentials return 401 and cross-account read update and delete return non-mutating 404. +advances: [C-AUTH] +context: + topics: [tenancy, forged credentials] + required: [PROJECT.md, ARCHITECTURE.md] +evidence: + required: [integration] +delivery: project-default diff --git a/test/fixtures/workflow/critical-auth-isolation.ts b/test/fixtures/workflow/critical-auth-isolation.ts index e2369ce..ff9f732 100644 --- a/test/fixtures/workflow/critical-auth-isolation.ts +++ b/test/fixtures/workflow/critical-auth-isolation.ts @@ -24,9 +24,10 @@ export function createAuthIsolationFixture(): { }): AuthResponse; snapshot(): ReadonlyArray>; } { + const itemKey = (owner: Item["owner"], id: string): string => `${owner}\u0000${id}`; const items = new Map([ - ["item-a", { id: "item-a", owner: "account-a", title: "Private A" }], - ["item-b", { id: "item-b", owner: "account-b", title: "Private B" }], + [itemKey("account-a", "item-a"), { id: "item-a", owner: "account-a", title: "Private A" }], + [itemKey("account-b", "item-b"), { id: "item-b", owner: "account-b", title: "Private B" }], ]); return { @@ -35,21 +36,22 @@ export function createAuthIsolationFixture(): { if (principal === undefined) { return { status: 401, body: { error: "unauthorized" } }; } - const item = items.get(input.itemId); + const key = itemKey(principal, input.itemId); + const item = items.get(key); if (input.method === "POST") { - if (item !== undefined) return { status: 409, body: { error: "already_exists" } }; if (typeof input.title !== "string" || input.title.trim().length === 0) { return { status: 400, body: { error: "invalid_title" } }; } + if (item !== undefined) return { status: 409, body: { error: "already_exists" } }; const created: Item = { id: input.itemId, owner: principal, title: input.title.trim(), }; - items.set(created.id, created); + items.set(key, created); return { status: 201, body: { id: created.id, title: created.title } }; } - if (item === undefined || item.owner !== principal) { + if (item === undefined) { // Deliberately conceal whether another account owns this identifier. return { status: 404, body: { error: "not_found" } }; } @@ -57,7 +59,7 @@ export function createAuthIsolationFixture(): { return { status: 200, body: { id: item.id, title: item.title } }; } if (input.method === "DELETE") { - items.delete(item.id); + items.delete(key); return { status: 204, body: {} }; } if (typeof input.title !== "string" || input.title.trim().length === 0) { diff --git a/test/workflow.test.ts b/test/workflow.test.ts index 5ea9c73..4d70997 100644 --- a/test/workflow.test.ts +++ b/test/workflow.test.ts @@ -100,6 +100,50 @@ test("critical auth fixture conceals and preserves another account's item", () = assert.deepEqual(api.snapshot(), before); }); +test("critical auth fixture prevents cross-tenant identifier collision probes", () => { + const api = createAuthIsolationFixture(); + const accountABefore = api + .snapshot() + .find((item) => item.owner === "account-a" && item.id === "item-a"); + + assert.deepEqual( + api.request({ + token: "fixture-token-b", + method: "POST", + itemId: "item-a", + title: "Private B with tenant-local id", + }), + { status: 201, body: { id: "item-a", title: "Private B with tenant-local id" } }, + ); + assert.deepEqual(api.request({ token: "fixture-token-a", method: "GET", itemId: "item-a" }), { + status: 200, + body: { id: "item-a", title: "Private A" }, + }); + assert.deepEqual(api.request({ token: "fixture-token-b", method: "GET", itemId: "item-a" }), { + status: 200, + body: { id: "item-a", title: "Private B with tenant-local id" }, + }); + assert.deepEqual( + api.snapshot().find((item) => item.owner === "account-a" && item.id === "item-a"), + accountABefore, + ); +}); + +test("critical auth fixture validates title before tenant-local collision state", () => { + const api = createAuthIsolationFixture(); + const before = api.snapshot(); + + assert.deepEqual( + api.request({ token: "fixture-token-b", method: "POST", itemId: "item-a", title: "" }), + { status: 400, body: { error: "invalid_title" } }, + ); + assert.deepEqual( + api.request({ token: "fixture-token-b", method: "POST", itemId: "unused", title: "" }), + { status: 400, body: { error: "invalid_title" } }, + ); + assert.deepEqual(api.snapshot(), before); +}); + test("critical auth fixture derives new item ownership from the principal", () => { const api = createAuthIsolationFixture(); assert.equal( From c3eb0b7716bb0b124e6b5b25ccd186e1b5b5246f Mon Sep 17 00:00:00 2001 From: Factory CP-06 final correction Date: Tue, 29 Sep 2026 17:54:27 +0000 Subject: [PATCH 09/45] no-mistakes(review): Enforce scoped CTX provenance --- docs/probes/CP-06.md | 11 +++++--- src/core/context.ts | 27 ++++++++++++++++++ test/context.test.ts | 66 +++++++++++++++++++++++++++++--------------- 3 files changed, 78 insertions(+), 26 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index cbdabaf..75aa6f4 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -8,10 +8,12 @@ unchanged in this branch's history): review intake (also preserved unchanged in history): `cdaf5e5c03d36f1c9b6bb0115793766a384cc178`. -The final correction was performed in the isolated worktree reported by both -`pwd -P` and `git rev-parse --show-toplevel` as -`/home/ansh/.treehouse/Factory-86709f/8/Factory`. It did not touch PR #8, PR #9, -their branches or worker state, a real Firstmate home, routing, shipping +The implementation correction was performed in the isolated Firstmate-selected +Factory worktree. Independent validation and publication review run from the +dedicated no-mistakes validation worktree selected for this replacement branch; +tracked proof identifies exact commands and Git SHAs rather than relying on +durable absolute paths. The correction did not touch PR #8, PR #9, their +branches or worker state, a real Firstmate home, routing, shipping configuration, live accounts, deployed systems, or the shared no-mistakes daemon. @@ -87,6 +89,7 @@ npm test npm run proof:cp06:ten-pack npm run proof:cp06:p07 npm run proof:cp06:worker +git diff --check ``` The exact runtime manifests are: diff --git a/src/core/context.ts b/src/core/context.ts index 8897a02..67fa172 100644 --- a/src/core/context.ts +++ b/src/core/context.ts @@ -143,6 +143,7 @@ export async function generateContextPack(options: { input.task.context.required.length > 0 ? input.task.context.required : input.project.documents.required; + const retrievalIdentities = selectedRetrievalIdentities(input.sources, retrievalDocuments); const packArgs = [ "pack", contextQuery(input.task), @@ -190,6 +191,7 @@ export async function generateContextPack(options: { generation, input.sources, allSourceDigests, + retrievalIdentities, ); const markdown = renderPack({ @@ -406,12 +408,31 @@ function assertSameSnapshot(path: string, captured: string | undefined, actual: } } +function selectedRetrievalIdentities(sources: SourceFile[], selectedPaths: string[]): Set { + const identities = new Set(); + const missing: string[] = []; + for (const path of selectedPaths) { + const source = sources.find((candidate) => candidate.aliases.includes(path)); + if (source === undefined) missing.push(path); + else identities.add(source.identity); + } + if (missing.length > 0) { + blocked("CTX retrieval scope did not match the validated mandatory sources", { + reason: "RETRIEVAL_SCOPE_INVALID", + selected_documents: selectedPaths, + missing_documents: missing, + }); + } + return identities; +} + async function verifiedRanges( root: string, pack: Record, generation: number, mandatory: SourceFile[], digests: Record, + allowedIdentities: Set, ): Promise { if (!Array.isArray(pack.items)) { blocked("CTX pack omitted its items array", { reason: "CTX_PACK_INVALID" }); @@ -474,6 +495,12 @@ async function verifiedRanges( end_offset: endOffset, }); } + if (!allowedIdentities.has(document.identity)) { + blocked("CTX excerpt provenance was outside the selected retrieval scope", { + reason: "RETRIEVAL_SCOPE_WIDENED", + path, + }); + } assertSameSnapshot(path, snapshots.get(document.identity), document.sha256); assertSameSnapshot(path, digests[path], document.sha256); snapshots.set(document.identity, document.sha256); diff --git a/test/context.test.ts b/test/context.test.ts index e1ebaf4..8e3b698 100644 --- a/test/context.test.ts +++ b/test/context.test.ts @@ -67,7 +67,7 @@ evidence: delivery: project-default `; -test("context writes exact mandatory sources, merges overlapping excerpts, and records provenance", async () => { +test("context writes exact mandatory sources and records provenance", async () => { const fixture = await seeded("success with spaces"); try { const result = await factory(fixture, { mode: "semantic" }); @@ -91,10 +91,7 @@ test("context writes exact mandatory sources, merges overlapping excerpts, and r assert.ok(pack.includes(architecture)); assert.match(pack, /Source: `PROJECT\.md`/); assert.match(pack, /CTX generation: `7`/); - assert.equal( - count(pack, "The overlapping excerpt sentinel appears exactly once in the source."), - 1, - ); + assert.doesNotMatch(pack, /The overlapping excerpt sentinel/); await assert.rejects(stat(join(fixture.root, "CONTEXT-PWNED"))); const receipt = parseObject(await readFile(join(fixture.root, payload.receipt_path), "utf8")); @@ -107,7 +104,7 @@ test("context writes exact mandatory sources, merges overlapping excerpts, and r receipt.source_digests["ARCHITECTURE.md"], payload.source_digests["ARCHITECTURE.md"], ); - assert.match(receipt.source_digests["docs/EXTRA.md"], /^[a-f0-9]{64}$/); + assert.equal(receipt.source_digests["docs/EXTRA.md"], undefined); assert.equal(receipt.ctx.index_generation, 7); assert.equal(receipt.ctx.retrieval_mode, "HYBRID_SEMANTIC"); } finally { @@ -142,6 +139,33 @@ test("context limits CTX retrieval to the task-local source selection", async () } }); +test("context blocks CTX provenance outside the task-local source selection", async () => { + const fixture = await seeded("source-filter-adversarial"); + try { + await writeFile( + join(fixture.root, "docs/TARGET.md"), + "# Target authority\n\nOnly this task-local source is relevant.\n", + ); + await writeFile( + join(fixture.root, ".factory/tasks/CTX-001.yaml"), + taskContract.replace( + "required: [PROJECT.md, ARCHITECTURE.md]", + "required: [PROJECT.md, ARCHITECTURE.md, docs/TARGET.md]", + ), + ); + + const result = await factory(fixture, { mode: "source-filter-adversarial" }); + assert.equal(result.exit, 3, result.stderr || result.stdout); + const payload = parseObject(result.stdout); + assert.equal(payload.error.code, "CONTEXT_BLOCKED"); + assert.equal(payload.error.details.reason, "RETRIEVAL_SCOPE_WIDENED"); + assert.equal(payload.error.details.path, "docs/EXTRA.md"); + await assertNoPack(fixture.root); + } finally { + await fixture.dispose(); + } +}); + test("empty task-local selection constrains CTX to mandatory global authority", async () => { const fixture = await seeded("empty-source-selection"); try { @@ -427,19 +451,15 @@ test("context deduplicates mandatory and retrieved symlink identities", async () const payload = parseObject(result.stdout); const pack = await readFile(join(fixture.root, payload.pack_path), "utf8"); assert.equal(count(pack, await readFile(join(fixture.root, "PROJECT.md"), "utf8")), 1); + assert.doesNotMatch(pack, /The overlapping excerpt sentinel/); + assert.ok(pack.includes("docs/PROJECT-ALIAS.md")); assert.equal( - count(pack, "The overlapping excerpt sentinel appears exactly once in the source."), - 1, + payload.source_digests["docs/PROJECT-ALIAS.md"], + createHash("sha256") + .update(await readFile(join(fixture.root, "docs/PROJECT-ALIAS.md"))) + .digest("hex"), ); - for (const path of ["docs/PROJECT-ALIAS.md", "docs/EXTRA-ALIAS.md"]) { - assert.ok(pack.includes(path)); - assert.equal( - payload.source_digests[path], - createHash("sha256") - .update(await readFile(join(fixture.root, path))) - .digest("hex"), - ); - } + assert.equal(payload.source_digests["docs/EXTRA-ALIAS.md"], undefined); } finally { await fixture.dispose(); } @@ -620,15 +640,17 @@ if (command === "doctor") { if (command === "pack") { const selectedDocuments = args.flatMap((arg, index) => arg === "--document" ? [args[index + 1]] : []); const sourceFiltered = mode === "source-filter" && selectedDocuments.includes("docs/TARGET.md"); + const adversarialScope = mode === "source-filter-adversarial"; const globalsOnly = mode === "empty-source-selection" && selectedDocuments.length === 2 && selectedDocuments.includes("PROJECT.md") && selectedDocuments.includes("ARCHITECTURE.md"); - const relative = sourceFiltered ? "docs/TARGET.md" : globalsOnly ? "PROJECT.md" : "docs/EXTRA.md"; + const relative = adversarialScope ? "docs/EXTRA.md" : sourceFiltered ? "docs/TARGET.md" : globalsOnly ? "PROJECT.md" : selectedDocuments[0] || "docs/EXTRA.md"; const full = fs.readFileSync(path.join(root, relative), "utf8"); - const firstEnd = sourceFiltered ? full.length : full.indexOf("Gamma context ends here."); - const secondStart = sourceFiltered ? 0 : full.indexOf("The overlapping excerpt sentinel"); + const sentinelEnd = full.indexOf("Gamma context ends here."); + const firstEnd = sourceFiltered || globalsOnly || sentinelEnd < 0 ? full.length : sentinelEnd; + const sentinelStart = full.indexOf("The overlapping excerpt sentinel"); + const secondStart = sourceFiltered || globalsOnly || sentinelStart < 0 ? 0 : sentinelStart; const item = (start, end, text = full.slice(start, end)) => ({source:{source_type:"excerpt",text,provenance:{document_path:relative,document_sha256:hash(full),start_line:1,end_line:6,start_offset:start,end_offset:end,range_sha256:hash(text),index_generation:7}},estimated_tokens:Math.ceil(Buffer.byteLength(text)/3)}); - const items = sourceFiltered || globalsOnly ? [item(0, full.length)] : [item(0, firstEnd), item(secondStart, full.length), item(0, firstEnd)]; + const items = sourceFiltered || globalsOnly || sentinelEnd < 0 ? [item(0, full.length)] : [item(0, firstEnd), item(secondStart, full.length), item(0, firstEnd)]; if (mode === "aliases") { - items[1].source.provenance.document_path = "docs/EXTRA-ALIAS.md"; const original = fs.readFileSync(path.join(root, "docs/PROJECT-ALIAS.md"), "utf8"); items.push({source:{text:original,provenance:{document_path:"docs/PROJECT-ALIAS.md",document_sha256:hash(original),start_offset:0,end_offset:[...original].length,index_generation:7}}}); } From 6d02e466ab220530dc7d1bd35c211e9e1770a570 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 29 Sep 2026 18:50:40 +0000 Subject: [PATCH 10/45] fix(cp06): enforce isolated worker credential reads --- docs/probes/CP-06.md | 13 ++++-- scripts/cp06-worker-audit.mjs | 8 ++++ scripts/replay-cp06-worker.mjs | 73 ++++++++++++++++++++++++++++++--- test/cp06-worker-audit.test.mjs | 30 +++++++++++++- 4 files changed, 113 insertions(+), 11 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 75aa6f4..c2a7cb4 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -185,10 +185,15 @@ ID and event order, requires `isError: false`, forbids offset/limit and all othe reads/tools, compares the returned UTF-8 bytes and SHA-256 to the tracked exact original, and requires the final response after the completion. Deterministic protocol tests reject omitted, failed, partial, mismatched, out-of-order, and -extra-project cases. The genuine Pi process uses a disposable HOME and agent -directory, links rather than copies the caller-authorized credential file, -verifies that file remains byte-identical, deletes the evaluation home, and -retains no raw transcript. Model wording is nondeterministic; the protocol and +extra-project cases. The genuine Pi process runs in a disposable Linux +user/mount namespace. The caller-authorized credential file is bind-remounted +read-only (never copied), and both direct writes and writes through the +evaluation symlink are attempted and denied before Pi starts. The runner fails +safely when `unshare` or read-only mount operations are unavailable; it does not +silently fall back to an unsandboxed credential. It uses a disposable HOME and +agent directory, verifies that file remains byte-identical, deletes the +evaluation home, and retains no raw transcript. Native Windows/macOS mount +isolation is unverified. Model wording is nondeterministic; the protocol and source-byte oracle are not. ## P-07 finding, repair, and retest diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs index 1890cea..8f96665 100644 --- a/scripts/cp06-worker-audit.mjs +++ b/scripts/cp06-worker-audit.mjs @@ -1,6 +1,14 @@ import { createHash } from "node:crypto"; import { resolve } from "node:path"; +export function isExpectedOriginalReference(value, options) { + if (typeof value !== "string" || value.trim().length === 0) return false; + const reference = value.trim(); + const expectedPath = resolve(options.root, options.expectedOriginalPath); + const referenceRoot = options.referenceRoot ?? options.root; + return resolve(referenceRoot, reference) === expectedPath; +} + export function auditReadEvents(events, options) { const expectedPath = resolve(options.root, options.expectedOriginalPath); const expectedBytes = Buffer.isBuffer(options.expectedOriginal) diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index c60eabb..3551971 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -5,9 +5,35 @@ import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promis import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; -import { auditReadEvents } from "./cp06-worker-audit.mjs"; +import { auditReadEvents, isExpectedOriginalReference } from "./cp06-worker-audit.mjs"; -const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const scriptPath = fileURLToPath(import.meta.url); +const root = resolve(dirname(scriptPath), ".."); +if (process.env.CP06_MOUNT_ISOLATION_CHILD !== "1") { + const isolated = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + scriptPath, + ], + { + cwd: root, + encoding: "utf8", + env: { ...process.env, CP06_MOUNT_ISOLATION_CHILD: "1" }, + timeout: 600_000, + }, + ); + process.stdout.write(isolated.stdout ?? ""); + process.stderr.write(isolated.stderr ?? ""); + if (isolated.error !== undefined) throw isolated.error; + process.exit(isolated.status ?? 1); +} const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); const workerRoot = join(outputRoot, "worker"); const fixtureRoot = join(outputRoot, "ten-pack", "repo"); @@ -21,7 +47,7 @@ const contract = await readFile(contractPath, "utf8"); const pack = await readFile(initialPackPath, "utf8"); const original = await readFile(originalPath); const decisiveConstraint = "request-supplied owner fields are ignored"; -const prompt = `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; +const prompt = `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE, set missing_source to the exact canonical relative path docs/AUTH.md, and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; const provider = process.env.CP06_PI_PROVIDER ?? "openai-codex"; const model = process.env.CP06_PI_MODEL ?? "gpt-5.6-sol"; @@ -52,7 +78,37 @@ const credentialSource = resolve( join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "auth.json"), ); const credentialHashBefore = await fileHash(credentialSource); -await symlink(credentialSource, join(evaluationAgentDir, "auth.json")); +const mount = spawnSync("mount", ["--bind", credentialSource, credentialSource], { + cwd: root, + encoding: "utf8", +}); +if (mount.status !== 0) { + throw new Error(`read-only credential isolation unavailable: ${mount.stderr ?? "mount failed"}`); +} +const remount = spawnSync("mount", ["-o", "remount,bind,ro", credentialSource], { + cwd: root, + encoding: "utf8", +}); +if (remount.status !== 0) { + throw new Error( + `read-only credential isolation unavailable: ${remount.stderr ?? "remount failed"}`, + ); +} +const evaluationCredential = join(evaluationAgentDir, "auth.json"); +await symlink(credentialSource, evaluationCredential); +for (const target of [credentialSource, evaluationCredential]) { + try { + await writeFile(target, "unauthorized mutation"); + throw new Error(`read-only credential isolation permitted a write to ${target}`); + } catch (error) { + if ( + error instanceof Error && + error.message.startsWith("read-only credential isolation permitted") + ) { + throw error; + } + } +} let result; let credentialHashAfter; try { @@ -94,7 +150,11 @@ const constraintText = Array.isArray(response.corrected_constraints) : ""; if ( response.status !== "MISSING_SOURCE" || - response.missing_source !== "docs/AUTH.md" || + !isExpectedOriginalReference(response.missing_source, { + root, + referenceRoot: fixtureRoot, + expectedOriginalPath: originalPath, + }) || response.broad_scan !== false || !requiredConstraints.every((constraint) => constraint.test(constraintText)) ) { @@ -133,7 +193,8 @@ const manifest = { isolation: { disposable_home: true, disposable_pi_agent_dir: true, - credential_mode: "symlink to caller-authorized source; integrity checked; removed after run", + credential_mode: + "Linux user/mount namespace; source bind-remounted read-only; direct and symlink writes denied; no copy", credential_source_unchanged: credentialHashAfter === credentialHashBefore, raw_transcript_retained: false, }, diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs index 5fdbf17..6e484da 100644 --- a/test/cp06-worker-audit.test.mjs +++ b/test/cp06-worker-audit.test.mjs @@ -2,7 +2,7 @@ import { strict as assert } from "node:assert"; import { readFile } from "node:fs/promises"; import { resolve } from "node:path"; import test from "node:test"; -import { auditReadEvents } from "../scripts/cp06-worker-audit.mjs"; +import { auditReadEvents, isExpectedOriginalReference } from "../scripts/cp06-worker-audit.mjs"; const root = process.cwd(); const originalPath = resolve(root, "test/fixtures/cp06-context/docs/AUTH.md"); @@ -106,6 +106,34 @@ test("worker read audit rejects another project read", () => { assert.equal(auditReadEvents(events, options).ok, false); }); +test("worker missing-source identity accepts only the canonical original path", () => { + const fixtureRoot = resolve(root, "test/fixtures/cp06-context"); + assert.equal( + isExpectedOriginalReference("docs/AUTH.md", { + root, + referenceRoot: fixtureRoot, + expectedOriginalPath: originalPath, + }), + true, + ); + assert.equal( + isExpectedOriginalReference("docs/OTHER.md", { + root, + referenceRoot: fixtureRoot, + expectedOriginalPath: originalPath, + }), + false, + ); + assert.equal( + isExpectedOriginalReference("unrelated but nonempty", { + root, + referenceRoot: fixtureRoot, + expectedOriginalPath: originalPath, + }), + false, + ); +}); + test("worker read audit accepts one completed exact-original read before the response", () => { const audit = auditReadEvents([start, successfulEnd(), finalMessage], options); assert.equal(audit.ok, true); From 385b80eb6dd3432fcaf063842ade70aa59c90c8d Mon Sep 17 00:00:00 2001 From: Factory CP-06 final correction Date: Tue, 29 Sep 2026 19:07:06 +0000 Subject: [PATCH 11/45] no-mistakes(review): Harden CP-06 credential isolation --- docs/probes/CP-06.md | 20 ++-- scripts/cp06-credential-isolation.mjs | 144 ++++++++++++++++++++++++ scripts/replay-cp06-worker.mjs | 55 ++++----- test/cp06-credential-isolation.test.mjs | 60 ++++++++++ 4 files changed, 237 insertions(+), 42 deletions(-) create mode 100644 scripts/cp06-credential-isolation.mjs create mode 100644 test/cp06-credential-isolation.test.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index c2a7cb4..fedd8dd 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -186,15 +186,17 @@ reads/tools, compares the returned UTF-8 bytes and SHA-256 to the tracked exact original, and requires the final response after the completion. Deterministic protocol tests reject omitted, failed, partial, mismatched, out-of-order, and extra-project cases. The genuine Pi process runs in a disposable Linux -user/mount namespace. The caller-authorized credential file is bind-remounted -read-only (never copied), and both direct writes and writes through the -evaluation symlink are attempted and denied before Pi starts. The runner fails -safely when `unshare` or read-only mount operations are unavailable; it does not -silently fall back to an unsandboxed credential. It uses a disposable HOME and -agent directory, verifies that file remains byte-identical, deletes the -evaluation home, and retains no raw transcript. Native Windows/macOS mount -isolation is unverified. Model wording is nondeterministic; the protocol and -source-byte oracle are not. +user/mount namespace. Before any live credential is exposed, an equivalent dummy +credential namespace proves direct writes and atomic replacement are denied for +both source and target. The caller-authorized credential file is then +bind-remounted read-only onto itself and onto a regular isolated `auth.json` +mountpoint (never copied, never symlinked), with non-writing read-only mount +checks before Pi starts. The runner fails safely when `unshare` or read-only +mount operations are unavailable; it does not silently fall back to an +unsandboxed credential. It uses a disposable HOME and agent directory, verifies +that file remains byte-identical, deletes the evaluation home, and retains no +raw transcript. Native Windows/macOS mount isolation is unverified. Model +wording is nondeterministic; the protocol and source-byte oracle are not. ## P-07 finding, repair, and retest diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs new file mode 100644 index 0000000..74bb883 --- /dev/null +++ b/scripts/cp06-credential-isolation.mjs @@ -0,0 +1,144 @@ +import { spawnSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import { constants } from "node:fs"; +import { access, lstat, mkdir, mkdtemp, readFile, rm, rename, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join, resolve } from "node:path"; + +export async function proveDummyCredentialIsolation(options = {}) { + const root = resolve(options.root ?? process.cwd()); + const proofRoot = await mkdtemp( + join(resolve(options.tmpRoot ?? tmpdir()), "factory-cp06-auth-proof-"), + ); + const source = join(proofRoot, "dummy-source-auth.json"); + const target = join(proofRoot, "dummy-target-auth.json"); + await writeFile(source, JSON.stringify({ fixture: "dummy", token: "not-real" })); + await writeFile(target, ""); + const hashBefore = await fileHash(source); + try { + bindReadOnlyOntoSelf(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + bindReadOnlyFile(source, target, { root, spawnSyncImpl: options.spawnSyncImpl }); + await assertDestructiveCredentialWritesDenied([source, target]); + assertReadOnlyMount(source, { spawnSyncImpl: options.spawnSyncImpl }); + assertReadOnlyMount(target, { spawnSyncImpl: options.spawnSyncImpl }); + const hashAfter = await fileHash(source); + if (hashAfter !== hashBefore) { + throw new Error("dummy credential source changed during isolation proof"); + } + return { source, target, source_unchanged: true, atomic_replacement_denied: true }; + } finally { + unmountQuietly(target, { root, spawnSyncImpl: options.spawnSyncImpl }); + unmountQuietly(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + await rm(proofRoot, { recursive: true, force: true }); + } +} + +export async function mountLiveCredentialReadOnly(options) { + const root = resolve(options.root ?? process.cwd()); + const source = resolve(options.source); + const target = resolve(options.target); + const sourceStat = await lstat(source); + if (!sourceStat.isFile()) throw new Error("Pi credential source is not a regular file"); + await mkdir(dirname(target), { recursive: true }); + await writeFile(target, "", { flag: "w", mode: 0o600 }); + await access(target, constants.R_OK | constants.W_OK); + try { + bindReadOnlyOntoSelf(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + bindReadOnlyFile(source, target, { root, spawnSyncImpl: options.spawnSyncImpl }); + assertReadOnlyMount(source, { spawnSyncImpl: options.spawnSyncImpl }); + assertReadOnlyMount(target, { spawnSyncImpl: options.spawnSyncImpl }); + return { + source, + target, + cleanup() { + unmountQuietly(target, { root, spawnSyncImpl: options.spawnSyncImpl }); + unmountQuietly(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + }, + }; + } catch (error) { + unmountQuietly(target, { root, spawnSyncImpl: options.spawnSyncImpl }); + unmountQuietly(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + throw error; + } +} + +export async function assertDestructiveCredentialWritesDenied(paths, operations = {}) { + const write = operations.writeFile ?? writeFile; + const move = operations.rename ?? rename; + const remove = operations.rm ?? rm; + for (const target of paths) { + await expectDenied(`write to ${target}`, () => write(target, "unauthorized mutation")); + const replacement = join(dirname(target), `.replacement-${process.pid}-${Date.now()}`); + await write(replacement, "unauthorized replacement"); + try { + await expectDenied(`atomic replacement of ${target}`, () => move(replacement, target)); + } finally { + await remove(replacement, { force: true }); + } + } +} + +function bindReadOnlyOntoSelf(path, options) { + checkedSpawn("mount", ["--bind", path, path], options); + checkedSpawn("mount", ["-o", "remount,bind,ro", path], options); +} + +function bindReadOnlyFile(source, target, options) { + checkedSpawn("mount", ["--bind", source, target], options); + checkedSpawn("mount", ["-o", "remount,bind,ro", target], options); +} + +function assertReadOnlyMount(path, options = {}) { + const result = (options.spawnSyncImpl ?? spawnSync)( + "findmnt", + ["--noheadings", "--output", "OPTIONS", "--target", path], + { + encoding: "utf8", + }, + ); + if (result.status !== 0) { + throw new Error( + `read-only credential isolation unavailable: ${result.stderr ?? "findmnt failed"}`, + ); + } + const optionSets = result.stdout + .trim() + .split(/\r?\n/) + .map((line) => line.split(",")); + if (!optionSets.some((options) => options.includes("ro"))) { + throw new Error(`credential mount is not read-only: ${path}`); + } +} + +function checkedSpawn(command, args, options = {}) { + const result = (options.spawnSyncImpl ?? spawnSync)(command, args, { + cwd: options.root, + encoding: "utf8", + }); + if (result.error !== undefined) throw result.error; + if (result.status !== 0) { + throw new Error( + `read-only credential isolation unavailable: ${result.stderr ?? `${command} failed`}`, + ); + } + return result; +} + +function unmountQuietly(path, options = {}) { + (options.spawnSyncImpl ?? spawnSync)("umount", [path], { cwd: options.root, encoding: "utf8" }); +} + +async function expectDenied(label, action) { + try { + await action(); + } catch { + return; + } + throw new Error(`read-only credential isolation permitted ${label}`); +} + +async function fileHash(path) { + return createHash("sha256") + .update(await readFile(path)) + .digest("hex"); +} diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 3551971..799d91a 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -1,10 +1,14 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; +import { + mountLiveCredentialReadOnly, + proveDummyCredentialIsolation, +} from "./cp06-credential-isolation.mjs"; import { auditReadEvents, isExpectedOriginalReference } from "./cp06-worker-audit.mjs"; const scriptPath = fileURLToPath(import.meta.url); @@ -78,37 +82,13 @@ const credentialSource = resolve( join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "auth.json"), ); const credentialHashBefore = await fileHash(credentialSource); -const mount = spawnSync("mount", ["--bind", credentialSource, credentialSource], { - cwd: root, - encoding: "utf8", -}); -if (mount.status !== 0) { - throw new Error(`read-only credential isolation unavailable: ${mount.stderr ?? "mount failed"}`); -} -const remount = spawnSync("mount", ["-o", "remount,bind,ro", credentialSource], { - cwd: root, - encoding: "utf8", -}); -if (remount.status !== 0) { - throw new Error( - `read-only credential isolation unavailable: ${remount.stderr ?? "remount failed"}`, - ); -} +const dummyIsolationProof = await proveDummyCredentialIsolation({ root }); const evaluationCredential = join(evaluationAgentDir, "auth.json"); -await symlink(credentialSource, evaluationCredential); -for (const target of [credentialSource, evaluationCredential]) { - try { - await writeFile(target, "unauthorized mutation"); - throw new Error(`read-only credential isolation permitted a write to ${target}`); - } catch (error) { - if ( - error instanceof Error && - error.message.startsWith("read-only credential isolation permitted") - ) { - throw error; - } - } -} +const liveCredentialMount = await mountLiveCredentialReadOnly({ + root, + source: credentialSource, + target: evaluationCredential, +}); let result; let credentialHashAfter; try { @@ -120,6 +100,7 @@ try { }); credentialHashAfter = await fileHash(credentialSource); } finally { + liveCredentialMount.cleanup(); await rm(evaluationHome, { recursive: true, force: true }); } if (credentialHashAfter !== credentialHashBefore) { @@ -194,7 +175,15 @@ const manifest = { disposable_home: true, disposable_pi_agent_dir: true, credential_mode: - "Linux user/mount namespace; source bind-remounted read-only; direct and symlink writes denied; no copy", + "Linux user/mount namespace; live source and isolated auth regular file bind-mounted read-only; no symlink; no copy", + dummy_destructive_probe: { + source_unchanged: dummyIsolationProof.source_unchanged, + atomic_replacement_denied: dummyIsolationProof.atomic_replacement_denied, + }, + live_mount_checks: { + non_writing_read_only_mount_checks: true, + destructive_real_source_probe: false, + }, credential_source_unchanged: credentialHashAfter === credentialHashBefore, raw_transcript_retained: false, }, @@ -211,7 +200,7 @@ const manifest = { response, result: "pass", limitation: - "Live model wording is nondeterministic; the script deterministically checks actual read tool-call events and required corrected constraints.", + "Live model wording is nondeterministic; credential mount isolation requires Linux unshare/mount support and fails safe before Pi is invoked when unsupported; the script deterministically checks actual read tool-call events and required corrected constraints.", }; const manifestPath = join(workerRoot, "evidence.json"); await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); diff --git a/test/cp06-credential-isolation.test.mjs b/test/cp06-credential-isolation.test.mjs new file mode 100644 index 0000000..d106c93 --- /dev/null +++ b/test/cp06-credential-isolation.test.mjs @@ -0,0 +1,60 @@ +import { strict as assert } from "node:assert"; +import { lstat, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; +import { + assertDestructiveCredentialWritesDenied, + mountLiveCredentialReadOnly, +} from "../scripts/cp06-credential-isolation.mjs"; + +test("live credential mount uses a regular target without source writes", async () => { + const root = await mkdtemp(join(tmpdir(), "factory-cp06-live-mount-test-")); + const source = join(root, "auth-source.json"); + const target = join(root, "agent", "auth.json"); + const sourceBytes = '{"live":true}\n'; + await writeFile(source, sourceBytes); + const calls = []; + const spawnSyncImpl = (command, args) => { + calls.push([command, ...args]); + if (command === "findmnt") return { status: 0, stdout: "ro,nosuid,nodev\n", stderr: "" }; + return { status: 0, stdout: "", stderr: "" }; + }; + try { + const mount = await mountLiveCredentialReadOnly({ root, source, target, spawnSyncImpl }); + try { + assert.equal(await readFile(source, "utf8"), sourceBytes); + assert.equal((await lstat(target)).isFile(), true); + assert.equal((await lstat(target)).isSymbolicLink(), false); + assert.deepEqual(calls.slice(0, 4), [ + ["mount", "--bind", source, source], + ["mount", "-o", "remount,bind,ro", source], + ["mount", "--bind", source, target], + ["mount", "-o", "remount,bind,ro", target], + ]); + } finally { + mount.cleanup(); + } + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("destructive dummy proof requires denied atomic replacement", async () => { + const calls = []; + const protectedPaths = new Set(["/dummy/source-auth.json", "/dummy/target-auth.json"]); + await assertDestructiveCredentialWritesDenied([...protectedPaths], { + async writeFile(path) { + calls.push(["write", path]); + if (protectedPaths.has(path)) throw new Error("EROFS"); + }, + async rename(from, to) { + calls.push(["rename", from, to]); + if (protectedPaths.has(to)) throw new Error("EROFS"); + }, + async rm(path) { + calls.push(["rm", path]); + }, + }); + assert.equal(calls.filter(([operation]) => operation === "rename").length, 2); +}); From 752152e8c2b2cda627ad3f87583786844016c5ad Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Wed, 30 Sep 2026 17:36:07 +0000 Subject: [PATCH 12/45] fix(cp06): harden read-only Pi authentication --- docs/probes/CP-06.md | 54 ++-- package.json | 1 + scripts/cp06-auth-security.mjs | 411 ++++++++++++++++++++++++ scripts/cp06-credential-isolation.mjs | 248 ++++++++++---- scripts/cp06-dummy-sdk-proof.mjs | 226 +++++++++++++ scripts/cp06-isolation-adversary.mjs | 123 +++++++ scripts/cp06-isolation-syscalls.c | 46 +++ scripts/cp06-namespace-supervisor.mjs | 166 ++++++++++ scripts/cp06-readonly-credentials.mjs | 143 +++++++++ scripts/cp06-sdk-worker.mjs | 287 +++++++++++++++++ scripts/cp06-seccomp-exec.c | 123 +++++++ scripts/cp06-worker-audit.mjs | 11 +- scripts/prove-cp06-auth-security.mjs | 32 ++ scripts/replay-cp06-worker.mjs | 324 ++++++------------- test/cp06-credential-isolation.test.mjs | 115 ++++++- test/cp06-readonly-credentials.test.mjs | 126 ++++++++ test/cp06-worker-audit.test.mjs | 15 + 17 files changed, 2139 insertions(+), 312 deletions(-) create mode 100644 scripts/cp06-auth-security.mjs create mode 100644 scripts/cp06-dummy-sdk-proof.mjs create mode 100644 scripts/cp06-isolation-adversary.mjs create mode 100644 scripts/cp06-isolation-syscalls.c create mode 100644 scripts/cp06-namespace-supervisor.mjs create mode 100644 scripts/cp06-readonly-credentials.mjs create mode 100644 scripts/cp06-sdk-worker.mjs create mode 100644 scripts/cp06-seccomp-exec.c create mode 100644 scripts/prove-cp06-auth-security.mjs create mode 100644 test/cp06-readonly-credentials.test.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index fedd8dd..7a5ebf8 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -17,6 +17,14 @@ branches or worker state, a real Firstmate home, routing, shipping configuration, live accounts, deployed systems, or the shared no-mistakes daemon. +## Read-only Pi authentication correction + +The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Unsupported Linux x86-64 user/mount namespaces, libseccomp, helper compilation, Pi version, or a cleanup failure block; there is no unsandboxed fallback. Windows and macOS are not claimed. + +The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and exactly the built-in `read` tool. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact hashes/counts and parsed output, never credential contents or raw events. + +Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is attempted only after that DUMMY gate and fail-closed preflight pass. No destructive operation is ever directed at the real credential source or mounted target. + ## Normative result The PR #8 measurement is a **failure**, not a P-06 pass. Its predeclared oracle @@ -166,11 +174,13 @@ not fabricate completeness. ## Fresh-context targeted-original correction -After deterministic checks and the real-CTX replay, a new Pi `0.85.1` process -runs with `--no-session`, `--no-context-files`, no discovered extensions or -skills, the Factory skill explicitly selected, and only the `read` tool. Its -initial handoff contains exactly the AUTH-before contract, bounded missing-AUTH -pack, and a short deterministic evidence statement—no prior transcript. +After deterministic checks and the real-CTX replay, a new Pi `0.85.1` SDK +session runs with an injected read-only credential store and model runtime, an +in-memory session/settings manager, an explicit resource loader containing no +extensions, skills, prompts, context files, or themes, and exactly the built-in +`read` tool. Its initial handoff contains exactly the AUTH-before contract, +bounded missing-AUTH pack, and a short deterministic evidence statement—no +prior transcript. The fresh-worker handoff deliberately omits the concrete rule that request-supplied owner fields are ignored at creation time while retaining the @@ -182,21 +192,25 @@ missing creation-time integration evidence. The audit correlates `tool_execution_start` and `tool_execution_end` by tool-call ID and event order, requires `isError: false`, forbids offset/limit and all other -reads/tools, compares the returned UTF-8 bytes and SHA-256 to the tracked exact -original, and requires the final response after the completion. Deterministic -protocol tests reject omitted, failed, partial, mismatched, out-of-order, and -extra-project cases. The genuine Pi process runs in a disposable Linux -user/mount namespace. Before any live credential is exposed, an equivalent dummy -credential namespace proves direct writes and atomic replacement are denied for -both source and target. The caller-authorized credential file is then -bind-remounted read-only onto itself and onto a regular isolated `auth.json` -mountpoint (never copied, never symlinked), with non-writing read-only mount -checks before Pi starts. The runner fails safely when `unshare` or read-only -mount operations are unavailable; it does not silently fall back to an -unsandboxed credential. It uses a disposable HOME and agent directory, verifies -that file remains byte-identical, deletes the evaluation home, and retains no -raw transcript. Native Windows/macOS mount isolation is unverified. Model -wording is nondeterministic; the protocol and source-byte oracle are not. +start or completion events, compares the returned UTF-8 bytes and SHA-256 to the +tracked exact original, and requires the final response after the completion. +Deterministic protocol tests reject omitted, failed, partial, mismatched, +out-of-order, malformed-orphan, and extra-project cases. + +The genuine Pi SDK process runs in a private Linux user/mount namespace only +after the complete DUMMY auth and OS preflight passes. A supervisor pins the +regular source inode with `O_NOFOLLOW`, creates distinct source/target bind +mounts with read-only VFS flags, and closes the pin before launch. The evaluated +child has all capability sets empty, locked securebits, `no_new_privs`, and an +inherited seccomp filter blocking mount and namespace escapes, alternate inode +handles, ptrace/process-memory access, and namespace-bearing child calls. Its +credential store rejects refresh/mutation callbacks before entry; expired or +near-expiry OAuth is therefore blocked before a provider request or persistence. +Setup, helper/version, auth-validity, timeout, audit, and cleanup failures are +reported as blocked/nonzero with no unsandboxed fallback. The evaluation state +is removed and no raw transcript is retained. Native Windows/macOS isolation is +unsupported rather than claimed. Model wording is nondeterministic; the +protocol, exact bytes, and isolation assertions are not. ## P-07 finding, repair, and retest diff --git a/package.json b/package.json index 457eff4..74c9862 100644 --- a/package.json +++ b/package.json @@ -15,6 +15,7 @@ "lint": "node scripts/lint.mjs", "proof:cp06:ten-pack": "npm run build && node scripts/replay-cp06-ten-pack.mjs", "proof:cp06:p07": "node scripts/replay-cp06-p07.mjs", + "proof:cp06:auth-security": "node scripts/prove-cp06-auth-security.mjs", "proof:cp06:worker": "node scripts/replay-cp06-worker.mjs", "factory": "node dist/src/cli.js" }, diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs new file mode 100644 index 0000000..edb8805 --- /dev/null +++ b/scripts/cp06-auth-security.mjs @@ -0,0 +1,411 @@ +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { accessSync, constants, readFileSync, realpathSync } from "node:fs"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { delimiter, dirname, join, resolve } from "node:path"; + +export class Cp06IsolationUnsupportedError extends Error { + constructor(message) { + super(message); + this.name = "Cp06IsolationUnsupportedError"; + this.code = "CP06_ISOLATION_UNSUPPORTED"; + } +} + +export async function runCp06SecurityPreflight(options = {}) { + const root = resolve(options.root ?? process.cwd()); + const outputRoot = resolve(options.outputRoot ?? join(root, ".factory/state/cp06-auth-security")); + await mkdir(outputRoot, { recursive: true }); + const prerequisites = verifyIsolationPrerequisites({ + root, + spawnSyncImpl: options.spawnSyncImpl, + }); + const binaries = compileIsolationHelpers({ root, outputRoot }); + const pi = resolvePiPackage(); + const unsafe = await runIsolationProbe({ + root, + outputRoot, + mode: "unsafe-probe", + binaries, + }); + assertUnsafeControl(unsafe); + const hardened = await runIsolationProbe({ + root, + outputRoot, + mode: "hardened-probe", + binaries, + }); + assertHardenedProof(hardened); + const sdk = runDummySdkProof({ root, helper: binaries.helper, piRoot: pi.root }); + return { + schema_version: 1, + gate: "CP-06-read-only-auth-security", + result: "pass", + tested_sha: commandText(spawnSync, "git", ["rev-parse", "HEAD"], root), + recorded_at: new Date().toISOString(), + reviewer: options.reviewer ?? "Pi CP-06 read-only auth correction worker", + command: "npm run proof:cp06:auth-security", + exit_code: 0, + pi_version: pi.version, + platform: prerequisites, + binaries: { + seccomp_helper_sha256: sha256(readFileSync(binaries.helper)), + syscall_probe_sha256: sha256(readFileSync(binaries.syscallProbe)), + }, + unsafe_control: summarizeUnsafe(unsafe), + hardened_child: summarizeHardened(hardened), + dummy_sdk: sdk, + }; +} + +export function verifyIsolationPrerequisites(options = {}) { + const root = resolve(options.root ?? process.cwd()); + const spawn = options.spawnSyncImpl ?? spawnSync; + if (process.platform !== "linux" || process.arch !== "x64") { + throw unsupported("CP-06 proof runner supports Linux x86-64 only"); + } + const probes = [ + ["unshare", ["--user", "--map-root-user", "--mount", "--propagation", "private", "true"]], + ["mount", ["--version"]], + ["umount", ["--version"]], + ["findmnt", ["--version"]], + ["setpriv", ["--version"]], + ["nsenter", ["--version"]], + ["cc", ["--version"]], + ]; + for (const [command, args] of probes) { + const result = spawn(command, args, { cwd: root, encoding: "utf8", timeout: 30_000 }); + if (result.error !== undefined || result.status !== 0) { + throw unsupported(`required isolation probe failed: ${command}`); + } + } + return { + os: process.platform, + architecture: process.arch, + kernel: commandText(spawn, "uname", ["-r"], root), + user_mount_namespace: true, + libseccomp: true, + }; +} + +export function compileIsolationHelpers(options) { + const root = resolve(options.root); + const outputRoot = resolve(options.outputRoot); + const helper = join(outputRoot, "cp06-seccomp-exec"); + const syscallProbe = join(outputRoot, "cp06-isolation-syscalls"); + checked( + "cc", + [ + "-std=c11", + "-O2", + "-Wall", + "-Wextra", + "-Werror", + join(root, "scripts/cp06-seccomp-exec.c"), + "-Wl,-l:libseccomp.so.2", + "-o", + helper, + ], + root, + ); + checked( + "cc", + [ + "-std=c11", + "-O2", + "-Wall", + "-Wextra", + "-Werror", + join(root, "scripts/cp06-isolation-syscalls.c"), + "-o", + syscallProbe, + ], + root, + ); + return { helper, syscallProbe }; +} + +export async function runIsolatedWorker(options) { + const result = runNamespace({ + root: options.root, + mode: "worker", + source: options.source, + target: options.target, + helper: options.binaries.helper, + syscallProbe: options.binaries.syscallProbe, + childArgs: [options.inputPath], + timeout: options.timeout ?? 660_000, + environment: options.environment, + }); + if (result.status !== 0) { + const error = new Error( + result.status === 73 + ? "CP-06 credential namespace setup is unsupported" + : `CP-06 isolated SDK worker exited ${result.status ?? "without status"}`, + ); + error.code = result.status === 73 ? "CP06_ISOLATION_UNSUPPORTED" : "CP06_WORKER_FAILED"; + error.exitCode = result.status ?? 70; + error.stderr = result.stderr; + throw error; + } + return parseCompactJson(result.stdout, "isolated SDK worker"); +} + +async function runIsolationProbe({ root, outputRoot, mode, binaries }) { + const proofRoot = await mkdtemp(join(outputRoot, `DUMMY-${mode}-`)); + const source = join(proofRoot, "source", "DUMMY-auth.json"); + const target = join(proofRoot, "agent", "DUMMY-auth.json"); + await mkdir(dirname(source), { recursive: true }); + await writeFile(source, "DUMMY ORIGINAL", { mode: 0o600 }); + try { + const result = runNamespace({ + root, + mode, + source, + target, + helper: binaries.helper, + syscallProbe: binaries.syscallProbe, + childArgs: [], + timeout: 90_000, + }); + if (result.status !== 0) { + throw unsupported(`${mode} failed: ${String(result.stderr).trim() || result.status}`); + } + return parseCompactJson(result.stdout, mode); + } finally { + await rm(proofRoot, { recursive: true, force: true }); + } +} + +function runNamespace(options) { + const args = [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + join(options.root, "scripts/cp06-namespace-supervisor.mjs"), + options.mode, + resolve(options.source), + resolve(options.target), + resolve(options.helper), + resolve(options.syscallProbe), + ...options.childArgs.map((value) => resolve(value)), + ]; + return spawnSync("unshare", args, { + cwd: options.root, + encoding: "utf8", + timeout: options.timeout, + maxBuffer: 1024 * 1024, + env: options.environment ?? process.env, + }); +} + +function assertUnsafeControl(proof) { + assert(proof.mode === "unsafe-probe", "unsafe control mode mismatch"); + assert(proof.mount_ids_distinct === true, "unsafe control lacked distinct mounts"); + assert(proof.source_unchanged === false, "unsafe control did not mutate DUMMY source"); + assert(proof.child.remount_target_rw.exit === 0, "unsafe remount did not succeed"); + assert(proof.child.write_after_remount.exit === 0, "unsafe post-remount write failed"); + assert(proof.child.unmount_target.exit === 0, "unsafe unmount did not succeed"); + assert(proof.child.bind_parent_alias.exit === 0, "unsafe parent bind did not succeed"); + assert(proof.child.alias_underlying_write.exit === 0, "unsafe alias write did not succeed"); + assert( + proof.child.identity.status.CapEff !== "0000000000000000", + "unsafe control unexpectedly had no capabilities", + ); +} + +function assertHardenedProof(proof) { + assert(proof.mode === "hardened-probe", "hardened proof mode mismatch"); + assert(proof.mount_ids_distinct === true, "hardened proof lacked distinct mounts"); + assert(proof.source_unchanged === true, "hardened child changed DUMMY source"); + const status = proof.child.identity.status; + for (const name of ["CapInh", "CapPrm", "CapEff", "CapBnd", "CapAmb"]) + assert(status[name] === "0000000000000000", `${name} was not empty`); + assert(status.NoNewPrivs === "1", "no_new_privs was not active"); + assert(status.Seccomp === "2", "seccomp filter mode was not active"); + for (const name of [ + "direct_write_target", + "direct_write_source", + "unlink_target", + "rename_over_target", + "symlink_replacement", + "hardlink_target", + "nested_user_mount_namespace", + "nsenter_self", + "nsenter_supervisor", + "uid_change", + "remount_target_rw", + "write_after_remount", + "unmount_target", + "bind_parent_alias", + ]) { + assert(proof.child[name].exit !== 0, `${name} unexpectedly succeeded`); + } + assert(proof.child.alias_underlying_write.skipped === true, "alias write was attempted"); + assert(proof.child.ordinary_node_child.exit === 0, "ordinary child process was blocked"); + assert( + proof.child.ordinary_node_child.stdout === "DUMMY-child-ok", + "ordinary child output wrong", + ); + const syscalls = proof.child.direct_namespace_syscalls; + assert(syscalls.exit === 0, "direct syscall probe failed to execute"); + for (const expected of [ + "setns=-1:1", + "unshare=-1:1", + "clone=-1:1", + "clone3=-1:38", + "mount=-1:1", + "umount2=-1:1", + ]) { + assert(syscalls.stdout.includes(expected), `direct syscall was not filtered: ${expected}`); + } +} + +function runDummySdkProof({ root, helper, piRoot }) { + const result = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + "setpriv", + "--no-new-privs", + "--bounding-set=-all", + "--inh-caps=-all", + "--ambient-caps=-all", + "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", + "--", + helper, + process.execPath, + join(root, "scripts/cp06-dummy-sdk-proof.mjs"), + piRoot, + ], + { + cwd: root, + encoding: "utf8", + env: { + PATH: process.env.PATH ?? "/usr/bin:/bin", + LANG: process.env.LANG ?? "C.UTF-8", + PI_OFFLINE: "1", + PI_SKIP_VERSION_CHECK: "1", + PI_TELEMETRY: "0", + }, + timeout: 120_000, + maxBuffer: 1024 * 1024, + }, + ); + if (result.error !== undefined || result.status !== 0) { + throw unsupported(`DUMMY SDK proof failed: ${result.error?.message ?? result.stderr}`); + } + return parseCompactJson(result.stdout, "DUMMY SDK proof"); +} + +function resolvePiPackage() { + const executable = realpathSync(findExecutable("pi")); + const root = findPiPackageRoot(executable); + const metadata = JSON.parse(readFileSync(join(root, "package.json"), "utf8")); + const version = checked("pi", ["--version"], root).stdout.trim(); + if ( + metadata.name !== "@earendil-works/pi-coding-agent" || + metadata.version !== "0.85.1" || + version !== "0.85.1" + ) { + throw unsupported("exact Pi 0.85.1 is unavailable"); + } + return { root, version }; +} + +function findPiPackageRoot(executable) { + let directory = dirname(executable); + for (let depth = 0; depth < 5; depth += 1) { + try { + const metadata = JSON.parse(readFileSync(join(directory, "package.json"), "utf8")); + if (metadata.name === "@earendil-works/pi-coding-agent") return directory; + } catch {} + const parent = dirname(directory); + if (parent === directory) break; + directory = parent; + } + throw unsupported("Pi package root could not be resolved from its executable"); +} + +function findExecutable(name) { + for (const directory of String(process.env.PATH ?? "").split(delimiter)) { + if (!directory) continue; + const candidate = join(directory, name); + try { + accessSync(candidate, constants.X_OK); + return candidate; + } catch {} + } + throw unsupported(`required executable is unavailable: ${name}`); +} + +function checked(command, args, cwd) { + const result = spawnSync(command, args, { cwd, encoding: "utf8", timeout: 60_000 }); + if (result.error !== undefined || result.status !== 0) { + throw unsupported(`required command failed: ${command}`); + } + return result; +} + +function commandText(spawn, command, args, cwd) { + const result = spawn(command, args, { cwd, encoding: "utf8", timeout: 30_000 }); + if (result.error !== undefined || result.status !== 0) { + throw unsupported(`required isolation probe failed: ${command}`); + } + return result.stdout.trim(); +} + +function parseCompactJson(value, label) { + try { + return JSON.parse(String(value).trim()); + } catch (cause) { + throw unsupported(`${label} did not return compact JSON`, { cause }); + } +} + +function summarizeUnsafe(proof) { + return { + retained_capabilities: proof.child.identity.status.CapEff, + remount_succeeded: proof.child.remount_target_rw.exit === 0, + unmount_succeeded: proof.child.unmount_target.exit === 0, + parent_bind_succeeded: proof.child.bind_parent_alias.exit === 0, + dummy_source_changed: !proof.source_unchanged, + }; +} + +function summarizeHardened(proof) { + return { + capability_sets_zero: true, + no_new_privs: true, + seccomp_filter: true, + ordinary_child_allowed: true, + direct_writes_denied: true, + replacement_denied: true, + remount_unmount_denied: true, + namespace_entry_creation_denied: true, + uid_change_denied: true, + alternate_parent_bind_denied: true, + dummy_source_unchanged: proof.source_unchanged, + }; +} + +function assert(condition, message) { + if (!condition) throw new Error(`CP-06 security assertion failed: ${message}`); +} + +function unsupported(message) { + return new Cp06IsolationUnsupportedError(message); +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs index 74bb883..a142eda 100644 --- a/scripts/cp06-credential-isolation.mjs +++ b/scripts/cp06-credential-isolation.mjs @@ -1,117 +1,232 @@ -import { spawnSync } from "node:child_process"; import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; import { constants } from "node:fs"; -import { access, lstat, mkdir, mkdtemp, readFile, rm, rename, writeFile } from "node:fs/promises"; +import { + link, + lstat, + mkdir, + mkdtemp, + open, + readFile, + rename, + rm, + symlink, + unlink, + writeFile, +} from "node:fs/promises"; import { tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; +const MAX_CREDENTIAL_BYTES = 1024 * 1024; +const REQUIRED_VFS_OPTIONS = ["ro", "nosuid", "nodev", "noexec"]; + export async function proveDummyCredentialIsolation(options = {}) { const root = resolve(options.root ?? process.cwd()); const proofRoot = await mkdtemp( join(resolve(options.tmpRoot ?? tmpdir()), "factory-cp06-auth-proof-"), ); - const source = join(proofRoot, "dummy-source-auth.json"); - const target = join(proofRoot, "dummy-target-auth.json"); - await writeFile(source, JSON.stringify({ fixture: "dummy", token: "not-real" })); - await writeFile(target, ""); + const source = join(proofRoot, "DUMMY-source-auth.json"); + const target = join(proofRoot, "agent", "DUMMY-target-auth.json"); + await writeFile(source, JSON.stringify({ fixture: "DUMMY", token: "DUMMY-NOT-REAL" })); const hashBefore = await fileHash(source); + let mount; try { - bindReadOnlyOntoSelf(source, { root, spawnSyncImpl: options.spawnSyncImpl }); - bindReadOnlyFile(source, target, { root, spawnSyncImpl: options.spawnSyncImpl }); + mount = await mountLiveCredentialReadOnly({ + root, + source, + target, + spawnSyncImpl: options.spawnSyncImpl, + }); await assertDestructiveCredentialWritesDenied([source, target]); - assertReadOnlyMount(source, { spawnSyncImpl: options.spawnSyncImpl }); - assertReadOnlyMount(target, { spawnSyncImpl: options.spawnSyncImpl }); const hashAfter = await fileHash(source); if (hashAfter !== hashBefore) { - throw new Error("dummy credential source changed during isolation proof"); + throw new Error("DUMMY credential source changed during isolation proof"); } - return { source, target, source_unchanged: true, atomic_replacement_denied: true }; + return { + source, + target, + source_unchanged: true, + destructive_operations_denied: true, + mount_ids: mount.mountIds, + }; } finally { - unmountQuietly(target, { root, spawnSyncImpl: options.spawnSyncImpl }); - unmountQuietly(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + mount?.cleanup(); await rm(proofRoot, { recursive: true, force: true }); } } +/** + * Pin one regular credential inode, bind it over both its source path and a fresh + * regular target, and make both mount instances read-only before closing the FD. + * This function must run only in the private setup namespace. + */ export async function mountLiveCredentialReadOnly(options) { const root = resolve(options.root ?? process.cwd()); - const source = resolve(options.source); - const target = resolve(options.target); - const sourceStat = await lstat(source); - if (!sourceStat.isFile()) throw new Error("Pi credential source is not a regular file"); - await mkdir(dirname(target), { recursive: true }); - await writeFile(target, "", { flag: "w", mode: 0o600 }); - await access(target, constants.R_OK | constants.W_OK); + const source = resolveRequiredPath(options.source, "credential source"); + const target = resolveRequiredPath(options.target, "credential target"); + const maxBytes = options.maxBytes ?? MAX_CREDENTIAL_BYTES; + const spawn = options.spawnSyncImpl ?? spawnSync; + const statPath = options.lstatImpl ?? lstat; + let handle; + let sourceMounted = false; + let targetMounted = false; try { - bindReadOnlyOntoSelf(source, { root, spawnSyncImpl: options.spawnSyncImpl }); - bindReadOnlyFile(source, target, { root, spawnSyncImpl: options.spawnSyncImpl }); - assertReadOnlyMount(source, { spawnSyncImpl: options.spawnSyncImpl }); - assertReadOnlyMount(target, { spawnSyncImpl: options.spawnSyncImpl }); + handle = await open(source, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC); + const pinned = await handle.stat({ bigint: true }); + const named = await statPath(source, { bigint: true }); + assertPinnedRegularFile(pinned, named, maxBytes); + + await mkdir(dirname(target), { recursive: true, mode: 0o700 }); + await writeFile(target, "", { flag: "wx", mode: 0o600 }); + const pinnedPath = `/proc/${process.pid}/fd/${handle.fd}`; + + checkedSpawn("mount", ["--bind", pinnedPath, source], { root, spawn }); + sourceMounted = true; + checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", source], { + root, + spawn, + }); + checkedSpawn("mount", ["--bind", pinnedPath, target], { root, spawn }); + targetMounted = true; + checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", target], { + root, + spawn, + }); + + const sourceMount = inspectReadOnlyMount(source, { spawn }); + const targetMount = inspectReadOnlyMount(target, { spawn }); + if (sourceMount.id === targetMount.id) { + throw new Error("credential source and target did not receive distinct mount instances"); + } + for (const path of [source, target]) { + const mounted = await statPath(path, { bigint: true }); + if ( + !mounted.isFile() || + mounted.dev !== pinned.dev || + mounted.ino !== pinned.ino || + mounted.size !== pinned.size + ) { + throw new Error(`credential mount identity changed: ${path}`); + } + } + await handle.close(); + handle = undefined; + + let cleaned = false; return { source, target, + sourceIdentity: { + dev: String(pinned.dev), + ino: String(pinned.ino), + size: Number(pinned.size), + }, + mountIds: { source: sourceMount.id, target: targetMount.id }, cleanup() { - unmountQuietly(target, { root, spawnSyncImpl: options.spawnSyncImpl }); - unmountQuietly(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + if (cleaned) return; + cleaned = true; + const failures = []; + if (targetMounted) unmount(target, { root, spawn, failures }); + if (sourceMounted) unmount(source, { root, spawn, failures }); + if (failures.length > 0) { + throw new Error(`credential mount cleanup failed: ${failures.join("; ")}`); + } }, }; } catch (error) { - unmountQuietly(target, { root, spawnSyncImpl: options.spawnSyncImpl }); - unmountQuietly(source, { root, spawnSyncImpl: options.spawnSyncImpl }); + const cleanupFailures = []; + if (targetMounted) unmount(target, { root, spawn, failures: cleanupFailures }); + if (sourceMounted) unmount(source, { root, spawn, failures: cleanupFailures }); + if (cleanupFailures.length > 0) { + throw new AggregateError( + [error, ...cleanupFailures.map((message) => new Error(message))], + "credential mount setup and cleanup failed", + ); + } throw error; + } finally { + await handle?.close(); } } export async function assertDestructiveCredentialWritesDenied(paths, operations = {}) { const write = operations.writeFile ?? writeFile; const move = operations.rename ?? rename; - const remove = operations.rm ?? rm; + const remove = operations.unlink ?? unlink; + const makeSymlink = operations.symlink ?? symlink; + const makeHardlink = operations.link ?? link; + const cleanup = operations.rm ?? rm; for (const target of paths) { - await expectDenied(`write to ${target}`, () => write(target, "unauthorized mutation")); - const replacement = join(dirname(target), `.replacement-${process.pid}-${Date.now()}`); - await write(replacement, "unauthorized replacement"); + await expectDenied(`write to ${target}`, () => write(target, "DUMMY unauthorized mutation")); + await expectDenied(`unlink of ${target}`, () => remove(target)); + const nonce = `${process.pid}-${Date.now()}-${Math.random().toString(16).slice(2)}`; + const replacement = join(dirname(target), `.DUMMY-replacement-${nonce}`); + const symbolic = join(dirname(target), `.DUMMY-symlink-${nonce}`); + const hard = join(dirname(target), `.DUMMY-hardlink-${nonce}`); + await write(replacement, "DUMMY unauthorized replacement"); try { await expectDenied(`atomic replacement of ${target}`, () => move(replacement, target)); + await expectDenied(`symlink replacement of ${target}`, async () => { + await remove(target); + await makeSymlink("/tmp/DUMMY-does-not-exist", target); + }); + await expectDenied(`hardlink of ${target}`, () => makeHardlink(target, hard)); } finally { - await remove(replacement, { force: true }); + await cleanup(replacement, { force: true }); + await cleanup(symbolic, { force: true }); + await cleanup(hard, { force: true }); } } } -function bindReadOnlyOntoSelf(path, options) { - checkedSpawn("mount", ["--bind", path, path], options); - checkedSpawn("mount", ["-o", "remount,bind,ro", path], options); -} - -function bindReadOnlyFile(source, target, options) { - checkedSpawn("mount", ["--bind", source, target], options); - checkedSpawn("mount", ["-o", "remount,bind,ro", target], options); -} - -function assertReadOnlyMount(path, options = {}) { - const result = (options.spawnSyncImpl ?? spawnSync)( +export function inspectReadOnlyMount(path, options = {}) { + const spawn = options.spawn ?? options.spawnSyncImpl ?? spawnSync; + const result = spawn( "findmnt", - ["--noheadings", "--output", "OPTIONS", "--target", path], - { - encoding: "utf8", - }, + ["--json", "--noheadings", "--output", "ID,TARGET,VFS-OPTIONS", "--target", path], + { encoding: "utf8" }, ); + if (result.error !== undefined) throw result.error; if (result.status !== 0) { throw new Error( `read-only credential isolation unavailable: ${result.stderr ?? "findmnt failed"}`, ); } - const optionSets = result.stdout - .trim() - .split(/\r?\n/) - .map((line) => line.split(",")); - if (!optionSets.some((options) => options.includes("ro"))) { - throw new Error(`credential mount is not read-only: ${path}`); + let entry; + try { + const parsed = JSON.parse(result.stdout); + [entry] = parsed.filesystems ?? []; + } catch (cause) { + throw new Error("read-only credential mount inspection returned invalid JSON", { cause }); + } + const optionsList = String(entry?.["vfs-options"] ?? "").split(","); + if ( + entry === undefined || + resolve(entry.target) !== resolve(path) || + !REQUIRED_VFS_OPTIONS.every((option) => optionsList.includes(option)) + ) { + throw new Error(`credential mount lacks required read-only VFS flags: ${path}`); } + return { id: String(entry.id), target: resolve(entry.target), options: optionsList }; } -function checkedSpawn(command, args, options = {}) { - const result = (options.spawnSyncImpl ?? spawnSync)(command, args, { +function assertPinnedRegularFile(pinned, named, maxBytes) { + if (!pinned.isFile() || !named.isFile()) { + throw new Error("Pi credential source is not a regular file"); + } + if (pinned.dev !== named.dev || pinned.ino !== named.ino) { + throw new Error("Pi credential source changed while it was pinned"); + } + if (pinned.nlink !== 1n) { + throw new Error("Pi credential source must have exactly one hard link"); + } + if (pinned.size <= 0n || pinned.size > BigInt(maxBytes)) { + throw new Error("Pi credential source size is outside the allowed bound"); + } +} + +function checkedSpawn(command, args, options) { + const result = options.spawn(command, args, { cwd: options.root, encoding: "utf8", }); @@ -124,8 +239,21 @@ function checkedSpawn(command, args, options = {}) { return result; } -function unmountQuietly(path, options = {}) { - (options.spawnSyncImpl ?? spawnSync)("umount", [path], { cwd: options.root, encoding: "utf8" }); +function unmount(path, options) { + const result = options.spawn("umount", [path], { + cwd: options.root, + encoding: "utf8", + }); + if (result.error !== undefined || result.status !== 0) { + options.failures.push( + `${path}: ${result.error?.message ?? result.stderr?.trim() ?? `exit ${result.status}`}`, + ); + } +} + +function resolveRequiredPath(value, label) { + if (typeof value !== "string" || value.length === 0) throw new Error(`${label} is missing`); + return resolve(value); } async function expectDenied(label, action) { diff --git a/scripts/cp06-dummy-sdk-proof.mjs b/scripts/cp06-dummy-sdk-proof.mjs new file mode 100644 index 0000000..c5611d4 --- /dev/null +++ b/scripts/cp06-dummy-sdk-proof.mjs @@ -0,0 +1,226 @@ +#!/usr/bin/env node +import { strict as assert } from "node:assert"; +import { createHash } from "node:crypto"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { pathToFileURL } from "node:url"; +import { Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; +import { auditReadEvents } from "./cp06-worker-audit.mjs"; + +const [piRootArg] = process.argv.slice(2); +if (!piRootArg) process.exit(64); +const piRoot = resolve(piRootArg); +const sdk = await import(pathToFileURL(join(piRoot, "dist/index.js"))); +const ai = await import( + pathToFileURL(join(piRoot, "node_modules/@earendil-works/pi-ai/dist/index.js")) +); +const originalFetch = globalThis.fetch; +let networkCalls = 0; +globalThis.fetch = async () => { + networkCalls += 1; + throw new Error("DUMMY proof forbids network"); +}; +const proofRoot = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-sdk-")); +try { + const authCases = []; + for (const scenario of [ + { name: "expired", expires: 0, expected: "blocked" }, + { name: "near_expiry", expires: Date.now() + 60_000, expected: "blocked" }, + { name: "unexpired", expires: Date.now() + 600_000, expected: "resolved" }, + ]) { + authCases.push(await authCase(scenario)); + } + const worker = await fauxWorkerProof(); + assert.equal(networkCalls, 0); + console.log( + JSON.stringify({ + schema_version: 1, + result: "pass", + auth_cases: authCases, + sdk_worker: worker, + network_calls: networkCalls, + }), + ); +} finally { + globalThis.fetch = originalFetch; + await rm(proofRoot, { recursive: true, force: true }); +} + +async function authCase(scenario) { + const providerId = "openai-codex"; + const path = join(proofRoot, `DUMMY-${scenario.name}.json`); + await writeFile( + path, + JSON.stringify({ + [providerId]: { + type: "oauth", + access: "DUMMY-ACCESS", + refresh: "DUMMY-REFRESH", + expires: scenario.expires, + }, + }), + ); + const before = sha256(await readFile(path)); + const store = await Cp06ReadOnlyCredentialStore.load({ path, providerId }); + const runtime = await sdk.ModelRuntime.create({ + credentials: store, + modelsPath: null, + refreshOnCreate: false, + allowModelNetwork: false, + }); + const provider = runtime.getProvider(providerId); + assert.ok(provider?.auth?.oauth); + let refreshCallbacks = 0; + let toAuthCalls = 0; + provider.auth = { + oauth: { + async refresh() { + refreshCallbacks += 1; + return { + type: "oauth", + access: "DUMMY-NEW", + refresh: "DUMMY-NEW", + expires: Date.now() + 3_600_000, + }; + }, + async toAuth(credential) { + toAuthCalls += 1; + assert.equal(credential.access, "DUMMY-ACCESS"); + return { apiKey: "DUMMY-RESOLVED" }; + }, + }, + }; + let result = "resolved"; + try { + const resolved = await runtime.getAuth(providerId, { minOAuthValidityMs: 300_000 }); + assert.ok(resolved); + } catch { + result = "blocked"; + } + assert.equal(result, scenario.expected); + assert.equal(refreshCallbacks, 0); + assert.equal(toAuthCalls, scenario.expected === "resolved" ? 1 : 0); + assert.equal(sha256(await readFile(path)), before); + return { + scenario: scenario.name, + result, + refresh_callbacks: refreshCallbacks, + to_auth_calls: toAuthCalls, + credential_store: store.audit(), + persistence_operations: 0, + source_unchanged: true, + }; +} + +async function fauxWorkerProof() { + const workerRoot = join(proofRoot, "DUMMY-worker"); + const originalPath = join(workerRoot, "DUMMY-original.md"); + const authPath = join(workerRoot, "DUMMY-auth.json"); + await import("node:fs/promises").then(({ mkdir }) => mkdir(workerRoot)); + const original = + "# DUMMY authority\n\nrequest-supplied owner fields are ignored; ownership comes only from the authenticated principal.\n"; + await writeFile(originalPath, original); + const faux = ai.fauxProvider({ + provider: "DUMMY-readonly", + models: [{ id: "DUMMY-model" }], + tokensPerSecond: 100_000, + }); + const providerId = faux.provider.id; + await writeFile( + authPath, + JSON.stringify({ + [providerId]: { + type: "oauth", + access: "DUMMY-ACCESS", + refresh: "DUMMY-REFRESH", + expires: Date.now() + 600_000, + }, + }), + ); + const store = await Cp06ReadOnlyCredentialStore.load({ path: authPath, providerId }); + let refreshCallbacks = 0; + faux.provider.auth = { + oauth: { + async refresh() { + refreshCallbacks += 1; + throw new Error("DUMMY refresh must not run"); + }, + async toAuth() { + return { apiKey: "DUMMY-RESOLVED" }; + }, + }, + }; + const runtime = await sdk.ModelRuntime.create({ + credentials: store, + modelsPath: null, + refreshOnCreate: false, + allowModelNetwork: false, + }); + runtime.registerNativeProvider(faux.provider); + const model = runtime.getModel(providerId, "DUMMY-model"); + assert.ok(model); + faux.setResponses([ + ai.fauxAssistantMessage(ai.fauxToolCall("read", { path: originalPath }, { id: "DUMMY-read" }), { + stopReason: "toolUse", + }), + ai.fauxAssistantMessage("DUMMY complete", { stopReason: "stop" }), + ]); + const resourceLoader = { + getExtensions: () => ({ extensions: [], errors: [], runtime: sdk.createExtensionRuntime() }), + getSkills: () => ({ skills: [], diagnostics: [] }), + getPrompts: () => ({ prompts: [], diagnostics: [] }), + getThemes: () => ({ themes: [], diagnostics: [] }), + getAgentsFiles: () => ({ agentsFiles: [] }), + getSystemPrompt: () => "Use only read and read the supplied DUMMY original exactly once.", + getSystemPromptSource: () => undefined, + getAppendSystemPrompt: () => [], + getAppendSystemPromptSources: () => [], + extendResources: () => {}, + reload: async () => {}, + }; + const { session } = await sdk.createAgentSession({ + cwd: workerRoot, + agentDir: workerRoot, + model, + modelRuntime: runtime, + resourceLoader, + tools: ["read"], + sessionManager: sdk.SessionManager.inMemory(workerRoot), + settingsManager: sdk.SettingsManager.inMemory({ + compaction: { enabled: false }, + retry: { enabled: false }, + }), + }); + const events = []; + const unsubscribe = session.subscribe((event) => events.push(structuredClone(event))); + try { + await session.prompt(`Read ${originalPath} exactly once.`, { expandPromptTemplates: false }); + } finally { + unsubscribe(); + session.dispose(); + } + const audit = auditReadEvents(events, { + root: workerRoot, + expectedOriginalPath: originalPath, + expectedOriginal: original, + decisiveText: ["request-supplied owner fields are ignored"], + }); + assert.equal(audit.ok, true); + assert.deepEqual(session.getActiveToolNames(), ["read"]); + assert.equal(session.sessionFile, undefined); + assert.equal(refreshCallbacks, 0); + return { + active_tools: session.getActiveToolNames(), + in_memory_session: session.sessionFile === undefined, + event_count: events.length, + event_sha256: sha256(JSON.stringify(events)), + read_audit: audit.summary, + refresh_callbacks: refreshCallbacks, + credential_store: store.audit(), + }; +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} diff --git a/scripts/cp06-isolation-adversary.mjs b/scripts/cp06-isolation-adversary.mjs new file mode 100644 index 0000000..e8bac4f --- /dev/null +++ b/scripts/cp06-isolation-adversary.mjs @@ -0,0 +1,123 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { + linkSync, + mkdirSync, + readFileSync, + renameSync, + symlinkSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { dirname, join } from "node:path"; + +const [securityMode, source, target, syscallProbe, expectedHash, supervisorPid] = + process.argv.slice(2); +if ( + !["unsafe", "hardened"].includes(securityMode) || + ![source, target, syscallProbe, expectedHash, supervisorPid].every(Boolean) +) + process.exit(64); +const proofRoot = dirname(dirname(target)); +const results = {}; + +function jsAttempt(name, action) { + try { + action(); + results[name] = { exit: 0 }; + } catch (error) { + results[name] = { exit: 1, code: error?.code ?? null }; + } +} + +function commandAttempt(name, command, args, options = {}) { + const result = spawnSync(command, args, { + encoding: "utf8", + timeout: 15_000, + ...options, + }); + results[name] = { + exit: result.status, + signal: result.signal, + stdout: String(result.stdout ?? "").trim(), + stderr: String(result.stderr ?? "").trim(), + }; + return result; +} + +jsAttempt("direct_write_target", () => writeFileSync(target, "DUMMY BYPASS target")); +jsAttempt("direct_write_source", () => writeFileSync(source, "DUMMY BYPASS source")); +jsAttempt("unlink_target", () => unlinkSync(target)); +const replacement = join(dirname(target), "DUMMY-replacement"); +writeFileSync(replacement, "DUMMY replacement"); +jsAttempt("rename_over_target", () => renameSync(replacement, target)); +jsAttempt("symlink_replacement", () => { + try { + unlinkSync(target); + } catch {} + symlinkSync("/tmp/DUMMY-does-not-exist", target); +}); +jsAttempt("hardlink_target", () => linkSync(target, join(proofRoot, "DUMMY-hardlink"))); + +if (securityMode === "hardened") { + commandAttempt("direct_namespace_syscalls", syscallProbe, [target]); +} else { + results.direct_namespace_syscalls = { exit: null, skipped: true }; +} +commandAttempt("ordinary_node_child", process.execPath, [ + "-e", + "process.stdout.write('DUMMY-child-ok')", +]); +commandAttempt("nested_user_mount_namespace", "unshare", [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "true", +]); +commandAttempt("nsenter_self", "nsenter", ["-t", String(process.pid), "-m", "true"]); +commandAttempt("nsenter_supervisor", "nsenter", [ + "-t", + supervisorPid, + "-m", + "mount", + "-o", + "remount,bind,rw", + target, +]); +commandAttempt("uid_change", "setpriv", ["--reuid=1", "--regid=1", "--clear-groups", "id"]); +commandAttempt("remount_target_rw", "mount", ["-o", "remount,bind,rw", target]); +jsAttempt("write_after_remount", () => writeFileSync(target, "DUMMY BYPASS remount")); +const unmountTarget = commandAttempt("unmount_target", "umount", [target]); +if (unmountTarget.status === 0) { + commandAttempt("restore_target_bind", "mount", ["--bind", source, target]); +} else { + results.restore_target_bind = { exit: null, skipped: true }; +} +const alias = join(proofRoot, "DUMMY-alias"); +mkdirSync(alias, { recursive: true }); +const bindParent = commandAttempt("bind_parent_alias", "mount", ["--bind", dirname(source), alias]); +if (bindParent.status === 0) { + jsAttempt("alias_underlying_write", () => + writeFileSync(join(alias, source.split("/").at(-1)), "DUMMY BYPASS alias"), + ); +} else { + results.alias_underlying_write = { exit: null, skipped: true }; +} + +const status = Object.fromEntries( + readFileSync("/proc/self/status", "utf8") + .split("\n") + .filter((line) => /^(Cap(Inh|Prm|Eff|Bnd|Amb)|NoNewPrivs|Seccomp):/.test(line)) + .map((line) => line.split(/:\s+/, 2)), +); +const finalBytes = readFileSync(source); +results.identity = { uid: process.getuid(), gid: process.getgid(), status }; +results.source = { + expected_sha256: expectedHash, + final_sha256: createHash("sha256").update(finalBytes).digest("hex"), + unchanged: createHash("sha256").update(finalBytes).digest("hex") === expectedHash, +}; +console.log(JSON.stringify(results)); diff --git a/scripts/cp06-isolation-syscalls.c b/scripts/cp06-isolation-syscalls.c new file mode 100644 index 0000000..85019d9 --- /dev/null +++ b/scripts/cp06-isolation-syscalls.c @@ -0,0 +1,46 @@ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#if !defined(__linux__) || !defined(__x86_64__) +#error "CP-06 isolation syscall probe currently supports Linux x86-64 only" +#endif + +static void result(const char *name, long value) { + printf("%s=%ld:%d\n", name, value, errno); +} + +int main(int argc, char **argv) { + if (argc != 2) return 64; + int namespace_fd = open("/proc/self/ns/mnt", O_RDONLY | O_CLOEXEC); + if (namespace_fd < 0) return 70; + + errno = 0; + result("setns", syscall(SYS_setns, namespace_fd, 0)); + errno = 0; + result("unshare", syscall(SYS_unshare, CLONE_NEWUSER)); + errno = 0; + long child = syscall(SYS_clone, (unsigned long)(CLONE_NEWUSER | SIGCHLD), 0, 0, 0, 0); + if (child == 0) _exit(0); + result("clone", child); +#ifdef SYS_clone3 + struct clone_args arguments = {.flags = CLONE_NEWUSER, .exit_signal = SIGCHLD}; + errno = 0; + child = syscall(SYS_clone3, &arguments, sizeof(arguments)); + if (child == 0) _exit(0); + result("clone3", child); +#endif + errno = 0; + result("mount", syscall(SYS_mount, argv[1], argv[1], 0, MS_REMOUNT | MS_BIND, 0)); + errno = 0; + result("umount2", syscall(SYS_umount2, argv[1], 0)); + close(namespace_fd); + return 0; +} diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs new file mode 100644 index 0000000..235e143 --- /dev/null +++ b/scripts/cp06-namespace-supervisor.mjs @@ -0,0 +1,166 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { mountLiveCredentialReadOnly } from "./cp06-credential-isolation.mjs"; + +const [mode, sourceArg, targetArg, helperArg, syscallProbeArg, ...childArgs] = + process.argv.slice(2); +if ( + !["unsafe-probe", "hardened-probe", "worker"].includes(mode) || + !sourceArg || + !targetArg || + !helperArg || + !syscallProbeArg +) { + process.stderr.write("invalid CP-06 namespace supervisor invocation\n"); + process.exit(64); +} +if (process.platform !== "linux" || process.arch !== "x64") { + process.stderr.write("CP-06 isolation supports Linux x86-64 only\n"); + process.exit(73); +} + +const root = resolve(new URL("..", import.meta.url).pathname); +const source = resolve(sourceArg); +const target = resolve(targetArg); +const helper = resolve(helperArg); +const syscallProbe = resolve(syscallProbeArg); +const beforeHash = sha256(readFileSync(source)); +let mount; +let childResult; +let cleanupError; +try { + mount = await mountLiveCredentialReadOnly({ root, source, target }); + const command = childCommand(); + childResult = spawnSync(command.command, command.args, { + cwd: root, + encoding: "utf8", + env: childEnvironment(mode, target), + timeout: mode === "worker" ? 600_000 : 60_000, + maxBuffer: 1024 * 1024, + }); +} catch (error) { + process.stderr.write(`CP-06 isolation setup failed: ${safeError(error)}\n`); + process.exit(73); +} finally { + try { + mount?.cleanup(); + } catch (error) { + cleanupError = error; + } +} +const afterHash = sha256(readFileSync(source)); +if (cleanupError !== undefined) { + process.stderr.write(`CP-06 isolation cleanup failed: ${safeError(cleanupError)}\n`); + process.exit(74); +} +if (childResult?.error !== undefined) { + process.stderr.write(`CP-06 isolated child launch failed: ${safeError(childResult.error)}\n`); + process.exit(70); +} +if (childResult?.status !== 0) { + process.stderr.write(childResult?.stderr ?? ""); + process.stderr.write(`CP-06 isolated child exited ${childResult?.status ?? "without status"}\n`); + process.exit(childResult?.status ?? 70); +} +if (mode === "worker" && beforeHash !== afterHash) { + process.stderr.write("CP-06 credential source changed during worker execution\n"); + process.exit(74); +} + +let childEvidence; +try { + childEvidence = JSON.parse(String(childResult.stdout).trim()); +} catch { + process.stderr.write("CP-06 isolated child returned non-compact output\n"); + process.exit(70); +} +console.log( + JSON.stringify({ + schema_version: 1, + mode, + source_unchanged: beforeHash === afterHash, + mount_ids_distinct: mount.mountIds.source !== mount.mountIds.target, + cleanup: "pass", + child: childEvidence, + }), +); + +function childCommand() { + if (mode === "unsafe-probe") { + return { + command: process.execPath, + args: [ + resolve(root, "scripts/cp06-isolation-adversary.mjs"), + "unsafe", + source, + target, + syscallProbe, + beforeHash, + String(process.pid), + ], + }; + } + const program = + mode === "hardened-probe" + ? [ + process.execPath, + resolve(root, "scripts/cp06-isolation-adversary.mjs"), + "hardened", + source, + target, + syscallProbe, + beforeHash, + String(process.pid), + ] + : [process.execPath, resolve(root, "scripts/cp06-sdk-worker.mjs"), target, ...childArgs]; + return { + command: "setpriv", + args: [ + "--no-new-privs", + "--bounding-set=-all", + "--inh-caps=-all", + "--ambient-caps=-all", + "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", + "--", + helper, + ...program, + ], + }; +} + +function childEnvironment(childMode, credentialTarget) { + const environment = { + PATH: process.env.PATH ?? "/usr/bin:/bin", + LANG: process.env.LANG ?? "C.UTF-8", + LC_ALL: process.env.LC_ALL ?? "C.UTF-8", + PI_OFFLINE: "1", + PI_SKIP_VERSION_CHECK: "1", + PI_TELEMETRY: "0", + }; + if (childMode === "worker") { + environment.CP06_CREDENTIAL_TARGET = credentialTarget; + for (const name of [ + "SSL_CERT_FILE", + "SSL_CERT_DIR", + "NODE_EXTRA_CA_CERTS", + "HTTP_PROXY", + "HTTPS_PROXY", + "NO_PROXY", + ]) { + if (process.env[name] !== undefined) environment[name] = process.env[name]; + } + } + return environment; +} + +function sha256(bytes) { + return createHash("sha256").update(bytes).digest("hex"); +} + +function safeError(error) { + if (error instanceof AggregateError) return error.errors.map(safeError).join("; "); + return error instanceof Error ? error.message : String(error); +} diff --git a/scripts/cp06-readonly-credentials.mjs b/scripts/cp06-readonly-credentials.mjs new file mode 100644 index 0000000..ad165d8 --- /dev/null +++ b/scripts/cp06-readonly-credentials.mjs @@ -0,0 +1,143 @@ +import { constants } from "node:fs"; +import { open } from "node:fs/promises"; + +const DEFAULT_MAX_BYTES = 1024 * 1024; + +export class Cp06AuthBlockedError extends Error { + constructor(message, options) { + super(message, options); + this.name = "Cp06AuthBlockedError"; + this.code = "CP06_AUTH_BLOCKED"; + } +} + +/** + * A deliberately tiny pi-ai CredentialStore for the CP-06 proof worker. + * It exposes one OAuth record and rejects every mutation before entering the + * callback that provider auth resolution would use to refresh the record. + */ +export class Cp06ReadOnlyCredentialStore { + #providerId; + #credential; + #audit = { reads: 0, lists: 0, modify_denials: 0, delete_denials: 0 }; + + static async load(options) { + const path = options?.path; + const providerId = options?.providerId; + const maxBytes = options?.maxBytes ?? DEFAULT_MAX_BYTES; + if (typeof path !== "string" || path.length === 0) { + throw blocked("credential target path is missing"); + } + if (typeof providerId !== "string" || providerId.length === 0) { + throw blocked("expected OAuth provider is missing"); + } + if (!Number.isSafeInteger(maxBytes) || maxBytes <= 0) { + throw blocked("credential size bound is invalid"); + } + + let handle; + try { + handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC); + const before = await handle.stat({ bigint: true }); + if (!before.isFile()) throw blocked("credential target is not a regular file"); + if (before.size <= 0n || before.size > BigInt(maxBytes)) { + throw blocked("credential target size is outside the allowed bound"); + } + const bytes = await handle.readFile(); + const after = await handle.stat({ bigint: true }); + if ( + before.dev !== after.dev || + before.ino !== after.ino || + before.size !== after.size || + bytes.length !== Number(before.size) + ) { + throw blocked("credential target changed while it was read"); + } + let parsed; + try { + parsed = JSON.parse(bytes.toString("utf8")); + } catch (cause) { + throw blocked("credential target is not valid JSON", { cause }); + } + const credential = parsed?.[providerId]; + validateOAuthCredential(credential); + return new Cp06ReadOnlyCredentialStore(providerId, credential); + } catch (cause) { + if (cause instanceof Cp06AuthBlockedError) throw cause; + throw blocked("credential target could not be opened read-only", { cause }); + } finally { + await handle?.close(); + } + } + + constructor(providerId, credential) { + this.#providerId = providerId; + this.#credential = structuredClone(credential); + } + + assertUsableOAuth(options = {}) { + const now = options.now ?? Date.now(); + const minValidityMs = options.minValidityMs ?? 300_000; + if (!Number.isFinite(now) || !Number.isFinite(minValidityMs) || minValidityMs < 0) { + throw blocked("OAuth validity preflight parameters are invalid"); + } + const remainingValidityMs = this.#credential.expires - now; + if (remainingValidityMs < minValidityMs) { + throw blocked("OAuth credential is expired or too near expiry for a no-refresh run"); + } + return { + providerId: this.#providerId, + type: "oauth", + expires: this.#credential.expires, + remainingValidityMs, + }; + } + + async read(providerId, options) { + options?.signal?.throwIfAborted(); + this.#audit.reads += 1; + return providerId === this.#providerId ? structuredClone(this.#credential) : undefined; + } + + async list(options) { + options?.signal?.throwIfAborted(); + this.#audit.lists += 1; + return [{ providerId: this.#providerId, type: "oauth" }]; + } + + async modify(_providerId, _callback, options) { + options?.signal?.throwIfAborted(); + this.#audit.modify_denials += 1; + throw blocked("CP-06 read-only credential store denies refresh and mutation"); + } + + async delete(_providerId, options) { + options?.signal?.throwIfAborted(); + this.#audit.delete_denials += 1; + throw blocked("CP-06 read-only credential store denies credential deletion"); + } + + audit() { + return { ...this.#audit }; + } +} + +function validateOAuthCredential(credential) { + if ( + credential === null || + typeof credential !== "object" || + Array.isArray(credential) || + credential.type !== "oauth" || + typeof credential.access !== "string" || + credential.access.length === 0 || + typeof credential.refresh !== "string" || + credential.refresh.length === 0 || + !Number.isFinite(credential.expires) + ) { + throw blocked("expected provider does not contain one usable OAuth credential"); + } +} + +function blocked(message, options) { + return new Cp06AuthBlockedError(message, options); +} diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs new file mode 100644 index 0000000..c75d496 --- /dev/null +++ b/scripts/cp06-sdk-worker.mjs @@ -0,0 +1,287 @@ +#!/usr/bin/env node +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { accessSync, constants, readFileSync, realpathSync } from "node:fs"; +import { readFile } from "node:fs/promises"; +import { delimiter, dirname, join, resolve } from "node:path"; +import { pathToFileURL } from "node:url"; +import { Cp06AuthBlockedError, Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; +import { auditReadEvents, isExpectedOriginalReference } from "./cp06-worker-audit.mjs"; + +const [credentialTargetArg, inputPathArg] = process.argv.slice(2); +if (!credentialTargetArg || !inputPathArg) process.exit(64); + +try { + const credentialTarget = resolve(credentialTargetArg); + const input = JSON.parse(await readFile(resolve(inputPathArg), "utf8")); + validateInput(input); + const pi = resolvePiPackage(); + const sdk = await import(pathToFileURL(join(pi.root, "dist/index.js"))); + assertSdkSurface(sdk); + + const contract = await readFile(input.contract_path, "utf8"); + const pack = await readFile(input.pack_path, "utf8"); + const original = await readFile(input.original_path); + const prompt = buildPrompt({ contract, pack, originalPath: input.original_path }); + const store = await Cp06ReadOnlyCredentialStore.load({ + path: credentialTarget, + providerId: input.provider, + }); + const minOAuthValidityMs = input.timeout_ms + 300_000; + const validity = store.assertUsableOAuth({ minValidityMs: minOAuthValidityMs }); + + const runtime = await sdk.ModelRuntime.create({ + credentials: store, + modelsPath: null, + refreshOnCreate: false, + allowModelNetwork: false, + }); + const model = runtime.getModel(input.provider, input.model); + if (model === undefined) throw blocked("configured Pi model is not built in to Pi 0.85.1"); + const auth = await runtime.getAuth(model, { minOAuthValidityMs }); + if (auth === undefined) throw blocked("expected OAuth provider did not resolve authentication"); + + const resourceLoader = emptyResourceLoader(sdk, input.system_prompt); + const settingsManager = sdk.SettingsManager.inMemory({ + compaction: { enabled: false }, + retry: { + enabled: false, + maxRetries: 0, + provider: { timeoutMs: input.timeout_ms, maxRetries: 0, maxRetryDelayMs: 0 }, + }, + defaultProjectTrust: "never", + enableAnalytics: false, + enableInstallTelemetry: false, + }); + const { session, extensionsResult } = await sdk.createAgentSession({ + cwd: input.root, + agentDir: dirname(credentialTarget), + model, + modelRuntime: runtime, + resourceLoader, + tools: ["read"], + sessionManager: sdk.SessionManager.inMemory(input.root), + settingsManager, + thinkingLevel: "high", + }); + if ((extensionsResult.extensions?.length ?? 0) !== 0 || session.sessionFile !== undefined) { + throw blocked("SDK worker unexpectedly loaded an extension or persistent session"); + } + if (JSON.stringify(session.getActiveToolNames()) !== JSON.stringify(["read"])) { + throw blocked("SDK worker active tool set is not exactly read"); + } + + const events = []; + const unsubscribe = session.subscribe((event) => events.push(structuredClone(event))); + let timedOut = false; + let timer; + try { + await Promise.race([ + session.prompt(prompt, { expandPromptTemplates: false, source: "rpc" }), + new Promise((_, reject) => { + timer = setTimeout(() => { + timedOut = true; + void session.abort(); + reject(new Error("CP06_WORKER_TIMEOUT")); + }, input.timeout_ms); + }), + ]); + } finally { + clearTimeout(timer); + if (timedOut || session.isStreaming) await session.abort(); + unsubscribe(); + session.dispose(); + } + + const responseText = finalAssistantText(events); + const response = parseResponse(responseText); + const readAudit = auditReadEvents(events, { + root: input.root, + expectedOriginalPath: input.original_path, + expectedOriginal: original, + decisiveText: ["request-supplied owner fields are ignored"], + }); + if (!readAudit.ok) { + throw blocked(`SDK event audit failed: ${readAudit.reason}`); + } + const corrected = Array.isArray(response.corrected_constraints) + ? response.corrected_constraints.join(" ") + : ""; + if ( + response.status !== "MISSING_SOURCE" || + !isExpectedOriginalReference(response.missing_source, { + root: input.root, + referenceRoot: input.fixture_root, + expectedOriginalPath: input.original_path, + }) || + response.broad_scan !== false || + !/ownership comes only from the authenticated principal/i.test(corrected) || + !/request-supplied owner fields? (?:are )?ignored/i.test(corrected) + ) { + throw blocked("SDK worker response did not satisfy the targeted-read oracle"); + } + + console.log( + JSON.stringify({ + schema_version: 1, + result: "pass", + pi_version: pi.version, + provider: input.provider, + model: input.model, + oauth_preflight: { + minimum_validity_ms: minOAuthValidityMs, + remaining_validity_ms: validity.remainingValidityMs, + refreshed: false, + }, + credential_store: store.audit(), + session: { in_memory: true, active_tools: session.getActiveToolNames() }, + event_stream: { + sha256: sha256(JSON.stringify(events)), + event_count: events.length, + read_audit: readAudit.summary, + }, + response, + response_sha256: sha256(responseText), + raw_transcript_retained: false, + }), + ); +} catch (error) { + if (error instanceof Cp06AuthBlockedError || error?.code === "CP06_AUTH_BLOCKED") { + process.stderr.write(`CP06_AUTH_BLOCKED: ${error.message}\n`); + process.exit(75); + } + process.stderr.write(`CP06_WORKER_FAILED: ${safeMessage(error)}\n`); + process.exit(70); +} + +function validateInput(input) { + if (input?.schema_version !== 1) throw blocked("worker input schema is unsupported"); + for (const key of ["root", "fixture_root", "contract_path", "pack_path", "original_path"]) + if (typeof input[key] !== "string" || !input[key].startsWith("/")) { + throw blocked(`worker input ${key} must be absolute`); + } + if (input.provider !== "openai-codex") throw blocked("worker provider must be openai-codex"); + if (typeof input.model !== "string" || input.model.length === 0) { + throw blocked("worker model is missing"); + } + if (!Number.isSafeInteger(input.timeout_ms) || input.timeout_ms < 1_000) { + throw blocked("worker timeout is invalid"); + } + if (typeof input.system_prompt !== "string" || input.system_prompt.length === 0) { + throw blocked("worker system prompt is missing"); + } +} + +function resolvePiPackage() { + const piExecutable = findExecutable("pi"); + const cli = realpathSync(piExecutable); + const packageRoot = findPiPackageRoot(cli); + const metadata = JSON.parse(readFileSync(join(packageRoot, "package.json"), "utf8")); + const version = spawnSync(piExecutable, ["--version"], { encoding: "utf8", timeout: 10_000 }); + if ( + metadata.name !== "@earendil-works/pi-coding-agent" || + metadata.version !== "0.85.1" || + version.status !== 0 || + version.stdout.trim() !== "0.85.1" + ) { + throw blocked("exact Pi 0.85.1 is required"); + } + return { root: packageRoot, version: metadata.version }; +} + +function findPiPackageRoot(executable) { + let directory = dirname(executable); + for (let depth = 0; depth < 5; depth += 1) { + try { + const metadata = JSON.parse(readFileSync(join(directory, "package.json"), "utf8")); + if (metadata.name === "@earendil-works/pi-coding-agent") return directory; + } catch {} + const parent = dirname(directory); + if (parent === directory) break; + directory = parent; + } + throw blocked("Pi package root could not be resolved from its executable"); +} + +function findExecutable(name) { + for (const directory of String(process.env.PATH ?? "").split(delimiter)) { + if (!directory) continue; + const candidate = join(directory, name); + try { + accessSync(candidate, constants.X_OK); + return candidate; + } catch {} + } + throw blocked("Pi executable is unavailable"); +} + +function assertSdkSurface(sdk) { + for (const name of [ + "ModelRuntime", + "createAgentSession", + "createExtensionRuntime", + "SessionManager", + "SettingsManager", + ]) { + if (sdk[name] === undefined) throw blocked(`Pi SDK export is unavailable: ${name}`); + } +} + +function emptyResourceLoader(sdk, systemPrompt) { + return { + getExtensions: () => ({ extensions: [], errors: [], runtime: sdk.createExtensionRuntime() }), + getSkills: () => ({ skills: [], diagnostics: [] }), + getPrompts: () => ({ prompts: [], diagnostics: [] }), + getThemes: () => ({ themes: [], diagnostics: [] }), + getAgentsFiles: () => ({ agentsFiles: [] }), + getSystemPrompt: () => systemPrompt, + getSystemPromptSource: () => undefined, + getAppendSystemPrompt: () => [], + getAppendSystemPromptSources: () => [], + extendResources: () => {}, + reload: async () => {}, + }; +} + +function buildPrompt({ contract, pack, originalPath }) { + return `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE, set missing_source to the exact canonical relative path docs/AUTH.md, and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; +} + +function finalAssistantText(events) { + const messages = events + .filter((event) => event.type === "message_end" && event.message?.role === "assistant") + .map((event) => event.message); + const assistant = messages.at(-1); + if (assistant === undefined) + throw blocked("SDK event stream omitted the final assistant message"); + if (typeof assistant.content === "string") return assistant.content; + return Array.isArray(assistant.content) + ? assistant.content + .filter((block) => block?.type === "text" && typeof block.text === "string") + .map((block) => block.text) + .join("") + : ""; +} + +function parseResponse(output) { + try { + return JSON.parse(output.trim()); + } catch { + const fenced = output.trim().match(/```(?:json)?\s*([\s\S]*?)```/); + if (fenced === null) throw blocked("SDK worker did not return the required JSON response"); + return JSON.parse(fenced[1]); + } +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + +function blocked(message) { + return new Cp06AuthBlockedError(message); +} + +function safeMessage(error) { + if (error?.message === "CP06_WORKER_TIMEOUT") return "worker timed out and was aborted"; + return error instanceof Error ? error.message : String(error); +} diff --git a/scripts/cp06-seccomp-exec.c b/scripts/cp06-seccomp-exec.c new file mode 100644 index 0000000..5ead15a --- /dev/null +++ b/scripts/cp06-seccomp-exec.c @@ -0,0 +1,123 @@ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#if !defined(__linux__) || !defined(__x86_64__) +#error "CP-06 isolation helper currently supports Linux x86-64 only" +#endif + +typedef void *scmp_filter_ctx; +struct scmp_arg_cmp { + unsigned int arg; + int op; + uint64_t datum_a; + uint64_t datum_b; +}; +extern scmp_filter_ctx seccomp_init(uint32_t def_action); +extern int seccomp_rule_add(scmp_filter_ctx, uint32_t, int, unsigned int, ...); +extern int seccomp_rule_add_array(scmp_filter_ctx, uint32_t, int, unsigned int, + const struct scmp_arg_cmp *); +extern int seccomp_load(scmp_filter_ctx); +extern void seccomp_release(scmp_filter_ctx); +extern int seccomp_syscall_resolve_name(const char *); + +#define SCMP_ACT_ALLOW 0x7fff0000U +#define SCMP_ACT_ERRNO(x) (0x00050000U | ((x) & 0x0000ffffU)) +#define SCMP_CMP_MASKED_EQ 7 + +static void fail(const char *message) { + perror(message); + _exit(70); +} + +static void deny(scmp_filter_ctx ctx, const char *name, int error) { + int nr = seccomp_syscall_resolve_name(name); + if (nr >= 0 && seccomp_rule_add(ctx, SCMP_ACT_ERRNO(error), nr, 0) < 0) { + fprintf(stderr, "cannot install seccomp rule for %s\n", name); + _exit(70); + } +} + +static void close_inherited_fds(void) { +#ifdef SYS_close_range + if (syscall(SYS_close_range, 3U, ~0U, 0U) == 0) return; + if (errno != ENOSYS) fail("close_range"); +#endif + long limit = sysconf(_SC_OPEN_MAX); + if (limit < 0 || limit > 1048576) limit = 1048576; + for (int fd = 3; fd < limit; ++fd) close(fd); +} + +int main(int argc, char **argv) { + if (argc < 2) { + fprintf(stderr, "usage: cp06-seccomp-exec COMMAND [ARG...]\n"); + return 64; + } + + pid_t parent = getppid(); + if (prctl(PR_SET_PDEATHSIG, SIGKILL) != 0) fail("PR_SET_PDEATHSIG"); + if (getppid() != parent || parent == 1) { + fprintf(stderr, "CP-06 supervisor disappeared before isolation\n"); + return 70; + } + close_inherited_fds(); + if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0) fail("PR_SET_NO_NEW_PRIVS"); + + scmp_filter_ctx ctx = seccomp_init(SCMP_ACT_ALLOW); + if (ctx == NULL) { + fprintf(stderr, "seccomp_init failed\n"); + return 70; + } + const char *blocked[] = { + "mount", "umount2", "pivot_root", + "move_mount", "open_tree", "fsopen", + "fsconfig", "fsmount", "fspick", + "mount_setattr", "unshare", "setns", + "open_by_handle_at", "name_to_handle_at", "ptrace", + "process_vm_readv", "process_vm_writev", "pidfd_getfd", + "bpf", "perf_event_open", "personality", + "kexec_load", "kexec_file_load", + }; + for (size_t i = 0; i < sizeof(blocked) / sizeof(blocked[0]); ++i) { + deny(ctx, blocked[i], EPERM); + } + + /* Node/libc can fall back to clone(2), where namespace bits are filtered. */ + deny(ctx, "clone3", ENOSYS); + int clone_nr = seccomp_syscall_resolve_name("clone"); + const uint64_t namespace_flags[] = { + CLONE_NEWNS, CLONE_NEWCGROUP, CLONE_NEWUTS, CLONE_NEWIPC, + CLONE_NEWUSER, CLONE_NEWPID, CLONE_NEWNET, + }; + for (size_t i = 0; + clone_nr >= 0 && i < sizeof(namespace_flags) / sizeof(namespace_flags[0]); + ++i) { + struct scmp_arg_cmp comparison = { + .arg = 0, + .op = SCMP_CMP_MASKED_EQ, + .datum_a = namespace_flags[i], + .datum_b = namespace_flags[i], + }; + if (seccomp_rule_add_array(ctx, SCMP_ACT_ERRNO(EPERM), clone_nr, 1, + &comparison) < 0) { + fprintf(stderr, "cannot install clone namespace filter\n"); + seccomp_release(ctx); + return 70; + } + } + + if (seccomp_load(ctx) < 0) { + seccomp_release(ctx); + fail("seccomp_load"); + } + seccomp_release(ctx); + execvp(argv[1], &argv[1]); + fail("execvp"); +} diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs index 8f96665..ebe1126 100644 --- a/scripts/cp06-worker-audit.mjs +++ b/scripts/cp06-worker-audit.mjs @@ -25,6 +25,7 @@ export function auditReadEvents(events, options) { const readStarts = starts.filter(({ event }) => event.toolName === "read"); const readEnds = ends.filter(({ event }) => event.toolName === "read"); const otherToolCalls = starts.filter(({ event }) => event.toolName !== "read"); + const otherToolCompletions = ends.filter(({ event }) => event.toolName !== "read"); const resolvedReads = readStarts.map(({ event, index }) => ({ event, index, @@ -40,6 +41,7 @@ export function auditReadEvents(events, options) { project_read_paths: originalReads.map((entry) => entry.resolvedPath), rejected_read_paths: rejectedReads.map((entry) => entry.path), other_tool_calls: otherToolCalls.map(({ event }) => event.toolName), + other_tool_completions: otherToolCompletions.map(({ event }) => event.toolName), completed_successfully: false, exact_original: false, expected_sha256: sha256(expectedBytes), @@ -52,8 +54,13 @@ export function auditReadEvents(events, options) { }; const reject = (reason) => ({ ok: false, reason, summary }); - if (otherToolCalls.length > 0) { - return reject(`unexpected tool calls ${summary.other_tool_calls.join(", ")}`); + if (otherToolCalls.length > 0 || otherToolCompletions.length > 0) { + return reject( + `unexpected tool events ${[ + ...summary.other_tool_calls, + ...summary.other_tool_completions, + ].join(", ")}`, + ); } if (readStarts.length !== 1 || originalReads.length !== 1 || rejectedReads.length > 0) { return reject( diff --git a/scripts/prove-cp06-auth-security.mjs b/scripts/prove-cp06-auth-security.mjs new file mode 100644 index 0000000..2c50956 --- /dev/null +++ b/scripts/prove-cp06-auth-security.mjs @@ -0,0 +1,32 @@ +#!/usr/bin/env node +import { writeFile } from "node:fs/promises"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { Cp06IsolationUnsupportedError, runCp06SecurityPreflight } from "./cp06-auth-security.mjs"; + +const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); +try { + const proof = await runCp06SecurityPreflight({ + root, + outputRoot: resolve(outputRoot, "auth-security"), + reviewer: process.env.CP06_REVIEWER, + }); + const path = resolve(outputRoot, "auth-security", "evidence.json"); + await writeFile(path, `${JSON.stringify(proof, null, 2)}\n`); + console.log( + JSON.stringify({ + ok: true, + gate: "CP-06-read-only-auth-security", + evidence_path: path.startsWith(`${root}/`) ? path.slice(root.length + 1) : path, + result: proof.result, + }), + ); +} catch (error) { + if (error instanceof Cp06IsolationUnsupportedError) { + process.stderr.write(`CP06_ISOLATION_UNSUPPORTED: ${error.message}\n`); + process.exit(73); + } + process.stderr.write(`CP06_SECURITY_PROOF_FAILED: ${error.message}\n`); + process.exit(1); +} diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 799d91a..4c2ddfb 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -6,142 +6,94 @@ import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { - mountLiveCredentialReadOnly, - proveDummyCredentialIsolation, -} from "./cp06-credential-isolation.mjs"; -import { auditReadEvents, isExpectedOriginalReference } from "./cp06-worker-audit.mjs"; + Cp06IsolationUnsupportedError, + runCp06SecurityPreflight, + runIsolatedWorker, +} from "./cp06-auth-security.mjs"; -const scriptPath = fileURLToPath(import.meta.url); -const root = resolve(dirname(scriptPath), ".."); -if (process.env.CP06_MOUNT_ISOLATION_CHILD !== "1") { - const isolated = spawnSync( - "unshare", - [ - "--user", - "--map-root-user", - "--mount", - "--propagation", - "private", - "--", - process.execPath, - scriptPath, - ], - { - cwd: root, - encoding: "utf8", - env: { ...process.env, CP06_MOUNT_ISOLATION_CHILD: "1" }, - timeout: 600_000, - }, - ); - process.stdout.write(isolated.stdout ?? ""); - process.stderr.write(isolated.stderr ?? ""); - if (isolated.error !== undefined) throw isolated.error; - process.exit(isolated.status ?? 1); -} +const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); const workerRoot = join(outputRoot, "worker"); +const securityRoot = join(outputRoot, "auth-security"); const fixtureRoot = join(outputRoot, "ten-pack", "repo"); const initialPackPath = join(outputRoot, "ten-pack", "raw", "auth-worker-missing.pack.md"); const contractPath = join(root, "test/fixtures/cp06-context/cases/AUTH-worker-before.yaml"); const originalPath = join(fixtureRoot, "docs/AUTH.md"); +const provider = "openai-codex"; +const model = process.env.CP06_PI_MODEL ?? "gpt-5.6-sol"; +const timeoutMs = 600_000; +const systemPrompt = + "You are a bounded CP-06 reviewer. Use exactly the read tool, read only the supplied exact original once, return only the requested JSON, and do not attempt any file mutation or broad scan."; + await rm(workerRoot, { recursive: true, force: true }); await mkdir(workerRoot, { recursive: true }); +let security; +try { + // This DUMMY-only gate includes the unsafe control, hardened evaluated-child + // attacks, vendor refresh counters, and offline SDK read audit. It completes + // before the real credential path is opened or any provider request is made. + security = await runCp06SecurityPreflight({ + root, + outputRoot: securityRoot, + reviewer: process.env.CP06_REVIEWER, + }); +} catch (error) { + await recordBlocked(error, "dummy-security-preflight"); + if (error instanceof Cp06IsolationUnsupportedError) process.exit(73); + throw error; +} -const contract = await readFile(contractPath, "utf8"); -const pack = await readFile(initialPackPath, "utf8"); -const original = await readFile(originalPath); -const decisiveConstraint = "request-supplied owner fields are ignored"; -const prompt = `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE, set missing_source to the exact canonical relative path docs/AUTH.md, and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; - -const provider = process.env.CP06_PI_PROVIDER ?? "openai-codex"; -const model = process.env.CP06_PI_MODEL ?? "gpt-5.6-sol"; -const args = [ - "--provider", +const workerInput = { + schema_version: 1, + root, + fixture_root: fixtureRoot, + contract_path: contractPath, + pack_path: initialPackPath, + original_path: originalPath, provider, - "--model", model, - "--thinking", - "high", - "--mode", - "json", - "--no-session", - "--no-context-files", - "--no-extensions", - "--no-skills", - "--skill", - "skills/factory", - "--tools", - "read", - prompt, -]; -const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-worker-")); -const evaluationAgentDir = join(evaluationHome, "pi-agent"); -await mkdir(evaluationAgentDir); + timeout_ms: timeoutMs, + system_prompt: systemPrompt, +}; +const inputPath = join(workerRoot, "input.json"); +await writeFile(inputPath, `${JSON.stringify(workerInput, null, 2)}\n`); +const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-sdk-worker-")); +const credentialTarget = join(evaluationHome, "pi-agent", "auth.json"); const credentialSource = resolve( process.env.CP06_PI_AUTH_FILE ?? join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "auth.json"), ); -const credentialHashBefore = await fileHash(credentialSource); -const dummyIsolationProof = await proveDummyCredentialIsolation({ root }); -const evaluationCredential = join(evaluationAgentDir, "auth.json"); -const liveCredentialMount = await mountLiveCredentialReadOnly({ - root, - source: credentialSource, - target: evaluationCredential, -}); -let result; -let credentialHashAfter; +const binaries = { + helper: join(securityRoot, "cp06-seccomp-exec"), + syscallProbe: join(securityRoot, "cp06-isolation-syscalls"), +}; +let isolated; try { - result = spawnSync("pi", args, { - cwd: root, - encoding: "utf8", - env: isolatedPiEnvironment(evaluationHome, evaluationAgentDir), - timeout: 600_000, + isolated = await runIsolatedWorker({ + root, + source: credentialSource, + target: credentialTarget, + binaries, + inputPath, + timeout: timeoutMs + 60_000, }); - credentialHashAfter = await fileHash(credentialSource); +} catch (error) { + await recordBlocked(error, "isolated-sdk-worker"); + process.stderr.write(`${error.code ?? "CP06_WORKER_FAILED"}: ${error.message}\n`); + process.exit(error.exitCode ?? 1); } finally { - liveCredentialMount.cleanup(); await rm(evaluationHome, { recursive: true, force: true }); } -if (credentialHashAfter !== credentialHashBefore) { - throw new Error("fresh Pi worker modified the source credential file"); -} -if (result.error !== undefined || result.status !== 0) { - process.stderr.write(result.stderr ?? ""); - throw new Error(`fresh Pi worker exited ${result.status}`); -} -const events = parseEventStream(result.stdout ?? ""); -const responseText = finalAssistantText(events); -const response = parseResponse(responseText); -const readAudit = auditReadEvents(events, { - root, - expectedOriginalPath: originalPath, - expectedOriginal: original, - decisiveText: [decisiveConstraint], -}); -if (!readAudit.ok) { - throw new Error(`fresh worker violated the event-stream read oracle: ${readAudit.reason}`); -} -const requiredConstraints = [ - /ownership comes only from the authenticated principal/i, - /request-supplied owner fields? (?:are )?ignored/i, -]; -const constraintText = Array.isArray(response.corrected_constraints) - ? response.corrected_constraints.join(" ") - : ""; if ( - response.status !== "MISSING_SOURCE" || - !isExpectedOriginalReference(response.missing_source, { - root, - referenceRoot: fixtureRoot, - expectedOriginalPath: originalPath, - }) || - response.broad_scan !== false || - !requiredConstraints.every((constraint) => constraint.test(constraintText)) + isolated.mode !== "worker" || + isolated.source_unchanged !== true || + isolated.mount_ids_distinct !== true || + isolated.cleanup !== "pass" || + isolated.child?.result !== "pass" ) { - throw new Error( - `fresh worker response did not satisfy the targeted-read oracle: ${responseText}`, - ); + const error = new Error("isolated SDK worker omitted a required safety assertion"); + await recordBlocked(error, "isolated-sdk-worker-audit"); + throw error; } const testedSha = textCommand("git", ["rev-parse", "HEAD"]); @@ -150,15 +102,16 @@ const manifest = { gate: "P-06-fresh-worker", tested_sha: testedSha, recorded_at: new Date().toISOString(), - reviewer: process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker", + reviewer: process.env.CP06_REVIEWER ?? "Pi CP-06 read-only auth correction worker", worker: { - pi_version: textCommand("pi", ["--version"]), + pi_version: isolated.child.pi_version, + interface: "public SDK with injected CredentialStore", provider, model, - no_session: true, - no_context_files: true, - mode: "json", - tools: ["read"], + in_memory_session: isolated.child.session.in_memory, + tools: isolated.child.session.active_tools, + refresh_on_create: false, + model_catalog_network: false, }, bounded_handoff: { contract_path: relative(contractPath), @@ -167,40 +120,33 @@ const manifest = { pack_sha256: await fileHash(initialPackPath), original_path: relative(originalPath), original_sha256: await fileHash(originalPath), - deliberately_omitted_constraint_sha256: sha256(decisiveConstraint), + deliberately_omitted_constraint_sha256: sha256("request-supplied owner fields are ignored"), evidence: "in-memory auth 401 and concealed non-mutating cross-account 404 observations", - prompt_sha256: sha256(prompt), }, isolation: { - disposable_home: true, - disposable_pi_agent_dir: true, - credential_mode: - "Linux user/mount namespace; live source and isolated auth regular file bind-mounted read-only; no symlink; no copy", - dummy_destructive_probe: { - source_unchanged: dummyIsolationProof.source_unchanged, - atomic_replacement_denied: dummyIsolationProof.atomic_replacement_denied, - }, - live_mount_checks: { - non_writing_read_only_mount_checks: true, - destructive_real_source_probe: false, - }, - credential_source_unchanged: credentialHashAfter === credentialHashBefore, + supported_platform: "Linux x86-64", + dummy_security_preflight: security, + source_and_target_distinct_read_only_mounts: isolated.mount_ids_distinct, + evaluated_child_capabilities: "all sets empty", + evaluated_child_no_new_privs: true, + evaluated_child_seccomp: true, + credential_store: "injected callback-denying read-only CredentialStore", + oauth_preflight: isolated.child.oauth_preflight, + credential_store_audit: isolated.child.credential_store, + credential_source_unchanged: isolated.source_unchanged, + cleanup: isolated.cleanup, + destructive_real_source_probe: false, raw_transcript_retained: false, }, - event_stream: { - sha256: sha256(result.stdout ?? ""), - event_count: events.length, - read_audit: readAudit.summary, - }, + event_stream: isolated.child.event_stream, command: - "pi --mode json --no-session --no-context-files --no-extensions --no-skills --skill skills/factory --tools read ", - exit_code: result.status, - output_sha256: sha256(responseText), - stderr_sha256: sha256(result.stderr ?? ""), - response, + "Pi 0.85.1 createAgentSession({modelRuntime: injectedReadOnlyRuntime, tools: ['read'], inMemorySession}) inside capability-free seccomp child", + exit_code: 0, + response_sha256: isolated.child.response_sha256, + response: isolated.child.response, result: "pass", limitation: - "Live model wording is nondeterministic; credential mount isolation requires Linux unshare/mount support and fails safe before Pi is invoked when unsupported; the script deterministically checks actual read tool-call events and required corrected constraints.", + "Live model wording is nondeterministic. The proof runner is Linux x86-64 only and exits blocked before credential access/provider use when user/mount namespaces, libseccomp, exact Pi 0.85.1, or adequate unexpired OAuth validity are unavailable.", }; const manifestPath = join(workerRoot, "evidence.json"); await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); @@ -211,62 +157,29 @@ console.log( tested_sha: testedSha, manifest_path: relative(manifestPath), manifest_sha256: await fileHash(manifestPath), - exit_code: result.status, - status: response.status, - reads: readAudit.summary.project_read_paths, - broad_scan: response.broad_scan, + exit_code: 0, + status: manifest.response.status, + reads: manifest.event_stream.read_audit.project_read_paths, + broad_scan: manifest.response.broad_scan, }), ); -function parseEventStream(output) { - const events = []; - for (const [index, line] of output.split(/\r?\n/).entries()) { - if (line.trim().length === 0) continue; - try { - events.push(JSON.parse(line)); - } catch (error) { - throw new Error(`Pi JSON event stream line ${index + 1} is not valid JSON: ${error.message}`); - } - } - return events; -} - -function finalAssistantText(events) { - const agentEnd = events.findLast((event) => event.type === "agent_end"); - const messages = Array.isArray(agentEnd?.messages) - ? agentEnd.messages - : events - .filter((event) => event.type === "message_end" && event.message?.role === "assistant") - .map((event) => event.message); - const assistant = messages.findLast((message) => message.role === "assistant"); - if (assistant === undefined) - throw new Error("Pi JSON event stream did not include a final assistant message"); - return messageText(assistant); -} - -function messageText(message) { - if (typeof message.content === "string") return message.content; - if (!Array.isArray(message.content)) return ""; - return message.content - .filter((block) => block?.type === "text" && typeof block.text === "string") - .map((block) => block.text) - .join(""); -} - -function parseResponse(output) { - const trimmed = output.trim(); - try { - return JSON.parse(trimmed); - } catch { - const fenced = trimmed.match(/```(?:json)?\s*([\s\S]*?)```/); - if (fenced === null) throw new Error(`fresh worker did not return JSON: ${output}`); - return JSON.parse(fenced[1]); - } +async function recordBlocked(error, stage) { + const status = { + schema_version: 1, + gate: "P-06-fresh-worker", + result: "blocked", + stage, + code: error?.code ?? "CP06_WORKER_FAILED", + recorded_at: new Date().toISOString(), + raw_transcript_retained: false, + }; + await writeFile(join(workerRoot, "blocked.json"), `${JSON.stringify(status, null, 2)}\n`); } function textCommand(command, args) { const result = spawnSync(command, args, { cwd: root, encoding: "utf8" }); - if (result.status !== 0) throw new Error(`${command} failed: ${result.stderr}`); + if (result.status !== 0) throw new Error(`${command} failed`); return result.stdout.trim(); } @@ -281,26 +194,3 @@ async function fileHash(path) { function sha256(value) { return createHash("sha256").update(value).digest("hex"); } - -function isolatedPiEnvironment(home, agentDir) { - const environment = { - HOME: home, - PI_CODING_AGENT_DIR: agentDir, - PI_SKIP_VERSION_CHECK: "1", - PI_TELEMETRY: "0", - }; - for (const name of [ - "PATH", - "LANG", - "LC_ALL", - "SSL_CERT_FILE", - "SSL_CERT_DIR", - "NODE_EXTRA_CA_CERTS", - "HTTP_PROXY", - "HTTPS_PROXY", - "NO_PROXY", - ]) { - if (process.env[name] !== undefined) environment[name] = process.env[name]; - } - return environment; -} diff --git a/test/cp06-credential-isolation.test.mjs b/test/cp06-credential-isolation.test.mjs index d106c93..cffe2f0 100644 --- a/test/cp06-credential-isolation.test.mjs +++ b/test/cp06-credential-isolation.test.mjs @@ -8,39 +8,114 @@ import { mountLiveCredentialReadOnly, } from "../scripts/cp06-credential-isolation.mjs"; -test("live credential mount uses a regular target without source writes", async () => { +test("live credential mount pins one inode behind distinct read-only regular mounts", async () => { const root = await mkdtemp(join(tmpdir(), "factory-cp06-live-mount-test-")); const source = join(root, "auth-source.json"); const target = join(root, "agent", "auth.json"); const sourceBytes = '{"live":true}\n'; await writeFile(source, sourceBytes); + const sourceStat = await lstat(source, { bigint: true }); const calls = []; + let mountId = 40; const spawnSyncImpl = (command, args) => { calls.push([command, ...args]); - if (command === "findmnt") return { status: 0, stdout: "ro,nosuid,nodev\n", stderr: "" }; + if (command === "findmnt") { + const path = args.at(-1); + return { + status: 0, + stdout: JSON.stringify({ + filesystems: [ + { + id: ++mountId, + target: path, + "vfs-options": "ro,nosuid,nodev,noexec,relatime", + }, + ], + }), + stderr: "", + }; + } return { status: 0, stdout: "", stderr: "" }; }; + const lstatImpl = async (path, options) => (path === target ? sourceStat : lstat(path, options)); try { - const mount = await mountLiveCredentialReadOnly({ root, source, target, spawnSyncImpl }); + const mount = await mountLiveCredentialReadOnly({ + root, + source, + target, + spawnSyncImpl, + lstatImpl, + }); try { assert.equal(await readFile(source, "utf8"), sourceBytes); - assert.equal((await lstat(target)).isFile(), true); - assert.equal((await lstat(target)).isSymbolicLink(), false); - assert.deepEqual(calls.slice(0, 4), [ - ["mount", "--bind", source, source], - ["mount", "-o", "remount,bind,ro", source], - ["mount", "--bind", source, target], - ["mount", "-o", "remount,bind,ro", target], - ]); + assert.notEqual(mount.mountIds.source, mount.mountIds.target); + assert.match(calls[0][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); + assert.deepEqual(calls[0].slice(0, 2), ["mount", "--bind"]); + assert.equal(calls[0][3], source); + assert.deepEqual(calls[1], ["mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", source]); + assert.match(calls[2][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); + assert.equal(calls[2][3], target); + assert.deepEqual(calls[3], ["mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", target]); } finally { mount.cleanup(); } + assert.deepEqual(calls.slice(-2), [ + ["umount", target], + ["umount", source], + ]); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("mount setup failure reports cleanup failure instead of discarding it", async () => { + const root = await mkdtemp(join(tmpdir(), "factory-cp06-cleanup-test-")); + const source = join(root, "auth-source.json"); + const target = join(root, "agent", "auth.json"); + await writeFile(source, "DUMMY"); + let mountCalls = 0; + const spawnSyncImpl = (command) => { + if (command === "mount" && ++mountCalls === 2) { + return { status: 32, stdout: "", stderr: "DUMMY remount failure" }; + } + if (command === "umount") { + return { status: 32, stdout: "", stderr: "DUMMY cleanup failure" }; + } + return { status: 0, stdout: "", stderr: "" }; + }; + try { + await assert.rejects( + mountLiveCredentialReadOnly({ root, source, target, spawnSyncImpl }), + AggregateError, + ); } finally { await rm(root, { recursive: true, force: true }); } }); -test("destructive dummy proof requires denied atomic replacement", async () => { +test("unsupported isolation fails before credential access or provider work", async () => { + const { verifyIsolationPrerequisites, Cp06IsolationUnsupportedError } = + await import("../scripts/cp06-auth-security.mjs"); + let invocations = 0; + assert.throws( + () => + verifyIsolationPrerequisites({ + root: process.cwd(), + spawnSyncImpl(command) { + invocations += 1; + return { + status: command === "unshare" ? 1 : 0, + stdout: "", + stderr: "DUMMY namespaces unavailable", + }; + }, + }), + Cp06IsolationUnsupportedError, + ); + assert.equal(invocations, 1); +}); + +test("destructive DUMMY proof requires denied write unlink rename symlink and hardlink", async () => { const calls = []; const protectedPaths = new Set(["/dummy/source-auth.json", "/dummy/target-auth.json"]); await assertDestructiveCredentialWritesDenied([...protectedPaths], { @@ -50,11 +125,25 @@ test("destructive dummy proof requires denied atomic replacement", async () => { }, async rename(from, to) { calls.push(["rename", from, to]); - if (protectedPaths.has(to)) throw new Error("EROFS"); + if (protectedPaths.has(to)) throw new Error("EBUSY"); + }, + async unlink(path) { + calls.push(["unlink", path]); + if (protectedPaths.has(path)) throw new Error("EBUSY"); + }, + async symlink(from, to) { + calls.push(["symlink", from, to]); + }, + async link(from, to) { + calls.push(["link", from, to]); + throw new Error("EXDEV"); }, async rm(path) { calls.push(["rm", path]); }, }); assert.equal(calls.filter(([operation]) => operation === "rename").length, 2); + assert.equal(calls.filter(([operation]) => operation === "unlink").length, 4); + assert.equal(calls.filter(([operation]) => operation === "link").length, 2); + assert.equal(calls.filter(([operation]) => operation === "symlink").length, 0); }); diff --git a/test/cp06-readonly-credentials.test.mjs b/test/cp06-readonly-credentials.test.mjs new file mode 100644 index 0000000..b69059e --- /dev/null +++ b/test/cp06-readonly-credentials.test.mjs @@ -0,0 +1,126 @@ +import { strict as assert } from "node:assert"; +import { mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; +import { + Cp06AuthBlockedError, + Cp06ReadOnlyCredentialStore, +} from "../scripts/cp06-readonly-credentials.mjs"; + +const providerId = "openai-codex"; + +async function withCredential(credential, action) { + const root = await mkdtemp(join(tmpdir(), "factory-cp06-credential-store-")); + const path = join(root, "DUMMY-auth.json"); + await writeFile( + path, + `${JSON.stringify({ + [providerId]: credential, + "ignored-provider": { type: "api_key", key: "DUMMY-IGNORED" }, + })}\n`, + { mode: 0o600 }, + ); + try { + return await action({ path, root }); + } finally { + await rm(root, { recursive: true, force: true }); + } +} + +function oauth(expires) { + return { + type: "oauth", + access: "DUMMY-ACCESS", + refresh: "DUMMY-REFRESH", + expires, + }; +} + +for (const [name, expires] of [ + ["expired", 0], + ["near-expiry", 1_060_000], +]) { + test(`${name} OAuth is rejected before refresh callback and persistence`, async () => { + await withCredential(oauth(expires), async ({ path }) => { + const before = await readFile(path); + const store = await Cp06ReadOnlyCredentialStore.load({ path, providerId }); + assert.throws( + () => store.assertUsableOAuth({ now: 1_000_000, minValidityMs: 300_000 }), + Cp06AuthBlockedError, + ); + let callbackCalls = 0; + await assert.rejects( + store.modify(providerId, async () => { + callbackCalls += 1; + return oauth(9_999_999); + }), + Cp06AuthBlockedError, + ); + assert.equal(callbackCalls, 0); + assert.deepEqual(await readFile(path), before); + assert.deepEqual(store.audit(), { + reads: 0, + lists: 0, + modify_denials: 1, + delete_denials: 0, + }); + }); + }); +} + +test("unexpired OAuth resolves from a clone without mutation", async () => { + await withCredential(oauth(2_000_000), async ({ path }) => { + const before = await readFile(path); + const store = await Cp06ReadOnlyCredentialStore.load({ path, providerId }); + const info = store.assertUsableOAuth({ now: 1_000_000, minValidityMs: 300_000 }); + assert.equal(info.remainingValidityMs, 1_000_000); + const credential = await store.read(providerId); + credential.access = "DUMMY-MUTATED-CLONE"; + assert.equal((await store.read(providerId)).access, "DUMMY-ACCESS"); + assert.equal(await store.read("ignored-provider"), undefined); + assert.deepEqual(await store.list(), [{ providerId, type: "oauth" }]); + assert.deepEqual(await readFile(path), before); + }); +}); + +test("delete is denied without persistence", async () => { + await withCredential(oauth(2_000_000), async ({ path }) => { + const before = await readFile(path); + const store = await Cp06ReadOnlyCredentialStore.load({ path, providerId }); + await assert.rejects(store.delete(providerId), Cp06AuthBlockedError); + assert.deepEqual(await readFile(path), before); + assert.equal(store.audit().delete_denials, 1); + }); +}); + +test("credential store rejects API keys, malformed OAuth, symlinks, and oversized files", async () => { + await withCredential({ type: "api_key", key: "DUMMY" }, async ({ path, root }) => { + await assert.rejects( + Cp06ReadOnlyCredentialStore.load({ path, providerId }), + Cp06AuthBlockedError, + ); + const link = join(root, "DUMMY-link.json"); + await symlink(path, link); + await assert.rejects( + Cp06ReadOnlyCredentialStore.load({ path: link, providerId }), + Cp06AuthBlockedError, + ); + const large = join(root, "DUMMY-large.json"); + await writeFile(large, "x".repeat(1025)); + await assert.rejects( + Cp06ReadOnlyCredentialStore.load({ path: large, providerId, maxBytes: 1024 }), + Cp06AuthBlockedError, + ); + }); + + await withCredential( + { type: "oauth", access: "DUMMY", refresh: "DUMMY", expires: "later" }, + async ({ path }) => { + await assert.rejects( + Cp06ReadOnlyCredentialStore.load({ path, providerId }), + Cp06AuthBlockedError, + ); + }, + ); +}); diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs index 6e484da..174f870 100644 --- a/test/cp06-worker-audit.test.mjs +++ b/test/cp06-worker-audit.test.mjs @@ -78,6 +78,21 @@ for (const [name, events] of [ ], ["completion before start", [successfulEnd(), start, finalMessage]], ["response before completion", [start, finalMessage, successfulEnd()]], + [ + "orphan non-read completion", + [ + start, + successfulEnd(), + { + type: "tool_execution_end", + toolName: "bash", + toolCallId: "orphan-bash", + isError: false, + result: { content: [{ type: "text", text: "unexpected" }] }, + }, + finalMessage, + ], + ], ]) { test(`worker read audit rejects ${name}`, () => { assert.equal(auditReadEvents(events, options).ok, false); From b835733794d5f9bc845171cb45c891901c80f305 Mon Sep 17 00:00:00 2001 From: Firstmate CP06 Validator Date: Wed, 30 Sep 2026 18:16:21 +0000 Subject: [PATCH 13/45] no-mistakes(review): Clarify CP-06 WSL2 proof limits --- docs/probes/CP-06.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 7a5ebf8..314c271 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -19,7 +19,7 @@ daemon. ## Read-only Pi authentication correction -The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Unsupported Linux x86-64 user/mount namespaces, libseccomp, helper compilation, Pi version, or a cleanup failure block; there is no unsandboxed fallback. Windows and macOS are not claimed. +The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and exactly the built-in `read` tool. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact hashes/counts and parsed output, never credential contents or raw events. From 2ec7c2682c87d9b60a5e812d17ec0fba73978f61 Mon Sep 17 00:00:00 2001 From: Firstmate CP06 Validator Date: Wed, 30 Sep 2026 18:21:49 +0000 Subject: [PATCH 14/45] no-mistakes(review): Cover source destructive isolation attacks --- scripts/cp06-auth-security.mjs | 5 +++++ scripts/cp06-isolation-adversary.mjs | 11 +++++++++++ 2 files changed, 16 insertions(+) diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index edb8805..6a60cf2 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -231,9 +231,13 @@ function assertHardenedProof(proof) { "direct_write_target", "direct_write_source", "unlink_target", + "unlink_source", "rename_over_target", + "rename_over_source", "symlink_replacement", + "symlink_source_replacement", "hardlink_target", + "hardlink_source", "nested_user_mount_namespace", "nsenter_self", "nsenter_supervisor", @@ -245,6 +249,7 @@ function assertHardenedProof(proof) { ]) { assert(proof.child[name].exit !== 0, `${name} unexpectedly succeeded`); } + assert(proof.child.source.unchanged === true, "hardened child source hash changed"); assert(proof.child.alias_underlying_write.skipped === true, "alias write was attempted"); assert(proof.child.ordinary_node_child.exit === 0, "ordinary child process was blocked"); assert( diff --git a/scripts/cp06-isolation-adversary.mjs b/scripts/cp06-isolation-adversary.mjs index e8bac4f..c50fe10 100644 --- a/scripts/cp06-isolation-adversary.mjs +++ b/scripts/cp06-isolation-adversary.mjs @@ -49,16 +49,27 @@ function commandAttempt(name, command, args, options = {}) { jsAttempt("direct_write_target", () => writeFileSync(target, "DUMMY BYPASS target")); jsAttempt("direct_write_source", () => writeFileSync(source, "DUMMY BYPASS source")); jsAttempt("unlink_target", () => unlinkSync(target)); +jsAttempt("unlink_source", () => unlinkSync(source)); const replacement = join(dirname(target), "DUMMY-replacement"); writeFileSync(replacement, "DUMMY replacement"); jsAttempt("rename_over_target", () => renameSync(replacement, target)); +const sourceReplacement = join(dirname(source), "DUMMY-source-replacement"); +writeFileSync(sourceReplacement, "DUMMY source replacement"); +jsAttempt("rename_over_source", () => renameSync(sourceReplacement, source)); jsAttempt("symlink_replacement", () => { try { unlinkSync(target); } catch {} symlinkSync("/tmp/DUMMY-does-not-exist", target); }); +jsAttempt("symlink_source_replacement", () => { + try { + unlinkSync(source); + } catch {} + symlinkSync("/tmp/DUMMY-does-not-exist", source); +}); jsAttempt("hardlink_target", () => linkSync(target, join(proofRoot, "DUMMY-hardlink"))); +jsAttempt("hardlink_source", () => linkSync(source, join(proofRoot, "DUMMY-source-hardlink"))); if (securityMode === "hardened") { commandAttempt("direct_namespace_syscalls", syscallProbe, [target]); From 2e36a8f355252c307f5770a7fbcd40cb271c9e19 Mon Sep 17 00:00:00 2001 From: Firstmate CP06 Validator Date: Wed, 30 Sep 2026 18:36:53 +0000 Subject: [PATCH 15/45] no-mistakes(test): Restore offline CTX model lookup --- scripts/replay-cp06-ten-pack.mjs | 46 +++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index 1014a32..1353a7c 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -1,6 +1,6 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; -import { readFileSync, writeFileSync } from "node:fs"; +import { existsSync, readFileSync, writeFileSync } from "node:fs"; import { cp, mkdir, readFile, rm, stat, writeFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; @@ -14,6 +14,7 @@ const fixtureRoot = join(proofRoot, "repo"); const rawRoot = join(proofRoot, "raw"); const cli = join(root, "dist/src/cli.js"); const ctx = resolveTool(process.env.CTX_BIN ?? "ctx"); +configureOfflineCtxModelDir(); const reviewer = process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker"; const commands = []; @@ -500,6 +501,49 @@ console.log( }), ); +function configureOfflineCtxModelDir() { + if (process.env.CTX_MODEL_DIR !== undefined && process.env.CTX_MODEL_DIR !== "") return; + const modelName = "BAAI/bge-small-en-v1.5"; + const revision = "52398278842ec682c6f32300af41344b1c0b0bb2"; + for (const candidate of candidateModelRoots()) { + const modelDirectory = join(candidate, modelKey(modelName, revision)); + const manifestPath = join(modelDirectory, "manifest.json"); + if (!existsSync(manifestPath)) continue; + try { + const manifest = JSON.parse(readFileSync(manifestPath, "utf8")); + if ( + manifest.provider === "fastembed" && + manifest.model_name === modelName && + manifest.revision === revision && + manifest.dimensions === 384 + ) { + process.env.CTX_MODEL_DIR = candidate; + return; + } + } catch { + // Keep the proof fail-closed; the later CTX semantic assertion reports the blocker. + } + } +} + +function candidateModelRoots() { + const candidates = []; + const home = process.env.HOME; + if (home !== undefined && home !== "") candidates.push(join(home, ".cache/ctx/models")); + const homeMatch = root.match(/^\/(?:home|Users)\/[^/]+/u); + if (homeMatch !== null) candidates.push(join(homeMatch[0], ".cache/ctx/models")); + return [...new Set(candidates)]; +} + +function modelKey(modelName, revision) { + const readable = modelName.replace(/[^A-Za-z0-9._-]+/g, "--").replace(/^-|-$/g, ""); + const digest = createHash("sha256") + .update(`${modelName}\0${revision}`) + .digest("hex") + .slice(0, 12); + return `${readable}-${digest}`; +} + function factoryContext(id, taskId, expectedExit) { const result = run( id, From 3abaa4362fe6b829d2e0d66f014e727530c58669 Mon Sep 17 00:00:00 2001 From: Firstmate CP06 Validator Date: Wed, 30 Sep 2026 18:42:02 +0000 Subject: [PATCH 16/45] no-mistakes(document): Document CP-06 auth-security proof --- docs/probes/CP-06.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 314c271..6d0ca5a 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -94,6 +94,7 @@ npm run format:check npm run lint npm run typecheck npm test +npm run proof:cp06:auth-security npm run proof:cp06:ten-pack npm run proof:cp06:p07 npm run proof:cp06:worker @@ -103,6 +104,7 @@ git diff --check The exact runtime manifests are: ```text +.factory/state/cp06-correction/auth-security/evidence.json .factory/state/cp06-correction/ten-pack/evidence.json .factory/state/cp06-correction/p07/evidence.json .factory/state/cp06-correction/worker/evidence.json From 33b2724e3316272aac1c563086d2ce5e77d496ca Mon Sep 17 00:00:00 2001 From: Firstmate CP06 Validator Date: Wed, 30 Sep 2026 18:48:31 +0000 Subject: [PATCH 17/45] no-mistakes(review): Enforce exact original read arguments --- scripts/cp06-worker-audit.mjs | 6 +++--- test/cp06-worker-audit.test.mjs | 12 ++++++++++++ 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs index ebe1126..782be9f 100644 --- a/scripts/cp06-worker-audit.mjs +++ b/scripts/cp06-worker-audit.mjs @@ -75,11 +75,11 @@ export function auditReadEvents(events, options) { if (typeof callId !== "string" || callId.length === 0) { return reject("original read start omitted toolCallId"); } - if (start.event.args?.offset !== undefined && start.event.args.offset !== 1) { - return reject("original read started outside the first line"); + if (start.event.args?.offset !== undefined) { + return reject("original read used an explicit offset"); } if (start.event.args?.limit !== undefined) { - return reject("original read used a partial line limit"); + return reject("original read used an explicit limit"); } const matchingEnds = readEnds.filter(({ event }) => event.toolCallId === callId); diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs index 174f870..8a8bf47 100644 --- a/test/cp06-worker-audit.test.mjs +++ b/test/cp06-worker-audit.test.mjs @@ -55,6 +55,18 @@ for (const [name, events] of [ finalMessage, ], ], + [ + "explicit first-line offset with complete bytes", + [{ ...start, args: { path: originalPath, offset: 1 } }, successfulEnd(), finalMessage], + ], + [ + "explicit limit with complete bytes", + [ + { ...start, args: { path: originalPath, limit: Number.MAX_SAFE_INTEGER } }, + successfulEnd(), + finalMessage, + ], + ], [ "wrong starting range", [ From 641d7ca7698dee6badf4b9ee34e1c20c92668b58 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Thu, 1 Oct 2026 12:40:03 +0000 Subject: [PATCH 18/45] fix(cp06): enforce worker outcome and cleanup --- BUILD_PLAN.md | 2 +- docs/probes/CP-06.md | 63 +++++- package.json | 1 + scripts/cp06-auth-security.mjs | 13 ++ scripts/cp06-dummy-outcome-proof.mjs | 128 ++++++++++++ scripts/cp06-dummy-sdk-proof.mjs | 5 + scripts/cp06-dummy-worker-hook.mjs | 90 +++++++++ scripts/cp06-namespace-supervisor.mjs | 184 ++++++++++------- scripts/cp06-sdk-worker.mjs | 27 +-- scripts/cp06-worker-lifecycle.mjs | 34 ++++ scripts/cp06-worker-outcome.mjs | 37 ++++ scripts/prove-cp06-worker-cleanup.mjs | 139 +++++++++++++ scripts/replay-cp06-worker.mjs | 277 ++++++++++++++------------ test/cp06-worker-lifecycle.test.mjs | 116 +++++++++++ test/cp06-worker-outcome.test.mjs | 69 +++++++ 15 files changed, 960 insertions(+), 225 deletions(-) create mode 100644 scripts/cp06-dummy-outcome-proof.mjs create mode 100644 scripts/cp06-dummy-worker-hook.mjs create mode 100644 scripts/cp06-worker-lifecycle.mjs create mode 100644 scripts/cp06-worker-outcome.mjs create mode 100644 scripts/prove-cp06-worker-cleanup.mjs create mode 100644 test/cp06-worker-lifecycle.test.mjs create mode 100644 test/cp06-worker-outcome.test.mjs diff --git a/BUILD_PLAN.md b/BUILD_PLAN.md index 4750e78..34f77af 100644 --- a/BUILD_PLAN.md +++ b/BUILD_PLAN.md @@ -89,7 +89,7 @@ - [x] Create tests/fixtures for critical auth isolation and normal API behavior, plus one simulated reviewer finding that causes repair and retest. - [x] Run an actual worker task with only task/pack/evidence sources in a fresh context; inspect whether it needs missing docs. -- [x] **Gate:** P-06 and P-07 via the tracked ten-pack, protocol-negative/fresh-worker, and P-07 replays in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. +- [ ] **Gate:** P-06 and P-07 via the tracked ten-pack, protocol-negative/fresh-worker, and P-07 replays in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. PR11 independent review rejected its A1 outcome and C1 cleanup claims; the correction requires final-head proofs and another independent review. DUMMY SDK controls are not live semantic acceptance, and strict-agent PARTIAL retrieval is not COMPLETE. ## CP-07 — Thin Pi command and status convenience diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 6d0ca5a..c5c574d 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -214,6 +214,64 @@ is removed and no raw transcript is retained. Native Windows/macOS isolation is unsupported rather than claimed. Model wording is nondeterministic; the protocol, exact bytes, and isolation assertions are not. +## PR11 A1/C1 correction and proof boundary + +The independent review of PR11 at `33b2724e3316272aac1c563086d2ce5e77d496ca` +reproduced two additional failures: a local faux SDK response with +`evidence_gaps: []` was accepted, and an expired-auth exit skipped the outer +runner's `finally`, leaving its evaluation directory and empty mount placeholder. +Neither finding demonstrated credential mutation; the DUMMY S1/S2 controls passed. + +The worker now requires an executable structured creation-evidence disposition: + +```json +{ + "evidence_gaps": [ + { + "id": "creation-time-ownership", + "status": "unverified", + "missing_checks": ["principal-derived-owner", "request-owner-ignored"] + } + ] +} +``` + +The check order is immaterial, but both checks must remain unverified. Missing, +empty, duplicate, wrong, or contradictory dispositions fail; recovering a +requirement by reading its original is not a passing behavior test. Extra fields +or prose in this gap are rejected rather than interpreted as an alternate +verification channel. The existing exact-original read and source-scope oracle +remain mandatory. + +`npm run proof:cp06:auth-security` additionally runs the actual SDK worker with a +local faux provider in the inherited capability-free/seccomp context. It records +a valid structured positive, missing/empty/contradictory gap negatives, expired +and near-expiry preflight denial, and an SDK timeout. Every case is explicitly +`fixture_origin: true`, `semantic_acceptance: false`; these are protocol and +infrastructure proofs, never live-model acceptance. Default AuthStorage, refresh, +and network entry are instrumented and must remain zero. The full read still +uses the actual built-in SDK read tool, not a fabricated completion event. + +The runner and namespace supervisor await cleanup before setting an exit code. +When execution and cleanup both fail, the cleanup error retains the primary +cause and its exit identity. `npm run proof:cp06:worker-cleanup` executes the full +runner with task-local expired/near-expiry DUMMY auth and missing-source setup, +asserts the blocked result, unchanged DUMMY source, absence of a pass manifest, +and removal of the evaluation home. Unit tests inject setup, launch, timeout, +audit and cleanup failures at the executable supervisor boundary, including +compound failure; they are distinct from actual kernel/SDK proof. No real +credential read, mount, refresh or model request is needed for these regressions. + +The strict-agent CTX audit is a separate qualification. PR11's ordinary Factory +STANDARD/HYBRID_SEMANTIC sample passed 10/10 with zero irrelevant provenance; +replaying those selections with `--strict-agent --require-semantic` reported ten +PARTIAL results because strict cross-document discovery required filtered-out +normative entries. This is not strict COMPLETE evidence. Do not widen filters, +hide omissions, or label the DUMMY worker as semantic acceptance to cure it. +Final-head manifests must be regenerated after committing the correction. A real +fresh-worker run and independent acceptance remain blocked/unverified unless +separately performed safely; the prior PR11 report is not approval of this head. + ## P-07 finding, repair, and retest `test/fixtures/workflow/history/finding.json` is the minimal structured reviewer @@ -245,8 +303,9 @@ no-mistakes nor changes its ownership boundary. ## Acceptance and residual limitations -P-06 passes for the corrected, predeclared ten-pack sample; the original PR #8 -sample remains explicitly failed. P-07 passes because the structured finding, +P-06's STANDARD ten-pack subgate passes when the corrected, predeclared sample +is replayed successfully; this is not the full fresh-worker or independent +CP-06 gate. The original PR #8 sample remains explicitly failed. P-07 passes because the structured finding, fail-before, repair, final behavior retest, and auth retest are executable and bound to the replayed SHA. CP-06 makes no change to Firstmate dispatch, delivery, or no-mistakes policy. diff --git a/package.json b/package.json index 74c9862..8b04e9a 100644 --- a/package.json +++ b/package.json @@ -17,6 +17,7 @@ "proof:cp06:p07": "node scripts/replay-cp06-p07.mjs", "proof:cp06:auth-security": "node scripts/prove-cp06-auth-security.mjs", "proof:cp06:worker": "node scripts/replay-cp06-worker.mjs", + "proof:cp06:worker-cleanup": "node scripts/prove-cp06-worker-cleanup.mjs", "factory": "node dist/src/cli.js" }, "engines": { diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 6a60cf2..7e7e6fd 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -146,6 +146,19 @@ export async function runIsolatedWorker(options) { error.code = result.status === 73 ? "CP06_ISOLATION_UNSUPPORTED" : "CP06_WORKER_FAILED"; error.exitCode = result.status ?? 70; error.stderr = result.stderr; + // Preserve the supervisor's compact failure identity, including both causes + // when child execution and mount cleanup fail. Do not persist raw stderr. + try { + const failure = JSON.parse(String(result.stderr).trim()); + if (failure.schema_version === 1 && failure.exit_code === result.status) { + error.workerFailure = { + code: failure.code, + exit_code: failure.exit_code, + causes: failure.causes, + }; + if (failure.code === "CP06_CLEANUP_FAILED") error.code = failure.code; + } + } catch {} throw error; } return parseCompactJson(result.stdout, "isolated SDK worker"); diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs new file mode 100644 index 0000000..6ae6c58 --- /dev/null +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -0,0 +1,128 @@ +import { strict as assert } from "node:assert"; +import { spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { withCleanup } from "./cp06-worker-lifecycle.mjs"; + +/** Exercise the actual SDK worker executable, with only a local faux provider. */ +export async function proveDummyWorkerOutcome({ piRoot }) { + const root = resolve(fileURLToPath(new URL("..", import.meta.url))); + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-outcome-")); + return withCleanup( + async () => { + await mkdir(join(directory, "docs")); + const originalPath = join(directory, "docs/AUTH.md"); + await writeFile( + originalPath, + "# DUMMY authority\n\nOwnership comes only from the authenticated principal; request-supplied owner fields are ignored.\n", + ); + const contractPath = join(directory, "DUMMY-contract.yaml"); + const packPath = join(directory, "DUMMY-pack.md"); + await writeFile( + contractPath, + "DUMMY critical AUTH ownership task; creation evidence is unverified\n", + ); + await writeFile( + packPath, + "DUMMY globals: offline, preserve tenant privacy, no secrets in context\n", + ); + const cases = []; + for (const [scenario, expectedExit] of [ + ["valid", 0], + ["empty-gap", 75], + ["missing-gap", 75], + ["contradictory-gap", 75], + ["expired", 75], + ["near-expiry", 75], + ["timeout", 70], + ]) { + const auth = join(directory, `DUMMY-${scenario}-auth.json`); + const counters = join(directory, `DUMMY-${scenario}-counters.json`); + const inputPath = join(directory, `DUMMY-${scenario}-input.json`); + const authBytes = JSON.stringify({ + "openai-codex": { + type: "oauth", + access: "DUMMY-ACCESS", + refresh: "DUMMY-REFRESH", + expires: + scenario === "expired" + ? 0 + : Date.now() + (scenario === "near-expiry" ? 60_000 : 3_600_000), + }, + }); + await writeFile(auth, authBytes); + await writeFile( + inputPath, + JSON.stringify({ + schema_version: 1, + root: directory, + fixture_root: directory, + contract_path: contractPath, + pack_path: packPath, + original_path: originalPath, + provider: "openai-codex", + model: "DUMMY-model", + timeout_ms: scenario === "timeout" ? 1_000 : 20_000, + system_prompt: "DUMMY ONLY local exact-original read and structured evidence gap", + }), + ); + const result = spawnSync( + process.execPath, + [ + "--import", + join(root, "scripts/cp06-dummy-worker-hook.mjs"), + join(root, "scripts/cp06-sdk-worker.mjs"), + auth, + inputPath, + ], + { + cwd: root, + encoding: "utf8", + timeout: 40_000, + maxBuffer: 1024 * 1024, + env: { + PATH: process.env.PATH, + HOME: directory, + TMPDIR: directory, + PI_OFFLINE: "1", + PI_SKIP_VERSION_CHECK: "1", + CP06_DUMMY_PI_ROOT: piRoot, + CP06_DUMMY_COUNTERS: counters, + CP06_DUMMY_SCENARIO: scenario, + }, + }, + ); + assert.equal(result.error, undefined); + assert.equal(result.status, expectedExit, `${scenario}: ${result.stderr}`); + const counts = JSON.parse(await readFile(counters, "utf8")); + assert.equal(counts.default_storage, 0); + assert.equal(counts.network, 0); + assert.equal(counts.refresh, 0); + assert.equal(await readFile(auth, "utf8"), authBytes); + if (["expired", "near-expiry"].includes(scenario)) assert.equal(counts.runtime_create, 0); + if (["empty-gap", "missing-gap", "contradictory-gap"].includes(scenario)) + assert.match(result.stderr, /SDK worker outcome failed/); + if (scenario === "timeout") assert.match(result.stderr, /worker timed out and was aborted/); + const output = scenario === "valid" ? JSON.parse(result.stdout) : undefined; + if (output) { + assert.equal(output.result, "pass"); + assert.equal(output.event_stream.read_audit.exact_original, true); + assert.equal(output.event_stream.read_audit.read_count, 1); + assert.equal(output.response.evidence_gaps[0].status, "unverified"); + } else assert.equal(result.stdout, ""); + cases.push({ + scenario, + exit_code: result.status, + expected_exit_code: expectedExit, + ...counts, + source_unchanged: true, + read_audit: output?.event_stream.read_audit, + }); + } + return { fixture_origin: true, semantic_acceptance: false, cases }; + }, + () => rm(directory, { recursive: true, force: true }), + ); +} diff --git a/scripts/cp06-dummy-sdk-proof.mjs b/scripts/cp06-dummy-sdk-proof.mjs index c5611d4..0f86f70 100644 --- a/scripts/cp06-dummy-sdk-proof.mjs +++ b/scripts/cp06-dummy-sdk-proof.mjs @@ -7,6 +7,7 @@ import { join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; import { auditReadEvents } from "./cp06-worker-audit.mjs"; +import { proveDummyWorkerOutcome } from "./cp06-dummy-outcome-proof.mjs"; const [piRootArg] = process.argv.slice(2); if (!piRootArg) process.exit(64); @@ -32,6 +33,7 @@ try { authCases.push(await authCase(scenario)); } const worker = await fauxWorkerProof(); + const outcomeProof = await proveDummyWorkerOutcome({ piRoot }); assert.equal(networkCalls, 0); console.log( JSON.stringify({ @@ -39,6 +41,9 @@ try { result: "pass", auth_cases: authCases, sdk_worker: worker, + actual_sdk_outcomes: outcomeProof, + fixture_origin: true, + semantic_acceptance: false, network_calls: networkCalls, }), ); diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs new file mode 100644 index 0000000..b41a254 --- /dev/null +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -0,0 +1,90 @@ +// Development-only local provider instrumentation. Never a live semantic proof. +import { readFileSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { pathToFileURL } from "node:url"; + +const piRoot = process.env.CP06_DUMMY_PI_ROOT; +const countersPath = process.env.CP06_DUMMY_COUNTERS; +const scenario = process.env.CP06_DUMMY_SCENARIO; +if (!piRoot || !countersPath || !scenario) throw new Error("DUMMY proof configuration missing"); +const sdk = await import(pathToFileURL(join(piRoot, "dist/index.js"))); +const { AuthStorage } = await import(pathToFileURL(join(piRoot, "dist/core/auth-storage.js"))); +const ai = await import( + pathToFileURL(join(piRoot, "node_modules/@earendil-works/pi-ai/dist/index.js")) +); +const input = JSON.parse(readFileSync(process.argv[3], "utf8")); +const counts = { + fixture_origin: true, + default_storage: 0, + network: 0, + refresh: 0, + to_auth: 0, + runtime_create: 0, +}; +AuthStorage.create = () => { + counts.default_storage++; + throw new Error("DUMMY forbids default AuthStorage"); +}; +globalThis.fetch = async () => { + counts.network++; + throw new Error("DUMMY forbids network"); +}; +const create = sdk.ModelRuntime.create; +sdk.ModelRuntime.create = async function (options) { + counts.runtime_create++; + if ( + !options.credentials || + options.modelsPath !== null || + options.refreshOnCreate !== false || + options.allowModelNetwork !== false + ) + throw new Error("DUMMY unsafe runtime options"); + const runtime = await create.call(this, options); + const faux = ai.fauxProvider({ + provider: "openai-codex", + models: [{ id: "DUMMY-model" }], + tokensPerSecond: scenario === "timeout" ? 1 : 100_000, + }); + faux.provider.auth = { + oauth: { + async refresh() { + counts.refresh++; + throw new Error("DUMMY refresh forbidden"); + }, + async toAuth(credential) { + counts.to_auth++; + if (credential.access !== "DUMMY-ACCESS") throw new Error("DUMMY credential required"); + return { apiKey: "DUMMY-LOCAL" }; + }, + }, + }; + runtime.registerNativeProvider(faux.provider); + const gap = { + id: "creation-time-ownership", + status: "unverified", + missing_checks: ["principal-derived-owner", "request-owner-ignored"], + }; + const response = { + status: "MISSING_SOURCE", + missing_source: "docs/AUTH.md", + reads: [input.original_path], + corrected_constraints: [ + "ownership comes only from the authenticated principal; request-supplied owner fields are ignored", + ], + evidence_gaps: + scenario === "empty-gap" + ? [] + : [{ ...gap, ...(scenario === "contradictory-gap" ? { status: "pass" } : {}) }], + broad_scan: false, + }; + if (scenario === "missing-gap") delete response.evidence_gaps; + faux.setResponses([ + ai.fauxAssistantMessage( + ai.fauxToolCall("read", { path: input.original_path }, { id: "DUMMY-actual-worker-read" }), + { stopReason: "toolUse" }, + ), + ai.fauxAssistantMessage(JSON.stringify(response), { stopReason: "stop" }), + ]); + return runtime; +}; +process.on("exit", (exit) => writeFileSync(countersPath, JSON.stringify({ ...counts, exit }))); diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 235e143..9a8fbe7 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -3,92 +3,121 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; import { readFileSync } from "node:fs"; import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; import { mountLiveCredentialReadOnly } from "./cp06-credential-isolation.mjs"; +import { withCleanup } from "./cp06-worker-lifecycle.mjs"; -const [mode, sourceArg, targetArg, helperArg, syscallProbeArg, ...childArgs] = - process.argv.slice(2); -if ( - !["unsafe-probe", "hardened-probe", "worker"].includes(mode) || - !sourceArg || - !targetArg || - !helperArg || - !syscallProbeArg -) { - process.stderr.write("invalid CP-06 namespace supervisor invocation\n"); - process.exit(64); -} -if (process.platform !== "linux" || process.arch !== "x64") { - process.stderr.write("CP-06 isolation supports Linux x86-64 only\n"); - process.exit(73); -} +const root = resolve(fileURLToPath(new URL("..", import.meta.url))); -const root = resolve(new URL("..", import.meta.url).pathname); -const source = resolve(sourceArg); -const target = resolve(targetArg); -const helper = resolve(helperArg); -const syscallProbe = resolve(syscallProbeArg); -const beforeHash = sha256(readFileSync(source)); -let mount; -let childResult; -let cleanupError; -try { - mount = await mountLiveCredentialReadOnly({ root, source, target }); - const command = childCommand(); - childResult = spawnSync(command.command, command.args, { - cwd: root, - encoding: "utf8", - env: childEnvironment(mode, target), - timeout: mode === "worker" ? 600_000 : 60_000, - maxBuffer: 1024 * 1024, - }); -} catch (error) { - process.stderr.write(`CP-06 isolation setup failed: ${safeError(error)}\n`); - process.exit(73); -} finally { - try { - mount?.cleanup(); - } catch (error) { - cleanupError = error; +// Injectable OS operations permit deterministic failure/cleanup tests without +// credentials, namespaces or an installed SDK in the ordinary unit suite. +export async function superviseCredentialChild(options, dependencies = {}) { + const { mode, source, target, helper, syscallProbe, childArgs = [] } = options; + if ( + !["unsafe-probe", "hardened-probe", "worker"].includes(mode) || + ![source, target, helper, syscallProbe].every( + (value) => typeof value === "string" && value.length > 0, + ) + ) + throw failure("invalid CP-06 namespace supervisor invocation", 64); + if (process.platform !== "linux" || process.arch !== "x64") { + throw failure("CP-06 isolation supports Linux x86-64 only", 73); } -} -const afterHash = sha256(readFileSync(source)); -if (cleanupError !== undefined) { - process.stderr.write(`CP-06 isolation cleanup failed: ${safeError(cleanupError)}\n`); - process.exit(74); -} -if (childResult?.error !== undefined) { - process.stderr.write(`CP-06 isolated child launch failed: ${safeError(childResult.error)}\n`); - process.exit(70); -} -if (childResult?.status !== 0) { - process.stderr.write(childResult?.stderr ?? ""); - process.stderr.write(`CP-06 isolated child exited ${childResult?.status ?? "without status"}\n`); - process.exit(childResult?.status ?? 70); -} -if (mode === "worker" && beforeHash !== afterHash) { - process.stderr.write("CP-06 credential source changed during worker execution\n"); - process.exit(74); -} + const mountCredential = dependencies.mount ?? mountLiveCredentialReadOnly; + const spawn = dependencies.spawn ?? spawnSync; + const beforeHash = sha256(readFileSync(source)); + let mount; + const childResult = await withCleanup( + async () => { + try { + mount = await mountCredential({ root, source, target }); + } catch (cause) { + throw failure(`CP-06 isolation setup failed: ${safeError(cause)}`, 73, cause); + } + const command = childCommand({ + mode, + source, + target, + helper, + syscallProbe, + childArgs, + beforeHash, + }); + const result = spawn(command.command, command.args, { + cwd: root, + encoding: "utf8", + env: childEnvironment(mode, target), + timeout: mode === "worker" ? 600_000 : 60_000, + maxBuffer: 1024 * 1024, + }); + // Throw inside the cleanup boundary so a cleanup error retains this cause. + if (result.error !== undefined) { + throw failure( + `CP-06 isolated child launch failed: ${safeError(result.error)}`, + 70, + result.error, + ); + } + if (result.status !== 0) { + throw failure( + `CP-06 isolated child exited ${result.status ?? "without status"}: ${String(result.stderr ?? "").trim()}`, + result.status ?? 70, + ); + } + return result; + }, + () => mount?.cleanup(), + ); -let childEvidence; -try { - childEvidence = JSON.parse(String(childResult.stdout).trim()); -} catch { - process.stderr.write("CP-06 isolated child returned non-compact output\n"); - process.exit(70); -} -console.log( - JSON.stringify({ + const afterHash = sha256(readFileSync(source)); + if (mode === "worker" && beforeHash !== afterHash) { + throw failure("CP-06 credential source changed during worker execution", 74); + } + let childEvidence; + try { + childEvidence = JSON.parse(String(childResult.stdout).trim()); + } catch { + throw failure("CP-06 isolated child returned non-compact output", 70); + } + return { schema_version: 1, mode, source_unchanged: beforeHash === afterHash, mount_ids_distinct: mount.mountIds.source !== mount.mountIds.target, cleanup: "pass", child: childEvidence, - }), -); + }; +} -function childCommand() { +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + const [mode, source, target, helper, syscallProbe, ...childArgs] = process.argv.slice(2); + try { + console.log( + JSON.stringify( + await superviseCredentialChild({ mode, source, target, helper, syscallProbe, childArgs }), + ), + ); + } catch (error) { + process.stderr.write( + `${JSON.stringify({ + schema_version: 1, + code: error.code ?? "CP06_WORKER_FAILED", + exit_code: error.exitCode ?? 70, + message: safeError(error), + causes: + error instanceof AggregateError + ? error.errors.map((cause) => ({ + code: cause?.code ?? "CP06_WORKER_FAILED", + exit_code: cause?.exitCode ?? null, + })) + : [], + })}\n`, + ); + process.exitCode = error.exitCode ?? 70; + } +} + +function childCommand({ mode, source, target, helper, syscallProbe, childArgs, beforeHash }) { if (mode === "unsafe-probe") { return { command: process.execPath, @@ -156,11 +185,16 @@ function childEnvironment(childMode, credentialTarget) { return environment; } +function failure(message, exitCode, cause) { + return Object.assign(new Error(message, { cause }), { exitCode, code: "CP06_WORKER_FAILED" }); +} + function sha256(bytes) { return createHash("sha256").update(bytes).digest("hex"); } function safeError(error) { - if (error instanceof AggregateError) return error.errors.map(safeError).join("; "); + if (error instanceof AggregateError) + return `${error.message}: ${error.errors.map(safeError).join("; ")}`; return error instanceof Error ? error.message : String(error); } diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs index c75d496..29c5e33 100644 --- a/scripts/cp06-sdk-worker.mjs +++ b/scripts/cp06-sdk-worker.mjs @@ -6,7 +6,8 @@ import { readFile } from "node:fs/promises"; import { delimiter, dirname, join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { Cp06AuthBlockedError, Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; -import { auditReadEvents, isExpectedOriginalReference } from "./cp06-worker-audit.mjs"; +import { auditReadEvents } from "./cp06-worker-audit.mjs"; +import { auditWorkerOutcome } from "./cp06-worker-outcome.mjs"; const [credentialTargetArg, inputPathArg] = process.argv.slice(2); if (!credentialTargetArg || !inputPathArg) process.exit(64); @@ -104,22 +105,12 @@ try { if (!readAudit.ok) { throw blocked(`SDK event audit failed: ${readAudit.reason}`); } - const corrected = Array.isArray(response.corrected_constraints) - ? response.corrected_constraints.join(" ") - : ""; - if ( - response.status !== "MISSING_SOURCE" || - !isExpectedOriginalReference(response.missing_source, { - root: input.root, - referenceRoot: input.fixture_root, - expectedOriginalPath: input.original_path, - }) || - response.broad_scan !== false || - !/ownership comes only from the authenticated principal/i.test(corrected) || - !/request-supplied owner fields? (?:are )?ignored/i.test(corrected) - ) { - throw blocked("SDK worker response did not satisfy the targeted-read oracle"); - } + const outcome = auditWorkerOutcome(response, { + root: input.root, + referenceRoot: input.fixture_root, + expectedOriginalPath: input.original_path, + }); + if (!outcome.ok) throw blocked(`SDK worker outcome failed: ${outcome.reason}`); console.log( JSON.stringify({ @@ -244,7 +235,7 @@ function emptyResourceLoader(sdk, systemPrompt) { } function buildPrompt({ contract, pack, originalPath }) { - return `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE, set missing_source to the exact canonical relative path docs/AUTH.md, and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; + return `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE, set missing_source to the exact canonical relative path docs/AUTH.md, and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. The evidence_gaps field must be exactly [{"id":"creation-time-ownership","status":"unverified","missing_checks":["principal-derived-owner","request-owner-ignored"]}]: the supplied deterministic evidence does not test those creation-time behaviors. Recovering a requirement by reading is not evidence that the behavior passed. Do not add a passed/verified disposition or prose fields to this structured gap. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; } function finalAssistantText(events) { diff --git a/scripts/cp06-worker-lifecycle.mjs b/scripts/cp06-worker-lifecycle.mjs new file mode 100644 index 0000000..db2cf8e --- /dev/null +++ b/scripts/cp06-worker-lifecycle.mjs @@ -0,0 +1,34 @@ +export class Cp06CleanupError extends AggregateError { + constructor(primary, cleanup) { + super( + primary === undefined ? [cleanup] : [primary, cleanup], + "CP-06 evaluation cleanup failed", + { + cause: primary ?? cleanup, + }, + ); + this.code = "CP06_CLEANUP_FAILED"; + this.exitCode = 74; + } +} + +/** Await cleanup on success and failure; never lose the primary error. */ +export async function withCleanup(action, cleanup) { + let result; + let primary; + let failed = false; + try { + result = await action(); + } catch (error) { + failed = true; + primary = error; + } finally { + try { + await cleanup(); + } catch (error) { + throw new Cp06CleanupError(primary, error); + } + } + if (failed) throw primary; + return result; +} diff --git a/scripts/cp06-worker-outcome.mjs b/scripts/cp06-worker-outcome.mjs new file mode 100644 index 0000000..d9a05fd --- /dev/null +++ b/scripts/cp06-worker-outcome.mjs @@ -0,0 +1,37 @@ +import { isExpectedOriginalReference } from "./cp06-worker-audit.mjs"; + +export function auditWorkerOutcome(response, options) { + const corrected = Array.isArray(response?.corrected_constraints) + ? response.corrected_constraints.join(" ") + : ""; + if ( + response?.status !== "MISSING_SOURCE" || + !isExpectedOriginalReference(response.missing_source, options) || + response.broad_scan !== false || + !/ownership comes only from the authenticated principal/i.test(corrected) || + !/request-supplied owner fields? (?:are )?ignored/i.test(corrected) + ) { + return { ok: false, reason: "response did not satisfy the targeted-read oracle" }; + } + const gaps = response.evidence_gaps; + const gap = Array.isArray(gaps) && gaps.length === 1 ? gaps[0] : undefined; + const checks = gap?.missing_checks; + if ( + gap === null || + typeof gap !== "object" || + Array.isArray(gap) || + Object.keys(gap).sort().join(",") !== "id,missing_checks,status" || + gap.id !== "creation-time-ownership" || + gap.status !== "unverified" || + !Array.isArray(checks) || + checks.length !== 2 || + !checks.includes("principal-derived-owner") || + !checks.includes("request-owner-ignored") + ) { + return { + ok: false, + reason: "creation-time ownership evidence must remain explicitly unverified", + }; + } + return { ok: true }; +} diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs new file mode 100644 index 0000000..44b08af --- /dev/null +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -0,0 +1,139 @@ +#!/usr/bin/env node +import { strict as assert } from "node:assert"; +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { withCleanup } from "./cp06-worker-lifecycle.mjs"; + +const root = resolve(fileURLToPath(new URL("..", import.meta.url))); +const output = resolve( + root, + process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction", + "worker-cleanup", +); +await mkdir(output, { recursive: true }); +const scratch = await mkdtemp(join(output, "DUMMY-cleanup-")); +const cases = await withCleanup( + async () => { + const cases = []; + for (const [scenario, expectedExit] of [ + ["expired", 75], + ["near-expiry", 75], + ["missing-source-setup", 70], + ]) { + const home = join(scratch, scenario); + const temporary = join(home, "tmp"); + const proofOutput = join(home, "proofs"); + const source = join(home, "DUMMY-auth.json"); + await mkdir(temporary, { recursive: true }); + const sourceBytes = JSON.stringify({ + "openai-codex": { + type: "oauth", + access: "DUMMY-ACCESS", + refresh: "DUMMY-REFRESH", + expires: scenario === "expired" ? 0 : Date.now() + 60_000, + }, + }); + if (scenario !== "missing-source-setup") await writeFile(source, sourceBytes); + await mkdir(join(proofOutput, "ten-pack/raw"), { recursive: true }); + await mkdir(join(proofOutput, "ten-pack/repo/docs"), { recursive: true }); + await writeFile( + join(proofOutput, "ten-pack/raw/auth-worker-missing.pack.md"), + "DUMMY cleanup-only handoff; no semantic acceptance claimed.\n", + ); + await writeFile( + join(proofOutput, "ten-pack/repo/docs/AUTH.md"), + await readFile(join(root, "test/fixtures/cp06-context/docs/AUTH.md")), + ); + const args = [join(root, "scripts/replay-cp06-worker.mjs")]; + const result = spawnSync(process.execPath, args, { + cwd: root, + encoding: "utf8", + timeout: 180_000, + maxBuffer: 1024 * 1024, + env: { + PATH: process.env.PATH, + HOME: home, + TMPDIR: temporary, + PI_OFFLINE: "1", + PI_SKIP_VERSION_CHECK: "1", + CP06_PI_AUTH_FILE: source, + CP06_OUTPUT: proofOutput, + CP06_REVIEWER: "DUMMY cleanup regression", + }, + }); + assert.equal(result.error, undefined); + assert.equal(result.status, expectedExit, result.stderr); + const blocked = JSON.parse(await readFile(join(proofOutput, "worker/blocked.json"), "utf8")); + assert.equal(blocked.result, "blocked"); + assert.equal(blocked.stage, "isolated-sdk-worker"); + assert.deepEqual(await readdir(temporary), [], `${scenario}: evaluation home leaked`); + if (scenario !== "missing-source-setup") + assert.equal(await readFile(source, "utf8"), sourceBytes); + await assert.rejects(readFile(join(proofOutput, "worker/evidence.json")), { code: "ENOENT" }); + cases.push({ + scenario, + command: [process.execPath, ...args], + exit_code: result.status, + expected_exit_code: expectedExit, + stage: blocked.stage, + worker_failure: blocked.worker_failure, + evaluation_home_removed: true, + source_unchanged: scenario === "missing-source-setup" ? null : true, + fixture_origin: true, + semantic_acceptance: false, + }); + } + return cases; + }, + () => rm(scratch, { recursive: true, force: true }), +); +const git = spawnSync("git", ["rev-parse", "HEAD"], { cwd: root, encoding: "utf8" }); +assert.equal(git.status, 0); +const sources = [ + "scripts/replay-cp06-worker.mjs", + "scripts/cp06-namespace-supervisor.mjs", + "scripts/cp06-worker-lifecycle.mjs", + "scripts/prove-cp06-worker-cleanup.mjs", +]; +const manifest = { + schema_version: 1, + gate: "CP06-DUMMY-worker-cleanup", + result: "pass", + tested_sha: git.stdout.trim(), + recorded_at: new Date().toISOString(), + reviewer: process.env.CP06_REVIEWER ?? "CP06 correction worker", + command: "npm run proof:cp06:worker-cleanup", + exit_code: 0, + fixture_origin: true, + semantic_acceptance: false, + cases, + source_sha256: Object.fromEntries( + await Promise.all( + sources.map(async (path) => [ + path, + createHash("sha256") + .update(await readFile(join(root, path))) + .digest("hex"), + ]), + ), + ), + limitations: [ + "Actual full-runner expired/near-expiry and missing-source setup cases use DUMMY auth only. Launch, timeout, audit and compound cleanup failures are additionally injected at the executable supervisor interface in the unit suite; SDK timeout is exercised by the DUMMY auth-security proof.", + ], +}; +await writeFile(join(output, "evidence.json"), `${JSON.stringify(manifest, null, 2)}\n`); +console.log( + JSON.stringify({ + result: "pass", + gate: manifest.gate, + tested_sha: manifest.tested_sha, + cases: cases.map(({ scenario, exit_code, evaluation_home_removed }) => ({ + scenario, + exit_code, + evaluation_home_removed, + })), + }), +); diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 4c2ddfb..d3a4448 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -5,6 +5,7 @@ import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; +import { withCleanup } from "./cp06-worker-lifecycle.mjs"; import { Cp06IsolationUnsupportedError, runCp06SecurityPreflight, @@ -25,144 +26,153 @@ const timeoutMs = 600_000; const systemPrompt = "You are a bounded CP-06 reviewer. Use exactly the read tool, read only the supplied exact original once, return only the requested JSON, and do not attempt any file mutation or broad scan."; -await rm(workerRoot, { recursive: true, force: true }); -await mkdir(workerRoot, { recursive: true }); -let security; try { - // This DUMMY-only gate includes the unsafe control, hardened evaluated-child - // attacks, vendor refresh counters, and offline SDK read audit. It completes - // before the real credential path is opened or any provider request is made. - security = await runCp06SecurityPreflight({ - root, - outputRoot: securityRoot, - reviewer: process.env.CP06_REVIEWER, - }); -} catch (error) { - await recordBlocked(error, "dummy-security-preflight"); - if (error instanceof Cp06IsolationUnsupportedError) process.exit(73); - throw error; -} - -const workerInput = { - schema_version: 1, - root, - fixture_root: fixtureRoot, - contract_path: contractPath, - pack_path: initialPackPath, - original_path: originalPath, - provider, - model, - timeout_ms: timeoutMs, - system_prompt: systemPrompt, -}; -const inputPath = join(workerRoot, "input.json"); -await writeFile(inputPath, `${JSON.stringify(workerInput, null, 2)}\n`); -const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-sdk-worker-")); -const credentialTarget = join(evaluationHome, "pi-agent", "auth.json"); -const credentialSource = resolve( - process.env.CP06_PI_AUTH_FILE ?? - join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "auth.json"), -); -const binaries = { - helper: join(securityRoot, "cp06-seccomp-exec"), - syscallProbe: join(securityRoot, "cp06-isolation-syscalls"), -}; -let isolated; -try { - isolated = await runIsolatedWorker({ - root, - source: credentialSource, - target: credentialTarget, - binaries, - inputPath, - timeout: timeoutMs + 60_000, - }); + await main(); } catch (error) { - await recordBlocked(error, "isolated-sdk-worker"); process.stderr.write(`${error.code ?? "CP06_WORKER_FAILED"}: ${error.message}\n`); - process.exit(error.exitCode ?? 1); -} finally { - await rm(evaluationHome, { recursive: true, force: true }); -} -if ( - isolated.mode !== "worker" || - isolated.source_unchanged !== true || - isolated.mount_ids_distinct !== true || - isolated.cleanup !== "pass" || - isolated.child?.result !== "pass" -) { - const error = new Error("isolated SDK worker omitted a required safety assertion"); - await recordBlocked(error, "isolated-sdk-worker-audit"); - throw error; + process.exitCode = error.exitCode ?? (error instanceof Cp06IsolationUnsupportedError ? 73 : 1); } -const testedSha = textCommand("git", ["rev-parse", "HEAD"]); -const manifest = { - schema_version: 1, - gate: "P-06-fresh-worker", - tested_sha: testedSha, - recorded_at: new Date().toISOString(), - reviewer: process.env.CP06_REVIEWER ?? "Pi CP-06 read-only auth correction worker", - worker: { - pi_version: isolated.child.pi_version, - interface: "public SDK with injected CredentialStore", +async function main() { + await rm(workerRoot, { recursive: true, force: true }); + await mkdir(workerRoot, { recursive: true }); + let security; + try { + // This DUMMY-only gate includes the unsafe control, hardened evaluated-child + // attacks, vendor refresh counters, and offline SDK read audit. It completes + // before the real credential path is opened or any provider request is made. + security = await runCp06SecurityPreflight({ + root, + outputRoot: securityRoot, + reviewer: process.env.CP06_REVIEWER, + }); + } catch (error) { + await recordBlocked(error, "dummy-security-preflight"); + throw error; + } + + const workerInput = { + schema_version: 1, + root, + fixture_root: fixtureRoot, + contract_path: contractPath, + pack_path: initialPackPath, + original_path: originalPath, provider, model, - in_memory_session: isolated.child.session.in_memory, - tools: isolated.child.session.active_tools, - refresh_on_create: false, - model_catalog_network: false, - }, - bounded_handoff: { - contract_path: relative(contractPath), - contract_sha256: await fileHash(contractPath), - pack_path: relative(initialPackPath), - pack_sha256: await fileHash(initialPackPath), - original_path: relative(originalPath), - original_sha256: await fileHash(originalPath), - deliberately_omitted_constraint_sha256: sha256("request-supplied owner fields are ignored"), - evidence: "in-memory auth 401 and concealed non-mutating cross-account 404 observations", - }, - isolation: { - supported_platform: "Linux x86-64", - dummy_security_preflight: security, - source_and_target_distinct_read_only_mounts: isolated.mount_ids_distinct, - evaluated_child_capabilities: "all sets empty", - evaluated_child_no_new_privs: true, - evaluated_child_seccomp: true, - credential_store: "injected callback-denying read-only CredentialStore", - oauth_preflight: isolated.child.oauth_preflight, - credential_store_audit: isolated.child.credential_store, - credential_source_unchanged: isolated.source_unchanged, - cleanup: isolated.cleanup, - destructive_real_source_probe: false, - raw_transcript_retained: false, - }, - event_stream: isolated.child.event_stream, - command: - "Pi 0.85.1 createAgentSession({modelRuntime: injectedReadOnlyRuntime, tools: ['read'], inMemorySession}) inside capability-free seccomp child", - exit_code: 0, - response_sha256: isolated.child.response_sha256, - response: isolated.child.response, - result: "pass", - limitation: - "Live model wording is nondeterministic. The proof runner is Linux x86-64 only and exits blocked before credential access/provider use when user/mount namespaces, libseccomp, exact Pi 0.85.1, or adequate unexpired OAuth validity are unavailable.", -}; -const manifestPath = join(workerRoot, "evidence.json"); -await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); -console.log( - JSON.stringify({ - ok: true, - gate: manifest.gate, + timeout_ms: timeoutMs, + system_prompt: systemPrompt, + }; + const inputPath = join(workerRoot, "input.json"); + await writeFile(inputPath, `${JSON.stringify(workerInput, null, 2)}\n`); + const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-sdk-worker-")); + const credentialTarget = join(evaluationHome, "pi-agent", "auth.json"); + const credentialSource = resolve( + process.env.CP06_PI_AUTH_FILE ?? + join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "auth.json"), + ); + const binaries = { + helper: join(securityRoot, "cp06-seccomp-exec"), + syscallProbe: join(securityRoot, "cp06-isolation-syscalls"), + }; + let isolated; + try { + isolated = await withCleanup( + () => + runIsolatedWorker({ + root, + source: credentialSource, + target: credentialTarget, + binaries, + inputPath, + timeout: timeoutMs + 60_000, + }), + () => rm(evaluationHome, { recursive: true, force: true }), + ); + } catch (error) { + await recordBlocked(error, "isolated-sdk-worker"); + throw error; + } + if ( + isolated.mode !== "worker" || + isolated.source_unchanged !== true || + isolated.mount_ids_distinct !== true || + isolated.cleanup !== "pass" || + isolated.child?.result !== "pass" + ) { + const error = new Error("isolated SDK worker omitted a required safety assertion"); + await recordBlocked(error, "isolated-sdk-worker-audit"); + throw error; + } + + const testedSha = textCommand("git", ["rev-parse", "HEAD"]); + const manifest = { + schema_version: 1, + gate: "P-06-fresh-worker", tested_sha: testedSha, - manifest_path: relative(manifestPath), - manifest_sha256: await fileHash(manifestPath), + recorded_at: new Date().toISOString(), + reviewer: process.env.CP06_REVIEWER ?? "Pi CP-06 read-only auth correction worker", + worker: { + pi_version: isolated.child.pi_version, + interface: "public SDK with injected CredentialStore", + provider, + model, + in_memory_session: isolated.child.session.in_memory, + tools: isolated.child.session.active_tools, + refresh_on_create: false, + model_catalog_network: false, + }, + bounded_handoff: { + contract_path: relative(contractPath), + contract_sha256: await fileHash(contractPath), + pack_path: relative(initialPackPath), + pack_sha256: await fileHash(initialPackPath), + original_path: relative(originalPath), + original_sha256: await fileHash(originalPath), + deliberately_omitted_constraint_sha256: sha256("request-supplied owner fields are ignored"), + evidence: "in-memory auth 401 and concealed non-mutating cross-account 404 observations", + }, + isolation: { + supported_platform: "Linux x86-64", + dummy_security_preflight: security, + source_and_target_distinct_read_only_mounts: isolated.mount_ids_distinct, + evaluated_child_capabilities: "all sets empty", + evaluated_child_no_new_privs: true, + evaluated_child_seccomp: true, + credential_store: "injected callback-denying read-only CredentialStore", + oauth_preflight: isolated.child.oauth_preflight, + credential_store_audit: isolated.child.credential_store, + credential_source_unchanged: isolated.source_unchanged, + cleanup: isolated.cleanup, + destructive_real_source_probe: false, + raw_transcript_retained: false, + }, + event_stream: isolated.child.event_stream, + command: + "Pi 0.85.1 createAgentSession({modelRuntime: injectedReadOnlyRuntime, tools: ['read'], inMemorySession}) inside capability-free seccomp child", exit_code: 0, - status: manifest.response.status, - reads: manifest.event_stream.read_audit.project_read_paths, - broad_scan: manifest.response.broad_scan, - }), -); + response_sha256: isolated.child.response_sha256, + response: isolated.child.response, + result: "pass", + limitation: + "Live model wording is nondeterministic. The proof runner is Linux x86-64 only and exits blocked before credential access/provider use when user/mount namespaces, libseccomp, exact Pi 0.85.1, or adequate unexpired OAuth validity are unavailable.", + }; + const manifestPath = join(workerRoot, "evidence.json"); + await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); + console.log( + JSON.stringify({ + ok: true, + gate: manifest.gate, + tested_sha: testedSha, + manifest_path: relative(manifestPath), + manifest_sha256: await fileHash(manifestPath), + exit_code: 0, + status: manifest.response.status, + reads: manifest.event_stream.read_audit.project_read_paths, + broad_scan: manifest.response.broad_scan, + }), + ); +} async function recordBlocked(error, stage) { const status = { @@ -171,6 +181,15 @@ async function recordBlocked(error, stage) { result: "blocked", stage, code: error?.code ?? "CP06_WORKER_FAILED", + cleanup_failed: error?.code === "CP06_CLEANUP_FAILED", + worker_failure: error?.workerFailure ?? null, + causes: + error instanceof AggregateError + ? error.errors.map((cause) => ({ + code: cause?.code ?? "CP06_WORKER_FAILED", + exit_code: cause?.exitCode ?? null, + })) + : [], recorded_at: new Date().toISOString(), raw_transcript_retained: false, }; diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs new file mode 100644 index 0000000..1355bfd --- /dev/null +++ b/test/cp06-worker-lifecycle.test.mjs @@ -0,0 +1,116 @@ +import { strict as assert } from "node:assert"; +import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; +import { superviseCredentialChild } from "../scripts/cp06-namespace-supervisor.mjs"; +import { Cp06CleanupError, withCleanup } from "../scripts/cp06-worker-lifecycle.mjs"; + +for (const scenario of [ + "success", + "expired", + "near-expiry", + "setup", + "launch", + "timeout", + "audit", + "cleanup", + "primary-and-cleanup", +]) { + test(`supervisor awaits cleanup and preserves failures: ${scenario}`, async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-lifecycle-")); + const source = join(directory, "DUMMY-source"); + const target = join(directory, "DUMMY-target"); + await writeFile(source, "DUMMY ONLY"); + let cleanups = 0; + let launches = 0; + const setupError = new Error("DUMMY setup failed before mount acquired"); + const cleanupError = new Error("DUMMY cleanup failed"); + const run = () => + superviseCredentialChild( + { + mode: "worker", + source, + target, + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + childArgs: ["/DUMMY-input"], + }, + { + async mount() { + if (scenario === "setup") throw setupError; + await writeFile(target, "DUMMY placeholder"); + return { + mountIds: { source: "1", target: "2" }, + async cleanup() { + await new Promise((done) => setTimeout(done, 5)); + cleanups++; + await rm(target); + if (["cleanup", "primary-and-cleanup"].includes(scenario)) throw cleanupError; + }, + }; + }, + spawn() { + launches++; + if (scenario === "launch") throw new Error("DUMMY spawn threw"); + if (scenario === "timeout") + return { + status: null, + error: Object.assign(new Error("DUMMY timeout"), { code: "ETIMEDOUT" }), + }; + if (["expired", "near-expiry", "primary-and-cleanup"].includes(scenario)) + return { status: 75, stderr: "DUMMY auth blocked" }; + return { + status: 0, + stdout: scenario === "audit" ? "invalid JSON" : '{"result":"pass"}', + }; + }, + }, + ); + try { + if (scenario === "success") { + const result = await run(); + assert.equal(result.cleanup, "pass"); + assert.equal(result.source_unchanged, true); + } else { + await assert.rejects(run, (error) => { + if (scenario === "setup") assert.equal(error.cause, setupError); + if (["expired", "near-expiry"].includes(scenario)) assert.equal(error.exitCode, 75); + if (scenario === "timeout") assert.equal(error.cause.code, "ETIMEDOUT"); + if (["cleanup", "primary-and-cleanup"].includes(scenario)) { + assert(error instanceof Cp06CleanupError); + assert.equal(error.exitCode, 74); + assert.equal(error.errors.at(-1), cleanupError); + if (scenario === "primary-and-cleanup") { + assert.equal(error.errors[0].exitCode, 75); + assert.match(error.errors[0].message, /DUMMY auth blocked/); + assert.equal(error.cause, error.errors[0]); + } + } + return true; + }); + } + assert.equal(cleanups, scenario === "setup" ? 0 : 1); + assert.equal(launches, scenario === "setup" ? 0 : 1); + await assert.rejects(stat(target), { code: "ENOENT" }); + assert.equal(await readFile(source, "utf8"), "DUMMY ONLY"); + } finally { + await rm(directory, { recursive: true, force: true }); + } + }); +} + +test("evaluation-home cleanup finishes before the blocked result is returned", async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-home-")); + const primary = Object.assign(new Error("DUMMY blocked"), { exitCode: 75 }); + await assert.rejects( + withCleanup( + async () => { + throw primary; + }, + () => rm(directory, { recursive: true }), + ), + (error) => error === primary, + ); + await assert.rejects(stat(directory), { code: "ENOENT" }); +}); diff --git a/test/cp06-worker-outcome.test.mjs b/test/cp06-worker-outcome.test.mjs new file mode 100644 index 0000000..1c51cbe --- /dev/null +++ b/test/cp06-worker-outcome.test.mjs @@ -0,0 +1,69 @@ +import { strict as assert } from "node:assert"; +import { resolve } from "node:path"; +import test from "node:test"; +import { auditWorkerOutcome } from "../scripts/cp06-worker-outcome.mjs"; + +const root = process.cwd(); +const options = { + root, + referenceRoot: resolve(root, "test/fixtures/cp06-context"), + expectedOriginalPath: resolve(root, "test/fixtures/cp06-context/docs/AUTH.md"), +}; +const gap = { + id: "creation-time-ownership", + status: "unverified", + missing_checks: ["principal-derived-owner", "request-owner-ignored"], +}; +const response = { + status: "MISSING_SOURCE", + missing_source: "docs/AUTH.md", + broad_scan: false, + corrected_constraints: [ + "Ownership comes only from the authenticated principal; request-supplied owner fields are ignored.", + ], + evidence_gaps: [gap], +}; + +test("worker outcome accepts the required structured unverified creation checks", () => { + assert.equal(auditWorkerOutcome(response, options).ok, true); + assert.equal( + auditWorkerOutcome( + { + ...response, + evidence_gaps: [{ ...gap, missing_checks: [...gap.missing_checks].reverse() }], + }, + options, + ).ok, + true, + ); +}); + +for (const [name, evidence_gaps] of [ + ["missing", undefined], + ["empty", []], + ["legacy prose without a check disposition", ["creation evidence missing"]], + ["contradictory passed disposition", [{ ...gap, status: "pass" }]], + ["contradictory verified flag", [{ ...gap, verified: true }]], + ["wrong gap", [{ ...gap, id: "read-time-ownership" }]], + ["omitted ignored-owner check", [{ ...gap, missing_checks: ["principal-derived-owner"] }]], + [ + "duplicate checks", + [{ ...gap, missing_checks: ["principal-derived-owner", "principal-derived-owner"] }], + ], + ["mixed unverified and passed", [gap, { ...gap, status: "pass" }]], +]) { + test(`worker outcome rejects ${name} creation evidence`, () => { + assert.equal(auditWorkerOutcome({ ...response, evidence_gaps }, options).ok, false); + }); +} + +test("creation evidence cannot replace source recovery or relax the read outcome", () => { + for (const change of [ + { status: "PASS" }, + { missing_source: "docs/OTHER.md" }, + { broad_scan: true }, + { corrected_constraints: [] }, + ]) + assert.equal(auditWorkerOutcome({ ...response, ...change }, options).ok, false); + assert.equal(auditWorkerOutcome(null, options).ok, false); +}); From 6b9b6e2e8b4ef11f48fb2ef0f3d09fd021692289 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Thu, 1 Oct 2026 13:03:12 +0000 Subject: [PATCH 19/45] no-mistakes(review): Enforce canonical missing source identity --- scripts/cp06-worker-audit.mjs | 6 +----- test/cp06-worker-audit.test.mjs | 16 ++++++++++++++++ test/cp06-worker-outcome.test.mjs | 2 ++ 3 files changed, 19 insertions(+), 5 deletions(-) diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs index 782be9f..366b19c 100644 --- a/scripts/cp06-worker-audit.mjs +++ b/scripts/cp06-worker-audit.mjs @@ -2,11 +2,7 @@ import { createHash } from "node:crypto"; import { resolve } from "node:path"; export function isExpectedOriginalReference(value, options) { - if (typeof value !== "string" || value.trim().length === 0) return false; - const reference = value.trim(); - const expectedPath = resolve(options.root, options.expectedOriginalPath); - const referenceRoot = options.referenceRoot ?? options.root; - return resolve(referenceRoot, reference) === expectedPath; + return value === (options.expectedOriginalReference ?? "docs/AUTH.md"); } export function auditReadEvents(events, options) { diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs index 8a8bf47..c562369 100644 --- a/test/cp06-worker-audit.test.mjs +++ b/test/cp06-worker-audit.test.mjs @@ -143,6 +143,22 @@ test("worker missing-source identity accepts only the canonical original path", }), true, ); + assert.equal( + isExpectedOriginalReference("./docs/AUTH.md", { + root, + referenceRoot: fixtureRoot, + expectedOriginalPath: originalPath, + }), + false, + ); + assert.equal( + isExpectedOriginalReference(originalPath, { + root, + referenceRoot: fixtureRoot, + expectedOriginalPath: originalPath, + }), + false, + ); assert.equal( isExpectedOriginalReference("docs/OTHER.md", { root, diff --git a/test/cp06-worker-outcome.test.mjs b/test/cp06-worker-outcome.test.mjs index 1c51cbe..41a20ea 100644 --- a/test/cp06-worker-outcome.test.mjs +++ b/test/cp06-worker-outcome.test.mjs @@ -61,6 +61,8 @@ test("creation evidence cannot replace source recovery or relax the read outcome for (const change of [ { status: "PASS" }, { missing_source: "docs/OTHER.md" }, + { missing_source: "./docs/AUTH.md" }, + { missing_source: resolve(root, "test/fixtures/cp06-context/docs/AUTH.md") }, { broad_scan: true }, { corrected_constraints: [] }, ]) From 949162642f2867d60c37fb772289815a074205ae Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Thu, 1 Oct 2026 13:13:08 +0000 Subject: [PATCH 20/45] no-mistakes(document): Document cleanup proof evidence --- docs/probes/CP-06.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index c5c574d..015b854 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -95,6 +95,7 @@ npm run lint npm run typecheck npm test npm run proof:cp06:auth-security +npm run proof:cp06:worker-cleanup npm run proof:cp06:ten-pack npm run proof:cp06:p07 npm run proof:cp06:worker @@ -105,6 +106,7 @@ The exact runtime manifests are: ```text .factory/state/cp06-correction/auth-security/evidence.json +.factory/state/cp06-correction/worker-cleanup/evidence.json .factory/state/cp06-correction/ten-pack/evidence.json .factory/state/cp06-correction/p07/evidence.json .factory/state/cp06-correction/worker/evidence.json From 0632643285eacd334635d72e4d4323e7dfae3eff Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 09:41:38 +0000 Subject: [PATCH 21/45] fix(cp06): preserve audit and cleanup failures --- BUILD_PLAN.md | 2 +- docs/probes/CP-06.md | 69 +++++++++++++------ scripts/cp06-auth-security.mjs | 51 +++++++------- scripts/cp06-dummy-outcome-proof.mjs | 1 + scripts/cp06-dummy-sdk-proof.mjs | 6 +- scripts/cp06-dummy-worker-hook.mjs | 6 +- scripts/cp06-namespace-supervisor.mjs | 73 ++++++++++---------- scripts/cp06-pi-dependency.mjs | 77 +++++++++++++++++++++ scripts/cp06-worker-audit.mjs | 10 +-- scripts/replay-cp06-worker.mjs | 22 +++--- test/cp06-pi-dependency.test.mjs | 68 ++++++++++++++++++ test/cp06-worker-audit.test.mjs | 22 ++++-- test/cp06-worker-lifecycle.test.mjs | 99 +++++++++++++++++++++++++-- 13 files changed, 393 insertions(+), 113 deletions(-) create mode 100644 scripts/cp06-pi-dependency.mjs create mode 100644 test/cp06-pi-dependency.test.mjs diff --git a/BUILD_PLAN.md b/BUILD_PLAN.md index 34f77af..034adb6 100644 --- a/BUILD_PLAN.md +++ b/BUILD_PLAN.md @@ -89,7 +89,7 @@ - [x] Create tests/fixtures for critical auth isolation and normal API behavior, plus one simulated reviewer finding that causes repair and retest. - [x] Run an actual worker task with only task/pack/evidence sources in a fresh context; inspect whether it needs missing docs. -- [ ] **Gate:** P-06 and P-07 via the tracked ten-pack, protocol-negative/fresh-worker, and P-07 replays in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. PR11 independent review rejected its A1 outcome and C1 cleanup claims; the correction requires final-head proofs and another independent review. DUMMY SDK controls are not live semantic acceptance, and strict-agent PARTIAL retrieval is not COMPLETE. +- [ ] **Gate:** P-06 and P-07 via the tracked ten-pack, protocol-negative/fresh-worker, and P-07 replays in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. The PR12 audit's C1 malformed-output-plus-cleanup finding is repaired at the supervisor audit boundary with propagated replay diagnostics; exact-final-head proofs and another independent review remain required. DUMMY SDK controls are not live semantic acceptance, and strict-agent PARTIAL retrieval is not COMPLETE. ## CP-07 — Thin Pi command and status convenience diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 015b854..7e61554 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -23,6 +23,8 @@ The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK pat The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and exactly the built-in `read` tool. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact hashes/counts and parsed output, never credential contents or raw events. +The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/`, prepends only that installation's `.bin` for the proof commands, and leaves the global installation/configuration unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the resolved package/entry paths and their SHA-256 digests. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. + Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is attempted only after that DUMMY gate and fail-closed preflight pass. No destructive operation is ever directed at the real credential source or mounted target. ## Normative result @@ -86,22 +88,30 @@ SHA, UTC date, reviewer, tool/index identity, and applicable source/contract/pack/receipt hashes, budgets, and provenance. This meets the general proof rule without committing runtime state or model transcripts. -Run in this order: +Run in this order (all package/cache paths are disposable and ignored): ```text -npm ci +npm_config_cache="$PWD/.factory/state/npm-cache" npm ci +mkdir -p .factory/state/cp06-sdk +npm_config_cache="$PWD/.factory/state/npm-cache-pi" npm install \ + --prefix .factory/state/cp06-sdk --package-lock=false --ignore-scripts \ + --no-audit --no-fund @earendil-works/pi-coding-agent@0.85.1 +PINNED_PI_BIN="$PWD/.factory/state/cp06-sdk/node_modules/.bin" +PATH="$PINNED_PI_BIN:$PATH" pi --version # must be exactly 0.85.1 npm run format:check npm run lint npm run typecheck npm test -npm run proof:cp06:auth-security -npm run proof:cp06:worker-cleanup +PATH="$PINNED_PI_BIN:$PATH" npm run proof:cp06:auth-security +PATH="$PINNED_PI_BIN:$PATH" npm run proof:cp06:worker-cleanup npm run proof:cp06:ten-pack npm run proof:cp06:p07 -npm run proof:cp06:worker +PATH="$PINNED_PI_BIN:$PATH" npm run proof:cp06:worker git diff --check ``` +The live worker command is omitted rather than passed when DUMMY protection, exact SDK selection, or non-refresh credential-validity preflight fails. + The exact runtime manifests are: ```text @@ -195,11 +205,14 @@ creation ownership and the ignored request-owner rule, and identifies the missing creation-time integration evidence. The audit correlates `tool_execution_start` and `tool_execution_end` by tool-call -ID and event order, requires `isError: false`, forbids offset/limit and all other -start or completion events, compares the returned UTF-8 bytes and SHA-256 to the -tracked exact original, and requires the final response after the completion. -Deterministic protocol tests reject omitted, failed, partial, mismatched, -out-of-order, malformed-orphan, and extra-project cases. +ID and event order, requires `isError: false`, and forbids all other start or +completion events. An omitted range is accepted; the SDK's explicit first-line +form is accepted only as `offset: 1` with a positive integer `limit`, and only +when the returned UTF-8 bytes and SHA-256 still equal the complete tracked exact +original. Offset-only, limit-only, wrong-start, and partial results fail. The +final response must follow the successful completion. Deterministic protocol +tests reject omitted, failed, partial, mismatched, out-of-order, +malformed-orphan, and extra-project cases. The genuine Pi SDK process runs in a private Linux user/mount namespace only after the complete DUMMY auth and OS preflight passes. A supervisor pins the @@ -256,13 +269,21 @@ uses the actual built-in SDK read tool, not a fabricated completion event. The runner and namespace supervisor await cleanup before setting an exit code. When execution and cleanup both fail, the cleanup error retains the primary -cause and its exit identity. `npm run proof:cp06:worker-cleanup` executes the full -runner with task-local expired/near-expiry DUMMY auth and missing-source setup, -asserts the blocked result, unchanged DUMMY source, absence of a pass manifest, -and removal of the evaluation home. Unit tests inject setup, launch, timeout, -audit and cleanup failures at the executable supervisor boundary, including -compound failure; they are distinct from actual kernel/SDK proof. No real -credential read, mount, refresh or model request is needed for these regressions. +cause and its exit identity. PR12 still parsed successful-child output after +cleanup, so malformed output plus cleanup failure lost the parse/audit cause. +The correction keeps child status, source-integrity verification, compact JSON +parsing, and evidence assembly inside the cleanup action. The shipped +actual-boundary regression separately triggers successful-child malformed +output and an independent awaited cleanup error, then verifies that the +supervisor JSON, namespace wrapper, and replay `blocked.json` retain both cause +identities; child-exit+cleanup, cleanup-only, audit-only, and success remain +separate controls. `npm run proof:cp06:worker-cleanup` executes the full runner +with task-local expired/near-expiry DUMMY auth and missing-source setup, asserts +the blocked result, unchanged DUMMY source, absence of a pass manifest, and +removal of the evaluation home. Unit tests also inject setup, launch, timeout, +audit and cleanup failures at the executable supervisor boundary; they are +distinct from actual kernel/SDK proof. No real credential read, mount, refresh +or model request is needed for these regressions. The strict-agent CTX audit is a separate qualification. PR11's ordinary Factory STANDARD/HYBRID_SEMANTIC sample passed 10/10 with zero irrelevant provenance; @@ -270,9 +291,12 @@ replaying those selections with `--strict-agent --require-semantic` reported ten PARTIAL results because strict cross-document discovery required filtered-out normative entries. This is not strict COMPLETE evidence. Do not widen filters, hide omissions, or label the DUMMY worker as semantic acceptance to cure it. -Final-head manifests must be regenerated after committing the correction. A real -fresh-worker run and independent acceptance remain blocked/unverified unless -separately performed safely; the prior PR11 report is not approval of this head. +Final-head manifests must be regenerated after committing the correction. The +fresh-worker gate is verified only by a successful final-head real model/SDK +manifest produced after the DUMMY gate through the hardened read-only path; a +fixture-origin stream or an earlier SHA cannot substitute. Independent +acceptance remains separate, and the prior PR11 report is not approval of this +head. ## P-07 finding, repair, and retest @@ -306,8 +330,9 @@ no-mistakes nor changes its ownership boundary. ## Acceptance and residual limitations P-06's STANDARD ten-pack subgate passes when the corrected, predeclared sample -is replayed successfully; this is not the full fresh-worker or independent -CP-06 gate. The original PR #8 sample remains explicitly failed. P-07 passes because the structured finding, +is replayed successfully; it does not substitute for the final-head real +fresh-worker manifest or independent CP-06 review. The original PR #8 sample +remains explicitly failed. P-07 passes because the structured finding, fail-before, repair, final behavior retest, and auth retest are executable and bound to the replayed SHA. CP-06 makes no change to Firstmate dispatch, delivery, or no-mistakes policy. diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 7e7e6fd..b6a7773 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -137,33 +137,35 @@ export async function runIsolatedWorker(options) { timeout: options.timeout ?? 660_000, environment: options.environment, }); - if (result.status !== 0) { - const error = new Error( - result.status === 73 - ? "CP-06 credential namespace setup is unsupported" - : `CP-06 isolated SDK worker exited ${result.status ?? "without status"}`, - ); - error.code = result.status === 73 ? "CP06_ISOLATION_UNSUPPORTED" : "CP06_WORKER_FAILED"; - error.exitCode = result.status ?? 70; - error.stderr = result.stderr; - // Preserve the supervisor's compact failure identity, including both causes - // when child execution and mount cleanup fail. Do not persist raw stderr. - try { - const failure = JSON.parse(String(result.stderr).trim()); - if (failure.schema_version === 1 && failure.exit_code === result.status) { - error.workerFailure = { - code: failure.code, - exit_code: failure.exit_code, - causes: failure.causes, - }; - if (failure.code === "CP06_CLEANUP_FAILED") error.code = failure.code; - } - } catch {} - throw error; - } + if (result.status !== 0) throw namespaceWorkerError(result); return parseCompactJson(result.stdout, "isolated SDK worker"); } +export function namespaceWorkerError(result) { + const error = new Error( + result.status === 73 + ? "CP-06 credential namespace setup is unsupported" + : `CP-06 isolated SDK worker exited ${result.status ?? "without status"}`, + ); + error.code = result.status === 73 ? "CP06_ISOLATION_UNSUPPORTED" : "CP06_WORKER_FAILED"; + error.exitCode = result.status ?? 70; + error.stderr = result.stderr; + // Preserve the supervisor's compact failure identity, including both causes + // when child execution or audit and mount cleanup fail. Do not persist raw stderr. + try { + const failure = JSON.parse(String(result.stderr).trim()); + if (failure.schema_version === 1 && failure.exit_code === result.status) { + error.workerFailure = { + code: failure.code, + exit_code: failure.exit_code, + causes: failure.causes, + }; + if (failure.code === "CP06_CLEANUP_FAILED") error.code = failure.code; + } + } catch {} + return error; +} + async function runIsolationProbe({ root, outputRoot, mode, binaries }) { const proofRoot = await mkdtemp(join(outputRoot, `DUMMY-${mode}-`)); const source = join(proofRoot, "source", "DUMMY-auth.json"); @@ -302,6 +304,7 @@ function runDummySdkProof({ root, helper, piRoot }) { "--", helper, process.execPath, + "--experimental-import-meta-resolve", join(root, "scripts/cp06-dummy-sdk-proof.mjs"), piRoot, ], diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index 6ae6c58..1256dd8 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -71,6 +71,7 @@ export async function proveDummyWorkerOutcome({ piRoot }) { const result = spawnSync( process.execPath, [ + "--experimental-import-meta-resolve", "--import", join(root, "scripts/cp06-dummy-worker-hook.mjs"), join(root, "scripts/cp06-sdk-worker.mjs"), diff --git a/scripts/cp06-dummy-sdk-proof.mjs b/scripts/cp06-dummy-sdk-proof.mjs index 0f86f70..5b56672 100644 --- a/scripts/cp06-dummy-sdk-proof.mjs +++ b/scripts/cp06-dummy-sdk-proof.mjs @@ -8,14 +8,13 @@ import { pathToFileURL } from "node:url"; import { Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; import { auditReadEvents } from "./cp06-worker-audit.mjs"; import { proveDummyWorkerOutcome } from "./cp06-dummy-outcome-proof.mjs"; +import { importPinnedPiAi } from "./cp06-pi-dependency.mjs"; const [piRootArg] = process.argv.slice(2); if (!piRootArg) process.exit(64); const piRoot = resolve(piRootArg); const sdk = await import(pathToFileURL(join(piRoot, "dist/index.js"))); -const ai = await import( - pathToFileURL(join(piRoot, "node_modules/@earendil-works/pi-ai/dist/index.js")) -); +const { module: ai, provenance: piAi } = await importPinnedPiAi(piRoot); const originalFetch = globalThis.fetch; let networkCalls = 0; globalThis.fetch = async () => { @@ -41,6 +40,7 @@ try { result: "pass", auth_cases: authCases, sdk_worker: worker, + sdk_dependency: piAi, actual_sdk_outcomes: outcomeProof, fixture_origin: true, semantic_acceptance: false, diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index b41a254..2286c5e 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -2,6 +2,7 @@ import { readFileSync, writeFileSync } from "node:fs"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; +import { importPinnedPiAi } from "./cp06-pi-dependency.mjs"; const piRoot = process.env.CP06_DUMMY_PI_ROOT; const countersPath = process.env.CP06_DUMMY_COUNTERS; @@ -9,12 +10,11 @@ const scenario = process.env.CP06_DUMMY_SCENARIO; if (!piRoot || !countersPath || !scenario) throw new Error("DUMMY proof configuration missing"); const sdk = await import(pathToFileURL(join(piRoot, "dist/index.js"))); const { AuthStorage } = await import(pathToFileURL(join(piRoot, "dist/core/auth-storage.js"))); -const ai = await import( - pathToFileURL(join(piRoot, "node_modules/@earendil-works/pi-ai/dist/index.js")) -); +const { module: ai, provenance: piAi } = await importPinnedPiAi(piRoot); const input = JSON.parse(readFileSync(process.argv[3], "utf8")); const counts = { fixture_origin: true, + sdk_dependency: piAi, default_storage: 0, network: 0, refresh: 0, diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 9a8fbe7..360c57d 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -27,7 +27,7 @@ export async function superviseCredentialChild(options, dependencies = {}) { const spawn = dependencies.spawn ?? spawnSync; const beforeHash = sha256(readFileSync(source)); let mount; - const childResult = await withCleanup( + return await withCleanup( async () => { try { mount = await mountCredential({ root, source, target }); @@ -64,29 +64,28 @@ export async function superviseCredentialChild(options, dependencies = {}) { result.status ?? 70, ); } - return result; + + const afterHash = sha256(readFileSync(source)); + if (mode === "worker" && beforeHash !== afterHash) { + throw failure("CP-06 credential source changed during worker execution", 74); + } + let childEvidence; + try { + childEvidence = JSON.parse(String(result.stdout).trim()); + } catch { + throw failure("CP-06 isolated child returned non-compact output", 70); + } + return { + schema_version: 1, + mode, + source_unchanged: beforeHash === afterHash, + mount_ids_distinct: mount.mountIds.source !== mount.mountIds.target, + cleanup: "pass", + child: childEvidence, + }; }, () => mount?.cleanup(), ); - - const afterHash = sha256(readFileSync(source)); - if (mode === "worker" && beforeHash !== afterHash) { - throw failure("CP-06 credential source changed during worker execution", 74); - } - let childEvidence; - try { - childEvidence = JSON.parse(String(childResult.stdout).trim()); - } catch { - throw failure("CP-06 isolated child returned non-compact output", 70); - } - return { - schema_version: 1, - mode, - source_unchanged: beforeHash === afterHash, - mount_ids_distinct: mount.mountIds.source !== mount.mountIds.target, - cleanup: "pass", - child: childEvidence, - }; } if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { @@ -98,21 +97,7 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur ), ); } catch (error) { - process.stderr.write( - `${JSON.stringify({ - schema_version: 1, - code: error.code ?? "CP06_WORKER_FAILED", - exit_code: error.exitCode ?? 70, - message: safeError(error), - causes: - error instanceof AggregateError - ? error.errors.map((cause) => ({ - code: cause?.code ?? "CP06_WORKER_FAILED", - exit_code: cause?.exitCode ?? null, - })) - : [], - })}\n`, - ); + process.stderr.write(`${JSON.stringify(supervisorFailureRecord(error))}\n`); process.exitCode = error.exitCode ?? 70; } } @@ -185,6 +170,22 @@ function childEnvironment(childMode, credentialTarget) { return environment; } +export function supervisorFailureRecord(error) { + return { + schema_version: 1, + code: error?.code ?? "CP06_WORKER_FAILED", + exit_code: error?.exitCode ?? 70, + message: safeError(error), + causes: + error instanceof AggregateError + ? error.errors.map((cause) => ({ + code: cause?.code ?? "CP06_WORKER_FAILED", + exit_code: cause?.exitCode ?? null, + })) + : [], + }; +} + function failure(message, exitCode, cause) { return Object.assign(new Error(message, { cause }), { exitCode, code: "CP06_WORKER_FAILED" }); } diff --git a/scripts/cp06-pi-dependency.mjs b/scripts/cp06-pi-dependency.mjs new file mode 100644 index 0000000..783d6fd --- /dev/null +++ b/scripts/cp06-pi-dependency.mjs @@ -0,0 +1,77 @@ +import { createHash } from "node:crypto"; +import { readFileSync, realpathSync } from "node:fs"; +import { dirname, isAbsolute, join, relative, resolve } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; + +const packageName = "@earendil-works/pi-ai"; +const supportedVersion = "0.85.1"; + +/** Resolve Pi's import-only pi-ai dependency from the explicitly selected Pi package. */ +export async function importPinnedPiAi(piRootArg) { + const piRoot = realpathSync(resolve(piRootArg)); + const piMetadataPath = join(piRoot, "package.json"); + const piMetadata = JSON.parse(readFileSync(piMetadataPath, "utf8")); + if ( + piMetadata.name !== "@earendil-works/pi-coding-agent" || + piMetadata.version !== supportedVersion || + piMetadata.dependencies?.[packageName] !== `^${supportedVersion}` + ) { + throw new Error("selected Pi package does not declare the supported pi-ai dependency"); + } + + const parent = pathToFileURL(piMetadataPath).href; + const entryUrl = import.meta.resolve(packageName, parent); + if (!entryUrl.startsWith("file:")) throw new Error("resolved pi-ai entry is not a local file"); + const entry = realpathSync(fileURLToPath(entryUrl)); + const dependencyRoot = findPackageRoot(entry, packageName); + const dependencyMetadataPath = join(dependencyRoot, "package.json"); + const dependencyMetadata = JSON.parse(readFileSync(dependencyMetadataPath, "utf8")); + if (dependencyMetadata.name !== packageName || dependencyMetadata.version !== supportedVersion) { + throw new Error("resolved pi-ai dependency is not exact version 0.85.1"); + } + + const selectedInstallNodeModules = dirname(dirname(piRoot)); + const nestedNodeModules = join(piRoot, "node_modules"); + if ( + !isWithin(selectedInstallNodeModules, dependencyRoot) && + !isWithin(nestedNodeModules, dependencyRoot) + ) { + throw new Error("resolved pi-ai dependency is outside the selected Pi installation"); + } + + return { + module: await import(entryUrl), + provenance: { + name: dependencyMetadata.name, + version: dependencyMetadata.version, + root: dependencyRoot, + entry, + resolution: "esm-import-condition", + package_sha256: sha256(readFileSync(dependencyMetadataPath)), + entry_sha256: sha256(readFileSync(entry)), + }, + }; +} + +function findPackageRoot(entry, expectedName) { + let directory = dirname(entry); + for (;;) { + try { + const metadata = JSON.parse(readFileSync(join(directory, "package.json"), "utf8")); + if (metadata.name === expectedName) return directory; + } catch {} + const parent = dirname(directory); + if (parent === directory) break; + directory = parent; + } + throw new Error("resolved pi-ai package root is unavailable"); +} + +function isWithin(parent, child) { + const path = relative(parent, child); + return path === "" || (!path.startsWith("..") && !isAbsolute(path)); +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs index 366b19c..411ceb7 100644 --- a/scripts/cp06-worker-audit.mjs +++ b/scripts/cp06-worker-audit.mjs @@ -71,11 +71,11 @@ export function auditReadEvents(events, options) { if (typeof callId !== "string" || callId.length === 0) { return reject("original read start omitted toolCallId"); } - if (start.event.args?.offset !== undefined) { - return reject("original read used an explicit offset"); - } - if (start.event.args?.limit !== undefined) { - return reject("original read used an explicit limit"); + const offset = start.event.args?.offset; + const limit = start.event.args?.limit; + const usesExplicitRange = offset !== undefined || limit !== undefined; + if (usesExplicitRange && (offset !== 1 || !Number.isSafeInteger(limit) || limit < 1)) { + return reject("original read used a range other than offset 1 with an explicit limit"); } const matchingEnds = readEnds.filter(({ event }) => event.toolCallId === callId); diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index d3a4448..5f623e4 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -26,11 +26,13 @@ const timeoutMs = 600_000; const systemPrompt = "You are a bounded CP-06 reviewer. Use exactly the read tool, read only the supplied exact original once, return only the requested JSON, and do not attempt any file mutation or broad scan."; -try { - await main(); -} catch (error) { - process.stderr.write(`${error.code ?? "CP06_WORKER_FAILED"}: ${error.message}\n`); - process.exitCode = error.exitCode ?? (error instanceof Cp06IsolationUnsupportedError ? 73 : 1); +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + try { + await main(); + } catch (error) { + process.stderr.write(`${error.code ?? "CP06_WORKER_FAILED"}: ${error.message}\n`); + process.exitCode = error.exitCode ?? (error instanceof Cp06IsolationUnsupportedError ? 73 : 1); + } } async function main() { @@ -175,7 +177,12 @@ async function main() { } async function recordBlocked(error, stage) { - const status = { + const status = createWorkerBlockedStatus(error, stage); + await writeFile(join(workerRoot, "blocked.json"), `${JSON.stringify(status, null, 2)}\n`); +} + +export function createWorkerBlockedStatus(error, stage, recordedAt = new Date().toISOString()) { + return { schema_version: 1, gate: "P-06-fresh-worker", result: "blocked", @@ -190,10 +197,9 @@ async function recordBlocked(error, stage) { exit_code: cause?.exitCode ?? null, })) : [], - recorded_at: new Date().toISOString(), + recorded_at: recordedAt, raw_transcript_retained: false, }; - await writeFile(join(workerRoot, "blocked.json"), `${JSON.stringify(status, null, 2)}\n`); } function textCommand(command, args) { diff --git a/test/cp06-pi-dependency.test.mjs b/test/cp06-pi-dependency.test.mjs new file mode 100644 index 0000000..d5bfe4d --- /dev/null +++ b/test/cp06-pi-dependency.test.mjs @@ -0,0 +1,68 @@ +import { strict as assert } from "node:assert"; +import { spawnSync } from "node:child_process"; +import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import test from "node:test"; +import { pathToFileURL } from "node:url"; + +const resolverUrl = pathToFileURL(resolve("scripts/cp06-pi-dependency.mjs")).href; + +for (const scenario of [ + { name: "exact hoisted import-only dependency", version: "0.85.1", expectedExit: 0 }, + { name: "newer hoisted import-only dependency", version: "0.99.2", expectedExit: 1 }, +]) { + test(`pinned Pi dependency resolver handles ${scenario.name}`, async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-pi-layout-")); + const piRoot = join(directory, "node_modules", "@earendil-works", "pi-coding-agent"); + const aiRoot = join(directory, "node_modules", "@earendil-works", "pi-ai"); + try { + await mkdir(piRoot, { recursive: true }); + await mkdir(join(aiRoot, "dist"), { recursive: true }); + await writeFile( + join(piRoot, "package.json"), + JSON.stringify({ + name: "@earendil-works/pi-coding-agent", + version: "0.85.1", + type: "module", + dependencies: { "@earendil-works/pi-ai": "^0.85.1" }, + }), + ); + await writeFile( + join(aiRoot, "package.json"), + JSON.stringify({ + name: "@earendil-works/pi-ai", + version: scenario.version, + type: "module", + exports: { ".": { import: "./dist/index.js" } }, + }), + ); + await writeFile(join(aiRoot, "dist/index.js"), 'export const marker = "DUMMY-import";\n'); + const result = spawnSync( + process.execPath, + [ + "--experimental-import-meta-resolve", + "--input-type=module", + "--eval", + `const {importPinnedPiAi}=await import(${JSON.stringify(resolverUrl)});const result=await importPinnedPiAi(process.argv[1]);console.log(JSON.stringify({marker:result.module.marker,provenance:result.provenance}));`, + piRoot, + ], + { encoding: "utf8", timeout: 30_000 }, + ); + assert.equal(result.status, scenario.expectedExit, result.stderr); + if (scenario.expectedExit === 0) { + const output = JSON.parse(result.stdout); + assert.equal(output.marker, "DUMMY-import"); + assert.equal(output.provenance.version, "0.85.1"); + assert.equal(output.provenance.resolution, "esm-import-condition"); + assert.equal(output.provenance.root, aiRoot); + assert.match(output.provenance.package_sha256, /^[a-f0-9]{64}$/); + assert.match(output.provenance.entry_sha256, /^[a-f0-9]{64}$/); + } else { + assert.match(result.stderr, /not exact version 0\.85\.1/); + } + } finally { + await rm(directory, { recursive: true, force: true }); + } + }); +} diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs index c562369..71698ec 100644 --- a/test/cp06-worker-audit.test.mjs +++ b/test/cp06-worker-audit.test.mjs @@ -177,10 +177,18 @@ test("worker missing-source identity accepts only the canonical original path", ); }); -test("worker read audit accepts one completed exact-original read before the response", () => { - const audit = auditReadEvents([start, successfulEnd(), finalMessage], options); - assert.equal(audit.ok, true); - assert.equal(audit.summary.completed_successfully, true); - assert.equal(audit.summary.exact_original, true); - assert.equal(audit.summary.expected_sha256, audit.summary.returned_sha256); -}); +for (const [name, acceptedStart] of [ + ["without an explicit range", start], + [ + "with the SDK first-line range when it returns the complete original", + { ...start, args: { path: originalPath, offset: 1, limit: 2_000 } }, + ], +]) { + test(`worker read audit accepts one completed exact-original read ${name}`, () => { + const audit = auditReadEvents([acceptedStart, successfulEnd(), finalMessage], options); + assert.equal(audit.ok, true); + assert.equal(audit.summary.completed_successfully, true); + assert.equal(audit.summary.exact_original, true); + assert.equal(audit.summary.expected_sha256, audit.summary.returned_sha256); + }); +} diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 1355bfd..7189d95 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -3,8 +3,13 @@ import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; -import { superviseCredentialChild } from "../scripts/cp06-namespace-supervisor.mjs"; +import { + superviseCredentialChild, + supervisorFailureRecord, +} from "../scripts/cp06-namespace-supervisor.mjs"; +import { namespaceWorkerError } from "../scripts/cp06-auth-security.mjs"; import { Cp06CleanupError, withCleanup } from "../scripts/cp06-worker-lifecycle.mjs"; +import { createWorkerBlockedStatus } from "../scripts/replay-cp06-worker.mjs"; for (const scenario of [ "success", @@ -16,6 +21,7 @@ for (const scenario of [ "audit", "cleanup", "primary-and-cleanup", + "audit-and-cleanup", ]) { test(`supervisor awaits cleanup and preserves failures: ${scenario}`, async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-lifecycle-")); @@ -46,7 +52,8 @@ for (const scenario of [ await new Promise((done) => setTimeout(done, 5)); cleanups++; await rm(target); - if (["cleanup", "primary-and-cleanup"].includes(scenario)) throw cleanupError; + if (["cleanup", "primary-and-cleanup", "audit-and-cleanup"].includes(scenario)) + throw cleanupError; }, }; }, @@ -62,7 +69,9 @@ for (const scenario of [ return { status: 75, stderr: "DUMMY auth blocked" }; return { status: 0, - stdout: scenario === "audit" ? "invalid JSON" : '{"result":"pass"}', + stdout: ["audit", "audit-and-cleanup"].includes(scenario) + ? "invalid JSON" + : '{"result":"pass"}', }; }, }, @@ -77,15 +86,21 @@ for (const scenario of [ if (scenario === "setup") assert.equal(error.cause, setupError); if (["expired", "near-expiry"].includes(scenario)) assert.equal(error.exitCode, 75); if (scenario === "timeout") assert.equal(error.cause.code, "ETIMEDOUT"); - if (["cleanup", "primary-and-cleanup"].includes(scenario)) { + if (["cleanup", "primary-and-cleanup", "audit-and-cleanup"].includes(scenario)) { assert(error instanceof Cp06CleanupError); assert.equal(error.exitCode, 74); assert.equal(error.errors.at(-1), cleanupError); + assert.equal(error.errors.length, scenario === "cleanup" ? 1 : 2); if (scenario === "primary-and-cleanup") { assert.equal(error.errors[0].exitCode, 75); assert.match(error.errors[0].message, /DUMMY auth blocked/); assert.equal(error.cause, error.errors[0]); } + if (scenario === "audit-and-cleanup") { + assert.equal(error.errors[0].exitCode, 70); + assert.match(error.errors[0].message, /non-compact output/); + assert.equal(error.cause, error.errors[0]); + } } return true; }); @@ -100,6 +115,82 @@ for (const scenario of [ }); } +test("audit and cleanup causes survive supervisor output and replay blocked recording", async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-output-chain-")); + const source = join(directory, "DUMMY-source"); + const cleanupError = new Error("DUMMY independent cleanup failure"); + await writeFile(source, "DUMMY ONLY"); + let cleanups = 0; + let caught; + try { + await superviseCredentialChild( + { + mode: "worker", + source, + target: join(directory, "DUMMY-target"), + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + }, + { + async mount() { + return { + mountIds: { source: "1", target: "2" }, + async cleanup() { + await new Promise((done) => setTimeout(done, 5)); + cleanups++; + throw cleanupError; + }, + }; + }, + spawn() { + return { status: 0, stdout: "DUMMY malformed child output" }; + }, + }, + ); + } catch (error) { + caught = error; + } + try { + assert(caught instanceof Cp06CleanupError); + assert.equal(cleanups, 1); + assert.equal(caught.errors.length, 2); + assert.equal(caught.errors[0].exitCode, 70); + assert.equal(caught.errors[1], cleanupError); + + const supervisorRecord = supervisorFailureRecord(caught); + assert.equal(supervisorRecord.code, "CP06_CLEANUP_FAILED"); + assert.equal(supervisorRecord.exit_code, 74); + assert.deepEqual( + supervisorRecord.causes.map(({ code, exit_code }) => ({ code, exit_code })), + [ + { code: "CP06_WORKER_FAILED", exit_code: 70 }, + { code: "CP06_WORKER_FAILED", exit_code: null }, + ], + ); + assert.match(supervisorRecord.message, /non-compact output/); + assert.match(supervisorRecord.message, /independent cleanup failure/); + + const workerError = namespaceWorkerError({ + status: 74, + stderr: JSON.stringify(supervisorRecord), + }); + const blocked = createWorkerBlockedStatus( + workerError, + "isolated-sdk-worker", + "2026-10-01T00:00:00.000Z", + ); + assert.equal(blocked.cleanup_failed, true); + assert.equal(blocked.code, "CP06_CLEANUP_FAILED"); + assert.deepEqual(blocked.worker_failure, { + code: "CP06_CLEANUP_FAILED", + exit_code: 74, + causes: supervisorRecord.causes, + }); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + test("evaluation-home cleanup finishes before the blocked result is returned", async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-home-")); const primary = Object.assign(new Error("DUMMY blocked"), { exitCode: 75 }); From 300bd38d4359468b62418868b76ca6a96bfc88f9 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 11:07:46 +0000 Subject: [PATCH 22/45] no-mistakes(review): Harden CP-06 credential isolation and evidence validation --- docs/probes/CP-06.md | 20 ++- scripts/cp06-auth-security.mjs | 80 +++++++++++- scripts/cp06-credential-isolation.mjs | 84 ++++++++++-- scripts/cp06-dummy-outcome-proof.mjs | 25 +++- scripts/cp06-dummy-worker-hook.mjs | 10 +- scripts/cp06-guarded-read.mjs | 79 ++++++++++++ scripts/cp06-isolation-adversary.mjs | 34 ++++- scripts/cp06-namespace-supervisor.mjs | 147 +++++++++++++++------ scripts/cp06-sdk-worker.mjs | 37 ++++-- scripts/cp06-worker-outcome.mjs | 28 +++- scripts/replay-cp06-worker.mjs | 17 ++- test/cp06-credential-isolation.test.mjs | 49 +++++-- test/cp06-guarded-read.test.mjs | 89 +++++++++++++ test/cp06-worker-lifecycle.test.mjs | 163 +++++++++++++++++++++--- test/cp06-worker-outcome.test.mjs | 16 +++ 15 files changed, 761 insertions(+), 117 deletions(-) create mode 100644 scripts/cp06-guarded-read.mjs create mode 100644 test/cp06-guarded-read.test.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 7e61554..e5a9523 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -19,9 +19,9 @@ daemon. ## Read-only Pi authentication correction -The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. +The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and exactly the built-in `read` tool. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact hashes/counts and parsed output, never credential contents or raw events. +The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before delegating to Pi's filesystem read. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/`, prepends only that installation's `.bin` for the proof commands, and leaves the global installation/configuration unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the resolved package/entry paths and their SHA-256 digests. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. @@ -191,8 +191,8 @@ not fabricate completeness. After deterministic checks and the real-CTX replay, a new Pi `0.85.1` SDK session runs with an injected read-only credential store and model runtime, an in-memory session/settings manager, an explicit resource loader containing no -extensions, skills, prompts, context files, or themes, and exactly the built-in -`read` tool. Its initial handoff contains exactly the AUTH-before contract, +extensions, skills, prompts, context files, or themes, and exactly one guarded +SDK `read` tool. Its initial handoff contains exactly the AUTH-before contract, bounded missing-AUTH pack, and a short deterministic evidence statement—no prior transcript. @@ -237,7 +237,9 @@ reproduced two additional failures: a local faux SDK response with runner's `finally`, leaving its evaluation directory and empty mount placeholder. Neither finding demonstrated credential mutation; the DUMMY S1/S2 controls passed. -The worker now requires an executable structured creation-evidence disposition: +The worker now requires an exact top-level response shape with one declared +canonical read, one corrected-constraint string, no secondary evidence channel, +and an executable structured creation-evidence disposition: ```json { @@ -265,11 +267,15 @@ and near-expiry preflight denial, and an SDK timeout. Every case is explicitly `fixture_origin: true`, `semantic_acceptance: false`; these are protocol and infrastructure proofs, never live-model acceptance. Default AuthStorage, refresh, and network entry are instrumented and must remain zero. The full read still -uses the actual built-in SDK read tool, not a fabricated completion event. +uses Pi's actual SDK read implementation behind the pre-execution allowlist, +not a fabricated completion event. The runner and namespace supervisor await cleanup before setting an exit code. When execution and cleanup both fail, the cleanup error retains the primary -cause and its exit identity. PR12 still parsed successful-child output after +cause and its exit identity. Supervisor output is an allowlisted stage/code/ +description protocol; launch, timeout, child exit, source-integrity audit, +output audit, and cleanup causes remain distinct without retaining child stderr +or arbitrary exception payloads. PR12 still parsed successful-child output after cleanup, so malformed output plus cleanup failure lost the parse/audit cause. The correction keeps child status, source-integrity verification, compact JSON parsing, and evidence assembly inside the cleanup action. The shipped diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index b6a7773..c08fb6c 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -148,19 +148,21 @@ export function namespaceWorkerError(result) { : `CP-06 isolated SDK worker exited ${result.status ?? "without status"}`, ); error.code = result.status === 73 ? "CP06_ISOLATION_UNSUPPORTED" : "CP06_WORKER_FAILED"; + error.stage = result.status === 73 ? "setup" : "namespace"; error.exitCode = result.status ?? 70; - error.stderr = result.stderr; - // Preserve the supervisor's compact failure identity, including both causes - // when child execution or audit and mount cleanup fail. Do not persist raw stderr. try { const failure = JSON.parse(String(result.stderr).trim()); - if (failure.schema_version === 1 && failure.exit_code === result.status) { + if (isSupervisorFailure(failure, result.status)) { error.workerFailure = { + schema_version: 1, + stage: failure.stage, code: failure.code, exit_code: failure.exit_code, - causes: failure.causes, + description: failure.description, + causes: failure.causes.map((cause) => ({ ...cause })), }; - if (failure.code === "CP06_CLEANUP_FAILED") error.code = failure.code; + error.code = failure.code; + error.stage = failure.stage; } } catch {} return error; @@ -218,6 +220,57 @@ function runNamespace(options) { }); } +function isSupervisorFailure(value, status) { + return ( + value?.schema_version === 1 && + value !== null && + typeof value === "object" && + !Array.isArray(value) && + Object.keys(value).sort().join(",") === + "causes,code,description,exit_code,schema_version,stage" && + value.exit_code === status && + typeof value.stage === "string" && + SUPERVISOR_FAILURE_CODES.get(value.code) === value.stage && + typeof value.description === "string" && + value.description === SUPERVISOR_FAILURE_DESCRIPTIONS[value.code] && + Array.isArray(value.causes) && + value.causes.every( + (cause) => + cause !== null && + typeof cause === "object" && + !Array.isArray(cause) && + Object.keys(cause).sort().join(",") === + "code,description,exit_code,stage" && + SUPERVISOR_FAILURE_CODES.get(cause.code) === cause.stage && + cause.description === SUPERVISOR_FAILURE_DESCRIPTIONS[cause.code] && + (cause.exit_code === null || Number.isSafeInteger(cause.exit_code)), + ) + ); +} + +const SUPERVISOR_FAILURE_DESCRIPTIONS = { + CP06_ISOLATION_SETUP_FAILED: "credential namespace setup failed", + CP06_CHILD_LAUNCH_FAILED: "isolated child launch failed", + CP06_CHILD_TIMEOUT: "isolated child timed out", + CP06_AUTH_BLOCKED: "credential preflight blocked the child", + CP06_CHILD_EXIT_FAILED: "isolated child exited unsuccessfully", + CP06_SOURCE_IDENTITY_CHANGED: "credential source identity changed", + CP06_CHILD_OUTPUT_INVALID: "isolated child output was invalid", + CP06_CLEANUP_FAILED: "credential namespace cleanup failed", + CP06_CREDENTIAL_CLEANUP_FAILED: "credential mount cleanup failed", +}; +const SUPERVISOR_FAILURE_CODES = new Map([ + ["CP06_ISOLATION_SETUP_FAILED", "setup"], + ["CP06_CHILD_LAUNCH_FAILED", "launch"], + ["CP06_CHILD_TIMEOUT", "timeout"], + ["CP06_AUTH_BLOCKED", "child-exit"], + ["CP06_CHILD_EXIT_FAILED", "child-exit"], + ["CP06_SOURCE_IDENTITY_CHANGED", "source-integrity"], + ["CP06_CHILD_OUTPUT_INVALID", "audit"], + ["CP06_CLEANUP_FAILED", "cleanup"], + ["CP06_CREDENTIAL_CLEANUP_FAILED", "cleanup"], +]); + function assertUnsafeControl(proof) { assert(proof.mode === "unsafe-probe", "unsafe control mode mismatch"); assert(proof.mount_ids_distinct === true, "unsafe control lacked distinct mounts"); @@ -226,6 +279,7 @@ function assertUnsafeControl(proof) { assert(proof.child.write_after_remount.exit === 0, "unsafe post-remount write failed"); assert(proof.child.unmount_target.exit === 0, "unsafe unmount did not succeed"); assert(proof.child.bind_parent_alias.exit === 0, "unsafe parent bind did not succeed"); + assert(proof.child.scratch_write.exit === 0, "unsafe scratch path was not writable"); assert(proof.child.alias_underlying_write.exit === 0, "unsafe alias write did not succeed"); assert( proof.child.identity.status.CapEff !== "0000000000000000", @@ -253,6 +307,16 @@ function assertHardenedProof(proof) { "symlink_source_replacement", "hardlink_target", "hardlink_source", + "create_target_parent_entry", + "create_source_parent_entry", + "unlink_target_parent", + "unlink_source_parent", + "rename_target_parent", + "rename_source_parent", + "symlink_in_target_parent", + "symlink_in_source_parent", + "hardlink_into_target_parent", + "hardlink_into_source_parent", "nested_user_mount_namespace", "nsenter_self", "nsenter_supervisor", @@ -266,6 +330,8 @@ function assertHardenedProof(proof) { } assert(proof.child.source.unchanged === true, "hardened child source hash changed"); assert(proof.child.alias_underlying_write.skipped === true, "alias write was attempted"); + assert(proof.child.scratch_write.exit === 0, "dedicated scratch path was not writable"); + assert(proof.parent_roots_read_only === true, "credential parent roots were not read-only"); assert(proof.child.ordinary_node_child.exit === 0, "ordinary child process was blocked"); assert( proof.child.ordinary_node_child.stdout === "DUMMY-child-ok", @@ -415,6 +481,8 @@ function summarizeHardened(proof) { namespace_entry_creation_denied: true, uid_change_denied: true, alternate_parent_bind_denied: true, + credential_parent_roots_read_only: proof.parent_roots_read_only, + dedicated_scratch_writable: proof.child.scratch_write.exit === 0, dummy_source_unchanged: proof.source_unchanged, }; } diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs index a142eda..24a2e03 100644 --- a/scripts/cp06-credential-isolation.mjs +++ b/scripts/cp06-credential-isolation.mjs @@ -67,9 +67,11 @@ export async function mountLiveCredentialReadOnly(options) { const maxBytes = options.maxBytes ?? MAX_CREDENTIAL_BYTES; const spawn = options.spawnSyncImpl ?? spawnSync; const statPath = options.lstatImpl ?? lstat; + const protectParents = options.protectParents !== false; let handle; let sourceMounted = false; let targetMounted = false; + const mountedParents = []; try { handle = await open(source, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC); const pinned = await handle.stat({ bigint: true }); @@ -78,6 +80,18 @@ export async function mountLiveCredentialReadOnly(options) { await mkdir(dirname(target), { recursive: true, mode: 0o700 }); await writeFile(target, "", { flag: "wx", mode: 0o600 }); + const parentMounts = []; + if (protectParents) { + for (const parent of [...new Set([dirname(source), dirname(target)])]) { + checkedSpawn("mount", ["--bind", parent, parent], { root, spawn }); + mountedParents.push(parent); + checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", parent], { + root, + spawn, + }); + parentMounts.push(inspectReadOnlyMount(parent, { spawn })); + } + } const pinnedPath = `/proc/${process.pid}/fd/${handle.fd}`; checkedSpawn("mount", ["--bind", pinnedPath, source], { root, spawn }); @@ -100,12 +114,7 @@ export async function mountLiveCredentialReadOnly(options) { } for (const path of [source, target]) { const mounted = await statPath(path, { bigint: true }); - if ( - !mounted.isFile() || - mounted.dev !== pinned.dev || - mounted.ino !== pinned.ino || - mounted.size !== pinned.size - ) { + if (!mounted.isFile() || !sameStatIdentity(pinned, mounted)) { throw new Error(`credential mount identity changed: ${path}`); } } @@ -116,18 +125,37 @@ export async function mountLiveCredentialReadOnly(options) { return { source, target, - sourceIdentity: { - dev: String(pinned.dev), - ino: String(pinned.ino), - size: Number(pinned.size), - }, + sourceIdentity: statIdentity(pinned), mountIds: { source: sourceMount.id, target: targetMount.id }, + parentMounts: parentMounts.map((entry) => ({ id: entry.id, target: entry.target })), + async verifyIntegrity() { + for (const path of [source, target]) { + const mounted = await statPath(path, { bigint: true }); + if (!mounted.isFile() || !sameStatIdentity(pinned, mounted)) return false; + } + const currentSourceMount = inspectReadOnlyMount(source, { spawn }); + const currentTargetMount = inspectReadOnlyMount(target, { spawn }); + if ( + currentSourceMount.id !== sourceMount.id || + currentTargetMount.id !== targetMount.id + ) { + return false; + } + for (const expected of parentMounts) { + const current = inspectReadOnlyMount(expected.target, { spawn }); + if (current.id !== expected.id) return false; + } + return true; + }, cleanup() { if (cleaned) return; cleaned = true; const failures = []; if (targetMounted) unmount(target, { root, spawn, failures }); if (sourceMounted) unmount(source, { root, spawn, failures }); + for (const parent of [...mountedParents].reverse()) { + unmount(parent, { root, spawn, failures }); + } if (failures.length > 0) { throw new Error(`credential mount cleanup failed: ${failures.join("; ")}`); } @@ -137,6 +165,9 @@ export async function mountLiveCredentialReadOnly(options) { const cleanupFailures = []; if (targetMounted) unmount(target, { root, spawn, failures: cleanupFailures }); if (sourceMounted) unmount(source, { root, spawn, failures: cleanupFailures }); + for (const parent of [...mountedParents].reverse()) { + unmount(parent, { root, spawn, failures: cleanupFailures }); + } if (cleanupFailures.length > 0) { throw new AggregateError( [error, ...cleanupFailures.map((message) => new Error(message))], @@ -156,13 +187,14 @@ export async function assertDestructiveCredentialWritesDenied(paths, operations const makeSymlink = operations.symlink ?? symlink; const makeHardlink = operations.link ?? link; const cleanup = operations.rm ?? rm; + const scratchRoot = operations.scratchRoot ?? tmpdir(); for (const target of paths) { await expectDenied(`write to ${target}`, () => write(target, "DUMMY unauthorized mutation")); await expectDenied(`unlink of ${target}`, () => remove(target)); const nonce = `${process.pid}-${Date.now()}-${Math.random().toString(16).slice(2)}`; - const replacement = join(dirname(target), `.DUMMY-replacement-${nonce}`); - const symbolic = join(dirname(target), `.DUMMY-symlink-${nonce}`); - const hard = join(dirname(target), `.DUMMY-hardlink-${nonce}`); + const replacement = join(scratchRoot, `.DUMMY-replacement-${nonce}`); + const symbolic = join(scratchRoot, `.DUMMY-symlink-${nonce}`); + const hard = join(scratchRoot, `.DUMMY-hardlink-${nonce}`); await write(replacement, "DUMMY unauthorized replacement"); try { await expectDenied(`atomic replacement of ${target}`, () => move(replacement, target)); @@ -210,6 +242,30 @@ export function inspectReadOnlyMount(path, options = {}) { return { id: String(entry.id), target: resolve(entry.target), options: optionsList }; } +export function statIdentity(stat) { + return { + dev: String(stat.dev), + ino: String(stat.ino), + size: String(stat.size), + nlink: String(stat.nlink), + mode: String(stat.mode), + mtime_ns: String(stat.mtimeNs), + ctime_ns: String(stat.ctimeNs), + }; +} + +function sameStatIdentity(expected, actual) { + return ( + expected.dev === actual.dev && + expected.ino === actual.ino && + expected.size === actual.size && + expected.nlink === actual.nlink && + expected.mode === actual.mode && + expected.mtimeNs === actual.mtimeNs && + expected.ctimeNs === actual.ctimeNs + ); +} + function assertPinnedRegularFile(pinned, named, maxBytes) { if (!pinned.isFile() || !named.isFile()) { throw new Error("Pi credential source is not a regular file"); diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index 1256dd8..1991931 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -1,6 +1,6 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; @@ -18,6 +18,7 @@ export async function proveDummyWorkerOutcome({ piRoot }) { originalPath, "# DUMMY authority\n\nOwnership comes only from the authenticated principal; request-supplied owner fields are ignored.\n", ); + const credentialAlias = join(directory, "DUMMY-credential-alias.json"); const contractPath = join(directory, "DUMMY-contract.yaml"); const packPath = join(directory, "DUMMY-pack.md"); await writeFile( @@ -34,6 +35,10 @@ export async function proveDummyWorkerOutcome({ piRoot }) { ["empty-gap", 75], ["missing-gap", 75], ["contradictory-gap", 75], + ["extra-evidence", 75], + ["missing-reads", 75], + ["credential-read", 75], + ["symlink-read", 75], ["expired", 75], ["near-expiry", 75], ["timeout", 70], @@ -53,6 +58,7 @@ export async function proveDummyWorkerOutcome({ piRoot }) { }, }); await writeFile(auth, authBytes); + if (scenario === "symlink-read") await symlink(auth, credentialAlias); await writeFile( inputPath, JSON.stringify({ @@ -62,6 +68,7 @@ export async function proveDummyWorkerOutcome({ piRoot }) { contract_path: contractPath, pack_path: packPath, original_path: originalPath, + symlink_path: credentialAlias, provider: "openai-codex", model: "DUMMY-model", timeout_ms: scenario === "timeout" ? 1_000 : 20_000, @@ -103,8 +110,21 @@ export async function proveDummyWorkerOutcome({ piRoot }) { assert.equal(counts.refresh, 0); assert.equal(await readFile(auth, "utf8"), authBytes); if (["expired", "near-expiry"].includes(scenario)) assert.equal(counts.runtime_create, 0); - if (["empty-gap", "missing-gap", "contradictory-gap"].includes(scenario)) + if ( + [ + "empty-gap", + "missing-gap", + "contradictory-gap", + "extra-evidence", + "missing-reads", + ].includes(scenario) + ) assert.match(result.stderr, /SDK worker outcome failed/); + if (["credential-read", "symlink-read"].includes(scenario)) { + assert.match(result.stderr, /SDK event audit failed/); + assert.equal(result.stdout.includes("DUMMY-ACCESS"), false); + assert.equal(result.stderr.includes("DUMMY-ACCESS"), false); + } if (scenario === "timeout") assert.match(result.stderr, /worker timed out and was aborted/); const output = scenario === "valid" ? JSON.parse(result.stdout) : undefined; if (output) { @@ -113,6 +133,7 @@ export async function proveDummyWorkerOutcome({ piRoot }) { assert.equal(output.event_stream.read_audit.read_count, 1); assert.equal(output.response.evidence_gaps[0].status, "unverified"); } else assert.equal(result.stdout, ""); + await rm(credentialAlias, { force: true }); cases.push({ scenario, exit_code: result.status, diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index 2286c5e..d1f7840 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -78,9 +78,17 @@ sdk.ModelRuntime.create = async function (options) { broad_scan: false, }; if (scenario === "missing-gap") delete response.evidence_gaps; + if (scenario === "extra-evidence") response.creation_evidence = { status: "passed" }; + if (scenario === "missing-reads") delete response.reads; + const readPath = + scenario === "credential-read" + ? process.argv[2] + : scenario === "symlink-read" + ? input.symlink_path + : input.original_path; faux.setResponses([ ai.fauxAssistantMessage( - ai.fauxToolCall("read", { path: input.original_path }, { id: "DUMMY-actual-worker-read" }), + ai.fauxToolCall("read", { path: readPath }, { id: "DUMMY-actual-worker-read" }), { stopReason: "toolUse" }, ), ai.fauxAssistantMessage(JSON.stringify(response), { stopReason: "stop" }), diff --git a/scripts/cp06-guarded-read.mjs b/scripts/cp06-guarded-read.mjs new file mode 100644 index 0000000..fd78fd9 --- /dev/null +++ b/scripts/cp06-guarded-read.mjs @@ -0,0 +1,79 @@ +import { realpathSync } from "node:fs"; +import { isAbsolute, relative, resolve } from "node:path"; + +export function canonicalWorkerPaths(input, credentialTarget, credentialSource) { + const root = canonicalExactPath(input.root, "root"); + const fixtureRoot = canonicalExactPath(input.fixture_root, "fixture_root"); + const contractPath = canonicalExactPath(input.contract_path, "contract_path"); + const packPath = canonicalExactPath(input.pack_path, "pack_path"); + const originalPath = canonicalExactPath(input.original_path, "original_path"); + if (!isWithin(fixtureRoot, originalPath)) { + throw new Error("worker original_path escapes its fixture root"); + } + const credentialPaths = [credentialTarget, credentialSource] + .filter((value) => typeof value === "string" && value.length > 0) + .map((value) => resolve(value)); + if (credentialPaths.includes(originalPath)) { + throw new Error("worker original_path overlaps a credential path"); + } + return { root, fixtureRoot, contractPath, packPath, originalPath }; +} + +export function createGuardedReadTool({ sdk, root, expectedOriginalPath }) { + if (typeof sdk?.createReadTool !== "function") { + throw new Error("Pi SDK guarded read constructor is unavailable"); + } + const canonicalOriginal = canonicalExactPath(expectedOriginalPath, "expected original"); + const read = sdk.createReadTool(root); + if (read?.name !== "read" || typeof read.execute !== "function") { + throw new Error("Pi SDK read tool surface is unsupported"); + } + return { + ...read, + async execute(toolCallId, args, ...rest) { + assertAllowedRead(args, canonicalOriginal); + return read.execute.call(read, toolCallId, args, ...rest); + }, + }; +} + +export function assertAllowedRead(args, expectedOriginalPath) { + if ( + args === null || + typeof args !== "object" || + Array.isArray(args) || + Object.keys(args).some((key) => !["path", "offset", "limit"].includes(key)) || + args.path !== expectedOriginalPath + ) { + throw new Error("read denied: only the canonical supplied original is allowed"); + } + const hasOffset = Object.hasOwn(args, "offset"); + const hasLimit = Object.hasOwn(args, "limit"); + if (hasOffset !== hasLimit) throw new Error("read denied: incomplete explicit range"); + if ( + hasOffset && + (args.offset !== 1 || !Number.isSafeInteger(args.limit) || args.limit < 1) + ) { + throw new Error("read denied: explicit range must start at one with a positive limit"); + } +} + +function canonicalExactPath(value, label) { + if (typeof value !== "string" || !isAbsolute(value)) { + throw new Error(`worker ${label} must be absolute`); + } + const resolved = resolve(value); + let canonical; + try { + canonical = realpathSync.native(resolved); + } catch { + throw new Error(`worker ${label} is unavailable`); + } + if (canonical !== resolved) throw new Error(`worker ${label} must already be canonical`); + return canonical; +} + +function isWithin(parent, child) { + const path = relative(parent, child); + return path === "" || (!path.startsWith("..") && !isAbsolute(path)); +} diff --git a/scripts/cp06-isolation-adversary.mjs b/scripts/cp06-isolation-adversary.mjs index c50fe10..dfbf41d 100644 --- a/scripts/cp06-isolation-adversary.mjs +++ b/scripts/cp06-isolation-adversary.mjs @@ -6,6 +6,7 @@ import { mkdirSync, readFileSync, renameSync, + rmdirSync, symlinkSync, unlinkSync, writeFileSync, @@ -50,10 +51,16 @@ jsAttempt("direct_write_target", () => writeFileSync(target, "DUMMY BYPASS targe jsAttempt("direct_write_source", () => writeFileSync(source, "DUMMY BYPASS source")); jsAttempt("unlink_target", () => unlinkSync(target)); jsAttempt("unlink_source", () => unlinkSync(source)); -const replacement = join(dirname(target), "DUMMY-replacement"); +jsAttempt("create_target_parent_entry", () => + writeFileSync(join(dirname(target), "DUMMY-parent-entry"), "DUMMY parent replacement"), +); +jsAttempt("create_source_parent_entry", () => + writeFileSync(join(dirname(source), "DUMMY-parent-entry"), "DUMMY parent replacement"), +); +const replacement = join(proofRoot, "DUMMY-target-replacement"); writeFileSync(replacement, "DUMMY replacement"); jsAttempt("rename_over_target", () => renameSync(replacement, target)); -const sourceReplacement = join(dirname(source), "DUMMY-source-replacement"); +const sourceReplacement = join(proofRoot, "DUMMY-source-replacement"); writeFileSync(sourceReplacement, "DUMMY source replacement"); jsAttempt("rename_over_source", () => renameSync(sourceReplacement, source)); jsAttempt("symlink_replacement", () => { @@ -72,6 +79,26 @@ jsAttempt("hardlink_target", () => linkSync(target, join(proofRoot, "DUMMY-hardl jsAttempt("hardlink_source", () => linkSync(source, join(proofRoot, "DUMMY-source-hardlink"))); if (securityMode === "hardened") { + jsAttempt("unlink_target_parent", () => rmdirSync(dirname(target))); + jsAttempt("unlink_source_parent", () => rmdirSync(dirname(source))); + jsAttempt("rename_target_parent", () => + renameSync(dirname(target), join(proofRoot, "DUMMY-renamed-target-parent")), + ); + jsAttempt("rename_source_parent", () => + renameSync(dirname(source), join(proofRoot, "DUMMY-renamed-source-parent")), + ); + jsAttempt("symlink_in_target_parent", () => + symlinkSync("/tmp/DUMMY-does-not-exist", join(dirname(target), "DUMMY-parent-symlink")), + ); + jsAttempt("symlink_in_source_parent", () => + symlinkSync("/tmp/DUMMY-does-not-exist", join(dirname(source), "DUMMY-parent-symlink")), + ); + jsAttempt("hardlink_into_target_parent", () => + linkSync(target, join(dirname(target), "DUMMY-parent-hardlink")), + ); + jsAttempt("hardlink_into_source_parent", () => + linkSync(source, join(dirname(source), "DUMMY-parent-hardlink")), + ); commandAttempt("direct_namespace_syscalls", syscallProbe, [target]); } else { results.direct_namespace_syscalls = { exit: null, skipped: true }; @@ -107,6 +134,9 @@ if (unmountTarget.status === 0) { } else { results.restore_target_bind = { exit: null, skipped: true }; } +jsAttempt("scratch_write", () => + writeFileSync(join(proofRoot, "DUMMY-writable-scratch"), "DUMMY scratch"), +); const alias = join(proofRoot, "DUMMY-alias"); mkdirSync(alias, { recursive: true }); const bindParent = commandAttempt("bind_parent_alias", "mount", ["--bind", dirname(source), alias]); diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 360c57d..d708fcd 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -19,20 +19,32 @@ export async function superviseCredentialChild(options, dependencies = {}) { (value) => typeof value === "string" && value.length > 0, ) ) - throw failure("invalid CP-06 namespace supervisor invocation", 64); + throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 64); if (process.platform !== "linux" || process.arch !== "x64") { - throw failure("CP-06 isolation supports Linux x86-64 only", 73); + throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73); } const mountCredential = dependencies.mount ?? mountLiveCredentialReadOnly; const spawn = dependencies.spawn ?? spawnSync; - const beforeHash = sha256(readFileSync(source)); + const beforeHash = mode === "worker" ? null : sha256(readFileSync(source)); let mount; return await withCleanup( async () => { try { - mount = await mountCredential({ root, source, target }); + mount = await mountCredential({ + root, + source, + target, + protectParents: mode !== "unsafe-probe", + }); } catch (cause) { - throw failure(`CP-06 isolation setup failed: ${safeError(cause)}`, 73, cause); + const setup = failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73, cause); + if (cause instanceof AggregateError) { + throw compoundFailure( + setup, + failure("cleanup", "CP06_CREDENTIAL_CLEANUP_FAILED", 74), + ); + } + throw setup; } const command = childCommand({ mode, @@ -43,43 +55,59 @@ export async function superviseCredentialChild(options, dependencies = {}) { childArgs, beforeHash, }); - const result = spawn(command.command, command.args, { - cwd: root, - encoding: "utf8", - env: childEnvironment(mode, target), - timeout: mode === "worker" ? 600_000 : 60_000, - maxBuffer: 1024 * 1024, - }); - // Throw inside the cleanup boundary so a cleanup error retains this cause. + let result; + try { + result = spawn(command.command, command.args, { + cwd: root, + encoding: "utf8", + env: childEnvironment(mode, source, target), + timeout: mode === "worker" ? 600_000 : 60_000, + maxBuffer: 1024 * 1024, + }); + } catch (cause) { + throw failure("launch", "CP06_CHILD_LAUNCH_FAILED", 70, cause); + } if (result.error !== undefined) { + const timeout = result.error?.code === "ETIMEDOUT"; throw failure( - `CP-06 isolated child launch failed: ${safeError(result.error)}`, + timeout ? "timeout" : "launch", + timeout ? "CP06_CHILD_TIMEOUT" : "CP06_CHILD_LAUNCH_FAILED", 70, result.error, ); } if (result.status !== 0) { throw failure( - `CP-06 isolated child exited ${result.status ?? "without status"}: ${String(result.stderr ?? "").trim()}`, + "child-exit", + result.status === 75 ? "CP06_AUTH_BLOCKED" : "CP06_CHILD_EXIT_FAILED", result.status ?? 70, ); } - const afterHash = sha256(readFileSync(source)); - if (mode === "worker" && beforeHash !== afterHash) { - throw failure("CP-06 credential source changed during worker execution", 74); + let sourceUnchanged; + try { + sourceUnchanged = + mode === "worker" + ? await mount.verifyIntegrity() + : beforeHash === sha256(readFileSync(source)); + } catch { + sourceUnchanged = false; + } + if (!sourceUnchanged) { + throw failure("source-integrity", "CP06_SOURCE_IDENTITY_CHANGED", 74); } let childEvidence; try { childEvidence = JSON.parse(String(result.stdout).trim()); } catch { - throw failure("CP-06 isolated child returned non-compact output", 70); + throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); } return { schema_version: 1, mode, - source_unchanged: beforeHash === afterHash, + source_unchanged: sourceUnchanged, mount_ids_distinct: mount.mountIds.source !== mount.mountIds.target, + parent_roots_read_only: mount.parentMounts.length > 0, cleanup: "pass", child: childEvidence, }; @@ -145,7 +173,7 @@ function childCommand({ mode, source, target, helper, syscallProbe, childArgs, b }; } -function childEnvironment(childMode, credentialTarget) { +function childEnvironment(childMode, credentialSource, credentialTarget) { const environment = { PATH: process.env.PATH ?? "/usr/bin:/bin", LANG: process.env.LANG ?? "C.UTF-8", @@ -155,6 +183,7 @@ function childEnvironment(childMode, credentialTarget) { PI_TELEMETRY: "0", }; if (childMode === "worker") { + environment.CP06_CREDENTIAL_SOURCE = credentialSource; environment.CP06_CREDENTIAL_TARGET = credentialTarget; for (const name of [ "SSL_CERT_FILE", @@ -171,31 +200,77 @@ function childEnvironment(childMode, credentialTarget) { } export function supervisorFailureRecord(error) { + const failure = classifyFailure(error); return { schema_version: 1, - code: error?.code ?? "CP06_WORKER_FAILED", - exit_code: error?.exitCode ?? 70, - message: safeError(error), + stage: failure.stage, + code: failure.code, + exit_code: failure.exit_code, + description: failure.description, causes: - error instanceof AggregateError - ? error.errors.map((cause) => ({ - code: cause?.code ?? "CP06_WORKER_FAILED", - exit_code: cause?.exitCode ?? null, - })) - : [], + error instanceof AggregateError ? error.errors.map((cause) => classifyFailure(cause)) : [], }; } -function failure(message, exitCode, cause) { - return Object.assign(new Error(message, { cause }), { exitCode, code: "CP06_WORKER_FAILED" }); +function failure(stage, code, exitCode, cause) { + return Object.assign(new Error(code, { cause }), { stage, exitCode, code }); +} + +function compoundFailure(primary, cleanup) { + return Object.assign(new AggregateError([primary, cleanup], "CP06_CLEANUP_FAILED"), { + stage: "cleanup", + exitCode: 74, + code: "CP06_CLEANUP_FAILED", + }); } function sha256(bytes) { return createHash("sha256").update(bytes).digest("hex"); } -function safeError(error) { - if (error instanceof AggregateError) - return `${error.message}: ${error.errors.map(safeError).join("; ")}`; - return error instanceof Error ? error.message : String(error); +function classifyFailure(error) { + const known = FAILURE_DESCRIPTIONS[error?.code]; + if (known !== undefined) { + return { + stage: error.stage ?? known.stage, + code: error.code, + exit_code: error.exitCode ?? null, + description: known.description, + }; + } + if (error instanceof AggregateError) { + return { + stage: "cleanup", + code: "CP06_CLEANUP_FAILED", + exit_code: error.exitCode ?? 74, + description: "credential namespace cleanup failed", + }; + } + return { + stage: "cleanup", + code: "CP06_CREDENTIAL_CLEANUP_FAILED", + exit_code: error?.exitCode ?? null, + description: "credential mount cleanup failed", + }; } + +const FAILURE_DESCRIPTIONS = { + CP06_ISOLATION_SETUP_FAILED: { + stage: "setup", + description: "credential namespace setup failed", + }, + CP06_CHILD_LAUNCH_FAILED: { stage: "launch", description: "isolated child launch failed" }, + CP06_CHILD_TIMEOUT: { stage: "timeout", description: "isolated child timed out" }, + CP06_AUTH_BLOCKED: { stage: "child-exit", description: "credential preflight blocked the child" }, + CP06_CHILD_EXIT_FAILED: { stage: "child-exit", description: "isolated child exited unsuccessfully" }, + CP06_SOURCE_IDENTITY_CHANGED: { + stage: "source-integrity", + description: "credential source identity changed", + }, + CP06_CHILD_OUTPUT_INVALID: { stage: "audit", description: "isolated child output was invalid" }, + CP06_CLEANUP_FAILED: { stage: "cleanup", description: "credential namespace cleanup failed" }, + CP06_CREDENTIAL_CLEANUP_FAILED: { + stage: "cleanup", + description: "credential mount cleanup failed", + }, +}; diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs index 29c5e33..ebb991e 100644 --- a/scripts/cp06-sdk-worker.mjs +++ b/scripts/cp06-sdk-worker.mjs @@ -6,6 +6,7 @@ import { readFile } from "node:fs/promises"; import { delimiter, dirname, join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { Cp06AuthBlockedError, Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; +import { createGuardedReadTool, canonicalWorkerPaths } from "./cp06-guarded-read.mjs"; import { auditReadEvents } from "./cp06-worker-audit.mjs"; import { auditWorkerOutcome } from "./cp06-worker-outcome.mjs"; @@ -16,14 +17,19 @@ try { const credentialTarget = resolve(credentialTargetArg); const input = JSON.parse(await readFile(resolve(inputPathArg), "utf8")); validateInput(input); + const paths = canonicalWorkerPaths( + input, + credentialTarget, + process.env.CP06_CREDENTIAL_SOURCE, + ); const pi = resolvePiPackage(); const sdk = await import(pathToFileURL(join(pi.root, "dist/index.js"))); assertSdkSurface(sdk); - const contract = await readFile(input.contract_path, "utf8"); - const pack = await readFile(input.pack_path, "utf8"); - const original = await readFile(input.original_path); - const prompt = buildPrompt({ contract, pack, originalPath: input.original_path }); + const contract = await readFile(paths.contractPath, "utf8"); + const pack = await readFile(paths.packPath, "utf8"); + const original = await readFile(paths.originalPath); + const prompt = buildPrompt({ contract, pack, originalPath: paths.originalPath }); const store = await Cp06ReadOnlyCredentialStore.load({ path: credentialTarget, providerId: input.provider, @@ -54,14 +60,20 @@ try { enableAnalytics: false, enableInstallTelemetry: false, }); + const guardedRead = createGuardedReadTool({ + sdk, + root: paths.root, + expectedOriginalPath: paths.originalPath, + }); const { session, extensionsResult } = await sdk.createAgentSession({ - cwd: input.root, + cwd: paths.root, agentDir: dirname(credentialTarget), model, modelRuntime: runtime, resourceLoader, tools: ["read"], - sessionManager: sdk.SessionManager.inMemory(input.root), + customTools: [guardedRead], + sessionManager: sdk.SessionManager.inMemory(paths.root), settingsManager, thinkingLevel: "high", }); @@ -97,8 +109,8 @@ try { const responseText = finalAssistantText(events); const response = parseResponse(responseText); const readAudit = auditReadEvents(events, { - root: input.root, - expectedOriginalPath: input.original_path, + root: paths.root, + expectedOriginalPath: paths.originalPath, expectedOriginal: original, decisiveText: ["request-supplied owner fields are ignored"], }); @@ -106,9 +118,9 @@ try { throw blocked(`SDK event audit failed: ${readAudit.reason}`); } const outcome = auditWorkerOutcome(response, { - root: input.root, - referenceRoot: input.fixture_root, - expectedOriginalPath: input.original_path, + root: paths.root, + referenceRoot: paths.fixtureRoot, + expectedOriginalPath: paths.originalPath, }); if (!outcome.ok) throw blocked(`SDK worker outcome failed: ${outcome.reason}`); @@ -211,6 +223,7 @@ function assertSdkSurface(sdk) { "ModelRuntime", "createAgentSession", "createExtensionRuntime", + "createReadTool", "SessionManager", "SettingsManager", ]) { @@ -274,5 +287,5 @@ function blocked(message) { function safeMessage(error) { if (error?.message === "CP06_WORKER_TIMEOUT") return "worker timed out and was aborted"; - return error instanceof Error ? error.message : String(error); + return "worker execution failed"; } diff --git a/scripts/cp06-worker-outcome.mjs b/scripts/cp06-worker-outcome.mjs index d9a05fd..a1af007 100644 --- a/scripts/cp06-worker-outcome.mjs +++ b/scripts/cp06-worker-outcome.mjs @@ -1,15 +1,33 @@ import { isExpectedOriginalReference } from "./cp06-worker-audit.mjs"; export function auditWorkerOutcome(response, options) { + const topLevelKeys = [ + "broad_scan", + "corrected_constraints", + "evidence_gaps", + "missing_source", + "reads", + "status", + ]; const corrected = Array.isArray(response?.corrected_constraints) - ? response.corrected_constraints.join(" ") - : ""; + ? response.corrected_constraints + : []; if ( - response?.status !== "MISSING_SOURCE" || + response === null || + typeof response !== "object" || + Array.isArray(response) || + Object.keys(response).sort().join(",") !== topLevelKeys.join(",") || + response.status !== "MISSING_SOURCE" || !isExpectedOriginalReference(response.missing_source, options) || response.broad_scan !== false || - !/ownership comes only from the authenticated principal/i.test(corrected) || - !/request-supplied owner fields? (?:are )?ignored/i.test(corrected) + !Array.isArray(response.reads) || + response.reads.length !== 1 || + response.reads[0] !== options.expectedOriginalPath || + corrected.length !== 1 || + typeof corrected[0] !== "string" || + !/ownership comes only from the authenticated principal/i.test(corrected[0]) || + !/request-supplied owner fields? (?:are )?ignored/i.test(corrected[0]) || + /\b(?:passed|verified|proven|satisfied)\b/i.test(corrected[0]) ) { return { ok: false, reason: "response did not satisfy the targeted-read oracle" }; } diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 5f623e4..149101d 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -99,6 +99,7 @@ async function main() { isolated.mode !== "worker" || isolated.source_unchanged !== true || isolated.mount_ids_distinct !== true || + isolated.parent_roots_read_only !== true || isolated.cleanup !== "pass" || isolated.child?.result !== "pass" ) { @@ -138,6 +139,7 @@ async function main() { supported_platform: "Linux x86-64", dummy_security_preflight: security, source_and_target_distinct_read_only_mounts: isolated.mount_ids_distinct, + credential_parent_roots_read_only: isolated.parent_roots_read_only, evaluated_child_capabilities: "all sets empty", evaluated_child_no_new_privs: true, evaluated_child_seccomp: true, @@ -182,18 +184,27 @@ async function recordBlocked(error, stage) { } export function createWorkerBlockedStatus(error, stage, recordedAt = new Date().toISOString()) { + const nestedWorkerFailure = + error?.workerFailure ?? + (error instanceof AggregateError + ? error.errors.find((cause) => cause?.workerFailure !== undefined)?.workerFailure + : undefined); return { schema_version: 1, gate: "P-06-fresh-worker", result: "blocked", stage, + failure_stage: error?.stage ?? null, code: error?.code ?? "CP06_WORKER_FAILED", - cleanup_failed: error?.code === "CP06_CLEANUP_FAILED", - worker_failure: error?.workerFailure ?? null, + cleanup_failed: + error?.code === "CP06_CLEANUP_FAILED" || + nestedWorkerFailure?.code === "CP06_CLEANUP_FAILED", + worker_failure: nestedWorkerFailure ?? null, causes: error instanceof AggregateError ? error.errors.map((cause) => ({ - code: cause?.code ?? "CP06_WORKER_FAILED", + stage: cause?.stage ?? "cleanup", + code: cause?.code ?? "CP06_CREDENTIAL_CLEANUP_FAILED", exit_code: cause?.exitCode ?? null, })) : [], diff --git a/test/cp06-credential-isolation.test.mjs b/test/cp06-credential-isolation.test.mjs index cffe2f0..4918081 100644 --- a/test/cp06-credential-isolation.test.mjs +++ b/test/cp06-credential-isolation.test.mjs @@ -17,16 +17,18 @@ test("live credential mount pins one inode behind distinct read-only regular mou const sourceStat = await lstat(source, { bigint: true }); const calls = []; let mountId = 40; + const mountIds = new Map(); const spawnSyncImpl = (command, args) => { calls.push([command, ...args]); if (command === "findmnt") { const path = args.at(-1); + if (!mountIds.has(path)) mountIds.set(path, ++mountId); return { status: 0, stdout: JSON.stringify({ filesystems: [ { - id: ++mountId, + id: mountIds.get(path), target: path, "vfs-options": "ro,nosuid,nodev,noexec,relatime", }, @@ -49,19 +51,48 @@ test("live credential mount pins one inode behind distinct read-only regular mou try { assert.equal(await readFile(source, "utf8"), sourceBytes); assert.notEqual(mount.mountIds.source, mount.mountIds.target); - assert.match(calls[0][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.deepEqual(calls[0].slice(0, 2), ["mount", "--bind"]); - assert.equal(calls[0][3], source); - assert.deepEqual(calls[1], ["mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", source]); - assert.match(calls[2][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.equal(calls[2][3], target); - assert.deepEqual(calls[3], ["mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", target]); + const mountCallsOnly = calls.filter(([command]) => command === "mount"); + assert.deepEqual( + mountCallsOnly.slice(0, 4), + [ + ["mount", "--bind", root, root], + ["mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", root], + ["mount", "--bind", join(root, "agent"), join(root, "agent")], + [ + "mount", + "-o", + "remount,bind,ro,nosuid,nodev,noexec", + join(root, "agent"), + ], + ], + ); + assert.equal(mount.parentMounts.length, 2); + assert.equal(await mount.verifyIntegrity(), true); + assert.match(mountCallsOnly[4][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); + assert.deepEqual(mountCallsOnly[4].slice(0, 2), ["mount", "--bind"]); + assert.equal(mountCallsOnly[4][3], source); + assert.deepEqual(mountCallsOnly[5], [ + "mount", + "-o", + "remount,bind,ro,nosuid,nodev,noexec", + source, + ]); + assert.match(mountCallsOnly[6][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); + assert.equal(mountCallsOnly[6][3], target); + assert.deepEqual(mountCallsOnly[7], [ + "mount", + "-o", + "remount,bind,ro,nosuid,nodev,noexec", + target, + ]); } finally { mount.cleanup(); } - assert.deepEqual(calls.slice(-2), [ + assert.deepEqual(calls.slice(-4), [ ["umount", target], ["umount", source], + ["umount", join(root, "agent")], + ["umount", root], ]); } finally { await rm(root, { recursive: true, force: true }); diff --git a/test/cp06-guarded-read.test.mjs b/test/cp06-guarded-read.test.mjs new file mode 100644 index 0000000..49a590c --- /dev/null +++ b/test/cp06-guarded-read.test.mjs @@ -0,0 +1,89 @@ +import { strict as assert } from "node:assert"; +import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; +import { + canonicalWorkerPaths, + createGuardedReadTool, +} from "../scripts/cp06-guarded-read.mjs"; + +test("guarded read denies noncanonical and unauthorized paths before tool access", async () => { + const root = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-read-")); + const fixtureRoot = join(root, "fixture"); + const original = join(fixtureRoot, "docs", "AUTH.md"); + const credential = join(root, "DUMMY-auth.json"); + const alias = join(root, "DUMMY-auth-alias"); + await mkdir(join(fixtureRoot, "docs"), { recursive: true }); + await writeFile(original, "DUMMY exact original"); + await writeFile(credential, "DUMMY secret canary"); + await symlink(credential, alias); + let accesses = 0; + const sdk = { + createReadTool() { + return { + name: "read", + async execute(_id, args) { + accesses += 1; + return { content: [{ type: "text", text: args.path }] }; + }, + }; + }, + }; + const tool = createGuardedReadTool({ sdk, root, expectedOriginalPath: original }); + try { + for (const args of [ + { path: credential }, + { path: alias }, + { path: join(root, "PROJECT.md") }, + { path: original, offset: 1 }, + { path: original, limit: 20 }, + { path: original, offset: 2, limit: 20 }, + { path: original, offset: 1, limit: 0 }, + { path: original, extra: true }, + ]) { + await assert.rejects(tool.execute("DUMMY-read", args)); + } + assert.equal(accesses, 0); + await tool.execute("DUMMY-read", { path: original }); + await tool.execute("DUMMY-read", { path: original, offset: 1, limit: 20 }); + assert.equal(accesses, 2); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("worker paths reject symlink escapes and credential overlap", async () => { + const root = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-paths-")); + const fixtureRoot = join(root, "fixture"); + const original = join(fixtureRoot, "AUTH.md"); + const contract = join(root, "contract.yaml"); + const pack = join(root, "pack.md"); + const alias = join(fixtureRoot, "AUTH-alias.md"); + await mkdir(fixtureRoot); + await writeFile(original, "DUMMY original"); + await writeFile(contract, "DUMMY contract"); + await writeFile(pack, "DUMMY pack"); + await symlink(original, alias); + const input = { + root, + fixture_root: fixtureRoot, + contract_path: contract, + pack_path: pack, + original_path: original, + }; + try { + assert.equal(canonicalWorkerPaths(input, join(root, "auth.json")).originalPath, original); + assert.throws(() => + canonicalWorkerPaths({ ...input, original_path: alias }, credentialPath(root)), + ); + assert.throws(() => canonicalWorkerPaths(input, original)); + assert.throws(() => canonicalWorkerPaths(input, join(root, "target-auth.json"), original)); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +function credentialPath(root) { + return join(root, "auth.json"); +} diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 7189d95..03433d8 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -1,5 +1,5 @@ import { strict as assert } from "node:assert"; -import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { lstat, mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; @@ -16,6 +16,7 @@ for (const scenario of [ "expired", "near-expiry", "setup", + "setup-and-cleanup", "launch", "timeout", "audit", @@ -45,9 +46,17 @@ for (const scenario of [ { async mount() { if (scenario === "setup") throw setupError; + if (scenario === "setup-and-cleanup") { + throw new AggregateError([setupError, cleanupError], "DUMMY setup cleanup"); + } await writeFile(target, "DUMMY placeholder"); return { + sourceIdentity: dummyIdentity(), mountIds: { source: "1", target: "2" }, + parentMounts: [{ id: "3", target: directory }], + async verifyIntegrity() { + return true; + }, async cleanup() { await new Promise((done) => setTimeout(done, 5)); cleanups++; @@ -57,6 +66,7 @@ for (const scenario of [ }, }; }, + inspectSource: dummyStat, spawn() { launches++; if (scenario === "launch") throw new Error("DUMMY spawn threw"); @@ -84,7 +94,21 @@ for (const scenario of [ } else { await assert.rejects(run, (error) => { if (scenario === "setup") assert.equal(error.cause, setupError); - if (["expired", "near-expiry"].includes(scenario)) assert.equal(error.exitCode, 75); + if (scenario === "setup-and-cleanup") { + assert(error instanceof AggregateError); + assert.equal(error.code, "CP06_CLEANUP_FAILED"); + assert.deepEqual( + supervisorFailureRecord(error).causes.map(({ stage, code }) => ({ stage, code })), + [ + { stage: "setup", code: "CP06_ISOLATION_SETUP_FAILED" }, + { stage: "cleanup", code: "CP06_CREDENTIAL_CLEANUP_FAILED" }, + ], + ); + } + if (["expired", "near-expiry"].includes(scenario)) { + assert.equal(error.exitCode, 75); + assert.equal(error.code, "CP06_AUTH_BLOCKED"); + } if (scenario === "timeout") assert.equal(error.cause.code, "ETIMEDOUT"); if (["cleanup", "primary-and-cleanup", "audit-and-cleanup"].includes(scenario)) { assert(error instanceof Cp06CleanupError); @@ -93,20 +117,20 @@ for (const scenario of [ assert.equal(error.errors.length, scenario === "cleanup" ? 1 : 2); if (scenario === "primary-and-cleanup") { assert.equal(error.errors[0].exitCode, 75); - assert.match(error.errors[0].message, /DUMMY auth blocked/); + assert.equal(error.errors[0].code, "CP06_AUTH_BLOCKED"); assert.equal(error.cause, error.errors[0]); } if (scenario === "audit-and-cleanup") { assert.equal(error.errors[0].exitCode, 70); - assert.match(error.errors[0].message, /non-compact output/); + assert.equal(error.errors[0].code, "CP06_CHILD_OUTPUT_INVALID"); assert.equal(error.cause, error.errors[0]); } } return true; }); } - assert.equal(cleanups, scenario === "setup" ? 0 : 1); - assert.equal(launches, scenario === "setup" ? 0 : 1); + assert.equal(cleanups, ["setup", "setup-and-cleanup"].includes(scenario) ? 0 : 1); + assert.equal(launches, ["setup", "setup-and-cleanup"].includes(scenario) ? 0 : 1); await assert.rejects(stat(target), { code: "ENOENT" }); assert.equal(await readFile(source, "utf8"), "DUMMY ONLY"); } finally { @@ -115,6 +139,53 @@ for (const scenario of [ }); } +test("worker source integrity uses metadata without reading credential bytes", async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-metadata-")); + const source = join(directory, "DUMMY-unreadable-source"); + await mkdir(source); + const stat = await lstat(source, { bigint: true }); + const sourceIdentity = { + dev: String(stat.dev), + ino: String(stat.ino), + size: String(stat.size), + nlink: String(stat.nlink), + mode: String(stat.mode), + mtime_ns: String(stat.mtimeNs), + ctime_ns: String(stat.ctimeNs), + }; + try { + const result = await superviseCredentialChild( + { + mode: "worker", + source, + target: join(directory, "DUMMY-target"), + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + }, + { + async mount() { + return { + sourceIdentity, + mountIds: { source: "1", target: "2" }, + parentMounts: [{ id: "3", target: directory }], + async verifyIntegrity() { + const current = await lstat(source, { bigint: true }); + return String(current.ino) === sourceIdentity.ino; + }, + cleanup() {}, + }; + }, + spawn() { + return { status: 0, stdout: '{"result":"pass"}' }; + }, + }, + ); + assert.equal(result.source_unchanged, true); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + test("audit and cleanup causes survive supervisor output and replay blocked recording", async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-output-chain-")); const source = join(directory, "DUMMY-source"); @@ -134,7 +205,12 @@ test("audit and cleanup causes survive supervisor output and replay blocked reco { async mount() { return { + sourceIdentity: dummyIdentity(), mountIds: { source: "1", target: "2" }, + parentMounts: [{ id: "3", target: directory }], + async verifyIntegrity() { + return true; + }, async cleanup() { await new Promise((done) => setTimeout(done, 5)); cleanups++; @@ -142,8 +218,13 @@ test("audit and cleanup causes survive supervisor output and replay blocked reco }, }; }, + inspectSource: dummyStat, spawn() { - return { status: 0, stdout: "DUMMY malformed child output" }; + return { + status: 0, + stdout: "DUMMY malformed child output", + stderr: "DUMMY-SECRET-MUST-NOT-SURVIVE", + }; }, }, ); @@ -160,15 +241,22 @@ test("audit and cleanup causes survive supervisor output and replay blocked reco const supervisorRecord = supervisorFailureRecord(caught); assert.equal(supervisorRecord.code, "CP06_CLEANUP_FAILED"); assert.equal(supervisorRecord.exit_code, 74); - assert.deepEqual( - supervisorRecord.causes.map(({ code, exit_code }) => ({ code, exit_code })), - [ - { code: "CP06_WORKER_FAILED", exit_code: 70 }, - { code: "CP06_WORKER_FAILED", exit_code: null }, - ], - ); - assert.match(supervisorRecord.message, /non-compact output/); - assert.match(supervisorRecord.message, /independent cleanup failure/); + assert.deepEqual(supervisorRecord.causes, [ + { + stage: "audit", + code: "CP06_CHILD_OUTPUT_INVALID", + exit_code: 70, + description: "isolated child output was invalid", + }, + { + stage: "cleanup", + code: "CP06_CREDENTIAL_CLEANUP_FAILED", + exit_code: null, + description: "credential mount cleanup failed", + }, + ]); + assert.equal(JSON.stringify(supervisorRecord).includes("DUMMY-SECRET"), false); + assert.equal(JSON.stringify(supervisorRecord).includes("independent cleanup"), false); const workerError = namespaceWorkerError({ status: 74, @@ -181,16 +269,51 @@ test("audit and cleanup causes survive supervisor output and replay blocked reco ); assert.equal(blocked.cleanup_failed, true); assert.equal(blocked.code, "CP06_CLEANUP_FAILED"); - assert.deepEqual(blocked.worker_failure, { - code: "CP06_CLEANUP_FAILED", - exit_code: 74, - causes: supervisorRecord.causes, + assert.deepEqual(blocked.worker_failure, supervisorRecord); + assert.equal(blocked.failure_stage, "cleanup"); + assert.equal(JSON.stringify(blocked).includes("DUMMY-SECRET"), false); + + const forged = namespaceWorkerError({ + status: 70, + stderr: JSON.stringify({ ...supervisorRecord, exit_code: 70, raw: "DUMMY-SECRET" }), }); + assert.equal(forged.workerFailure, undefined); + assert.equal( + JSON.stringify(createWorkerBlockedStatus(forged, "isolated-sdk-worker")).includes( + "DUMMY-SECRET", + ), + false, + ); } finally { await rm(directory, { recursive: true, force: true }); } }); +function dummyStat() { + return { + dev: 1n, + ino: 2n, + size: 10n, + nlink: 1n, + mode: 0o100600n, + mtimeNs: 3n, + ctimeNs: 4n, + }; +} + +function dummyIdentity() { + const stat = dummyStat(); + return { + dev: String(stat.dev), + ino: String(stat.ino), + size: String(stat.size), + nlink: String(stat.nlink), + mode: String(stat.mode), + mtime_ns: String(stat.mtimeNs), + ctime_ns: String(stat.ctimeNs), + }; +} + test("evaluation-home cleanup finishes before the blocked result is returned", async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-home-")); const primary = Object.assign(new Error("DUMMY blocked"), { exitCode: 75 }); diff --git a/test/cp06-worker-outcome.test.mjs b/test/cp06-worker-outcome.test.mjs index 41a20ea..df199de 100644 --- a/test/cp06-worker-outcome.test.mjs +++ b/test/cp06-worker-outcome.test.mjs @@ -17,6 +17,7 @@ const gap = { const response = { status: "MISSING_SOURCE", missing_source: "docs/AUTH.md", + reads: [options.expectedOriginalPath], broad_scan: false, corrected_constraints: [ "Ownership comes only from the authenticated principal; request-supplied owner fields are ignored.", @@ -64,7 +65,22 @@ test("creation evidence cannot replace source recovery or relax the read outcome { missing_source: "./docs/AUTH.md" }, { missing_source: resolve(root, "test/fixtures/cp06-context/docs/AUTH.md") }, { broad_scan: true }, + { reads: [] }, + { reads: [resolve(root, "PROJECT.md")] }, { corrected_constraints: [] }, + { + corrected_constraints: [ + ...response.corrected_constraints, + "creation behavior was independently verified", + ], + }, + { + corrected_constraints: [ + `${response.corrected_constraints[0]} These creation checks passed.`, + ], + }, + { creation_evidence: { status: "passed" } }, + { verified: true }, ]) assert.equal(auditWorkerOutcome({ ...response, ...change }, options).ok, false); assert.equal(auditWorkerOutcome(null, options).ok, false); From 693c4c6bcb2e097609144ff34d38cdd9609f16af Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 11:26:41 +0000 Subject: [PATCH 23/45] no-mistakes(review): Fix CP-06 source, read, oracle, and Pi pinning --- docs/probes/CP-06.md | 40 ++++++---- scripts/cp06-auth-security.mjs | 59 +++++--------- scripts/cp06-dummy-outcome-proof.mjs | 7 +- scripts/cp06-dummy-worker-hook.mjs | 6 +- scripts/cp06-guarded-read.mjs | 108 +++++++++++++++++++++++--- scripts/cp06-namespace-supervisor.mjs | 22 +++++- scripts/cp06-pi-install.mjs | 93 ++++++++++++++++++++++ scripts/cp06-sdk-worker.mjs | 71 +++++------------ scripts/cp06-worker-outcome.mjs | 13 ++-- scripts/prove-cp06-worker-cleanup.mjs | 2 + scripts/replay-cp06-worker.mjs | 1 + test/cp06-guarded-read.test.mjs | 26 +++++-- test/cp06-pi-dependency.test.mjs | 61 ++++++++++++++- test/cp06-worker-lifecycle.test.mjs | 45 ++++++++++- test/cp06-worker-outcome.test.mjs | 9 ++- 15 files changed, 426 insertions(+), 137 deletions(-) create mode 100644 scripts/cp06-pi-install.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index e5a9523..3cad318 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,9 +21,9 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before delegating to Pi's filesystem read. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. +The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. -The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/`, prepends only that installation's `.bin` for the proof commands, and leaves the global installation/configuration unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the resolved package/entry paths and their SHA-256 digests. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. +The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and hashes must agree; inherited `PATH`, a global installation, an unrelated same-version package, and substitutions are rejected without fallback. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths and SHA-256 digests. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is attempted only after that DUMMY gate and fail-closed preflight pass. No destructive operation is ever directed at the real credential source or mounted target. @@ -96,17 +96,18 @@ mkdir -p .factory/state/cp06-sdk npm_config_cache="$PWD/.factory/state/npm-cache-pi" npm install \ --prefix .factory/state/cp06-sdk --package-lock=false --ignore-scripts \ --no-audit --no-fund @earendil-works/pi-coding-agent@0.85.1 -PINNED_PI_BIN="$PWD/.factory/state/cp06-sdk/node_modules/.bin" -PATH="$PINNED_PI_BIN:$PATH" pi --version # must be exactly 0.85.1 +export CP06_PI_PACKAGE_ROOT="$PWD/.factory/state/cp06-sdk/node_modules/@earendil-works/pi-coding-agent" +export CP06_PI_BIN="$PWD/.factory/state/cp06-sdk/node_modules/.bin/pi" +"$CP06_PI_BIN" --version # must be exactly 0.85.1 npm run format:check npm run lint npm run typecheck npm test -PATH="$PINNED_PI_BIN:$PATH" npm run proof:cp06:auth-security -PATH="$PINNED_PI_BIN:$PATH" npm run proof:cp06:worker-cleanup +npm run proof:cp06:auth-security +npm run proof:cp06:worker-cleanup npm run proof:cp06:ten-pack npm run proof:cp06:p07 -PATH="$PINNED_PI_BIN:$PATH" npm run proof:cp06:worker +npm run proof:cp06:worker git diff --check ``` @@ -204,15 +205,21 @@ reports `MISSING_SOURCE`, performs exactly one read of the supplied creation ownership and the ignored request-owner rule, and identifies the missing creation-time integration evidence. -The audit correlates `tool_execution_start` and `tool_execution_end` by tool-call -ID and event order, requires `isError: false`, and forbids all other start or +The exact original is opened with `O_NOFOLLOW`, identity-checked, and snapshotted +before provider execution. At the actual tool boundary, the lexical path and +range are checked and the named path must still identify the pinned regular +file; Pi's SDK read implementation receives only the immutable snapshot, so a +replacement race cannot redirect the provider-visible bytes. The audit +correlates `tool_execution_start` and `tool_execution_end` by tool-call ID and +event order, requires `isError: false`, and forbids all other start or completion events. An omitted range is accepted; the SDK's explicit first-line form is accepted only as `offset: 1` with a positive integer `limit`, and only when the returned UTF-8 bytes and SHA-256 still equal the complete tracked exact original. Offset-only, limit-only, wrong-start, and partial results fail. The final response must follow the successful completion. Deterministic protocol tests reject omitted, failed, partial, mismatched, out-of-order, -malformed-orphan, and extra-project cases. +malformed-orphan, extra-project, credential-path, symlink-alias, and +post-construction replacement-race cases. The genuine Pi SDK process runs in a private Linux user/mount namespace only after the complete DUMMY auth and OS preflight passes. A supervisor pins the @@ -223,8 +230,11 @@ inherited seccomp filter blocking mount and namespace escapes, alternate inode handles, ptrace/process-memory access, and namespace-bearing child calls. Its credential store rejects refresh/mutation callbacks before entry; expired or near-expiry OAuth is therefore blocked before a provider request or persistence. -Setup, helper/version, auth-validity, timeout, audit, and cleanup failures are -reported as blocked/nonzero with no unsandboxed fallback. The evaluation state +A missing credential source is a sanitized `CP06_CREDENTIAL_SOURCE_MISSING` +missing-source result with exit 70, not an unsupported-isolation result; a +simultaneous cleanup failure retains both typed causes. Other setup, +helper/version, auth-validity, timeout, audit, and cleanup failures are reported +as blocked/nonzero with no unsandboxed fallback. The evaluation state is removed and no raw transcript is retained. Native Windows/macOS isolation is unsupported rather than claimed. Model wording is nondeterministic; the protocol, exact bytes, and isolation assertions are not. @@ -238,7 +248,7 @@ runner's `finally`, leaving its evaluation directory and empty mount placeholder Neither finding demonstrated credential mutation; the DUMMY S1/S2 controls passed. The worker now requires an exact top-level response shape with one declared -canonical read, one corrected-constraint string, no secondary evidence channel, +canonical read, two fixed corrected-constraint values, no secondary evidence channel, and an executable structured creation-evidence disposition: ```json @@ -253,7 +263,9 @@ and an executable structured creation-evidence disposition: } ``` -The check order is immaterial, but both checks must remain unverified. Missing, +The check order inside `missing_checks` is immaterial, but the two +`corrected_constraints` values are fixed canonical strings in fixed order so +that field cannot carry an additional free-text evidence disposition. Missing, empty, duplicate, wrong, or contradictory dispositions fail; recovering a requirement by reading its original is not a passing behavior test. Extra fields or prose in this gap are rejected rather than interpreted as an alternate diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index c08fb6c..9f21cd2 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -1,8 +1,9 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { accessSync, constants, readFileSync, realpathSync } from "node:fs"; +import { readFileSync } from "node:fs"; import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; -import { delimiter, dirname, join, resolve } from "node:path"; +import { dirname, join, resolve } from "node:path"; +import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; export class Cp06IsolationUnsupportedError extends Error { constructor(message) { @@ -21,7 +22,7 @@ export async function runCp06SecurityPreflight(options = {}) { spawnSyncImpl: options.spawnSyncImpl, }); const binaries = compileIsolationHelpers({ root, outputRoot }); - const pi = resolvePiPackage(); + const pi = resolvePiPackage(root, options); const unsafe = await runIsolationProbe({ root, outputRoot, @@ -47,6 +48,7 @@ export async function runCp06SecurityPreflight(options = {}) { command: "npm run proof:cp06:auth-security", exit_code: 0, pi_version: pi.version, + pi_install: pi.provenance, platform: prerequisites, binaries: { seccomp_helper_sha256: sha256(readFileSync(binaries.helper)), @@ -250,6 +252,7 @@ function isSupervisorFailure(value, status) { const SUPERVISOR_FAILURE_DESCRIPTIONS = { CP06_ISOLATION_SETUP_FAILED: "credential namespace setup failed", + CP06_CREDENTIAL_SOURCE_MISSING: "credential source is missing", CP06_CHILD_LAUNCH_FAILED: "isolated child launch failed", CP06_CHILD_TIMEOUT: "isolated child timed out", CP06_AUTH_BLOCKED: "credential preflight blocked the child", @@ -261,6 +264,7 @@ const SUPERVISOR_FAILURE_DESCRIPTIONS = { }; const SUPERVISOR_FAILURE_CODES = new Map([ ["CP06_ISOLATION_SETUP_FAILED", "setup"], + ["CP06_CREDENTIAL_SOURCE_MISSING", "missing-source"], ["CP06_CHILD_LAUNCH_FAILED", "launch"], ["CP06_CHILD_TIMEOUT", "timeout"], ["CP06_AUTH_BLOCKED", "child-exit"], @@ -394,45 +398,18 @@ function runDummySdkProof({ root, helper, piRoot }) { return parseCompactJson(result.stdout, "DUMMY SDK proof"); } -function resolvePiPackage() { - const executable = realpathSync(findExecutable("pi")); - const root = findPiPackageRoot(executable); - const metadata = JSON.parse(readFileSync(join(root, "package.json"), "utf8")); - const version = checked("pi", ["--version"], root).stdout.trim(); - if ( - metadata.name !== "@earendil-works/pi-coding-agent" || - metadata.version !== "0.85.1" || - version !== "0.85.1" - ) { - throw unsupported("exact Pi 0.85.1 is unavailable"); - } - return { root, version }; -} - -function findPiPackageRoot(executable) { - let directory = dirname(executable); - for (let depth = 0; depth < 5; depth += 1) { - try { - const metadata = JSON.parse(readFileSync(join(directory, "package.json"), "utf8")); - if (metadata.name === "@earendil-works/pi-coding-agent") return directory; - } catch {} - const parent = dirname(directory); - if (parent === directory) break; - directory = parent; - } - throw unsupported("Pi package root could not be resolved from its executable"); -} - -function findExecutable(name) { - for (const directory of String(process.env.PATH ?? "").split(delimiter)) { - if (!directory) continue; - const candidate = join(directory, name); - try { - accessSync(candidate, constants.X_OK); - return candidate; - } catch {} +function resolvePiPackage(root, options) { + try { + return resolvePinnedPiInstall({ + projectRoot: root, + packageRoot: options.piPackageRoot ?? process.env.CP06_PI_PACKAGE_ROOT, + executable: options.piExecutable ?? process.env.CP06_PI_BIN, + spawnSyncImpl: options.piSpawnSyncImpl, + environment: options.environment, + }); + } catch { + throw unsupported("explicit task-local Pi 0.85.1 installation is unavailable"); } - throw unsupported(`required executable is unavailable: ${name}`); } function checked(command, args, cwd) { diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index 1991931..424ca1b 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -2,7 +2,7 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join, resolve } from "node:path"; +import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; @@ -36,6 +36,7 @@ export async function proveDummyWorkerOutcome({ piRoot }) { ["missing-gap", 75], ["contradictory-gap", 75], ["extra-evidence", 75], + ["constraint-pass", 75], ["missing-reads", 75], ["credential-read", 75], ["symlink-read", 75], @@ -97,6 +98,9 @@ export async function proveDummyWorkerOutcome({ piRoot }) { PI_OFFLINE: "1", PI_SKIP_VERSION_CHECK: "1", CP06_DUMMY_PI_ROOT: piRoot, + CP06_PROJECT_ROOT: root, + CP06_PI_PACKAGE_ROOT: piRoot, + CP06_PI_BIN: join(dirname(dirname(piRoot)), ".bin", "pi"), CP06_DUMMY_COUNTERS: counters, CP06_DUMMY_SCENARIO: scenario, }, @@ -116,6 +120,7 @@ export async function proveDummyWorkerOutcome({ piRoot }) { "missing-gap", "contradictory-gap", "extra-evidence", + "constraint-pass", "missing-reads", ].includes(scenario) ) diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index d1f7840..1db0650 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -69,7 +69,8 @@ sdk.ModelRuntime.create = async function (options) { missing_source: "docs/AUTH.md", reads: [input.original_path], corrected_constraints: [ - "ownership comes only from the authenticated principal; request-supplied owner fields are ignored", + "ownership comes only from the authenticated principal", + "request-supplied owner fields are ignored", ], evidence_gaps: scenario === "empty-gap" @@ -79,6 +80,9 @@ sdk.ModelRuntime.create = async function (options) { }; if (scenario === "missing-gap") delete response.evidence_gaps; if (scenario === "extra-evidence") response.creation_evidence = { status: "passed" }; + if (scenario === "constraint-pass") { + response.corrected_constraints[1] += "; creation evidence status: pass"; + } if (scenario === "missing-reads") delete response.reads; const readPath = scenario === "credential-read" diff --git a/scripts/cp06-guarded-read.mjs b/scripts/cp06-guarded-read.mjs index fd78fd9..be53ba6 100644 --- a/scripts/cp06-guarded-read.mjs +++ b/scripts/cp06-guarded-read.mjs @@ -1,6 +1,17 @@ -import { realpathSync } from "node:fs"; +import { + closeSync, + constants, + fstatSync, + lstatSync, + openSync, + readSync, + realpathSync, +} from "node:fs"; import { isAbsolute, relative, resolve } from "node:path"; +const MAX_ORIGINAL_BYTES = 50 * 1024; +const MAX_ORIGINAL_LINES = 2_000; + export function canonicalWorkerPaths(input, credentialTarget, credentialSource) { const root = canonicalExactPath(input.root, "root"); const fixtureRoot = canonicalExactPath(input.fixture_root, "fixture_root"); @@ -24,20 +35,46 @@ export function createGuardedReadTool({ sdk, root, expectedOriginalPath }) { throw new Error("Pi SDK guarded read constructor is unavailable"); } const canonicalOriginal = canonicalExactPath(expectedOriginalPath, "expected original"); - const read = sdk.createReadTool(root); + const pinned = readPinnedRegularFile(canonicalOriginal); + const read = sdk.createReadTool(root, { + operations: { + async access(path) { + if (path !== canonicalOriginal) throw new Error("read denied"); + }, + async readFile(path) { + if (path !== canonicalOriginal) throw new Error("read denied"); + return Buffer.from(pinned.bytes); + }, + async detectImageMimeType() { + return null; + }, + }, + }); if (read?.name !== "read" || typeof read.execute !== "function") { throw new Error("Pi SDK read tool surface is unsupported"); } return { - ...read, - async execute(toolCallId, args, ...rest) { - assertAllowedRead(args, canonicalOriginal); - return read.execute.call(read, toolCallId, args, ...rest); + original: Buffer.from(pinned.bytes), + tool: { + ...read, + async execute(toolCallId, args, ...rest) { + assertAllowedRead(args, canonicalOriginal, pinned.lineCount); + let current; + try { + current = lstatSync(canonicalOriginal, { bigint: true }); + } catch { + throw new Error("read denied: supplied original changed"); + } + if (!current.isFile() || !sameIdentity(pinned.identity, current)) { + throw new Error("read denied: supplied original changed"); + } + return read.execute.call(read, toolCallId, args, ...rest); + }, }, }; } -export function assertAllowedRead(args, expectedOriginalPath) { +export function assertAllowedRead(args, expectedOriginalPath, completeLineLimit = 1) { if ( args === null || typeof args !== "object" || @@ -52,12 +89,65 @@ export function assertAllowedRead(args, expectedOriginalPath) { if (hasOffset !== hasLimit) throw new Error("read denied: incomplete explicit range"); if ( hasOffset && - (args.offset !== 1 || !Number.isSafeInteger(args.limit) || args.limit < 1) + (args.offset !== 1 || + !Number.isSafeInteger(args.limit) || + args.limit < completeLineLimit) ) { - throw new Error("read denied: explicit range must start at one with a positive limit"); + throw new Error("read denied: explicit range must cover the complete original"); + } +} + +function readPinnedRegularFile(path) { + const descriptor = openSync( + path, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); + try { + const before = fstatSync(descriptor, { bigint: true }); + const named = lstatSync(path, { bigint: true }); + if ( + !before.isFile() || + !sameIdentity(before, named) || + before.size <= 0n || + before.size > BigInt(MAX_ORIGINAL_BYTES) + ) { + throw new Error("supplied original is not one bounded pinned regular file"); + } + const bytes = Buffer.alloc(Number(before.size)); + let offset = 0; + while (offset < bytes.length) { + const count = readSync(descriptor, bytes, offset, bytes.length - offset, offset); + if (count === 0) throw new Error("supplied original ended while pinned"); + offset += count; + } + const after = fstatSync(descriptor, { bigint: true }); + if (!sameIdentity(before, after)) throw new Error("supplied original changed while pinned"); + const text = bytes.toString("utf8"); + if (!Buffer.from(text, "utf8").equals(bytes)) { + throw new Error("supplied original is not valid UTF-8 text"); + } + const lineCount = text.split("\n").length; + if (lineCount > MAX_ORIGINAL_LINES) { + throw new Error("supplied original exceeds the complete-read line bound"); + } + return { bytes, identity: before, lineCount }; + } finally { + closeSync(descriptor); } } +function sameIdentity(expected, actual) { + return ( + expected.dev === actual.dev && + expected.ino === actual.ino && + expected.size === actual.size && + expected.nlink === actual.nlink && + expected.mode === actual.mode && + expected.mtimeNs === actual.mtimeNs && + expected.ctimeNs === actual.ctimeNs + ); +} + function canonicalExactPath(value, label) { if (typeof value !== "string" || !isAbsolute(value)) { throw new Error(`worker ${label} must be absolute`); diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index d708fcd..5408992 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -37,7 +37,13 @@ export async function superviseCredentialChild(options, dependencies = {}) { protectParents: mode !== "unsafe-probe", }); } catch (cause) { - const setup = failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73, cause); + const missingSource = hasErrorCode(cause, "ENOENT"); + const setup = failure( + missingSource ? "missing-source" : "setup", + missingSource ? "CP06_CREDENTIAL_SOURCE_MISSING" : "CP06_ISOLATION_SETUP_FAILED", + missingSource ? 70 : 73, + cause, + ); if (cause instanceof AggregateError) { throw compoundFailure( setup, @@ -183,9 +189,12 @@ function childEnvironment(childMode, credentialSource, credentialTarget) { PI_TELEMETRY: "0", }; if (childMode === "worker") { + environment.CP06_PROJECT_ROOT = root; environment.CP06_CREDENTIAL_SOURCE = credentialSource; environment.CP06_CREDENTIAL_TARGET = credentialTarget; for (const name of [ + "CP06_PI_PACKAGE_ROOT", + "CP06_PI_BIN", "SSL_CERT_FILE", "SSL_CERT_DIR", "NODE_EXTRA_CA_CERTS", @@ -224,6 +233,13 @@ function compoundFailure(primary, cleanup) { }); } +function hasErrorCode(error, code) { + if (error?.code === code) return true; + return ( + error instanceof AggregateError && error.errors.some((cause) => hasErrorCode(cause, code)) + ); +} + function sha256(bytes) { return createHash("sha256").update(bytes).digest("hex"); } @@ -259,6 +275,10 @@ const FAILURE_DESCRIPTIONS = { stage: "setup", description: "credential namespace setup failed", }, + CP06_CREDENTIAL_SOURCE_MISSING: { + stage: "missing-source", + description: "credential source is missing", + }, CP06_CHILD_LAUNCH_FAILED: { stage: "launch", description: "isolated child launch failed" }, CP06_CHILD_TIMEOUT: { stage: "timeout", description: "isolated child timed out" }, CP06_AUTH_BLOCKED: { stage: "child-exit", description: "credential preflight blocked the child" }, diff --git a/scripts/cp06-pi-install.mjs b/scripts/cp06-pi-install.mjs new file mode 100644 index 0000000..39c584e --- /dev/null +++ b/scripts/cp06-pi-install.mjs @@ -0,0 +1,93 @@ +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { accessSync, constants, readFileSync, realpathSync } from "node:fs"; +import { isAbsolute, join, relative, resolve } from "node:path"; + +const PACKAGE_NAME = "@earendil-works/pi-coding-agent"; +const SUPPORTED_VERSION = "0.85.1"; + +export function resolvePinnedPiInstall(options) { + const projectRoot = realpathSync(resolve(options.projectRoot)); + const selectedRoot = resolve(projectRoot, ".factory/state/cp06-sdk"); + const expectedPackageRoot = join( + selectedRoot, + "node_modules", + "@earendil-works", + "pi-coding-agent", + ); + const expectedExecutable = join(selectedRoot, "node_modules", ".bin", "pi"); + requireExactPath(options.packageRoot, expectedPackageRoot, "Pi package root"); + requireExactPath(options.executable, expectedExecutable, "Pi executable"); + + const packageRoot = realpathSync(expectedPackageRoot); + if (packageRoot !== expectedPackageRoot) { + throw new Error("selected Pi package root is not canonical"); + } + accessSync(expectedExecutable, constants.X_OK); + const executableEntry = realpathSync(expectedExecutable); + if (!isWithin(packageRoot, executableEntry)) { + throw new Error("selected Pi executable resolves outside its package"); + } + + const metadataPath = join(packageRoot, "package.json"); + const sdkEntry = realpathSync(join(packageRoot, "dist/index.js")); + if (!isWithin(packageRoot, sdkEntry)) { + throw new Error("selected Pi SDK entry resolves outside its package"); + } + const metadata = JSON.parse(readFileSync(metadataPath, "utf8")); + const declaredBin = typeof metadata.bin === "string" ? metadata.bin : metadata.bin?.pi; + if ( + metadata.name !== PACKAGE_NAME || + metadata.version !== SUPPORTED_VERSION || + typeof declaredBin !== "string" || + realpathSync(resolve(packageRoot, declaredBin)) !== executableEntry + ) { + throw new Error("selected task-local Pi package is unsupported"); + } + + const spawn = options.spawnSyncImpl ?? spawnSync; + const version = spawn(expectedExecutable, ["--version"], { + cwd: projectRoot, + encoding: "utf8", + timeout: 10_000, + env: options.environment ?? process.env, + }); + if ( + version.error !== undefined || + version.status !== 0 || + version.stdout.trim() !== SUPPORTED_VERSION + ) { + throw new Error("selected task-local Pi executable version is unsupported"); + } + + return { + root: packageRoot, + executable: expectedExecutable, + version: SUPPORTED_VERSION, + provenance: { + install_root: selectedRoot, + package_root: packageRoot, + executable: expectedExecutable, + executable_entry: executableEntry, + sdk_entry: sdkEntry, + package_sha256: sha256(readFileSync(metadataPath)), + executable_sha256: sha256(readFileSync(executableEntry)), + sdk_entry_sha256: sha256(readFileSync(sdkEntry)), + }, + }; +} + +function requireExactPath(value, expected, label) { + if (typeof value !== "string" || !isAbsolute(value) || resolve(value) !== expected) { + throw new Error(`${label} must select the task-local CP-06 installation`); + } +} + +function isWithin(parent, child) { + const path = relative(parent, child); + return path === "" || (!path.startsWith("..") && !isAbsolute(path)); +} + +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs index ebb991e..83b5e37 100644 --- a/scripts/cp06-sdk-worker.mjs +++ b/scripts/cp06-sdk-worker.mjs @@ -1,12 +1,11 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; -import { spawnSync } from "node:child_process"; -import { accessSync, constants, readFileSync, realpathSync } from "node:fs"; import { readFile } from "node:fs/promises"; -import { delimiter, dirname, join, resolve } from "node:path"; +import { dirname, join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { Cp06AuthBlockedError, Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; -import { createGuardedReadTool, canonicalWorkerPaths } from "./cp06-guarded-read.mjs"; +import { canonicalWorkerPaths, createGuardedReadTool } from "./cp06-guarded-read.mjs"; +import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; import { auditReadEvents } from "./cp06-worker-audit.mjs"; import { auditWorkerOutcome } from "./cp06-worker-outcome.mjs"; @@ -26,9 +25,14 @@ try { const sdk = await import(pathToFileURL(join(pi.root, "dist/index.js"))); assertSdkSurface(sdk); + const guardedRead = createGuardedReadTool({ + sdk, + root: paths.root, + expectedOriginalPath: paths.originalPath, + }); const contract = await readFile(paths.contractPath, "utf8"); const pack = await readFile(paths.packPath, "utf8"); - const original = await readFile(paths.originalPath); + const original = guardedRead.original; const prompt = buildPrompt({ contract, pack, originalPath: paths.originalPath }); const store = await Cp06ReadOnlyCredentialStore.load({ path: credentialTarget, @@ -60,11 +64,6 @@ try { enableAnalytics: false, enableInstallTelemetry: false, }); - const guardedRead = createGuardedReadTool({ - sdk, - root: paths.root, - expectedOriginalPath: paths.originalPath, - }); const { session, extensionsResult } = await sdk.createAgentSession({ cwd: paths.root, agentDir: dirname(credentialTarget), @@ -72,7 +71,7 @@ try { modelRuntime: runtime, resourceLoader, tools: ["read"], - customTools: [guardedRead], + customTools: [guardedRead.tool], sessionManager: sdk.SessionManager.inMemory(paths.root), settingsManager, thinkingLevel: "high", @@ -129,6 +128,7 @@ try { schema_version: 1, result: "pass", pi_version: pi.version, + pi_install: pi.provenance, provider: input.provider, model: input.model, oauth_preflight: { @@ -176,46 +176,15 @@ function validateInput(input) { } function resolvePiPackage() { - const piExecutable = findExecutable("pi"); - const cli = realpathSync(piExecutable); - const packageRoot = findPiPackageRoot(cli); - const metadata = JSON.parse(readFileSync(join(packageRoot, "package.json"), "utf8")); - const version = spawnSync(piExecutable, ["--version"], { encoding: "utf8", timeout: 10_000 }); - if ( - metadata.name !== "@earendil-works/pi-coding-agent" || - metadata.version !== "0.85.1" || - version.status !== 0 || - version.stdout.trim() !== "0.85.1" - ) { - throw blocked("exact Pi 0.85.1 is required"); - } - return { root: packageRoot, version: metadata.version }; -} - -function findPiPackageRoot(executable) { - let directory = dirname(executable); - for (let depth = 0; depth < 5; depth += 1) { - try { - const metadata = JSON.parse(readFileSync(join(directory, "package.json"), "utf8")); - if (metadata.name === "@earendil-works/pi-coding-agent") return directory; - } catch {} - const parent = dirname(directory); - if (parent === directory) break; - directory = parent; - } - throw blocked("Pi package root could not be resolved from its executable"); -} - -function findExecutable(name) { - for (const directory of String(process.env.PATH ?? "").split(delimiter)) { - if (!directory) continue; - const candidate = join(directory, name); - try { - accessSync(candidate, constants.X_OK); - return candidate; - } catch {} + try { + return resolvePinnedPiInstall({ + projectRoot: process.env.CP06_PROJECT_ROOT, + packageRoot: process.env.CP06_PI_PACKAGE_ROOT, + executable: process.env.CP06_PI_BIN, + }); + } catch { + throw blocked("explicit task-local Pi 0.85.1 installation is required"); } - throw blocked("Pi executable is unavailable"); } function assertSdkSurface(sdk) { @@ -248,7 +217,7 @@ function emptyResourceLoader(sdk, systemPrompt) { } function buildPrompt({ contract, pack, originalPath }) { - return `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE, set missing_source to the exact canonical relative path docs/AUTH.md, and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. The evidence_gaps field must be exactly [{"id":"creation-time-ownership","status":"unverified","missing_checks":["principal-derived-owner","request-owner-ignored"]}]: the supplied deterministic evidence does not test those creation-time behaviors. Recovering a requirement by reading is not evidence that the behavior passed. Do not add a passed/verified disposition or prose fields to this structured gap. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; + return `You are a fresh task reviewer. You have no previous worker transcript. Review only the bounded handoff below. First inspect the task and pack. One concrete creation-time ownership rule was deliberately omitted from this handoff. Report MISSING_SOURCE, set missing_source to the exact canonical relative path docs/AUTH.md, and use the read tool exactly once, without offset or limit, on the supplied exact-original path; do not scan or read any other project file. Then return a concise JSON object with keys status, missing_source, reads, corrected_constraints, evidence_gaps, and broad_scan. The corrected_constraints field must be exactly ["ownership comes only from the authenticated principal","request-supplied owner fields are ignored"]. The evidence_gaps field must be exactly [{"id":"creation-time-ownership","status":"unverified","missing_checks":["principal-derived-owner","request-owner-ignored"]}]: the supplied deterministic evidence does not test those creation-time behaviors. Recovering a requirement by reading is not evidence that the behavior passed. Do not add a passed/verified disposition or prose fields to this structured gap. Do not modify files.\n\nEXACT TASK CONTRACT\n---\n${contract}\n---\n\nBOUNDED PACK\n---\n${pack}\n---\n\nEXISTING DETERMINISTIC EVIDENCE\n---\nThe in-memory auth fixture currently reports absent/forged credentials as 401 and concealed cross-account GET/PUT/DELETE as 404 with unchanged state. Review whether the handoff states every decisive acceptance constraint; do not assume unshown creation semantics.\n---\n\nSUPPLIED EXACT-ORIGINAL PATH (read only if needed)\n${originalPath}\n`; } function finalAssistantText(events) { diff --git a/scripts/cp06-worker-outcome.mjs b/scripts/cp06-worker-outcome.mjs index a1af007..5636c02 100644 --- a/scripts/cp06-worker-outcome.mjs +++ b/scripts/cp06-worker-outcome.mjs @@ -9,9 +9,7 @@ export function auditWorkerOutcome(response, options) { "reads", "status", ]; - const corrected = Array.isArray(response?.corrected_constraints) - ? response.corrected_constraints - : []; + const corrected = response?.corrected_constraints; if ( response === null || typeof response !== "object" || @@ -23,11 +21,10 @@ export function auditWorkerOutcome(response, options) { !Array.isArray(response.reads) || response.reads.length !== 1 || response.reads[0] !== options.expectedOriginalPath || - corrected.length !== 1 || - typeof corrected[0] !== "string" || - !/ownership comes only from the authenticated principal/i.test(corrected[0]) || - !/request-supplied owner fields? (?:are )?ignored/i.test(corrected[0]) || - /\b(?:passed|verified|proven|satisfied)\b/i.test(corrected[0]) + !Array.isArray(corrected) || + corrected.length !== 2 || + corrected[0] !== "ownership comes only from the authenticated principal" || + corrected[1] !== "request-supplied owner fields are ignored" ) { return { ok: false, reason: "response did not satisfy the targeted-read oracle" }; } diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index 44b08af..f7cc239 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -60,6 +60,8 @@ const cases = await withCleanup( PI_OFFLINE: "1", PI_SKIP_VERSION_CHECK: "1", CP06_PI_AUTH_FILE: source, + CP06_PI_PACKAGE_ROOT: process.env.CP06_PI_PACKAGE_ROOT, + CP06_PI_BIN: process.env.CP06_PI_BIN, CP06_OUTPUT: proofOutput, CP06_REVIEWER: "DUMMY cleanup regression", }, diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 149101d..f71d7d3 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -117,6 +117,7 @@ async function main() { reviewer: process.env.CP06_REVIEWER ?? "Pi CP-06 read-only auth correction worker", worker: { pi_version: isolated.child.pi_version, + pi_install: isolated.child.pi_install, interface: "public SDK with injected CredentialStore", provider, model, diff --git a/test/cp06-guarded-read.test.mjs b/test/cp06-guarded-read.test.mjs index 49a590c..daf9170 100644 --- a/test/cp06-guarded-read.test.mjs +++ b/test/cp06-guarded-read.test.mjs @@ -1,5 +1,5 @@ import { strict as assert } from "node:assert"; -import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises"; +import { mkdir, mkdtemp, rm, symlink, unlink, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; @@ -15,22 +15,27 @@ test("guarded read denies noncanonical and unauthorized paths before tool access const credential = join(root, "DUMMY-auth.json"); const alias = join(root, "DUMMY-auth-alias"); await mkdir(join(fixtureRoot, "docs"), { recursive: true }); - await writeFile(original, "DUMMY exact original"); + await writeFile(original, "DUMMY exact\noriginal"); await writeFile(credential, "DUMMY secret canary"); await symlink(credential, alias); let accesses = 0; + const exported = []; const sdk = { - createReadTool() { + createReadTool(_root, options) { return { name: "read", async execute(_id, args) { accesses += 1; - return { content: [{ type: "text", text: args.path }] }; + await options.operations.access(args.path); + const bytes = await options.operations.readFile(args.path); + exported.push(bytes.toString("utf8")); + return { content: [{ type: "text", text: bytes.toString("utf8") }] }; }, }; }, }; - const tool = createGuardedReadTool({ sdk, root, expectedOriginalPath: original }); + const guarded = createGuardedReadTool({ sdk, root, expectedOriginalPath: original }); + const tool = guarded.tool; try { for (const args of [ { path: credential }, @@ -40,14 +45,23 @@ test("guarded read denies noncanonical and unauthorized paths before tool access { path: original, limit: 20 }, { path: original, offset: 2, limit: 20 }, { path: original, offset: 1, limit: 0 }, + { path: original, offset: 1, limit: 1 }, { path: original, extra: true }, ]) { await assert.rejects(tool.execute("DUMMY-read", args)); } assert.equal(accesses, 0); - await tool.execute("DUMMY-read", { path: original }); + const complete = await tool.execute("DUMMY-read", { path: original }); await tool.execute("DUMMY-read", { path: original, offset: 1, limit: 20 }); + assert.equal(complete.content[0].text, "DUMMY exact\noriginal"); + assert.deepEqual(exported, ["DUMMY exact\noriginal", "DUMMY exact\noriginal"]); assert.equal(accesses, 2); + + await unlink(original); + await symlink(credential, original); + await assert.rejects(tool.execute("DUMMY-raced-read", { path: original })); + assert.equal(accesses, 2); + assert.equal(exported.includes("DUMMY secret canary"), false); } finally { await rm(root, { recursive: true, force: true }); } diff --git a/test/cp06-pi-dependency.test.mjs b/test/cp06-pi-dependency.test.mjs index d5bfe4d..9ff7c5a 100644 --- a/test/cp06-pi-dependency.test.mjs +++ b/test/cp06-pi-dependency.test.mjs @@ -1,13 +1,72 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises"; +import { chmod, mkdtemp, mkdir, rm, symlink, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import test from "node:test"; import { pathToFileURL } from "node:url"; +import { resolvePinnedPiInstall } from "../scripts/cp06-pi-install.mjs"; const resolverUrl = pathToFileURL(resolve("scripts/cp06-pi-dependency.mjs")).href; +test("Pi selection requires the explicit task-local package and executable paths", async () => { + const projectRoot = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-pi-install-")); + const installRoot = join(projectRoot, ".factory/state/cp06-sdk"); + const packageRoot = join( + installRoot, + "node_modules", + "@earendil-works", + "pi-coding-agent", + ); + const entry = join(packageRoot, "dist/cli.js"); + const bin = join(installRoot, "node_modules/.bin/pi"); + try { + await mkdir(join(packageRoot, "dist"), { recursive: true }); + await mkdir(join(installRoot, "node_modules/.bin"), { recursive: true }); + await writeFile( + join(packageRoot, "package.json"), + JSON.stringify({ + name: "@earendil-works/pi-coding-agent", + version: "0.85.1", + bin: { pi: "dist/cli.js" }, + }), + ); + await writeFile(entry, "#!/usr/bin/env node\n"); + await writeFile(join(packageRoot, "dist/index.js"), "export const sdk = 'DUMMY';\n"); + await chmod(entry, 0o755); + await symlink("../@earendil-works/pi-coding-agent/dist/cli.js", bin); + const invocations = []; + const selected = resolvePinnedPiInstall({ + projectRoot, + packageRoot, + executable: bin, + spawnSyncImpl(command, args) { + invocations.push([command, ...args]); + return { status: 0, stdout: "0.85.1\n", stderr: "" }; + }, + }); + assert.equal(selected.root, packageRoot); + assert.equal(selected.executable, bin); + assert.deepEqual(invocations, [[bin, "--version"]]); + assert.match(selected.provenance.package_sha256, /^[a-f0-9]{64}$/); + assert.match(selected.provenance.executable_sha256, /^[a-f0-9]{64}$/); + assert.match(selected.provenance.sdk_entry_sha256, /^[a-f0-9]{64}$/); + + for (const substitution of [ + { packageRoot: undefined, executable: bin }, + { packageRoot, executable: undefined }, + { packageRoot: join(projectRoot, "unrelated/pi-coding-agent"), executable: bin }, + { packageRoot, executable: entry }, + ]) { + assert.throws(() => + resolvePinnedPiInstall({ projectRoot, ...substitution, spawnSyncImpl() {} }), + ); + } + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + for (const scenario of [ { name: "exact hoisted import-only dependency", version: "0.85.1", expectedExit: 0 }, { name: "newer hoisted import-only dependency", version: "0.99.2", expectedExit: 1 }, diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 03433d8..46890b1 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -17,6 +17,8 @@ for (const scenario of [ "near-expiry", "setup", "setup-and-cleanup", + "missing-source", + "missing-source-and-cleanup", "launch", "timeout", "audit", @@ -32,6 +34,9 @@ for (const scenario of [ let cleanups = 0; let launches = 0; const setupError = new Error("DUMMY setup failed before mount acquired"); + const missingSourceError = Object.assign(new Error("DUMMY source missing"), { + code: "ENOENT", + }); const cleanupError = new Error("DUMMY cleanup failed"); const run = () => superviseCredentialChild( @@ -49,6 +54,13 @@ for (const scenario of [ if (scenario === "setup-and-cleanup") { throw new AggregateError([setupError, cleanupError], "DUMMY setup cleanup"); } + if (scenario === "missing-source") throw missingSourceError; + if (scenario === "missing-source-and-cleanup") { + throw new AggregateError( + [missingSourceError, cleanupError], + "DUMMY missing source cleanup", + ); + } await writeFile(target, "DUMMY placeholder"); return { sourceIdentity: dummyIdentity(), @@ -105,6 +117,29 @@ for (const scenario of [ ], ); } + if (scenario === "missing-source") { + assert.equal(error.code, "CP06_CREDENTIAL_SOURCE_MISSING"); + assert.equal(error.stage, "missing-source"); + assert.equal(error.exitCode, 70); + const wrapped = namespaceWorkerError({ + status: 70, + stderr: JSON.stringify(supervisorFailureRecord(error)), + }); + const blocked = createWorkerBlockedStatus(wrapped, "isolated-sdk-worker"); + assert.equal(blocked.code, "CP06_CREDENTIAL_SOURCE_MISSING"); + assert.equal(blocked.failure_stage, "missing-source"); + } + if (scenario === "missing-source-and-cleanup") { + const record = supervisorFailureRecord(error); + assert.equal(record.code, "CP06_CLEANUP_FAILED"); + assert.deepEqual( + record.causes.map(({ stage, code }) => ({ stage, code })), + [ + { stage: "missing-source", code: "CP06_CREDENTIAL_SOURCE_MISSING" }, + { stage: "cleanup", code: "CP06_CREDENTIAL_CLEANUP_FAILED" }, + ], + ); + } if (["expired", "near-expiry"].includes(scenario)) { assert.equal(error.exitCode, 75); assert.equal(error.code, "CP06_AUTH_BLOCKED"); @@ -129,8 +164,14 @@ for (const scenario of [ return true; }); } - assert.equal(cleanups, ["setup", "setup-and-cleanup"].includes(scenario) ? 0 : 1); - assert.equal(launches, ["setup", "setup-and-cleanup"].includes(scenario) ? 0 : 1); + const failedBeforeLaunch = [ + "setup", + "setup-and-cleanup", + "missing-source", + "missing-source-and-cleanup", + ].includes(scenario); + assert.equal(cleanups, failedBeforeLaunch ? 0 : 1); + assert.equal(launches, failedBeforeLaunch ? 0 : 1); await assert.rejects(stat(target), { code: "ENOENT" }); assert.equal(await readFile(source, "utf8"), "DUMMY ONLY"); } finally { diff --git a/test/cp06-worker-outcome.test.mjs b/test/cp06-worker-outcome.test.mjs index df199de..4c9bd31 100644 --- a/test/cp06-worker-outcome.test.mjs +++ b/test/cp06-worker-outcome.test.mjs @@ -20,7 +20,8 @@ const response = { reads: [options.expectedOriginalPath], broad_scan: false, corrected_constraints: [ - "Ownership comes only from the authenticated principal; request-supplied owner fields are ignored.", + "ownership comes only from the authenticated principal", + "request-supplied owner fields are ignored", ], evidence_gaps: [gap], }; @@ -76,9 +77,13 @@ test("creation evidence cannot replace source recovery or relax the read outcome }, { corrected_constraints: [ - `${response.corrected_constraints[0]} These creation checks passed.`, + response.corrected_constraints[0], + `${response.corrected_constraints[1]}; creation evidence status: pass`, ], }, + { + corrected_constraints: [...response.corrected_constraints].reverse(), + }, { creation_evidence: { status: "passed" } }, { verified: true }, ]) From d1cd3747054e12ce372471c183374a9df6c15fea Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 17:01:27 +0000 Subject: [PATCH 24/45] no-mistakes(review): Fix CP-06 DUMMY control, response parsing, and SDK provenance --- .factory/cp06-sdk-integrity.json | 18 +++++++++ docs/probes/CP-06.md | 2 +- package.json | 6 +-- scripts/cp06-dummy-outcome-proof.mjs | 8 +++- scripts/cp06-dummy-worker-hook.mjs | 8 +++- scripts/cp06-namespace-supervisor.mjs | 33 +++++++++++++-- scripts/cp06-package-integrity.mjs | 35 ++++++++++++++++ scripts/cp06-pi-dependency.mjs | 12 +++++- scripts/cp06-pi-install.mjs | 6 +++ scripts/cp06-sdk-worker.mjs | 8 ++-- test/cp06-pi-dependency.test.mjs | 58 ++++++++++++++++++++++----- test/cp06-worker-lifecycle.test.mjs | 37 +++++++++++++++++ 12 files changed, 204 insertions(+), 27 deletions(-) create mode 100644 .factory/cp06-sdk-integrity.json create mode 100644 scripts/cp06-package-integrity.mjs diff --git a/.factory/cp06-sdk-integrity.json b/.factory/cp06-sdk-integrity.json new file mode 100644 index 0000000..473fd4f --- /dev/null +++ b/.factory/cp06-sdk-integrity.json @@ -0,0 +1,18 @@ +{ + "schema_version": 1, + "source": "npm registry exact-version tarballs; verify each tarball SHA-512 SRI before extracting and recomputing the sorted [relative path, file SHA-256] manifest digest", + "packages": { + "@earendil-works/pi-coding-agent": { + "version": "0.85.1", + "tarball_integrity": "sha512-FGRN+OHbWaefBPGaTggAdLjrIHW+s2PzLyglz/5dfLzb9of7uuXMXYC0fJIeZTw+shS32o2cuQ9jF7YSDuL/oQ==", + "file_count": 1056, + "manifest_sha256": "ff28a78bb025ee9fc182929f8eec27908f5e20870a5b7d8b1b37a1ded8507ecf" + }, + "@earendil-works/pi-ai": { + "version": "0.85.1", + "tarball_integrity": "sha512-+VgVIJDkDO2efYJKEEqvPTH4zmnIaXdAppGbO+vKFA9qy5PdhFiAenuFAkU+oiCSfOC4dMHDyrjdQeL4ZoC5CQ==", + "file_count": 750, + "manifest_sha256": "3e1894e3f18b90f39c068cd0815f8559083c382e7d343efd1f961a8f07b0cefc" + } + } +} diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 3cad318..c7c1918 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -23,7 +23,7 @@ The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK pat The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. -The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and hashes must agree; inherited `PATH`, a global installation, an unrelated same-version package, and substitutions are rejected without fallback. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths and SHA-256 digests. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. +The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and complete published file manifests must agree with the independently pinned exact-version npm artifacts in `.factory/cp06-sdk-integrity.json`; inherited `PATH`, a global installation, an unrelated same-version package, and same-version file substitutions are rejected before SDK import without fallback. The pin records npm tarball SHA-512 SRI and sorted `[relative path, file SHA-256]` manifest SHA-256 and file count for both packages; it excludes nested installed `node_modules` from each package manifest, while separately verifying the resolved `pi-ai` package. It does not attest the registry itself, transitive packages other than `pi-ai`, or immutable files after this check. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths, trusted artifact pins, manifest digests, and SHA-256 entry digests. To reproduce a pin independently, download the named exact-version tarballs into ignored `.factory/state/`, compare each tarball's SHA-512 against the tracked SRI, extract into a disposable directory, sort all regular published files by relative path and compute SHA-256 of JSON-serialized `[path, SHA-256(file)]` pairs (with no extra files or symlinks); compare count and digest to the tracked pin. Never derive a new trusted pin from the selected installation under evaluation. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is attempted only after that DUMMY gate and fail-closed preflight pass. No destructive operation is ever directed at the real credential source or mounted target. diff --git a/package.json b/package.json index 8b04e9a..6ed3ea4 100644 --- a/package.json +++ b/package.json @@ -9,14 +9,14 @@ "scripts": { "build": "tsc -p tsconfig.json", "typecheck": "tsc -p tsconfig.json --noEmit", - "test": "npm run build && node --test \"dist/test/**/*.test.js\" \"test/**/*.test.mjs\"", + "test": "npm run build && node --experimental-import-meta-resolve --test \"dist/test/**/*.test.js\" \"test/**/*.test.mjs\"", "format": "prettier --write \"{src,test,scripts,.github,schemas}/**/*.{ts,js,mjs,yml,yaml,json}\" \"*.json\" \"docs/probes/*.md\" \"docs/superpowers/plans/*.md\"", "format:check": "prettier --check \"{src,test,scripts,.github,schemas}/**/*.{ts,js,mjs,yml,yaml,json}\" \"*.json\" \"docs/probes/*.md\" \"docs/superpowers/plans/*.md\"", "lint": "node scripts/lint.mjs", "proof:cp06:ten-pack": "npm run build && node scripts/replay-cp06-ten-pack.mjs", "proof:cp06:p07": "node scripts/replay-cp06-p07.mjs", - "proof:cp06:auth-security": "node scripts/prove-cp06-auth-security.mjs", - "proof:cp06:worker": "node scripts/replay-cp06-worker.mjs", + "proof:cp06:auth-security": "node --experimental-import-meta-resolve scripts/prove-cp06-auth-security.mjs", + "proof:cp06:worker": "node --experimental-import-meta-resolve scripts/replay-cp06-worker.mjs", "proof:cp06:worker-cleanup": "node scripts/prove-cp06-worker-cleanup.mjs", "factory": "node dist/src/cli.js" }, diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index 424ca1b..7d81594 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -32,6 +32,8 @@ export async function proveDummyWorkerOutcome({ piRoot }) { const cases = []; for (const [scenario, expectedExit] of [ ["valid", 0], + ["fenced-valid", 0], + ["extra-before-fence", 75], ["empty-gap", 75], ["missing-gap", 75], ["contradictory-gap", 75], @@ -131,7 +133,11 @@ export async function proveDummyWorkerOutcome({ piRoot }) { assert.equal(result.stderr.includes("DUMMY-ACCESS"), false); } if (scenario === "timeout") assert.match(result.stderr, /worker timed out and was aborted/); - const output = scenario === "valid" ? JSON.parse(result.stdout) : undefined; + if (scenario === "extra-before-fence") + assert.match(result.stderr, /SDK worker did not return the required JSON response/); + const output = ["valid", "fenced-valid"].includes(scenario) + ? JSON.parse(result.stdout) + : undefined; if (output) { assert.equal(output.result, "pass"); assert.equal(output.event_stream.read_audit.exact_original, true); diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index 1db0650..65a5a38 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -90,12 +90,18 @@ sdk.ModelRuntime.create = async function (options) { : scenario === "symlink-read" ? input.symlink_path : input.original_path; + const jsonResponse = JSON.stringify(response); + const responseText = scenario === "extra-before-fence" + ? `Creation verified and passed\n\x60\x60\x60json\n${jsonResponse}\n\x60\x60\x60` + : scenario === "fenced-valid" + ? `\x60\x60\x60json\n${jsonResponse}\n\x60\x60\x60` + : jsonResponse; faux.setResponses([ ai.fauxAssistantMessage( ai.fauxToolCall("read", { path: readPath }, { id: "DUMMY-actual-worker-read" }), { stopReason: "toolUse" }, ), - ai.fauxAssistantMessage(JSON.stringify(response), { stopReason: "stop" }), + ai.fauxAssistantMessage(responseText, { stopReason: "stop" }), ]); return runtime; }; diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 5408992..3126d31 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -1,8 +1,8 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { readFileSync } from "node:fs"; -import { resolve } from "node:path"; +import { existsSync, readFileSync, realpathSync } from "node:fs"; +import { dirname, join, relative, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { mountLiveCredentialReadOnly } from "./cp06-credential-isolation.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; @@ -20,6 +20,9 @@ export async function superviseCredentialChild(options, dependencies = {}) { ) ) throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 64); + if (mode === "unsafe-probe" && !isDisposableUnsafeControl(source, target)) { + throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 64); + } if (process.platform !== "linux" || process.arch !== "x64") { throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73); } @@ -99,7 +102,7 @@ export async function superviseCredentialChild(options, dependencies = {}) { } catch { sourceUnchanged = false; } - if (!sourceUnchanged) { + if (!sourceUnchanged && mode !== "unsafe-probe") { throw failure("source-integrity", "CP06_SOURCE_IDENTITY_CHANGED", 74); } let childEvidence; @@ -136,6 +139,22 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur } } +function isDisposableUnsafeControl(source, target) { + const base = join(root, ".factory/state/cp06-auth-security"); + const candidate = dirname(dirname(resolve(source))); + const name = relative(base, candidate); + if (!/^DUMMY-unsafe-probe-[^/]+$/.test(name)) return false; + if (resolve(source) !== join(candidate, "source/DUMMY-auth.json")) return false; + if (resolve(target) !== join(candidate, "agent/DUMMY-auth.json")) return false; + try { + return realpathSync(source) === resolve(source) && + realpathSync(candidate) === candidate && + (!existsSync(dirname(target)) || realpathSync(dirname(target)) === dirname(target)); + } catch { + return false; + } +} + function childCommand({ mode, source, target, helper, syscallProbe, childArgs, beforeHash }) { if (mode === "unsafe-probe") { return { @@ -163,7 +182,13 @@ function childCommand({ mode, source, target, helper, syscallProbe, childArgs, b beforeHash, String(process.pid), ] - : [process.execPath, resolve(root, "scripts/cp06-sdk-worker.mjs"), target, ...childArgs]; + : [ + process.execPath, + "--experimental-import-meta-resolve", + resolve(root, "scripts/cp06-sdk-worker.mjs"), + target, + ...childArgs, + ]; return { command: "setpriv", args: [ diff --git a/scripts/cp06-package-integrity.mjs b/scripts/cp06-package-integrity.mjs new file mode 100644 index 0000000..6198a0f --- /dev/null +++ b/scripts/cp06-package-integrity.mjs @@ -0,0 +1,35 @@ +import { createHash } from "node:crypto"; +import { readFileSync, readdirSync } from "node:fs"; +import { join, relative, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const pins = JSON.parse( + readFileSync(fileURLToPath(new URL("../.factory/cp06-sdk-integrity.json", import.meta.url)), "utf8"), +).packages; + +export function verifyPinnedPackage(root, name) { + const pin = pins[name]; + if (pin?.version !== "0.85.1") throw new Error("unsupported CP-06 package pin"); + const files = []; + function walk(directory) { + for (const entry of readdirSync(directory, { withFileTypes: true })) { + if (entry.name === "node_modules" && directory === root) continue; + const path = join(directory, entry.name); + if (entry.isDirectory()) walk(path); + else if (entry.isFile()) { + files.push([ + relative(root, path).split("\\").join("/"), + createHash("sha256").update(readFileSync(path)).digest("hex"), + ]); + } else throw new Error("selected CP-06 package contains an unsupported file entry"); + } + } + root = resolve(root); + walk(root); + files.sort((a, b) => (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0)); + const manifest = createHash("sha256").update(JSON.stringify(files)).digest("hex"); + if (files.length !== pin.file_count || manifest !== pin.manifest_sha256) { + throw new Error("selected CP-06 package differs from the pinned artifact"); + } + return { manifest_sha256: manifest, tarball_integrity: pin.tarball_integrity }; +} diff --git a/scripts/cp06-pi-dependency.mjs b/scripts/cp06-pi-dependency.mjs index 783d6fd..e140592 100644 --- a/scripts/cp06-pi-dependency.mjs +++ b/scripts/cp06-pi-dependency.mjs @@ -2,12 +2,13 @@ import { createHash } from "node:crypto"; import { readFileSync, realpathSync } from "node:fs"; import { dirname, isAbsolute, join, relative, resolve } from "node:path"; import { fileURLToPath, pathToFileURL } from "node:url"; +import { verifyPinnedPackage } from "./cp06-package-integrity.mjs"; const packageName = "@earendil-works/pi-ai"; const supportedVersion = "0.85.1"; /** Resolve Pi's import-only pi-ai dependency from the explicitly selected Pi package. */ -export async function importPinnedPiAi(piRootArg) { +export function resolvePinnedPiAi(piRootArg) { const piRoot = realpathSync(resolve(piRootArg)); const piMetadataPath = join(piRoot, "package.json"); const piMetadata = JSON.parse(readFileSync(piMetadataPath, "utf8")); @@ -39,9 +40,11 @@ export async function importPinnedPiAi(piRootArg) { throw new Error("resolved pi-ai dependency is outside the selected Pi installation"); } + const integrity = verifyPinnedPackage(dependencyRoot, packageName); return { - module: await import(entryUrl), + entryUrl, provenance: { + ...integrity, name: dependencyMetadata.name, version: dependencyMetadata.version, root: dependencyRoot, @@ -53,6 +56,11 @@ export async function importPinnedPiAi(piRootArg) { }; } +export async function importPinnedPiAi(piRootArg) { + const selected = resolvePinnedPiAi(piRootArg); + return { module: await import(selected.entryUrl), provenance: selected.provenance }; +} + function findPackageRoot(entry, expectedName) { let directory = dirname(entry); for (;;) { diff --git a/scripts/cp06-pi-install.mjs b/scripts/cp06-pi-install.mjs index 39c584e..41035c2 100644 --- a/scripts/cp06-pi-install.mjs +++ b/scripts/cp06-pi-install.mjs @@ -2,6 +2,8 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; import { accessSync, constants, readFileSync, realpathSync } from "node:fs"; import { isAbsolute, join, relative, resolve } from "node:path"; +import { verifyPinnedPackage } from "./cp06-package-integrity.mjs"; +import { resolvePinnedPiAi } from "./cp06-pi-dependency.mjs"; const PACKAGE_NAME = "@earendil-works/pi-coding-agent"; const SUPPORTED_VERSION = "0.85.1"; @@ -45,6 +47,8 @@ export function resolvePinnedPiInstall(options) { throw new Error("selected task-local Pi package is unsupported"); } + const packageIntegrity = verifyPinnedPackage(packageRoot, PACKAGE_NAME); + const piAi = resolvePinnedPiAi(packageRoot); const spawn = options.spawnSyncImpl ?? spawnSync; const version = spawn(expectedExecutable, ["--version"], { cwd: projectRoot, @@ -67,6 +71,8 @@ export function resolvePinnedPiInstall(options) { provenance: { install_root: selectedRoot, package_root: packageRoot, + package_artifact: packageIntegrity, + dependency: piAi.provenance, executable: expectedExecutable, executable_entry: executableEntry, sdk_entry: sdkEntry, diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs index 83b5e37..3ee6524 100644 --- a/scripts/cp06-sdk-worker.mjs +++ b/scripts/cp06-sdk-worker.mjs @@ -237,12 +237,12 @@ function finalAssistantText(events) { } function parseResponse(output) { + const trimmed = output.trim(); + const fenced = trimmed.match(/^```(?:json)?\s*\n([\s\S]*?)\n```$/); try { - return JSON.parse(output.trim()); + return JSON.parse(fenced ? fenced[1] : trimmed); } catch { - const fenced = output.trim().match(/```(?:json)?\s*([\s\S]*?)```/); - if (fenced === null) throw blocked("SDK worker did not return the required JSON response"); - return JSON.parse(fenced[1]); + throw blocked("SDK worker did not return the required JSON response"); } } diff --git a/test/cp06-pi-dependency.test.mjs b/test/cp06-pi-dependency.test.mjs index 9ff7c5a..ab833ee 100644 --- a/test/cp06-pi-dependency.test.mjs +++ b/test/cp06-pi-dependency.test.mjs @@ -1,6 +1,7 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { chmod, mkdtemp, mkdir, rm, symlink, writeFile } from "node:fs/promises"; +import { chmod, cp, mkdtemp, mkdir, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { existsSync } from "node:fs"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import test from "node:test"; @@ -36,7 +37,7 @@ test("Pi selection requires the explicit task-local package and executable paths await chmod(entry, 0o755); await symlink("../@earendil-works/pi-coding-agent/dist/cli.js", bin); const invocations = []; - const selected = resolvePinnedPiInstall({ + assert.throws(() => resolvePinnedPiInstall({ projectRoot, packageRoot, executable: bin, @@ -44,13 +45,8 @@ test("Pi selection requires the explicit task-local package and executable paths invocations.push([command, ...args]); return { status: 0, stdout: "0.85.1\n", stderr: "" }; }, - }); - assert.equal(selected.root, packageRoot); - assert.equal(selected.executable, bin); - assert.deepEqual(invocations, [[bin, "--version"]]); - assert.match(selected.provenance.package_sha256, /^[a-f0-9]{64}$/); - assert.match(selected.provenance.executable_sha256, /^[a-f0-9]{64}$/); - assert.match(selected.provenance.sdk_entry_sha256, /^[a-f0-9]{64}$/); + }), /pinned artifact/); + assert.deepEqual(invocations, []); for (const substitution of [ { packageRoot: undefined, executable: bin }, @@ -67,8 +63,48 @@ test("Pi selection requires the explicit task-local package and executable paths } }); +test("exact artifact succeeds and same-version selected entry substitutions fail before SDK use", async (t) => { + const installed = resolve(".factory/state/cp06-sdk/node_modules/@earendil-works"); + if (!existsSync(join(installed, "pi-ai"))) return t.skip("task-local CP-06 SDK is not installed"); + const projectRoot = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-pinned-artifact-")); + const selected = join(projectRoot, ".factory/state/cp06-sdk/node_modules"); + const packages = join(selected, "@earendil-works"); + const packageRoot = join(packages, "pi-coding-agent"); + const executable = join(selected, ".bin/pi"); + try { + await mkdir(packages, { recursive: true }); + await mkdir(join(selected, ".bin")); + for (const name of ["pi-coding-agent", "pi-ai"]) { + await cp(join(installed, name), join(packages, name), { recursive: true }); + } + const binPath = JSON.parse(await readFile(join(packageRoot, "package.json"))).bin; + await symlink(join(packageRoot, typeof binPath === "string" ? binPath : binPath.pi), executable); + let launches = 0; + const options = { + projectRoot, packageRoot, executable, + spawnSyncImpl() { launches++; return { status: 0, stdout: "0.85.1\n" }; }, + }; + const positive = resolvePinnedPiInstall(options); + assert.equal(positive.provenance.dependency.version, "0.85.1"); + assert.match(positive.provenance.package_artifact.manifest_sha256, /^[a-f0-9]{64}$/); + assert.equal(launches, 1); + const entry = join(packageRoot, "dist/index.js"); + const original = await readFile(entry); + await writeFile(entry, "DUMMY same-version SDK replacement"); + assert.throws(() => resolvePinnedPiInstall(options), /pinned artifact/); + assert.equal(launches, 1); + await writeFile(entry, original); + const aiEntry = join(packages, "pi-ai/dist/index.js"); + await writeFile(aiEntry, "DUMMY same-version dependency replacement"); + assert.throws(() => resolvePinnedPiInstall(options), /pinned artifact/); + assert.equal(launches, 1); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + for (const scenario of [ - { name: "exact hoisted import-only dependency", version: "0.85.1", expectedExit: 0 }, + { name: "substituted same-version import-only dependency", version: "0.85.1", expectedExit: 1 }, { name: "newer hoisted import-only dependency", version: "0.99.2", expectedExit: 1 }, ]) { test(`pinned Pi dependency resolver handles ${scenario.name}`, async () => { @@ -118,7 +154,7 @@ for (const scenario of [ assert.match(output.provenance.package_sha256, /^[a-f0-9]{64}$/); assert.match(output.provenance.entry_sha256, /^[a-f0-9]{64}$/); } else { - assert.match(result.stderr, /not exact version 0\.85\.1/); + assert.match(result.stderr, scenario.version === "0.85.1" ? /pinned artifact/ : /not exact version 0\.85\.1/); } } finally { await rm(directory, { recursive: true, force: true }); diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 46890b1..c6708bb 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -1,5 +1,6 @@ import { strict as assert } from "node:assert"; import { lstat, mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; @@ -180,6 +181,42 @@ for (const scenario of [ }); } +test("unsafe mode accepts only disposable DUMMY control and records observed mutation", async () => { + const base = join(process.cwd(), ".factory/state/cp06-auth-security"); + await mkdir(base, { recursive: true }); + const directory = await mkdtemp(join(base, "DUMMY-unsafe-probe-test-")); + const source = join(directory, "source/DUMMY-auth.json"); + const target = join(directory, "agent/DUMMY-auth.json"); + let launched = 0; + try { + await mkdir(join(directory, "source")); + await writeFile(source, "DUMMY original"); + const options = { mode: "unsafe-probe", source, target, helper: "/DUMMY-helper", syscallProbe: "/DUMMY-probe" }; + await assert.rejects( + superviseCredentialChild({ ...options, source: join(directory, "source/not-allowed") }, { + mount() { launched++; }, + }), + { code: "CP06_ISOLATION_SETUP_FAILED" }, + ); + assert.equal(launched, 0); + const result = await superviseCredentialChild(options, { + mount() { + return { mountIds: { source: "1", target: "2" }, parentMounts: [], cleanup() {} }; + }, + spawn() { + launched++; + writeFileSync(source, "DUMMY mutated"); + return { status: 0, stdout: '{"result":"DUMMY"}' }; + }, + }); + assert.equal(result.source_unchanged, false); + assert.equal(result.mode, "unsafe-probe"); + assert.equal(launched, 1); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + test("worker source integrity uses metadata without reading credential bytes", async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-metadata-")); const source = join(directory, "DUMMY-unreadable-source"); From 5cee2d80f6ebc85a808243accd0d101b90ddb0cb Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 17:25:53 +0000 Subject: [PATCH 25/45] no-mistakes(review): Pin both DUMMY probes to supervisor-owned fixtures --- docs/probes/CP-06.md | 2 + scripts/cp06-auth-security.mjs | 44 ++-- scripts/cp06-credential-isolation.mjs | 9 +- scripts/cp06-namespace-supervisor.mjs | 294 ++++++++++++++------------ scripts/cp06-probe-fixture.mjs | 135 ++++++++++++ scripts/prove-cp06-worker-cleanup.mjs | 8 +- test/cp06-worker-lifecycle.test.mjs | 119 ++++++++--- 7 files changed, 406 insertions(+), 205 deletions(-) create mode 100644 scripts/cp06-probe-fixture.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index c7c1918..4f0ad24 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,6 +21,8 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, pins the fixture directory and source inode through mount, adversary execution and integrity audit, then unmounts/removes the fixture. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. + The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and complete published file manifests must agree with the independently pinned exact-version npm artifacts in `.factory/cp06-sdk-integrity.json`; inherited `PATH`, a global installation, an unrelated same-version package, and same-version file substitutions are rejected before SDK import without fallback. The pin records npm tarball SHA-512 SRI and sorted `[relative path, file SHA-256]` manifest SHA-256 and file count for both packages; it excludes nested installed `node_modules` from each package manifest, while separately verifying the resolved `pi-ai` package. It does not attest the registry itself, transitive packages other than `pi-ai`, or immutable files after this check. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths, trusted artifact pins, manifest digests, and SHA-256 entry digests. To reproduce a pin independently, download the named exact-version tarballs into ignored `.factory/state/`, compare each tarball's SHA-512 against the tracked SRI, extract into a disposable directory, sort all regular published files by relative path and compute SHA-256 of JSON-serialized `[path, SHA-256(file)]` pairs (with no extra files or symlinks); compare count and digest to the tracked pin. Never derive a new trusted pin from the selected installation under evaluation. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 9f21cd2..3e678f3 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -1,8 +1,8 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; import { readFileSync } from "node:fs"; -import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; -import { dirname, join, resolve } from "node:path"; +import { mkdir } from "node:fs/promises"; +import { join, resolve } from "node:path"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; export class Cp06IsolationUnsupportedError extends Error { @@ -171,29 +171,20 @@ export function namespaceWorkerError(result) { } async function runIsolationProbe({ root, outputRoot, mode, binaries }) { - const proofRoot = await mkdtemp(join(outputRoot, `DUMMY-${mode}-`)); - const source = join(proofRoot, "source", "DUMMY-auth.json"); - const target = join(proofRoot, "agent", "DUMMY-auth.json"); - await mkdir(dirname(source), { recursive: true }); - await writeFile(source, "DUMMY ORIGINAL", { mode: 0o600 }); - try { - const result = runNamespace({ - root, - mode, - source, - target, - helper: binaries.helper, - syscallProbe: binaries.syscallProbe, - childArgs: [], - timeout: 90_000, - }); - if (result.status !== 0) { - throw unsupported(`${mode} failed: ${String(result.stderr).trim() || result.status}`); - } - return parseCompactJson(result.stdout, mode); - } finally { - await rm(proofRoot, { recursive: true, force: true }); + const result = runNamespace({ + root, + mode, + source: outputRoot, + target: "-", + helper: binaries.helper, + syscallProbe: binaries.syscallProbe, + childArgs: [], + timeout: 90_000, + }); + if (result.status !== 0) { + throw unsupported(`${mode} failed: ${String(result.stderr).trim() || result.status}`); } + return parseCompactJson(result.stdout, mode); } function runNamespace(options) { @@ -208,7 +199,7 @@ function runNamespace(options) { join(options.root, "scripts/cp06-namespace-supervisor.mjs"), options.mode, resolve(options.source), - resolve(options.target), + options.mode === "worker" ? resolve(options.target) : "-", resolve(options.helper), resolve(options.syscallProbe), ...options.childArgs.map((value) => resolve(value)), @@ -241,8 +232,7 @@ function isSupervisorFailure(value, status) { cause !== null && typeof cause === "object" && !Array.isArray(cause) && - Object.keys(cause).sort().join(",") === - "code,description,exit_code,stage" && + Object.keys(cause).sort().join(",") === "code,description,exit_code,stage" && SUPERVISOR_FAILURE_CODES.get(cause.code) === cause.stage && cause.description === SUPERVISOR_FAILURE_DESCRIPTIONS[cause.code] && (cause.exit_code === null || Number.isSafeInteger(cause.exit_code)), diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs index 24a2e03..2c92e30 100644 --- a/scripts/cp06-credential-isolation.mjs +++ b/scripts/cp06-credential-isolation.mjs @@ -1,6 +1,6 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { constants } from "node:fs"; +import { constants, realpathSync } from "node:fs"; import { link, lstat, @@ -135,10 +135,7 @@ export async function mountLiveCredentialReadOnly(options) { } const currentSourceMount = inspectReadOnlyMount(source, { spawn }); const currentTargetMount = inspectReadOnlyMount(target, { spawn }); - if ( - currentSourceMount.id !== sourceMount.id || - currentTargetMount.id !== targetMount.id - ) { + if (currentSourceMount.id !== sourceMount.id || currentTargetMount.id !== targetMount.id) { return false; } for (const expected of parentMounts) { @@ -234,7 +231,7 @@ export function inspectReadOnlyMount(path, options = {}) { const optionsList = String(entry?.["vfs-options"] ?? "").split(","); if ( entry === undefined || - resolve(entry.target) !== resolve(path) || + resolve(entry.target) !== realpathSync(path) || !REQUIRED_VFS_OPTIONS.every((option) => optionsList.includes(option)) ) { throw new Error(`credential mount lacks required read-only VFS flags: ${path}`); diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 3126d31..f66c41e 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -1,26 +1,32 @@ #!/usr/bin/env node -import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { existsSync, readFileSync, realpathSync } from "node:fs"; -import { dirname, join, relative, resolve } from "node:path"; +import { resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { mountLiveCredentialReadOnly } from "./cp06-credential-isolation.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; +import { createProbeFixture } from "./cp06-probe-fixture.mjs"; const root = resolve(fileURLToPath(new URL("..", import.meta.url))); // Injectable OS operations permit deterministic failure/cleanup tests without // credentials, namespaces or an installed SDK in the ordinary unit suite. export async function superviseCredentialChild(options, dependencies = {}) { - const { mode, source, target, helper, syscallProbe, childArgs = [] } = options; + const { + mode, + source: requestedSource, + target: requestedTarget, + helper, + syscallProbe, + childArgs = [], + } = options; if ( !["unsafe-probe", "hardened-probe", "worker"].includes(mode) || - ![source, target, helper, syscallProbe].every( + ![requestedSource, requestedTarget, helper, syscallProbe].every( (value) => typeof value === "string" && value.length > 0, ) ) throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 64); - if (mode === "unsafe-probe" && !isDisposableUnsafeControl(source, target)) { + if (mode !== "worker" && (requestedTarget !== "-" || childArgs.length !== 0)) { throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 64); } if (process.platform !== "linux" || process.arch !== "x64") { @@ -28,142 +34,139 @@ export async function superviseCredentialChild(options, dependencies = {}) { } const mountCredential = dependencies.mount ?? mountLiveCredentialReadOnly; const spawn = dependencies.spawn ?? spawnSync; - const beforeHash = mode === "worker" ? null : sha256(readFileSync(source)); + let probe; + if (mode !== "worker") { + try { + probe = await (dependencies.createProbeFixture ?? createProbeFixture)({ + root, + outputRoot: requestedSource, + }); + } catch (cause) { + throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73, cause); + } + } + const source = probe?.source ?? requestedSource; + const target = probe?.target ?? requestedTarget; + const beforeHash = probe?.beforeHash ?? null; let mount; return await withCleanup( - async () => { - try { - mount = await mountCredential({ - root, - source, - target, - protectParents: mode !== "unsafe-probe", - }); - } catch (cause) { - const missingSource = hasErrorCode(cause, "ENOENT"); - const setup = failure( - missingSource ? "missing-source" : "setup", - missingSource ? "CP06_CREDENTIAL_SOURCE_MISSING" : "CP06_ISOLATION_SETUP_FAILED", - missingSource ? 70 : 73, - cause, - ); - if (cause instanceof AggregateError) { - throw compoundFailure( - setup, - failure("cleanup", "CP06_CREDENTIAL_CLEANUP_FAILED", 74), - ); - } - throw setup; - } - const command = childCommand({ - mode, - source, - target, - helper, - syscallProbe, - childArgs, - beforeHash, - }); - let result; - try { - result = spawn(command.command, command.args, { - cwd: root, - encoding: "utf8", - env: childEnvironment(mode, source, target), - timeout: mode === "worker" ? 600_000 : 60_000, - maxBuffer: 1024 * 1024, - }); - } catch (cause) { - throw failure("launch", "CP06_CHILD_LAUNCH_FAILED", 70, cause); - } - if (result.error !== undefined) { - const timeout = result.error?.code === "ETIMEDOUT"; - throw failure( - timeout ? "timeout" : "launch", - timeout ? "CP06_CHILD_TIMEOUT" : "CP06_CHILD_LAUNCH_FAILED", - 70, - result.error, - ); - } - if (result.status !== 0) { - throw failure( - "child-exit", - result.status === 75 ? "CP06_AUTH_BLOCKED" : "CP06_CHILD_EXIT_FAILED", - result.status ?? 70, - ); - } - - let sourceUnchanged; - try { - sourceUnchanged = - mode === "worker" - ? await mount.verifyIntegrity() - : beforeHash === sha256(readFileSync(source)); - } catch { - sourceUnchanged = false; - } - if (!sourceUnchanged && mode !== "unsafe-probe") { - throw failure("source-integrity", "CP06_SOURCE_IDENTITY_CHANGED", 74); - } - let childEvidence; - try { - childEvidence = JSON.parse(String(result.stdout).trim()); - } catch { - throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); - } - return { - schema_version: 1, - mode, - source_unchanged: sourceUnchanged, - mount_ids_distinct: mount.mountIds.source !== mount.mountIds.target, - parent_roots_read_only: mount.parentMounts.length > 0, - cleanup: "pass", - child: childEvidence, - }; - }, - () => mount?.cleanup(), - ); -} + () => + withCleanup( + async () => { + try { + mount = await mountCredential({ + root, + source, + target, + protectParents: mode !== "unsafe-probe", + }); + } catch (cause) { + const missingSource = hasErrorCode(cause, "ENOENT"); + const setup = failure( + missingSource ? "missing-source" : "setup", + missingSource ? "CP06_CREDENTIAL_SOURCE_MISSING" : "CP06_ISOLATION_SETUP_FAILED", + missingSource ? 70 : 73, + cause, + ); + if (cause instanceof AggregateError) { + throw compoundFailure( + setup, + failure("cleanup", "CP06_CREDENTIAL_CLEANUP_FAILED", 74), + ); + } + throw setup; + } + const command = childCommand({ + mode, + source, + target, + helper, + syscallProbe, + childArgs, + beforeHash, + probe: probe !== undefined, + }); + let result; + try { + result = spawn(command.command, command.args, { + cwd: probe?.cwd ?? root, + encoding: "utf8", + env: childEnvironment(mode, source, target), + timeout: mode === "worker" ? 600_000 : 60_000, + maxBuffer: 1024 * 1024, + }); + } catch (cause) { + throw failure("launch", "CP06_CHILD_LAUNCH_FAILED", 70, cause); + } + if (result.error !== undefined) { + const timeout = result.error?.code === "ETIMEDOUT"; + throw failure( + timeout ? "timeout" : "launch", + timeout ? "CP06_CHILD_TIMEOUT" : "CP06_CHILD_LAUNCH_FAILED", + 70, + result.error, + ); + } + if (result.status !== 0) { + throw failure( + "child-exit", + result.status === 75 ? "CP06_AUTH_BLOCKED" : "CP06_CHILD_EXIT_FAILED", + result.status ?? 70, + ); + } -if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { - const [mode, source, target, helper, syscallProbe, ...childArgs] = process.argv.slice(2); - try { - console.log( - JSON.stringify( - await superviseCredentialChild({ mode, source, target, helper, syscallProbe, childArgs }), + let sourceUnchanged; + try { + sourceUnchanged = + mode === "worker" ? await mount.verifyIntegrity() : probe.sourceUnchanged(); + } catch { + sourceUnchanged = false; + } + if (!sourceUnchanged && mode !== "unsafe-probe") { + throw failure("source-integrity", "CP06_SOURCE_IDENTITY_CHANGED", 74); + } + let childEvidence; + try { + childEvidence = JSON.parse(String(result.stdout).trim()); + } catch { + throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); + } + return { + schema_version: 1, + mode, + source_unchanged: sourceUnchanged, + mount_ids_distinct: mount.mountIds.source !== mount.mountIds.target, + parent_roots_read_only: mount.parentMounts.length > 0, + cleanup: "pass", + child: childEvidence, + }; + }, + () => mount?.cleanup(), ), - ); - } catch (error) { - process.stderr.write(`${JSON.stringify(supervisorFailureRecord(error))}\n`); - process.exitCode = error.exitCode ?? 70; - } -} - -function isDisposableUnsafeControl(source, target) { - const base = join(root, ".factory/state/cp06-auth-security"); - const candidate = dirname(dirname(resolve(source))); - const name = relative(base, candidate); - if (!/^DUMMY-unsafe-probe-[^/]+$/.test(name)) return false; - if (resolve(source) !== join(candidate, "source/DUMMY-auth.json")) return false; - if (resolve(target) !== join(candidate, "agent/DUMMY-auth.json")) return false; - try { - return realpathSync(source) === resolve(source) && - realpathSync(candidate) === candidate && - (!existsSync(dirname(target)) || realpathSync(dirname(target)) === dirname(target)); - } catch { - return false; - } + () => probe?.cleanup(), + ); } -function childCommand({ mode, source, target, helper, syscallProbe, childArgs, beforeHash }) { +function childCommand({ + mode, + source, + target, + helper, + syscallProbe, + childArgs, + beforeHash, + probe, +}) { + const childSource = probe ? "source/DUMMY-auth.json" : source; + const childTarget = probe ? "agent/DUMMY-auth.json" : target; if (mode === "unsafe-probe") { return { command: process.execPath, args: [ resolve(root, "scripts/cp06-isolation-adversary.mjs"), "unsafe", - source, - target, + childSource, + childTarget, syscallProbe, beforeHash, String(process.pid), @@ -176,8 +179,8 @@ function childCommand({ mode, source, target, helper, syscallProbe, childArgs, b process.execPath, resolve(root, "scripts/cp06-isolation-adversary.mjs"), "hardened", - source, - target, + childSource, + childTarget, syscallProbe, beforeHash, String(process.pid), @@ -260,13 +263,7 @@ function compoundFailure(primary, cleanup) { function hasErrorCode(error, code) { if (error?.code === code) return true; - return ( - error instanceof AggregateError && error.errors.some((cause) => hasErrorCode(cause, code)) - ); -} - -function sha256(bytes) { - return createHash("sha256").update(bytes).digest("hex"); + return error instanceof AggregateError && error.errors.some((cause) => hasErrorCode(cause, code)); } function classifyFailure(error) { @@ -307,7 +304,10 @@ const FAILURE_DESCRIPTIONS = { CP06_CHILD_LAUNCH_FAILED: { stage: "launch", description: "isolated child launch failed" }, CP06_CHILD_TIMEOUT: { stage: "timeout", description: "isolated child timed out" }, CP06_AUTH_BLOCKED: { stage: "child-exit", description: "credential preflight blocked the child" }, - CP06_CHILD_EXIT_FAILED: { stage: "child-exit", description: "isolated child exited unsuccessfully" }, + CP06_CHILD_EXIT_FAILED: { + stage: "child-exit", + description: "isolated child exited unsuccessfully", + }, CP06_SOURCE_IDENTITY_CHANGED: { stage: "source-integrity", description: "credential source identity changed", @@ -319,3 +319,17 @@ const FAILURE_DESCRIPTIONS = { description: "credential mount cleanup failed", }, }; + +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + const [mode, source, target, helper, syscallProbe, ...childArgs] = process.argv.slice(2); + try { + console.log( + JSON.stringify( + await superviseCredentialChild({ mode, source, target, helper, syscallProbe, childArgs }), + ), + ); + } catch (error) { + process.stderr.write(`${JSON.stringify(supervisorFailureRecord(error))}\n`); + process.exitCode = error.exitCode ?? 70; + } +} diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs new file mode 100644 index 0000000..19455a8 --- /dev/null +++ b/scripts/cp06-probe-fixture.mjs @@ -0,0 +1,135 @@ +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { constants, openSync, closeSync, fstatSync, lstatSync, readSync } from "node:fs"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { join, relative, resolve } from "node:path"; + +const original = Buffer.from("DUMMY ORIGINAL"); + +export async function createProbeFixture({ root, outputRoot, spawn = spawnSync }) { + const state = resolve(root, ".factory/state"); + const selected = resolve(outputRoot); + const subpath = relative(state, selected); + if ( + subpath.startsWith("..") || + subpath === "" || + !subpath.split("/").includes("auth-security") || + selected !== outputRoot + ) + throw new Error("DUMMY proof root is outside task-local auth-security state"); + + const directoryFlags = constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW; + const projectFd = openSync(root, directoryFlags); + let factoryFd; + let stateFd; + try { + factoryFd = openSync(`/proc/${process.pid}/fd/${projectFd}/.factory`, directoryFlags); + stateFd = openSync(`/proc/${process.pid}/fd/${factoryFd}/state`, directoryFlags); + } finally { + if (factoryFd !== undefined) closeSync(factoryFd); + closeSync(projectFd); + } + let outputFd; + let traversedFd; + let rootFd; + let sourceFd; + let directory; + let mounted = false; + try { + for (const component of subpath.split("/")) { + const nextFd = openSync( + join(`/proc/${process.pid}/fd/${traversedFd ?? stateFd}`, component), + directoryFlags, + ); + if (traversedFd !== undefined) closeSync(traversedFd); + traversedFd = nextFd; + } + outputFd = traversedFd; + traversedFd = undefined; + if ( + lstatSync(selected, { bigint: true }).ino !== fstatSync(outputFd, { bigint: true }).ino || + lstatSync(selected, { bigint: true }).dev !== fstatSync(outputFd, { bigint: true }).dev + ) { + throw new Error("DUMMY proof root identity changed"); + } + directory = await mkdtemp(join(`/proc/${process.pid}/fd/${outputFd}`, "DUMMY-probe-")); + checkedMount(spawn, ["-t", "tmpfs", "-o", "mode=0700,nosuid,nodev,noexec", "tmpfs", directory]); + mounted = true; + rootFd = openSync(directory, directoryFlags); + const anchoredRoot = `/proc/${process.pid}/fd/${rootFd}`; + await mkdir(join(anchoredRoot, "source"), { mode: 0o700 }); + await mkdir(join(anchoredRoot, "agent"), { mode: 0o700 }); + const source = join(anchoredRoot, "source/DUMMY-auth.json"); + const target = join(anchoredRoot, "agent/DUMMY-auth.json"); + await writeFile(source, original, { flag: "wx", mode: 0o600 }); + sourceFd = openSync(source, constants.O_RDONLY | constants.O_NOFOLLOW); + const pinned = fstatSync(sourceFd, { bigint: true }); + if ( + !pinned.isFile() || + pinned.nlink !== 1n || + !sameInode(pinned, lstatSync(source, { bigint: true })) + ) { + throw new Error("DUMMY source identity changed"); + } + const beforeHash = hashPinned(sourceFd, original.length); + return { + source, + target, + cwd: anchoredRoot, + beforeHash, + sourceUnchanged() { + try { + const named = lstatSync(source, { bigint: true }); + return ( + sameInode(pinned, named) && + named.isFile() && + hashPinned(sourceFd, original.length) === beforeHash + ); + } catch { + return false; + } + }, + async cleanup() { + const alias = join(anchoredRoot, "DUMMY-alias"); + const mountedAlias = spawn("findmnt", ["--mountpoint", alias, "--noheadings"], { + encoding: "utf8", + timeout: 10_000, + }); + if (mountedAlias.status === 0) checkedMount(spawn, ["--", alias], "umount"); + closeSync(sourceFd); + closeSync(rootFd); + checkedMount(spawn, ["--", directory], "umount"); + mounted = false; + await rm(directory, { recursive: true, force: true }); + closeSync(outputFd); + closeSync(stateFd); + }, + }; + } catch (error) { + if (sourceFd !== undefined) closeSync(sourceFd); + if (rootFd !== undefined) closeSync(rootFd); + if (mounted) checkedMount(spawn, ["--", directory], "umount"); + if (directory !== undefined) await rm(directory, { recursive: true, force: true }); + if (outputFd !== undefined) closeSync(outputFd); + if (traversedFd !== undefined) closeSync(traversedFd); + closeSync(stateFd); + throw error; + } +} + +function hashPinned(fd, length) { + const buffer = Buffer.alloc(length); + if (readSync(fd, buffer, 0, length, 0) !== length) return null; + return createHash("sha256").update(buffer).digest("hex"); +} + +function sameInode(a, b) { + return a.dev === b.dev && a.ino === b.ino; +} + +function checkedMount(spawn, args, command = "mount") { + const result = spawn(command, args, { encoding: "utf8", timeout: 10_000 }); + if (result.error !== undefined || result.status !== 0) { + throw new Error("private DUMMY fixture mount unavailable"); + } +} diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index f7cc239..f290bb7 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -26,8 +26,9 @@ const cases = await withCleanup( const home = join(scratch, scenario); const temporary = join(home, "tmp"); const proofOutput = join(home, "proofs"); - const source = join(home, "DUMMY-auth.json"); + const source = join(home, "credential", "DUMMY-auth.json"); await mkdir(temporary, { recursive: true }); + await mkdir(join(home, "credential"), { recursive: true }); const sourceBytes = JSON.stringify({ "openai-codex": { type: "oauth", @@ -47,7 +48,10 @@ const cases = await withCleanup( join(proofOutput, "ten-pack/repo/docs/AUTH.md"), await readFile(join(root, "test/fixtures/cp06-context/docs/AUTH.md")), ); - const args = [join(root, "scripts/replay-cp06-worker.mjs")]; + const args = [ + "--experimental-import-meta-resolve", + join(root, "scripts/replay-cp06-worker.mjs"), + ]; const result = spawnSync(process.execPath, args, { cwd: root, encoding: "utf8", diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index c6708bb..267d2ad 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -1,6 +1,8 @@ import { strict as assert } from "node:assert"; -import { lstat, mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { lstat, mkdir, mkdtemp, readFile, rm, stat, symlink, writeFile } from "node:fs/promises"; import { writeFileSync } from "node:fs"; +import { spawnSync } from "node:child_process"; +import { resolve } from "node:path"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; @@ -181,38 +183,95 @@ for (const scenario of [ }); } -test("unsafe mode accepts only disposable DUMMY control and records observed mutation", async () => { - const base = join(process.cwd(), ".factory/state/cp06-auth-security"); +test("both probe modes require a supervisor-generated DUMMY fixture", async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-probe-test-")); + const source = join(directory, "DUMMY-auth.json"); + const secret = join(directory, "DUMMY-sentinel-secret.json"); + const outputRoot = resolve(".factory/state/cp06-correction/auth-security"); + await writeFile(secret, "DUMMY SECRET NEVER READ"); + try { + for (const mode of ["unsafe-probe", "hardened-probe"]) { + await writeFile(source, "DUMMY ORIGINAL"); + let prepared = 0; + let mounts = 0; + let launches = 0; + const options = { + mode, + source: outputRoot, + target: "-", + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + }; + const dependencies = { + async createProbeFixture({ outputRoot: selected }) { + prepared++; + assert.equal(selected, outputRoot); + return { + source, + target: join(directory, "DUMMY-target"), + beforeHash: "DUMMY-hash", + sourceUnchanged: () => mode !== "unsafe-probe", + cleanup() {}, + }; + }, + mount() { + mounts++; + return { mountIds: { source: "1", target: "2" }, parentMounts: [], cleanup() {} }; + }, + spawn() { + launches++; + if (mode === "unsafe-probe") writeFileSync(source, "DUMMY mutated"); + return { status: 0, stdout: '{"result":"DUMMY"}' }; + }, + }; + await assert.rejects(superviseCredentialChild({ ...options, target: secret }, dependencies), { + code: "CP06_ISOLATION_SETUP_FAILED", + }); + assert.equal(prepared, 0); + const result = await superviseCredentialChild(options, dependencies); + assert.equal(result.source_unchanged, mode === "hardened-probe"); + assert.equal(prepared, 1); + assert.equal(mounts, 1); + assert.equal(launches, 1); + } + assert.equal(await readFile(secret, "utf8"), "DUMMY SECRET NEVER READ"); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + +test("the probe executable refuses arbitrary and symlinked DUMMY roots before access", async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-probe-refusal-")); + const secret = join(directory, "DUMMY-sentinel-secret.json"); + const base = resolve(".factory/state/cp06-correction"); + const alias = join(base, "auth-security-DUMMY-alias"); await mkdir(base, { recursive: true }); - const directory = await mkdtemp(join(base, "DUMMY-unsafe-probe-test-")); - const source = join(directory, "source/DUMMY-auth.json"); - const target = join(directory, "agent/DUMMY-auth.json"); - let launched = 0; + await writeFile(secret, "DUMMY SECRET NEVER READ"); + await mkdir(join(directory, "auth-security")); + await symlink(directory, alias); try { - await mkdir(join(directory, "source")); - await writeFile(source, "DUMMY original"); - const options = { mode: "unsafe-probe", source, target, helper: "/DUMMY-helper", syscallProbe: "/DUMMY-probe" }; - await assert.rejects( - superviseCredentialChild({ ...options, source: join(directory, "source/not-allowed") }, { - mount() { launched++; }, - }), - { code: "CP06_ISOLATION_SETUP_FAILED" }, - ); - assert.equal(launched, 0); - const result = await superviseCredentialChild(options, { - mount() { - return { mountIds: { source: "1", target: "2" }, parentMounts: [], cleanup() {} }; - }, - spawn() { - launched++; - writeFileSync(source, "DUMMY mutated"); - return { status: 0, stdout: '{"result":"DUMMY"}' }; - }, - }); - assert.equal(result.source_unchanged, false); - assert.equal(result.mode, "unsafe-probe"); - assert.equal(launched, 1); + for (const mode of ["unsafe-probe", "hardened-probe"]) { + for (const requested of [secret, join(alias, "auth-security")]) { + const result = spawnSync( + process.execPath, + [ + resolve("scripts/cp06-namespace-supervisor.mjs"), + mode, + requested, + "-", + "/DUMMY-helper", + "/DUMMY-probe", + ], + { encoding: "utf8", timeout: 10_000 }, + ); + assert.equal(result.status, 73); + assert.equal(result.stdout, ""); + assert.equal(JSON.parse(result.stderr).code, "CP06_ISOLATION_SETUP_FAILED"); + } + } + assert.equal(await readFile(secret, "utf8"), "DUMMY SECRET NEVER READ"); } finally { + await rm(alias); await rm(directory, { recursive: true, force: true }); } }); From 2fb24cae42cfa8b64fe123e4de56781461f625b4 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 17:47:02 +0000 Subject: [PATCH 26/45] no-mistakes(review): Harden CP-06 proof boundaries and failure reporting --- docs/probes/CP-06.md | 2 +- scripts/cp06-auth-security.mjs | 103 +++++++++------- scripts/cp06-namespace-supervisor.mjs | 37 +++++- scripts/cp06-probe-fixture.mjs | 129 ++++++++++++++------ scripts/prove-cp06-auth-security.mjs | 8 +- test/cp06-worker-lifecycle.test.mjs | 150 +++++++++++++++++++++++- test/fixtures/cp06-probe-transition.mjs | 81 +++++++++++++ 7 files changed, 420 insertions(+), 90 deletions(-) create mode 100644 test/fixtures/cp06-probe-transition.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 4f0ad24..020b647 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,7 +21,7 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, pins the fixture directory and source inode through mount, adversary execution and integrity audit, then unmounts/removes the fixture. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is anchored to that directory. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, pins the fixture and source/target parent directories plus source inode through mount, adversary execution and complete-byte DUMMY integrity audit, then unmounts/removes the fixture. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 3e678f3..a0a7111 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -1,9 +1,9 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; import { readFileSync } from "node:fs"; -import { mkdir } from "node:fs/promises"; import { join, resolve } from "node:path"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; +import { openProbeOutput } from "./cp06-probe-fixture.mjs"; export class Cp06IsolationUnsupportedError extends Error { constructor(message) { @@ -16,48 +16,56 @@ export class Cp06IsolationUnsupportedError extends Error { export async function runCp06SecurityPreflight(options = {}) { const root = resolve(options.root ?? process.cwd()); const outputRoot = resolve(options.outputRoot ?? join(root, ".factory/state/cp06-auth-security")); - await mkdir(outputRoot, { recursive: true }); - const prerequisites = verifyIsolationPrerequisites({ - root, - spawnSyncImpl: options.spawnSyncImpl, - }); - const binaries = compileIsolationHelpers({ root, outputRoot }); - const pi = resolvePiPackage(root, options); - const unsafe = await runIsolationProbe({ - root, - outputRoot, - mode: "unsafe-probe", - binaries, - }); - assertUnsafeControl(unsafe); - const hardened = await runIsolationProbe({ - root, - outputRoot, - mode: "hardened-probe", - binaries, - }); - assertHardenedProof(hardened); - const sdk = runDummySdkProof({ root, helper: binaries.helper, piRoot: pi.root }); - return { - schema_version: 1, - gate: "CP-06-read-only-auth-security", - result: "pass", - tested_sha: commandText(spawnSync, "git", ["rev-parse", "HEAD"], root), - recorded_at: new Date().toISOString(), - reviewer: options.reviewer ?? "Pi CP-06 read-only auth correction worker", - command: "npm run proof:cp06:auth-security", - exit_code: 0, - pi_version: pi.version, - pi_install: pi.provenance, - platform: prerequisites, - binaries: { - seccomp_helper_sha256: sha256(readFileSync(binaries.helper)), - syscall_probe_sha256: sha256(readFileSync(binaries.syscallProbe)), - }, - unsafe_control: summarizeUnsafe(unsafe), - hardened_child: summarizeHardened(hardened), - dummy_sdk: sdk, - }; + const output = openProbeOutput({ root, outputRoot, create: true }); + try { + const prerequisites = verifyIsolationPrerequisites({ + root, + spawnSyncImpl: options.spawnSyncImpl, + }); + const binaries = compileIsolationHelpers({ + root, + outputRoot, + anchoredOutputRoot: output.anchor, + }); + const pi = resolvePiPackage(root, options); + const unsafe = await runIsolationProbe({ + root, + outputRoot, + mode: "unsafe-probe", + binaries, + }); + assertUnsafeControl(unsafe); + const hardened = await runIsolationProbe({ + root, + outputRoot, + mode: "hardened-probe", + binaries, + }); + assertHardenedProof(hardened); + const sdk = runDummySdkProof({ root, helper: binaries.helper, piRoot: pi.root }); + return { + schema_version: 1, + gate: "CP-06-read-only-auth-security", + result: "pass", + tested_sha: commandText(spawnSync, "git", ["rev-parse", "HEAD"], root), + recorded_at: new Date().toISOString(), + reviewer: options.reviewer ?? "Pi CP-06 read-only auth correction worker", + command: "npm run proof:cp06:auth-security", + exit_code: 0, + pi_version: pi.version, + pi_install: pi.provenance, + platform: prerequisites, + binaries: { + seccomp_helper_sha256: sha256(readFileSync(binaries.helper)), + syscall_probe_sha256: sha256(readFileSync(binaries.syscallProbe)), + }, + unsafe_control: summarizeUnsafe(unsafe), + hardened_child: summarizeHardened(hardened), + dummy_sdk: sdk, + }; + } finally { + output.close(); + } } export function verifyIsolationPrerequisites(options = {}) { @@ -95,6 +103,11 @@ export function compileIsolationHelpers(options) { const outputRoot = resolve(options.outputRoot); const helper = join(outputRoot, "cp06-seccomp-exec"); const syscallProbe = join(outputRoot, "cp06-isolation-syscalls"); + const anchoredHelper = join(options.anchoredOutputRoot ?? outputRoot, "cp06-seccomp-exec"); + const anchoredSyscallProbe = join( + options.anchoredOutputRoot ?? outputRoot, + "cp06-isolation-syscalls", + ); checked( "cc", [ @@ -106,7 +119,7 @@ export function compileIsolationHelpers(options) { join(root, "scripts/cp06-seccomp-exec.c"), "-Wl,-l:libseccomp.so.2", "-o", - helper, + anchoredHelper, ], root, ); @@ -120,7 +133,7 @@ export function compileIsolationHelpers(options) { "-Werror", join(root, "scripts/cp06-isolation-syscalls.c"), "-o", - syscallProbe, + anchoredSyscallProbe, ], root, ); diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index f66c41e..b031a9f 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -54,6 +54,7 @@ export async function superviseCredentialChild(options, dependencies = {}) { withCleanup( async () => { try { + probe?.assertSafePaths?.(); mount = await mountCredential({ root, source, @@ -76,6 +77,13 @@ export async function superviseCredentialChild(options, dependencies = {}) { } throw setup; } + if (probe !== undefined) { + try { + probe.assertSafePaths?.({ mounted: true }); + } catch (cause) { + throw failure("source-integrity", "CP06_SOURCE_IDENTITY_CHANGED", 74, cause); + } + } const command = childCommand({ mode, source, @@ -244,8 +252,7 @@ export function supervisorFailureRecord(error) { code: failure.code, exit_code: failure.exit_code, description: failure.description, - causes: - error instanceof AggregateError ? error.errors.map((cause) => classifyFailure(cause)) : [], + causes: error instanceof AggregateError ? leafCauses(error).map(classifyFailure) : [], }; } @@ -266,13 +273,31 @@ function hasErrorCode(error, code) { return error instanceof AggregateError && error.errors.some((cause) => hasErrorCode(cause, code)); } +function leafCauses(error, limit = 8) { + const leaves = []; + function visit(cause) { + if (leaves.length >= limit) return; + if (cause instanceof AggregateError && cause.errors.length > 0) { + for (const nested of cause.errors) visit(nested); + } else leaves.push(cause); + } + visit(error); + return leaves; +} + +function safeExitCode(error) { + return Number.isSafeInteger(error?.exitCode) && error.exitCode >= 1 && error.exitCode <= 255 + ? error.exitCode + : null; +} + function classifyFailure(error) { const known = FAILURE_DESCRIPTIONS[error?.code]; if (known !== undefined) { return { - stage: error.stage ?? known.stage, + stage: known.stage, code: error.code, - exit_code: error.exitCode ?? null, + exit_code: safeExitCode(error), description: known.description, }; } @@ -280,14 +305,14 @@ function classifyFailure(error) { return { stage: "cleanup", code: "CP06_CLEANUP_FAILED", - exit_code: error.exitCode ?? 74, + exit_code: safeExitCode(error) ?? 74, description: "credential namespace cleanup failed", }; } return { stage: "cleanup", code: "CP06_CREDENTIAL_CLEANUP_FAILED", - exit_code: error?.exitCode ?? null, + exit_code: safeExitCode(error), description: "credential mount cleanup failed", }; } diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index 19455a8..6f7e857 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -1,64 +1,94 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { constants, openSync, closeSync, fstatSync, lstatSync, readSync } from "node:fs"; +import { closeSync, constants, fstatSync, lstatSync, mkdirSync, openSync, readSync } from "node:fs"; import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { join, relative, resolve } from "node:path"; const original = Buffer.from("DUMMY ORIGINAL"); +const directoryFlags = constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW; -export async function createProbeFixture({ root, outputRoot, spawn = spawnSync }) { +export function openProbeOutput({ root, outputRoot, create = false }) { const state = resolve(root, ".factory/state"); const selected = resolve(outputRoot); const subpath = relative(state, selected); if ( - subpath.startsWith("..") || + selected !== outputRoot || subpath === "" || - !subpath.split("/").includes("auth-security") || - selected !== outputRoot + subpath === ".." || + subpath.startsWith("../") || + !(subpath === "cp06-auth-security" || subpath.split("/").includes("auth-security")) ) throw new Error("DUMMY proof root is outside task-local auth-security state"); - const directoryFlags = constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW; const projectFd = openSync(root, directoryFlags); - let factoryFd; - let stateFd; + let currentFd = projectFd; try { - factoryFd = openSync(`/proc/${process.pid}/fd/${projectFd}/.factory`, directoryFlags); - stateFd = openSync(`/proc/${process.pid}/fd/${factoryFd}/state`, directoryFlags); + const components = [".factory", "state", ...subpath.split("/")]; + let scanFd = projectFd; + try { + for (const component of components) { + const path = join(`/proc/${process.pid}/fd/${scanFd}`, component); + try { + const entry = lstatSync(path); + if (!entry.isDirectory()) throw new Error("DUMMY proof output contains a non-directory"); + const nextFd = openSync(path, directoryFlags); + if (scanFd !== projectFd) closeSync(scanFd); + scanFd = nextFd; + } catch (error) { + if (error?.code !== "ENOENT" || !create) throw error; + break; + } + } + } finally { + if (scanFd !== projectFd) closeSync(scanFd); + } + for (const component of components) { + const path = join(`/proc/${process.pid}/fd/${currentFd}`, component); + if (create) { + try { + mkdirSync(path, { mode: 0o700 }); + } catch (error) { + if (error?.code !== "EEXIST") throw error; + } + } + const nextFd = openSync(path, directoryFlags); + if (currentFd !== projectFd) closeSync(currentFd); + currentFd = nextFd; + } + const outputFd = currentFd; + currentFd = projectFd; + return { + path: selected, + anchor: `/proc/${process.pid}/fd/${outputFd}`, + identity: fstatSync(outputFd, { bigint: true }), + close() { + closeSync(outputFd); + }, + }; } finally { - if (factoryFd !== undefined) closeSync(factoryFd); + if (currentFd !== projectFd) closeSync(currentFd); closeSync(projectFd); } - let outputFd; - let traversedFd; +} + +export async function createProbeFixture({ root, outputRoot, spawn = spawnSync }) { + const output = openProbeOutput({ root, outputRoot }); let rootFd; + let sourceDirFd; + let agentDirFd; let sourceFd; let directory; let mounted = false; try { - for (const component of subpath.split("/")) { - const nextFd = openSync( - join(`/proc/${process.pid}/fd/${traversedFd ?? stateFd}`, component), - directoryFlags, - ); - if (traversedFd !== undefined) closeSync(traversedFd); - traversedFd = nextFd; - } - outputFd = traversedFd; - traversedFd = undefined; - if ( - lstatSync(selected, { bigint: true }).ino !== fstatSync(outputFd, { bigint: true }).ino || - lstatSync(selected, { bigint: true }).dev !== fstatSync(outputFd, { bigint: true }).dev - ) { - throw new Error("DUMMY proof root identity changed"); - } - directory = await mkdtemp(join(`/proc/${process.pid}/fd/${outputFd}`, "DUMMY-probe-")); + directory = await mkdtemp(join(output.anchor, "DUMMY-probe-")); checkedMount(spawn, ["-t", "tmpfs", "-o", "mode=0700,nosuid,nodev,noexec", "tmpfs", directory]); mounted = true; rootFd = openSync(directory, directoryFlags); const anchoredRoot = `/proc/${process.pid}/fd/${rootFd}`; await mkdir(join(anchoredRoot, "source"), { mode: 0o700 }); await mkdir(join(anchoredRoot, "agent"), { mode: 0o700 }); + sourceDirFd = openSync(join(anchoredRoot, "source"), directoryFlags); + agentDirFd = openSync(join(anchoredRoot, "agent"), directoryFlags); const source = join(anchoredRoot, "source/DUMMY-auth.json"); const target = join(anchoredRoot, "agent/DUMMY-auth.json"); await writeFile(source, original, { flag: "wx", mode: 0o600 }); @@ -72,18 +102,44 @@ export async function createProbeFixture({ root, outputRoot, spawn = spawnSync } throw new Error("DUMMY source identity changed"); } const beforeHash = hashPinned(sourceFd, original.length); + const sourceParent = fstatSync(sourceDirFd, { bigint: true }); + const targetParent = fstatSync(agentDirFd, { bigint: true }); return { source, target, cwd: anchoredRoot, beforeHash, + assertSafePaths({ mounted = false } = {}) { + if ( + !sameInode(sourceParent, lstatSync(join(anchoredRoot, "source"), { bigint: true })) || + !sameInode(targetParent, lstatSync(join(anchoredRoot, "agent"), { bigint: true })) || + !sameInode(pinned, lstatSync(source, { bigint: true })) + ) + throw new Error("DUMMY fixture path identity changed"); + if (mounted) { + if (!sameInode(pinned, lstatSync(target, { bigint: true }))) { + throw new Error("DUMMY fixture target identity changed"); + } + } else { + try { + lstatSync(target); + throw new Error("DUMMY fixture target was replaced"); + } catch (error) { + if (error?.code !== "ENOENT") throw error; + } + } + }, sourceUnchanged() { try { const named = lstatSync(source, { bigint: true }); + const current = fstatSync(sourceFd, { bigint: true }); return ( sameInode(pinned, named) && named.isFile() && - hashPinned(sourceFd, original.length) === beforeHash + sameInode(pinned, current) && + current.size === pinned.size && + named.size === pinned.size && + hashPinned(sourceFd, Number(pinned.size)) === beforeHash ); } catch { return false; @@ -97,22 +153,23 @@ export async function createProbeFixture({ root, outputRoot, spawn = spawnSync } }); if (mountedAlias.status === 0) checkedMount(spawn, ["--", alias], "umount"); closeSync(sourceFd); + closeSync(sourceDirFd); + closeSync(agentDirFd); closeSync(rootFd); checkedMount(spawn, ["--", directory], "umount"); mounted = false; await rm(directory, { recursive: true, force: true }); - closeSync(outputFd); - closeSync(stateFd); + output.close(); }, }; } catch (error) { if (sourceFd !== undefined) closeSync(sourceFd); + if (sourceDirFd !== undefined) closeSync(sourceDirFd); + if (agentDirFd !== undefined) closeSync(agentDirFd); if (rootFd !== undefined) closeSync(rootFd); if (mounted) checkedMount(spawn, ["--", directory], "umount"); if (directory !== undefined) await rm(directory, { recursive: true, force: true }); - if (outputFd !== undefined) closeSync(outputFd); - if (traversedFd !== undefined) closeSync(traversedFd); - closeSync(stateFd); + output.close(); throw error; } } diff --git a/scripts/prove-cp06-auth-security.mjs b/scripts/prove-cp06-auth-security.mjs index 2c50956..4e8cd88 100644 --- a/scripts/prove-cp06-auth-security.mjs +++ b/scripts/prove-cp06-auth-security.mjs @@ -3,6 +3,7 @@ import { writeFile } from "node:fs/promises"; import { dirname, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { Cp06IsolationUnsupportedError, runCp06SecurityPreflight } from "./cp06-auth-security.mjs"; +import { openProbeOutput } from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); @@ -13,7 +14,12 @@ try { reviewer: process.env.CP06_REVIEWER, }); const path = resolve(outputRoot, "auth-security", "evidence.json"); - await writeFile(path, `${JSON.stringify(proof, null, 2)}\n`); + const output = openProbeOutput({ root, outputRoot: resolve(outputRoot, "auth-security") }); + try { + await writeFile(resolve(output.anchor, "evidence.json"), `${JSON.stringify(proof, null, 2)}\n`); + } finally { + output.close(); + } console.log( JSON.stringify({ ok: true, diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 267d2ad..2b9c02e 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -1,5 +1,15 @@ import { strict as assert } from "node:assert"; -import { lstat, mkdir, mkdtemp, readFile, rm, stat, symlink, writeFile } from "node:fs/promises"; +import { + lstat, + mkdir, + mkdtemp, + readFile, + readdir, + rm, + stat, + symlink, + writeFile, +} from "node:fs/promises"; import { writeFileSync } from "node:fs"; import { spawnSync } from "node:child_process"; import { resolve } from "node:path"; @@ -276,6 +286,144 @@ test("the probe executable refuses arbitrary and symlinked DUMMY roots before ac } }); +test("proof entrypoint rejects a symlinked output without writing outside state", async () => { + const directory = await mkdtemp(join(resolve(".factory/state"), "DUMMY-output-reject-")); + const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-external-")); + const sentinel = join(external, "DUMMY-sentinel"); + await writeFile(sentinel, "DUMMY untouched"); + await symlink(external, join(directory, "auth-security")); + try { + const result = spawnSync( + process.execPath, + ["--experimental-import-meta-resolve", resolve("scripts/prove-cp06-auth-security.mjs")], + { + cwd: process.cwd(), + encoding: "utf8", + timeout: 15_000, + env: { ...process.env, CP06_OUTPUT: directory }, + }, + ); + assert.notEqual(result.status, 0); + assert.equal(result.stdout, ""); + assert.deepEqual(await readdir(external), ["DUMMY-sentinel"]); + assert.equal(await readFile(sentinel, "utf8"), "DUMMY untouched"); + } finally { + await rm(directory, { recursive: true, force: true }); + await rm(external, { recursive: true, force: true }); + } +}); + +test("probe transition rejects DUMMY substitutions before mount for both modes", async () => { + const directory = await mkdtemp(join(resolve(".factory/state"), "DUMMY-transition-")); + const sentinel = join(directory, "DUMMY-sentinel-secret.json"); + await writeFile(sentinel, "DUMMY-SENTINEL-SECRET"); + try { + for (const mode of ["unsafe-probe", "hardened-probe"]) { + for (const scenario of [ + "unchanged", + "append", + "truncate", + "rewrite", + "source-ancestor", + "target-slot", + ]) { + const result = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + resolve("test/fixtures/cp06-probe-transition.mjs"), + mode, + scenario, + sentinel, + ], + { cwd: process.cwd(), encoding: "utf8", timeout: 30_000 }, + ); + assert.equal(result.status, 0, `${mode}/${scenario}: ${result.stderr}`); + const proof = JSON.parse(result.stdout); + assert.equal(proof.sentinel_unchanged, true); + assert.equal(proof.mounts, 0); + assert.equal(proof.launches, 0); + if (scenario === "source-ancestor" || scenario === "target-slot") { + assert.equal(proof.failure.code, "CP06_ISOLATION_SETUP_FAILED"); + assert.equal(JSON.stringify(proof).includes("DUMMY-SENTINEL-SECRET"), false); + } + } + } + assert.equal(await readFile(sentinel, "utf8"), "DUMMY-SENTINEL-SECRET"); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + +test("nested audit, mount and fixture cleanup failures retain typed causes", async () => { + const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-nested-cleanup-")); + try { + let caught; + try { + await superviseCredentialChild( + { + mode: "unsafe-probe", + source: resolve(".factory/state/cp06-correction/auth-security"), + target: "-", + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + }, + { + createProbeFixture() { + return { + source: join(directory, "DUMMY-source"), + target: join(directory, "DUMMY-target"), + beforeHash: "DUMMY", + cleanup() { + throw new Error("DUMMY fixture secret"); + }, + }; + }, + mount() { + return { + mountIds: { source: "1", target: "2" }, + parentMounts: [], + cleanup() { + throw new Error("DUMMY mount secret"); + }, + }; + }, + spawn() { + return { status: 0, stdout: "DUMMY malformed", stderr: "DUMMY child secret" }; + }, + }, + ); + } catch (error) { + caught = error; + } + assert(caught instanceof AggregateError); + const record = supervisorFailureRecord(caught); + assert.equal(record.code, "CP06_CLEANUP_FAILED"); + assert.deepEqual( + record.causes.map(({ stage, code }) => [stage, code]), + [ + ["audit", "CP06_CHILD_OUTPUT_INVALID"], + ["cleanup", "CP06_CREDENTIAL_CLEANUP_FAILED"], + ["cleanup", "CP06_CREDENTIAL_CLEANUP_FAILED"], + ], + ); + const wrapped = namespaceWorkerError({ status: 74, stderr: JSON.stringify(record) }); + assert.deepEqual( + createWorkerBlockedStatus(wrapped, "isolated-sdk-worker").worker_failure, + record, + ); + assert.equal(JSON.stringify(record).includes("secret"), false); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + test("worker source integrity uses metadata without reading credential bytes", async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-metadata-")); const source = join(directory, "DUMMY-unreadable-source"); diff --git a/test/fixtures/cp06-probe-transition.mjs b/test/fixtures/cp06-probe-transition.mjs new file mode 100644 index 0000000..46672d0 --- /dev/null +++ b/test/fixtures/cp06-probe-transition.mjs @@ -0,0 +1,81 @@ +import { strict as assert } from "node:assert"; +import { + appendFile, + mkdir, + readFile, + rename, + symlink, + truncate, + writeFile, +} from "node:fs/promises"; +import { dirname, join, resolve } from "node:path"; +import { createProbeFixture } from "../../scripts/cp06-probe-fixture.mjs"; +import { + superviseCredentialChild, + supervisorFailureRecord, +} from "../../scripts/cp06-namespace-supervisor.mjs"; + +const [mode, scenario, sentinel] = process.argv.slice(2); +const root = process.cwd(); +const outputRoot = resolve(root, ".factory/state/cp06-correction/auth-security"); +if (!["unsafe-probe", "hardened-probe"].includes(mode)) process.exit(64); +const secretBefore = await readFile(sentinel); +let fixture; +let mounts = 0; +let launches = 0; +let failure; +if (["unchanged", "append", "truncate", "rewrite"].includes(scenario)) { + fixture = await createProbeFixture({ root, outputRoot }); + try { + assert.equal(fixture.sourceUnchanged(), true); + if (scenario === "append") await appendFile(fixture.source, " DUMMY appended"); + if (scenario === "truncate") await truncate(fixture.source, 5); + if (scenario === "rewrite") await writeFile(fixture.source, "DUMMY ALTERED!", { flag: "r+" }); + assert.equal(fixture.sourceUnchanged(), scenario === "unchanged"); + } finally { + await fixture.cleanup(); + } +} else { + try { + await superviseCredentialChild( + { + mode, + source: outputRoot, + target: "-", + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + }, + { + async createProbeFixture() { + fixture = await createProbeFixture({ root, outputRoot }); + if (scenario === "source-ancestor") { + const named = join(fixture.cwd, "source"); + await rename(named, join(fixture.cwd, "source-moved")); + await symlink(dirname(sentinel), named); + } else if (scenario === "target-slot") { + await symlink(sentinel, fixture.target); + } else throw new Error("unsupported DUMMY scenario"); + return fixture; + }, + mount() { + mounts++; + throw new Error("DUMMY mount must not run"); + }, + spawn() { + launches++; + throw new Error("DUMMY adversary must not run"); + }, + }, + ); + throw new Error("substituted DUMMY fixture was accepted"); + } catch (error) { + failure = supervisorFailureRecord(error); + assert.equal(failure.code, "CP06_ISOLATION_SETUP_FAILED"); + assert.equal(mounts, 0); + assert.equal(launches, 0); + } +} +assert.deepEqual(await readFile(sentinel), secretBefore); +console.log( + JSON.stringify({ mode, scenario, mounts, launches, failure, sentinel_unchanged: true }), +); From c92c2fe608aa24c714115a667ae7ac0f9401f03a Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 18:12:49 +0000 Subject: [PATCH 27/45] no-mistakes(review): Sanitize outputs; native FD-mount and exec handoff remain required --- docs/probes/CP-06.md | 4 +- scripts/cp06-auth-security.mjs | 29 ++++++-- scripts/cp06-probe-fixture.mjs | 6 +- scripts/replay-cp06-p07.mjs | 13 +++- scripts/replay-cp06-ten-pack.mjs | 9 ++- scripts/replay-cp06-worker.mjs | 16 ++-- test/cp06-proof-boundary.test.mjs | 117 ++++++++++++++++++++++++++++++ 7 files changed, 171 insertions(+), 23 deletions(-) create mode 100644 test/cp06-proof-boundary.test.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 020b647..42a5c43 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,13 +21,13 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is anchored to that directory. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, pins the fixture and source/target parent directories plus source inode through mount, adversary execution and complete-byte DUMMY integrity audit, then unmounts/removes the fixture. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is anchored to that directory. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. This does not yet pin the actual mount helper's pathname opens/target creation through a concurrent parent replacement, nor the compiled helpers' pathname execution and final hashes after output replacement. Those transitions remain unverified even if the ordinary DUMMY controls pass. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and complete published file manifests must agree with the independently pinned exact-version npm artifacts in `.factory/cp06-sdk-integrity.json`; inherited `PATH`, a global installation, an unrelated same-version package, and same-version file substitutions are rejected before SDK import without fallback. The pin records npm tarball SHA-512 SRI and sorted `[relative path, file SHA-256]` manifest SHA-256 and file count for both packages; it excludes nested installed `node_modules` from each package manifest, while separately verifying the resolved `pi-ai` package. It does not attest the registry itself, transitive packages other than `pi-ai`, or immutable files after this check. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths, trusted artifact pins, manifest digests, and SHA-256 entry digests. To reproduce a pin independently, download the named exact-version tarballs into ignored `.factory/state/`, compare each tarball's SHA-512 against the tracked SRI, extract into a disposable directory, sort all regular published files by relative path and compute SHA-256 of JSON-serialized `[path, SHA-256(file)]` pairs (with no extra files or symlinks); compare count and digest to the tracked pin. Never derive a new trusted pin from the selected installation under evaluation. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. -Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is attempted only after that DUMMY gate and fail-closed preflight pass. No destructive operation is ever directed at the real credential source or mounted target. +Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it must not be attempted until the mount and executable identity handoff gaps above are repaired and the final-head DUMMY gate and transition regressions pass. The existing `cp06-seccomp-exec` helper closes all inherited file descriptors; the current Node interface has no FD-relative mount API. Repair requires a bounded FD-based mount consumer and a seccomp-safe handoff that executes/hashes the same retained helper/probe identities without passing credential descriptors to the evaluated child. No unsupported lexical fallback is acceptable. No destructive operation is ever directed at the real credential source or mounted target. ## Normative result diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index a0a7111..de0ba01 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -183,8 +183,14 @@ export function namespaceWorkerError(result) { return error; } -async function runIsolationProbe({ root, outputRoot, mode, binaries }) { - const result = runNamespace({ +export async function runIsolationProbe({ + root, + outputRoot, + mode, + binaries, + runNamespaceImpl = runNamespace, +}) { + const result = runNamespaceImpl({ root, mode, source: outputRoot, @@ -194,8 +200,11 @@ async function runIsolationProbe({ root, outputRoot, mode, binaries }) { childArgs: [], timeout: 90_000, }); - if (result.status !== 0) { - throw unsupported(`${mode} failed: ${String(result.stderr).trim() || result.status}`); + if (result.error !== undefined || result.status !== 0) { + const failure = namespaceWorkerError(result); + const error = unsupported(`${mode} failed: ${failure.code}`); + error.workerFailure = failure.workerFailure; + throw error; } return parseCompactJson(result.stdout, mode); } @@ -358,8 +367,8 @@ function assertHardenedProof(proof) { } } -function runDummySdkProof({ root, helper, piRoot }) { - const result = spawnSync( +export function runDummySdkProof({ root, helper, piRoot, spawnSyncImpl = spawnSync }) { + const result = spawnSyncImpl( "unshare", [ "--user", @@ -396,7 +405,13 @@ function runDummySdkProof({ root, helper, piRoot }) { }, ); if (result.error !== undefined || result.status !== 0) { - throw unsupported(`DUMMY SDK proof failed: ${result.error?.message ?? result.stderr}`); + const code = + result.error?.code === "ETIMEDOUT" + ? "CP06_DUMMY_SDK_TIMEOUT" + : result.error !== undefined + ? "CP06_DUMMY_SDK_LAUNCH_FAILED" + : "CP06_DUMMY_SDK_CHILD_EXIT"; + throw unsupported(`DUMMY SDK proof failed: ${code}`); } return parseCompactJson(result.stdout, "DUMMY SDK proof"); } diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index 6f7e857..b1dd763 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -7,7 +7,7 @@ import { join, relative, resolve } from "node:path"; const original = Buffer.from("DUMMY ORIGINAL"); const directoryFlags = constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW; -export function openProbeOutput({ root, outputRoot, create = false }) { +export function openProbeOutput({ root, outputRoot, create = false, runner = false }) { const state = resolve(root, ".factory/state"); const selected = resolve(outputRoot); const subpath = relative(state, selected); @@ -16,7 +16,9 @@ export function openProbeOutput({ root, outputRoot, create = false }) { subpath === "" || subpath === ".." || subpath.startsWith("../") || - !(subpath === "cp06-auth-security" || subpath.split("/").includes("auth-security")) + !(runner + ? subpath === "cp06-correction" || subpath.startsWith("cp06-correction/") + : subpath === "cp06-auth-security" || subpath.split("/").includes("auth-security")) ) throw new Error("DUMMY proof root is outside task-local auth-security state"); diff --git a/scripts/replay-cp06-p07.mjs b/scripts/replay-cp06-p07.mjs index 83ce136..5f5bcfa 100644 --- a/scripts/replay-cp06-p07.mjs +++ b/scripts/replay-cp06-p07.mjs @@ -5,11 +5,13 @@ import { mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; +import { openProbeOutput } from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); -const rawRoot = join(outputRoot, "p07", "raw"); -await rm(join(outputRoot, "p07"), { recursive: true, force: true }); +const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); +const rawRoot = join(output.anchor, "p07", "raw"); +await rm(join(output.anchor, "p07"), { recursive: true, force: true }); await mkdir(rawRoot, { recursive: true }); const steps = []; @@ -65,7 +67,7 @@ const manifest = { "The historical module is a minimal retained reproduction, not a raw worker transcript.", ], }; -const manifestPath = join(outputRoot, "p07", "evidence.json"); +const manifestPath = join(output.anchor, "p07", "evidence.json"); await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); console.log( JSON.stringify({ @@ -121,7 +123,10 @@ function quote(parts) { } function relative(path) { - return path.startsWith(`${root}/`) ? path.slice(root.length + 1) : path; + const located = path.startsWith(`${output.anchor}/`) + ? join(outputRoot, path.slice(output.anchor.length + 1)) + : path; + return located.startsWith(`${root}/`) ? located.slice(root.length + 1) : located; } function sha256(text) { diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index 1353a7c..ee7a1d8 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -5,11 +5,13 @@ import { cp, mkdir, readFile, rm, stat, writeFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; +import { openProbeOutput } from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const fixtureSource = join(root, "test/fixtures/cp06-context"); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); -const proofRoot = join(outputRoot, "ten-pack"); +const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); +const proofRoot = join(output.anchor, "ten-pack"); const fixtureRoot = join(proofRoot, "repo"); const rawRoot = join(proofRoot, "raw"); const cli = join(root, "dist/src/cli.js"); @@ -678,7 +680,10 @@ function quote(parts) { } function relative(path) { - return path.startsWith(`${root}/`) ? path.slice(root.length + 1) : path; + const located = path.startsWith(`${output.anchor}/`) + ? join(outputRoot, path.slice(output.anchor.length + 1)) + : path; + return located.startsWith(`${root}/`) ? located.slice(root.length + 1) : located; } function slug(path) { diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index f71d7d3..dde70a9 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -6,6 +6,7 @@ import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; +import { openProbeOutput } from "./cp06-probe-fixture.mjs"; import { Cp06IsolationUnsupportedError, runCp06SecurityPreflight, @@ -14,7 +15,8 @@ import { const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); -const workerRoot = join(outputRoot, "worker"); +const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); +const workerRoot = join(output.anchor, "worker"); const securityRoot = join(outputRoot, "auth-security"); const fixtureRoot = join(outputRoot, "ten-pack", "repo"); const initialPackPath = join(outputRoot, "ten-pack", "raw", "auth-worker-missing.pack.md"); @@ -65,8 +67,8 @@ async function main() { timeout_ms: timeoutMs, system_prompt: systemPrompt, }; - const inputPath = join(workerRoot, "input.json"); - await writeFile(inputPath, `${JSON.stringify(workerInput, null, 2)}\n`); + const inputPath = join(outputRoot, "worker", "input.json"); + await writeFile(join(workerRoot, "input.json"), `${JSON.stringify(workerInput, null, 2)}\n`); const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-sdk-worker-")); const credentialTarget = join(evaluationHome, "pi-agent", "auth.json"); const credentialSource = resolve( @@ -198,8 +200,7 @@ export function createWorkerBlockedStatus(error, stage, recordedAt = new Date(). failure_stage: error?.stage ?? null, code: error?.code ?? "CP06_WORKER_FAILED", cleanup_failed: - error?.code === "CP06_CLEANUP_FAILED" || - nestedWorkerFailure?.code === "CP06_CLEANUP_FAILED", + error?.code === "CP06_CLEANUP_FAILED" || nestedWorkerFailure?.code === "CP06_CLEANUP_FAILED", worker_failure: nestedWorkerFailure ?? null, causes: error instanceof AggregateError @@ -221,7 +222,10 @@ function textCommand(command, args) { } function relative(path) { - return path.startsWith(`${root}/`) ? path.slice(root.length + 1) : path; + const located = path.startsWith(`${output.anchor}/`) + ? join(outputRoot, path.slice(output.anchor.length + 1)) + : path; + return located.startsWith(`${root}/`) ? located.slice(root.length + 1) : located; } async function fileHash(path) { diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs new file mode 100644 index 0000000..4452b21 --- /dev/null +++ b/test/cp06-proof-boundary.test.mjs @@ -0,0 +1,117 @@ +import { strict as assert } from "node:assert"; +import { spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, readdir, rm, symlink, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import test from "node:test"; +import { runDummySdkProof, runIsolationProbe } from "../scripts/cp06-auth-security.mjs"; + +const runners = [ + "scripts/replay-cp06-worker.mjs", + "scripts/replay-cp06-ten-pack.mjs", + "scripts/replay-cp06-p07.mjs", +]; + +test("all proof runners reject external and symlinked output before removing artifacts", async () => { + const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-external-output-")); + const base = resolve(".factory/state/cp06-correction"); + const alias = join(base, "DUMMY-output-alias"); + await mkdir(base, { recursive: true }); + for (const name of ["worker", "ten-pack", "p07"]) { + await mkdir(join(external, name)); + await writeFile(join(external, name, "DUMMY-sentinel"), "DUMMY-SECRET-UNCHANGED"); + } + await symlink(external, alias); + try { + for (const runner of runners) { + for (const output of [external, alias]) { + const result = spawnSync(process.execPath, [resolve(runner)], { + cwd: process.cwd(), + encoding: "utf8", + timeout: 15_000, + env: { ...process.env, CP06_OUTPUT: output }, + }); + assert.notEqual(result.status, 0, `${runner} accepted ${output}`); + assert.equal(result.stdout.includes("DUMMY-SECRET-UNCHANGED"), false); + assert.equal(result.stderr.includes("DUMMY-SECRET-UNCHANGED"), false); + for (const name of ["worker", "ten-pack", "p07"]) { + assert.deepEqual(await readdir(join(external, name)), ["DUMMY-sentinel"]); + assert.equal( + await readFile(join(external, name, "DUMMY-sentinel"), "utf8"), + "DUMMY-SECRET-UNCHANGED", + ); + } + } + } + } finally { + await rm(alias); + await rm(external, { recursive: true, force: true }); + } +}); + +test("failed DUMMY probes never forward child output or spawn exceptions", async () => { + const secret = "DUMMY-SECRET-STDOUT-STDERR"; + const structured = { + schema_version: 1, + stage: "audit", + code: "CP06_CHILD_OUTPUT_INVALID", + exit_code: 70, + description: "isolated child output was invalid", + causes: [], + }; + for (const result of [ + { status: 70, stderr: `${JSON.stringify(structured)}\n${secret}`, stdout: secret }, + { status: 70, stderr: JSON.stringify(structured), stdout: secret }, + { + status: null, + error: Object.assign(new Error(secret), { code: "ETIMEDOUT" }), + stderr: secret, + }, + ]) { + await assert.rejects( + runIsolationProbe({ + root: process.cwd(), + outputRoot: resolve(".factory/state/cp06-correction/auth-security"), + mode: "unsafe-probe", + binaries: { helper: "/DUMMY-helper", syscallProbe: "/DUMMY-probe" }, + runNamespaceImpl() { + return result; + }, + }), + (error) => { + assert.equal(error.code, "CP06_ISOLATION_UNSUPPORTED"); + assert.equal( + JSON.stringify({ message: error.message, workerFailure: error.workerFailure }).includes( + secret, + ), + false, + ); + if (result.stderr === JSON.stringify(structured)) + assert.deepEqual(error.workerFailure, structured); + else assert.equal(error.workerFailure, undefined); + return true; + }, + ); + } + for (const result of [ + { status: 1, stdout: secret, stderr: secret }, + { + status: null, + error: Object.assign(new Error(secret), { code: "ETIMEDOUT" }), + stderr: secret, + }, + ]) { + assert.throws( + () => + runDummySdkProof({ + root: process.cwd(), + helper: "/DUMMY-helper", + piRoot: "/DUMMY-pi", + spawnSyncImpl() { + return result; + }, + }), + (error) => error.code === "CP06_ISOLATION_UNSUPPORTED" && !error.message.includes(secret), + ); + } +}); From 036178f482724c212bf50cc5512e63e364512164 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 19:27:45 +0000 Subject: [PATCH 28/45] no-mistakes(review): Harden CP06 proof boundaries --- scripts/cp06-auth-security.mjs | 208 +++++++++++++++++++++++- scripts/cp06-credential-isolation.mjs | 83 +++++++--- scripts/cp06-probe-fixture.mjs | 59 ++++++- scripts/replay-cp06-p07.mjs | 19 ++- scripts/replay-cp06-ten-pack.mjs | 19 ++- scripts/replay-cp06-worker.mjs | 27 +-- test/cp06-credential-isolation.test.mjs | 62 +++---- test/cp06-proof-boundary.test.mjs | 144 +++++++++++++++- 8 files changed, 530 insertions(+), 91 deletions(-) diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index de0ba01..a52d503 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -1,6 +1,6 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { readFileSync } from "node:fs"; +import { closeSync, constants, openSync, readFileSync } from "node:fs"; import { join, resolve } from "node:path"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; import { openProbeOutput } from "./cp06-probe-fixture.mjs"; @@ -17,12 +17,13 @@ export async function runCp06SecurityPreflight(options = {}) { const root = resolve(options.root ?? process.cwd()); const outputRoot = resolve(options.outputRoot ?? join(root, ".factory/state/cp06-auth-security")); const output = openProbeOutput({ root, outputRoot, create: true }); + let binaries; try { const prerequisites = verifyIsolationPrerequisites({ root, spawnSyncImpl: options.spawnSyncImpl, }); - const binaries = compileIsolationHelpers({ + binaries = compileIsolationHelpers({ root, outputRoot, anchoredOutputRoot: output.anchor, @@ -43,7 +44,7 @@ export async function runCp06SecurityPreflight(options = {}) { }); assertHardenedProof(hardened); const sdk = runDummySdkProof({ root, helper: binaries.helper, piRoot: pi.root }); - return { + const proof = { schema_version: 1, gate: "CP-06-read-only-auth-security", result: "pass", @@ -63,7 +64,13 @@ export async function runCp06SecurityPreflight(options = {}) { hardened_child: summarizeHardened(hardened), dummy_sdk: sdk, }; + if (options.retainBinaries === true) { + Object.defineProperty(proof, "retainedBinaries", { value: binaries }); + binaries = undefined; + } + return proof; } finally { + binaries?.close?.(); output.close(); } } @@ -137,7 +144,20 @@ export function compileIsolationHelpers(options) { ], root, ); - return { helper, syscallProbe }; + const helperFd = openSync(anchoredHelper, constants.O_RDONLY | constants.O_NOFOLLOW); + const syscallProbeFd = openSync(anchoredSyscallProbe, constants.O_RDONLY | constants.O_NOFOLLOW); + let closed = false; + return { + helper: `/proc/${process.pid}/fd/${helperFd}`, + syscallProbe: `/proc/${process.pid}/fd/${syscallProbeFd}`, + outputPaths: { helper, syscallProbe }, + close() { + if (closed) return; + closed = true; + closeSync(helperFd); + closeSync(syscallProbeFd); + }, + }; } export async function runIsolatedWorker(options) { @@ -157,6 +177,21 @@ export async function runIsolatedWorker(options) { } export function namespaceWorkerError(result) { + if (result.error?.code === "ETIMEDOUT") { + const error = new Error("CP-06 isolated SDK worker timed out"); + error.code = "CP06_CHILD_TIMEOUT"; + error.stage = "timeout"; + error.exitCode = 70; + error.workerFailure = { + schema_version: 1, + stage: "timeout", + code: "CP06_CHILD_TIMEOUT", + exit_code: 70, + description: SUPERVISOR_FAILURE_DESCRIPTIONS.CP06_CHILD_TIMEOUT, + causes: [], + }; + return error; + } const error = new Error( result.status === 73 ? "CP-06 credential namespace setup is unsupported" @@ -413,7 +448,170 @@ export function runDummySdkProof({ root, helper, piRoot, spawnSyncImpl = spawnSy : "CP06_DUMMY_SDK_CHILD_EXIT"; throw unsupported(`DUMMY SDK proof failed: ${code}`); } - return parseCompactJson(result.stdout, "DUMMY SDK proof"); + return validateDummySdkProof(parseCompactJson(result.stdout, "DUMMY SDK proof")); +} + +function validateDummySdkProof(proof) { + assertExactKeys(proof, [ + "actual_sdk_outcomes", + "auth_cases", + "fixture_origin", + "network_calls", + "result", + "schema_version", + "sdk_dependency", + "sdk_worker", + "semantic_acceptance", + ]); + assert(proof.schema_version === 1, "DUMMY SDK schema mismatch"); + assert(proof.result === "pass", "DUMMY SDK proof did not pass"); + assert(proof.fixture_origin === true, "DUMMY SDK fixture origin missing"); + assert(proof.semantic_acceptance === false, "DUMMY SDK semantic acceptance was claimed"); + assert(proof.network_calls === 0, "DUMMY SDK attempted network access"); + validateAuthCases(proof.auth_cases); + validateSdkWorker(proof.sdk_worker); + validateSdkDependency(proof.sdk_dependency); + validateOutcomeProof(proof.actual_sdk_outcomes); + return proof; +} + +function validateAuthCases(cases) { + assert(Array.isArray(cases) && cases.length === 3, "DUMMY SDK auth cases incomplete"); + const expected = new Map([ + ["expired", { result: "blocked", toAuth: 0 }], + ["near_expiry", { result: "blocked", toAuth: 0 }], + ["unexpired", { result: "resolved", toAuth: 1 }], + ]); + const seen = new Set(); + for (const entry of cases) { + assertExactKeys(entry, [ + "credential_store", + "persistence_operations", + "refresh_callbacks", + "result", + "scenario", + "source_unchanged", + "to_auth_calls", + ]); + const expectation = expected.get(entry.scenario); + assert(expectation !== undefined && !seen.has(entry.scenario), "DUMMY SDK auth case mismatch"); + seen.add(entry.scenario); + assert(entry.result === expectation.result, "DUMMY SDK auth result mismatch"); + assert(entry.refresh_callbacks === 0, "DUMMY SDK refresh callback ran"); + assert(entry.to_auth_calls === expectation.toAuth, "DUMMY SDK toAuth count mismatch"); + assert(entry.persistence_operations === 0, "DUMMY SDK persistence occurred"); + assert(entry.source_unchanged === true, "DUMMY SDK auth source changed"); + validateCredentialAudit(entry.credential_store); + } +} + +function validateSdkWorker(worker) { + assertExactKeys(worker, [ + "active_tools", + "credential_store", + "event_count", + "event_sha256", + "in_memory_session", + "read_audit", + "refresh_callbacks", + ]); + assert( + Array.isArray(worker.active_tools) && worker.active_tools.join(",") === "read", + "DUMMY SDK tools mismatch", + ); + assert(worker.in_memory_session === true, "DUMMY SDK session was persisted"); + assert( + Number.isSafeInteger(worker.event_count) && worker.event_count > 0, + "DUMMY SDK event count missing", + ); + assert(isSha256(worker.event_sha256), "DUMMY SDK event digest missing"); + assert(worker.refresh_callbacks === 0, "DUMMY SDK worker refresh callback ran"); + validateCredentialAudit(worker.credential_store); + assert(worker.read_audit?.exact_original === true, "DUMMY SDK exact original read missing"); + assert(worker.read_audit?.read_count === 1, "DUMMY SDK read count mismatch"); +} + +function validateSdkDependency(dependency) { + assert( + dependency !== null && typeof dependency === "object" && !Array.isArray(dependency), + "DUMMY SDK dependency missing", + ); + assert( + typeof dependency.version === "string" && dependency.version.length > 0, + "DUMMY SDK dependency version missing", + ); + assert(isSha256(dependency.package_sha256), "DUMMY SDK dependency digest missing"); +} + +function validateOutcomeProof(outcomes) { + assertExactKeys(outcomes, ["cases", "fixture_origin", "semantic_acceptance"]); + assert(outcomes.fixture_origin === true, "DUMMY outcome fixture origin missing"); + assert(outcomes.semantic_acceptance === false, "DUMMY outcome semantic acceptance was claimed"); + assert(Array.isArray(outcomes.cases), "DUMMY outcome cases missing"); + const expected = new Map([ + ["valid", 0], + ["fenced-valid", 0], + ["extra-before-fence", 75], + ["empty-gap", 75], + ["missing-gap", 75], + ["contradictory-gap", 75], + ["extra-evidence", 75], + ["constraint-pass", 75], + ["missing-reads", 75], + ["credential-read", 75], + ["symlink-read", 75], + ["expired", 75], + ["near-expiry", 75], + ["timeout", 70], + ]); + const seen = new Set(); + for (const entry of outcomes.cases) { + assert( + entry !== null && typeof entry === "object" && !Array.isArray(entry), + "DUMMY outcome case malformed", + ); + const expectedExit = expected.get(entry.scenario); + assert( + expectedExit !== undefined && !seen.has(entry.scenario), + "DUMMY outcome scenario mismatch", + ); + seen.add(entry.scenario); + assert( + entry.exit_code === expectedExit && entry.expected_exit_code === expectedExit, + "DUMMY outcome exit mismatch", + ); + assert(entry.source_unchanged === true, "DUMMY outcome source changed"); + assert( + entry.default_storage === 0 && entry.network === 0 && entry.refresh === 0, + "DUMMY outcome side effect occurred", + ); + if (["expired", "near-expiry"].includes(entry.scenario)) { + assert(entry.runtime_create === 0, "DUMMY expired auth reached runtime creation"); + } + } + assert(seen.size === expected.size, "DUMMY outcome cases incomplete"); +} + +function validateCredentialAudit(audit) { + assertExactKeys(audit, ["delete_denials", "lists", "modify_denials", "reads"]); + for (const key of ["delete_denials", "lists", "modify_denials", "reads"]) { + assert(Number.isSafeInteger(audit[key]) && audit[key] >= 0, "DUMMY credential audit malformed"); + } +} + +function assertExactKeys(value, keys) { + assert( + value !== null && typeof value === "object" && !Array.isArray(value), + "DUMMY SDK proof object malformed", + ); + assert( + Object.keys(value).sort().join(",") === keys.slice().sort().join(","), + "DUMMY SDK proof keys mismatch", + ); +} + +function isSha256(value) { + return typeof value === "string" && /^[0-9a-f]{64}$/u.test(value); } function resolvePiPackage(root, options) { diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs index 2c92e30..dc122b9 100644 --- a/scripts/cp06-credential-isolation.mjs +++ b/scripts/cp06-credential-isolation.mjs @@ -15,7 +15,7 @@ import { writeFile, } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { dirname, join, resolve } from "node:path"; +import { basename, dirname, join, resolve } from "node:path"; const MAX_CREDENTIAL_BYTES = 1024 * 1024; const REQUIRED_VFS_OPTIONS = ["ro", "nosuid", "nodev", "noexec"]; @@ -50,7 +50,7 @@ export async function proveDummyCredentialIsolation(options = {}) { mount_ids: mount.mountIds, }; } finally { - mount?.cleanup(); + await mount?.cleanup(); await rm(proofRoot, { recursive: true, force: true }); } } @@ -69,20 +69,40 @@ export async function mountLiveCredentialReadOnly(options) { const statPath = options.lstatImpl ?? lstat; const protectParents = options.protectParents !== false; let handle; + let sourceParentHandle; + let targetParentHandle; let sourceMounted = false; let targetMounted = false; + let sourceMountPath = source; + let targetMountPath = target; const mountedParents = []; + let parentHandlesTransferred = false; try { - handle = await open(source, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC); + sourceParentHandle = await open( + dirname(source), + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); + const sourceParentPath = `/proc/${process.pid}/fd/${sourceParentHandle.fd}`; + sourceMountPath = join(sourceParentPath, basename(source)); + handle = await open( + sourceMountPath, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); const pinned = await handle.stat({ bigint: true }); - const named = await statPath(source, { bigint: true }); + const named = await statPath(sourceMountPath, { bigint: true }); assertPinnedRegularFile(pinned, named, maxBytes); await mkdir(dirname(target), { recursive: true, mode: 0o700 }); - await writeFile(target, "", { flag: "wx", mode: 0o600 }); + targetParentHandle = await open( + dirname(target), + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); + const targetParentPath = `/proc/${process.pid}/fd/${targetParentHandle.fd}`; + targetMountPath = join(targetParentPath, basename(target)); + await writeFile(targetMountPath, "", { flag: "wx", mode: 0o600 }); const parentMounts = []; if (protectParents) { - for (const parent of [...new Set([dirname(source), dirname(target)])]) { + for (const parent of [...new Set([sourceParentPath, targetParentPath])]) { checkedSpawn("mount", ["--bind", parent, parent], { root, spawn }); mountedParents.push(parent); checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", parent], { @@ -94,25 +114,25 @@ export async function mountLiveCredentialReadOnly(options) { } const pinnedPath = `/proc/${process.pid}/fd/${handle.fd}`; - checkedSpawn("mount", ["--bind", pinnedPath, source], { root, spawn }); + checkedSpawn("mount", ["--bind", pinnedPath, sourceMountPath], { root, spawn }); sourceMounted = true; - checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", source], { + checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", sourceMountPath], { root, spawn, }); - checkedSpawn("mount", ["--bind", pinnedPath, target], { root, spawn }); + checkedSpawn("mount", ["--bind", pinnedPath, targetMountPath], { root, spawn }); targetMounted = true; - checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", target], { + checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", targetMountPath], { root, spawn, }); - const sourceMount = inspectReadOnlyMount(source, { spawn }); - const targetMount = inspectReadOnlyMount(target, { spawn }); + const sourceMount = inspectReadOnlyMount(sourceMountPath, { spawn }); + const targetMount = inspectReadOnlyMount(targetMountPath, { spawn }); if (sourceMount.id === targetMount.id) { throw new Error("credential source and target did not receive distinct mount instances"); } - for (const path of [source, target]) { + for (const path of [sourceMountPath, targetMountPath]) { const mounted = await statPath(path, { bigint: true }); if (!mounted.isFile() || !sameStatIdentity(pinned, mounted)) { throw new Error(`credential mount identity changed: ${path}`); @@ -122,6 +142,7 @@ export async function mountLiveCredentialReadOnly(options) { handle = undefined; let cleaned = false; + parentHandlesTransferred = true; return { source, target, @@ -129,12 +150,12 @@ export async function mountLiveCredentialReadOnly(options) { mountIds: { source: sourceMount.id, target: targetMount.id }, parentMounts: parentMounts.map((entry) => ({ id: entry.id, target: entry.target })), async verifyIntegrity() { - for (const path of [source, target]) { + for (const path of [sourceMountPath, targetMountPath]) { const mounted = await statPath(path, { bigint: true }); if (!mounted.isFile() || !sameStatIdentity(pinned, mounted)) return false; } - const currentSourceMount = inspectReadOnlyMount(source, { spawn }); - const currentTargetMount = inspectReadOnlyMount(target, { spawn }); + const currentSourceMount = inspectReadOnlyMount(sourceMountPath, { spawn }); + const currentTargetMount = inspectReadOnlyMount(targetMountPath, { spawn }); if (currentSourceMount.id !== sourceMount.id || currentTargetMount.id !== targetMount.id) { return false; } @@ -144,15 +165,29 @@ export async function mountLiveCredentialReadOnly(options) { } return true; }, - cleanup() { + async cleanup() { if (cleaned) return; cleaned = true; const failures = []; - if (targetMounted) unmount(target, { root, spawn, failures }); - if (sourceMounted) unmount(source, { root, spawn, failures }); + if (targetMounted) unmount(targetMountPath, { root, spawn, failures }); + if (sourceMounted) unmount(sourceMountPath, { root, spawn, failures }); for (const parent of [...mountedParents].reverse()) { unmount(parent, { root, spawn, failures }); } + const closeFailures = []; + try { + await sourceParentHandle?.close(); + } catch (error) { + closeFailures.push(error?.message ?? "source parent close failed"); + } + try { + await targetParentHandle?.close(); + } catch (error) { + closeFailures.push(error?.message ?? "target parent close failed"); + } + sourceParentHandle = undefined; + targetParentHandle = undefined; + failures.push(...closeFailures); if (failures.length > 0) { throw new Error(`credential mount cleanup failed: ${failures.join("; ")}`); } @@ -160,8 +195,8 @@ export async function mountLiveCredentialReadOnly(options) { }; } catch (error) { const cleanupFailures = []; - if (targetMounted) unmount(target, { root, spawn, failures: cleanupFailures }); - if (sourceMounted) unmount(source, { root, spawn, failures: cleanupFailures }); + if (targetMounted) unmount(targetMountPath, { root, spawn, failures: cleanupFailures }); + if (sourceMounted) unmount(sourceMountPath, { root, spawn, failures: cleanupFailures }); for (const parent of [...mountedParents].reverse()) { unmount(parent, { root, spawn, failures: cleanupFailures }); } @@ -174,6 +209,10 @@ export async function mountLiveCredentialReadOnly(options) { throw error; } finally { await handle?.close(); + if (!parentHandlesTransferred) { + await sourceParentHandle?.close(); + await targetParentHandle?.close(); + } } } @@ -231,7 +270,7 @@ export function inspectReadOnlyMount(path, options = {}) { const optionsList = String(entry?.["vfs-options"] ?? "").split(","); if ( entry === undefined || - resolve(entry.target) !== realpathSync(path) || + ![resolve(path), realpathSync(path)].includes(resolve(entry.target)) || !REQUIRED_VFS_OPTIONS.every((option) => optionsList.includes(option)) ) { throw new Error(`credential mount lacks required read-only VFS flags: ${path}`); diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index b1dd763..3f46167 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -1,6 +1,15 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { closeSync, constants, fstatSync, lstatSync, mkdirSync, openSync, readSync } from "node:fs"; +import { + closeSync, + constants, + fstatSync, + lstatSync, + mkdirSync, + openSync, + readSync, + rmSync, +} from "node:fs"; import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { join, relative, resolve } from "node:path"; @@ -73,6 +82,54 @@ export function openProbeOutput({ root, outputRoot, create = false, runner = fal } } +export function openAnchoredDirectory( + parentAnchor, + relativePath, + { create = true, reset = false } = {}, +) { + const components = String(relativePath) + .split("/") + .filter((component) => component.length > 0); + if ( + components.length === 0 || + components.some( + (component) => component === "." || component === ".." || component.includes("\0"), + ) + ) { + throw new Error("DUMMY proof output child path is invalid"); + } + if (reset) rmSync(join(parentAnchor, components[0]), { recursive: true, force: true }); + let currentFd; + let currentAnchor = parentAnchor; + try { + for (const component of components) { + const path = join(currentAnchor, component); + if (create) { + try { + mkdirSync(path, { mode: 0o700 }); + } catch (error) { + if (error?.code !== "EEXIST") throw error; + } + } + const nextFd = openSync(path, directoryFlags); + if (currentFd !== undefined) closeSync(currentFd); + currentFd = nextFd; + currentAnchor = `/proc/${process.pid}/fd/${currentFd}`; + } + const outputFd = currentFd; + currentFd = undefined; + return { + anchor: `/proc/${process.pid}/fd/${outputFd}`, + identity: fstatSync(outputFd, { bigint: true }), + close() { + closeSync(outputFd); + }, + }; + } finally { + if (currentFd !== undefined) closeSync(currentFd); + } +} + export async function createProbeFixture({ root, outputRoot, spawn = spawnSync }) { const output = openProbeOutput({ root, outputRoot }); let rootFd; diff --git a/scripts/replay-cp06-p07.mjs b/scripts/replay-cp06-p07.mjs index 5f5bcfa..fa1fecc 100644 --- a/scripts/replay-cp06-p07.mjs +++ b/scripts/replay-cp06-p07.mjs @@ -1,18 +1,18 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; -import { writeFileSync } from "node:fs"; +import { realpathSync, writeFileSync } from "node:fs"; import { mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -import { openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); -const rawRoot = join(output.anchor, "p07", "raw"); -await rm(join(output.anchor, "p07"), { recursive: true, force: true }); -await mkdir(rawRoot, { recursive: true }); +const p07Root = openAnchoredDirectory(output.anchor, "p07", { reset: true }); +const rawDirectory = openAnchoredDirectory(p07Root.anchor, "raw"); +const rawRoot = rawDirectory.anchor; const steps = []; steps.push(run("build", "npm", ["run", "build"], 0)); @@ -67,7 +67,7 @@ const manifest = { "The historical module is a minimal retained reproduction, not a raw worker transcript.", ], }; -const manifestPath = join(output.anchor, "p07", "evidence.json"); +const manifestPath = join(p07Root.anchor, "evidence.json"); await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); console.log( JSON.stringify({ @@ -123,9 +123,10 @@ function quote(parts) { } function relative(path) { - const located = path.startsWith(`${output.anchor}/`) - ? join(outputRoot, path.slice(output.anchor.length + 1)) - : path; + let located = path; + try { + located = realpathSync(path); + } catch {} return located.startsWith(`${root}/`) ? located.slice(root.length + 1) : located; } diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index ee7a1d8..d50b225 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -1,27 +1,27 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; -import { existsSync, readFileSync, writeFileSync } from "node:fs"; +import { existsSync, readFileSync, realpathSync, writeFileSync } from "node:fs"; import { cp, mkdir, readFile, rm, stat, writeFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -import { openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const fixtureSource = join(root, "test/fixtures/cp06-context"); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); -const proofRoot = join(output.anchor, "ten-pack"); +const proofDirectory = openAnchoredDirectory(output.anchor, "ten-pack", { reset: true }); +const rawDirectory = openAnchoredDirectory(proofDirectory.anchor, "raw"); +const proofRoot = proofDirectory.anchor; const fixtureRoot = join(proofRoot, "repo"); -const rawRoot = join(proofRoot, "raw"); +const rawRoot = rawDirectory.anchor; const cli = join(root, "dist/src/cli.js"); const ctx = resolveTool(process.env.CTX_BIN ?? "ctx"); configureOfflineCtxModelDir(); const reviewer = process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker"; const commands = []; -await rm(proofRoot, { recursive: true, force: true }); -await mkdir(rawRoot, { recursive: true }); await cp(fixtureSource, fixtureRoot, { recursive: true }); await mkdir(join(fixtureRoot, ".factory/state"), { recursive: true }); const oracle = JSON.parse(await readFile(join(fixtureRoot, "oracle.json"), "utf8")); @@ -680,9 +680,10 @@ function quote(parts) { } function relative(path) { - const located = path.startsWith(`${output.anchor}/`) - ? join(outputRoot, path.slice(output.anchor.length + 1)) - : path; + let located = path; + try { + located = realpathSync(path); + } catch {} return located.startsWith(`${root}/`) ? located.slice(root.length + 1) : located; } diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index dde70a9..81126c2 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -1,12 +1,13 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; +import { realpathSync } from "node:fs"; import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; -import { openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; import { Cp06IsolationUnsupportedError, runCp06SecurityPreflight, @@ -16,7 +17,8 @@ import { const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); -const workerRoot = join(output.anchor, "worker"); +const workerDirectory = openAnchoredDirectory(output.anchor, "worker", { reset: true }); +const workerRoot = workerDirectory.anchor; const securityRoot = join(outputRoot, "auth-security"); const fixtureRoot = join(outputRoot, "ten-pack", "repo"); const initialPackPath = join(outputRoot, "ten-pack", "raw", "auth-worker-missing.pack.md"); @@ -38,8 +40,6 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur } async function main() { - await rm(workerRoot, { recursive: true, force: true }); - await mkdir(workerRoot, { recursive: true }); let security; try { // This DUMMY-only gate includes the unsafe control, hardened evaluated-child @@ -49,6 +49,7 @@ async function main() { root, outputRoot: securityRoot, reviewer: process.env.CP06_REVIEWER, + retainBinaries: true, }); } catch (error) { await recordBlocked(error, "dummy-security-preflight"); @@ -67,18 +68,15 @@ async function main() { timeout_ms: timeoutMs, system_prompt: systemPrompt, }; - const inputPath = join(outputRoot, "worker", "input.json"); - await writeFile(join(workerRoot, "input.json"), `${JSON.stringify(workerInput, null, 2)}\n`); + const inputPath = join(workerRoot, "input.json"); + await writeFile(inputPath, `${JSON.stringify(workerInput, null, 2)}\n`); const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-sdk-worker-")); const credentialTarget = join(evaluationHome, "pi-agent", "auth.json"); const credentialSource = resolve( process.env.CP06_PI_AUTH_FILE ?? join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "auth.json"), ); - const binaries = { - helper: join(securityRoot, "cp06-seccomp-exec"), - syscallProbe: join(securityRoot, "cp06-isolation-syscalls"), - }; + const binaries = security.retainedBinaries; let isolated; try { isolated = await withCleanup( @@ -96,6 +94,8 @@ async function main() { } catch (error) { await recordBlocked(error, "isolated-sdk-worker"); throw error; + } finally { + binaries?.close?.(); } if ( isolated.mode !== "worker" || @@ -222,9 +222,10 @@ function textCommand(command, args) { } function relative(path) { - const located = path.startsWith(`${output.anchor}/`) - ? join(outputRoot, path.slice(output.anchor.length + 1)) - : path; + let located = path; + try { + located = realpathSync(path); + } catch {} return located.startsWith(`${root}/`) ? located.slice(root.length + 1) : located; } diff --git a/test/cp06-credential-isolation.test.mjs b/test/cp06-credential-isolation.test.mjs index 4918081..adeed26 100644 --- a/test/cp06-credential-isolation.test.mjs +++ b/test/cp06-credential-isolation.test.mjs @@ -39,7 +39,8 @@ test("live credential mount pins one inode behind distinct read-only regular mou } return { status: 0, stdout: "", stderr: "" }; }; - const lstatImpl = async (path, options) => (path === target ? sourceStat : lstat(path, options)); + const lstatImpl = async (path, options) => + String(path).endsWith("/auth.json") ? sourceStat : lstat(path, options); try { const mount = await mountLiveCredentialReadOnly({ root, @@ -52,48 +53,51 @@ test("live credential mount pins one inode behind distinct read-only regular mou assert.equal(await readFile(source, "utf8"), sourceBytes); assert.notEqual(mount.mountIds.source, mount.mountIds.target); const mountCallsOnly = calls.filter(([command]) => command === "mount"); - assert.deepEqual( - mountCallsOnly.slice(0, 4), - [ - ["mount", "--bind", root, root], - ["mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", root], - ["mount", "--bind", join(root, "agent"), join(root, "agent")], - [ - "mount", - "-o", - "remount,bind,ro,nosuid,nodev,noexec", - join(root, "agent"), - ], - ], - ); + assert.match(mountCallsOnly[0][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); + assert.equal(mountCallsOnly[0][2], mountCallsOnly[0][3]); + assert.deepEqual(mountCallsOnly[1].slice(0, 3), [ + "mount", + "-o", + "remount,bind,ro,nosuid,nodev,noexec", + ]); + assert.match(mountCallsOnly[2][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); + assert.equal(mountCallsOnly[2][2], mountCallsOnly[2][3]); + assert.deepEqual(mountCallsOnly[3].slice(0, 3), [ + "mount", + "-o", + "remount,bind,ro,nosuid,nodev,noexec", + ]); assert.equal(mount.parentMounts.length, 2); assert.equal(await mount.verifyIntegrity(), true); assert.match(mountCallsOnly[4][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.deepEqual(mountCallsOnly[4].slice(0, 2), ["mount", "--bind"]); - assert.equal(mountCallsOnly[4][3], source); - assert.deepEqual(mountCallsOnly[5], [ + assert.match( + mountCallsOnly[4][3], + new RegExp(`^/proc/${process.pid}/fd/\\d+/auth-source\\.json$`), + ); + assert.deepEqual(mountCallsOnly[5].slice(0, 3), [ "mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", - source, ]); + assert.match( + mountCallsOnly[5][3], + new RegExp(`^/proc/${process.pid}/fd/\\d+/auth-source\\.json$`), + ); assert.match(mountCallsOnly[6][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.equal(mountCallsOnly[6][3], target); - assert.deepEqual(mountCallsOnly[7], [ + assert.match(mountCallsOnly[6][3], new RegExp(`^/proc/${process.pid}/fd/\\d+/auth\\.json$`)); + assert.deepEqual(mountCallsOnly[7].slice(0, 3), [ "mount", "-o", "remount,bind,ro,nosuid,nodev,noexec", - target, ]); + assert.match(mountCallsOnly[7][3], new RegExp(`^/proc/${process.pid}/fd/\\d+/auth\\.json$`)); } finally { - mount.cleanup(); + await mount.cleanup(); + } + for (const [command, path] of calls.slice(-4)) { + assert.equal(command, "umount"); + assert.match(path, new RegExp(`^/proc/${process.pid}/fd/\\d+`)); } - assert.deepEqual(calls.slice(-4), [ - ["umount", target], - ["umount", source], - ["umount", join(root, "agent")], - ["umount", root], - ]); } finally { await rm(root, { recursive: true, force: true }); } diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 4452b21..cfb8884 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -1,10 +1,20 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { mkdir, mkdtemp, readFile, readdir, rm, symlink, writeFile } from "node:fs/promises"; +import { + mkdir, + mkdtemp, + readFile, + readdir, + rename, + rm, + symlink, + writeFile, +} from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import test from "node:test"; import { runDummySdkProof, runIsolationProbe } from "../scripts/cp06-auth-security.mjs"; +import { openAnchoredDirectory, openProbeOutput } from "../scripts/cp06-probe-fixture.mjs"; const runners = [ "scripts/replay-cp06-worker.mjs", @@ -86,9 +96,12 @@ test("failed DUMMY probes never forward child output or spawn exceptions", async ), false, ); - if (result.stderr === JSON.stringify(structured)) + if (result.stderr === JSON.stringify(structured)) { assert.deepEqual(error.workerFailure, structured); - else assert.equal(error.workerFailure, undefined); + } else if (result.error?.code === "ETIMEDOUT") { + assert.equal(error.workerFailure.code, "CP06_CHILD_TIMEOUT"); + assert.equal(error.workerFailure.stage, "timeout"); + } else assert.equal(error.workerFailure, undefined); return true; }, ); @@ -115,3 +128,128 @@ test("failed DUMMY probes never forward child output or spawn exceptions", async ); } }); + +test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { + const validBase = { + schema_version: 1, + result: "pass", + auth_cases: [ + authCase("expired", "blocked", 0), + authCase("near_expiry", "blocked", 0), + authCase("unexpired", "resolved", 1), + ], + sdk_worker: { + active_tools: ["read"], + in_memory_session: true, + event_count: 2, + event_sha256: "0".repeat(64), + read_audit: { exact_original: true, read_count: 1 }, + refresh_callbacks: 0, + credential_store: credentialAudit(), + }, + sdk_dependency: { version: "0.85.1", package_sha256: "1".repeat(64) }, + actual_sdk_outcomes: { + fixture_origin: true, + semantic_acceptance: false, + cases: [ + outcome("valid", 0), + outcome("fenced-valid", 0), + outcome("extra-before-fence", 75), + outcome("empty-gap", 75), + outcome("missing-gap", 75), + outcome("contradictory-gap", 75), + outcome("extra-evidence", 75), + outcome("constraint-pass", 75), + outcome("missing-reads", 75), + outcome("credential-read", 75), + outcome("symlink-read", 75), + outcome("expired", 75, { runtime_create: 0 }), + outcome("near-expiry", 75, { runtime_create: 0 }), + outcome("timeout", 70), + ], + }, + fixture_origin: true, + semantic_acceptance: false, + network_calls: 0, + }; + for (const payload of [ + { ...validBase, auth_cases: [] }, + { ...validBase, auth_cases: [...validBase.auth_cases, authCase("expired", "blocked", 0)] }, + { ...validBase, extra: "DUMMY-SECRET" }, + { ...validBase, sdk_worker: { ...validBase.sdk_worker, active_tools: ["read", "write"] } }, + ]) { + assert.throws( + () => + runDummySdkProof({ + root: process.cwd(), + helper: "/DUMMY-helper", + piRoot: "/DUMMY-pi", + spawnSyncImpl() { + return { status: 0, stdout: JSON.stringify(payload), stderr: "" }; + }, + }), + (error) => !String(error.message).includes("DUMMY-SECRET"), + ); + } + const accepted = runDummySdkProof({ + root: process.cwd(), + helper: "/DUMMY-helper", + piRoot: "/DUMMY-pi", + spawnSyncImpl() { + return { status: 0, stdout: JSON.stringify(validBase), stderr: "" }; + }, + }); + assert.equal(accepted.result, "pass"); +}); + +test("anchored runner child directories do not follow substituted paths", async () => { + const root = process.cwd(); + const outputRoot = resolve(".factory/state/cp06-correction/DUMMY-child-anchor"); + const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-child-external-")); + const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); + const child = openAnchoredDirectory(output.anchor, "worker", { reset: true }); + try { + await rename(join(outputRoot, "worker"), join(outputRoot, "worker-original")); + await symlink(external, join(outputRoot, "worker")); + await writeFile(join(child.anchor, "input.json"), "DUMMY anchored write"); + assert.deepEqual(await readdir(external), []); + assert.equal( + await readFile(join(outputRoot, "worker-original", "input.json"), "utf8"), + "DUMMY anchored write", + ); + } finally { + child.close(); + output.close(); + await rm(outputRoot, { recursive: true, force: true }); + await rm(external, { recursive: true, force: true }); + } +}); + +function authCase(scenario, result, toAuth) { + return { + scenario, + result, + refresh_callbacks: 0, + to_auth_calls: toAuth, + credential_store: credentialAudit(), + persistence_operations: 0, + source_unchanged: true, + }; +} + +function credentialAudit() { + return { reads: 1, lists: 0, modify_denials: 1, delete_denials: 1 }; +} + +function outcome(scenario, exitCode, extra = {}) { + return { + scenario, + exit_code: exitCode, + expected_exit_code: exitCode, + default_storage: 0, + network: 0, + refresh: 0, + source_unchanged: true, + ...extra, + }; +} From 9afae5c97d4f46a1378f2814d94d4edf354138e6 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Fri, 2 Oct 2026 19:38:47 +0000 Subject: [PATCH 29/45] no-mistakes(review): Harden CP06 identity handoffs --- scripts/cp06-auth-security.mjs | 184 ++++++++++++++++-------- scripts/cp06-credential-isolation.mjs | 46 +++--- scripts/cp06-namespace-supervisor.mjs | 4 + scripts/cp06-probe-fixture.mjs | 8 +- test/cp06-credential-isolation.test.mjs | 83 ++++++++++- test/cp06-proof-boundary.test.mjs | 74 +++++++++- 6 files changed, 316 insertions(+), 83 deletions(-) diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index a52d503..b390159 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -1,6 +1,6 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { closeSync, constants, openSync, readFileSync } from "node:fs"; +import { closeSync, constants, mkdtempSync, openSync, readFileSync } from "node:fs"; import { join, resolve } from "node:path"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; import { openProbeOutput } from "./cp06-probe-fixture.mjs"; @@ -108,56 +108,70 @@ export function verifyIsolationPrerequisites(options = {}) { export function compileIsolationHelpers(options) { const root = resolve(options.root); const outputRoot = resolve(options.outputRoot); - const helper = join(outputRoot, "cp06-seccomp-exec"); - const syscallProbe = join(outputRoot, "cp06-isolation-syscalls"); - const anchoredHelper = join(options.anchoredOutputRoot ?? outputRoot, "cp06-seccomp-exec"); - const anchoredSyscallProbe = join( - options.anchoredOutputRoot ?? outputRoot, - "cp06-isolation-syscalls", + const artifactRoot = mkdtempSync(join(options.anchoredOutputRoot ?? outputRoot, "helpers-")); + const artifactFd = openSync( + artifactRoot, + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW, ); - checked( - "cc", - [ - "-std=c11", - "-O2", - "-Wall", - "-Wextra", - "-Werror", - join(root, "scripts/cp06-seccomp-exec.c"), - "-Wl,-l:libseccomp.so.2", - "-o", - anchoredHelper, - ], - root, - ); - checked( - "cc", - [ - "-std=c11", - "-O2", - "-Wall", - "-Wextra", - "-Werror", - join(root, "scripts/cp06-isolation-syscalls.c"), - "-o", - anchoredSyscallProbe, - ], - root, - ); - const helperFd = openSync(anchoredHelper, constants.O_RDONLY | constants.O_NOFOLLOW); - const syscallProbeFd = openSync(anchoredSyscallProbe, constants.O_RDONLY | constants.O_NOFOLLOW); - let closed = false; - return { - helper: `/proc/${process.pid}/fd/${helperFd}`, - syscallProbe: `/proc/${process.pid}/fd/${syscallProbeFd}`, - outputPaths: { helper, syscallProbe }, - close() { - if (closed) return; - closed = true; - closeSync(helperFd); - closeSync(syscallProbeFd); - }, - }; + const artifactAnchor = `/proc/${process.pid}/fd/${artifactFd}`; + const helper = join(artifactAnchor, "cp06-seccomp-exec"); + const syscallProbe = join(artifactAnchor, "cp06-isolation-syscalls"); + const spawn = options.spawnSyncImpl ?? spawnSync; + let helperFd; + let syscallProbeFd; + try { + checked( + "cc", + [ + "-std=c11", + "-O2", + "-Wall", + "-Wextra", + "-Werror", + join(root, "scripts/cp06-seccomp-exec.c"), + "-Wl,-l:libseccomp.so.2", + "-o", + helper, + ], + root, + spawn, + ); + checked( + "cc", + [ + "-std=c11", + "-O2", + "-Wall", + "-Wextra", + "-Werror", + join(root, "scripts/cp06-isolation-syscalls.c"), + "-o", + syscallProbe, + ], + root, + spawn, + ); + helperFd = openSync(helper, constants.O_RDONLY | constants.O_NOFOLLOW); + syscallProbeFd = openSync(syscallProbe, constants.O_RDONLY | constants.O_NOFOLLOW); + let closed = false; + closeSync(artifactFd); + return { + helper: `/proc/${process.pid}/fd/${helperFd}`, + syscallProbe: `/proc/${process.pid}/fd/${syscallProbeFd}`, + outputPaths: { helper, syscallProbe }, + close() { + if (closed) return; + closed = true; + closeSync(helperFd); + closeSync(syscallProbeFd); + }, + }; + } catch (error) { + if (helperFd !== undefined) closeSync(helperFd); + if (syscallProbeFd !== undefined) closeSync(syscallProbeFd); + closeSync(artifactFd); + throw error; + } } export async function runIsolatedWorker(options) { @@ -532,15 +546,32 @@ function validateSdkWorker(worker) { } function validateSdkDependency(dependency) { + assertExactKeys(dependency, [ + "entry", + "entry_sha256", + "manifest_sha256", + "name", + "package_sha256", + "resolution", + "root", + "tarball_integrity", + "version", + ]); + assert(dependency.name === "@earendil-works/pi-ai", "DUMMY SDK dependency name mismatch"); + assert(dependency.version === "0.85.1", "DUMMY SDK dependency version mismatch"); assert( - dependency !== null && typeof dependency === "object" && !Array.isArray(dependency), - "DUMMY SDK dependency missing", - ); - assert( - typeof dependency.version === "string" && dependency.version.length > 0, - "DUMMY SDK dependency version missing", + dependency.resolution === "esm-import-condition", + "DUMMY SDK dependency resolution mismatch", ); - assert(isSha256(dependency.package_sha256), "DUMMY SDK dependency digest missing"); + for (const key of ["entry", "root", "tarball_integrity"]) { + assert( + typeof dependency[key] === "string" && dependency[key].length > 0, + "DUMMY SDK dependency path missing", + ); + } + for (const key of ["entry_sha256", "manifest_sha256", "package_sha256"]) { + assert(isSha256(dependency[key]), "DUMMY SDK dependency digest missing"); + } } function validateOutcomeProof(outcomes) { @@ -575,18 +606,49 @@ function validateOutcomeProof(outcomes) { expectedExit !== undefined && !seen.has(entry.scenario), "DUMMY outcome scenario mismatch", ); + const expectedKeys = [ + "default_storage", + "exit", + "expected_exit_code", + "exit_code", + "fixture_origin", + "network", + "refresh", + "runtime_create", + "scenario", + "sdk_dependency", + "source_unchanged", + "to_auth", + ]; + if (["valid", "fenced-valid"].includes(entry.scenario)) expectedKeys.push("read_audit"); + assertExactKeys(entry, expectedKeys); seen.add(entry.scenario); assert( - entry.exit_code === expectedExit && entry.expected_exit_code === expectedExit, + entry.exit_code === expectedExit && + entry.expected_exit_code === expectedExit && + entry.exit === expectedExit, "DUMMY outcome exit mismatch", ); + assert(entry.fixture_origin === true, "DUMMY outcome fixture origin missing"); + validateSdkDependency(entry.sdk_dependency); assert(entry.source_unchanged === true, "DUMMY outcome source changed"); assert( entry.default_storage === 0 && entry.network === 0 && entry.refresh === 0, "DUMMY outcome side effect occurred", ); if (["expired", "near-expiry"].includes(entry.scenario)) { - assert(entry.runtime_create === 0, "DUMMY expired auth reached runtime creation"); + assert( + entry.runtime_create === 0 && entry.to_auth === 0, + "DUMMY expired auth reached runtime creation", + ); + } + if (["valid", "fenced-valid"].includes(entry.scenario)) { + assert( + entry.runtime_create === 1 && entry.to_auth === 1, + "DUMMY valid auth did not run exactly once", + ); + assert(entry.read_audit?.exact_original === true, "DUMMY outcome exact read missing"); + assert(entry.read_audit?.read_count === 1, "DUMMY outcome read count mismatch"); } } assert(seen.size === expected.size, "DUMMY outcome cases incomplete"); @@ -628,8 +690,8 @@ function resolvePiPackage(root, options) { } } -function checked(command, args, cwd) { - const result = spawnSync(command, args, { cwd, encoding: "utf8", timeout: 60_000 }); +function checked(command, args, cwd, spawn = spawnSync) { + const result = spawn(command, args, { cwd, encoding: "utf8", timeout: 60_000 }); if (result.error !== undefined || result.status !== 0) { throw unsupported(`required command failed: ${command}`); } diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs index dc122b9..e09dcee 100644 --- a/scripts/cp06-credential-isolation.mjs +++ b/scripts/cp06-credential-isolation.mjs @@ -71,6 +71,8 @@ export async function mountLiveCredentialReadOnly(options) { let handle; let sourceParentHandle; let targetParentHandle; + let closeSourceParentHandle = true; + let closeTargetParentHandle = true; let sourceMounted = false; let targetMounted = false; let sourceMountPath = source; @@ -78,12 +80,17 @@ export async function mountLiveCredentialReadOnly(options) { const mountedParents = []; let parentHandlesTransferred = false; try { - sourceParentHandle = await open( - dirname(source), - constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, - ); - const sourceParentPath = `/proc/${process.pid}/fd/${sourceParentHandle.fd}`; - sourceMountPath = join(sourceParentPath, basename(source)); + let sourceParentPath = options.sourceParentAnchor; + if (sourceParentPath === undefined) { + sourceParentHandle = await open( + dirname(source), + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); + sourceParentPath = `/proc/${process.pid}/fd/${sourceParentHandle.fd}`; + } else { + closeSourceParentHandle = false; + } + sourceMountPath = join(sourceParentPath, options.sourceLeaf ?? basename(source)); handle = await open( sourceMountPath, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, @@ -92,13 +99,18 @@ export async function mountLiveCredentialReadOnly(options) { const named = await statPath(sourceMountPath, { bigint: true }); assertPinnedRegularFile(pinned, named, maxBytes); - await mkdir(dirname(target), { recursive: true, mode: 0o700 }); - targetParentHandle = await open( - dirname(target), - constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, - ); - const targetParentPath = `/proc/${process.pid}/fd/${targetParentHandle.fd}`; - targetMountPath = join(targetParentPath, basename(target)); + let targetParentPath = options.targetParentAnchor; + if (targetParentPath === undefined) { + await mkdir(dirname(target), { recursive: true, mode: 0o700 }); + targetParentHandle = await open( + dirname(target), + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); + targetParentPath = `/proc/${process.pid}/fd/${targetParentHandle.fd}`; + } else { + closeTargetParentHandle = false; + } + targetMountPath = join(targetParentPath, options.targetLeaf ?? basename(target)); await writeFile(targetMountPath, "", { flag: "wx", mode: 0o600 }); const parentMounts = []; if (protectParents) { @@ -176,12 +188,12 @@ export async function mountLiveCredentialReadOnly(options) { } const closeFailures = []; try { - await sourceParentHandle?.close(); + if (closeSourceParentHandle) await sourceParentHandle?.close(); } catch (error) { closeFailures.push(error?.message ?? "source parent close failed"); } try { - await targetParentHandle?.close(); + if (closeTargetParentHandle) await targetParentHandle?.close(); } catch (error) { closeFailures.push(error?.message ?? "target parent close failed"); } @@ -210,8 +222,8 @@ export async function mountLiveCredentialReadOnly(options) { } finally { await handle?.close(); if (!parentHandlesTransferred) { - await sourceParentHandle?.close(); - await targetParentHandle?.close(); + if (closeSourceParentHandle) await sourceParentHandle?.close(); + if (closeTargetParentHandle) await targetParentHandle?.close(); } } } diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index b031a9f..2e5c475 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -60,6 +60,10 @@ export async function superviseCredentialChild(options, dependencies = {}) { source, target, protectParents: mode !== "unsafe-probe", + sourceParentAnchor: probe?.credentialAnchors?.sourceParent, + targetParentAnchor: probe?.credentialAnchors?.targetParent, + sourceLeaf: probe?.credentialAnchors?.sourceLeaf, + targetLeaf: probe?.credentialAnchors?.targetLeaf, }); } catch (cause) { const missingSource = hasErrorCode(cause, "ENOENT"); diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index 3f46167..6df932e 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -131,7 +131,7 @@ export function openAnchoredDirectory( } export async function createProbeFixture({ root, outputRoot, spawn = spawnSync }) { - const output = openProbeOutput({ root, outputRoot }); + const output = openProbeOutput({ root, outputRoot, create: true }); let rootFd; let sourceDirFd; let agentDirFd; @@ -168,6 +168,12 @@ export async function createProbeFixture({ root, outputRoot, spawn = spawnSync } target, cwd: anchoredRoot, beforeHash, + credentialAnchors: { + sourceParent: `/proc/${process.pid}/fd/${sourceDirFd}`, + targetParent: `/proc/${process.pid}/fd/${agentDirFd}`, + sourceLeaf: "DUMMY-auth.json", + targetLeaf: "DUMMY-auth.json", + }, assertSafePaths({ mounted = false } = {}) { if ( !sameInode(sourceParent, lstatSync(join(anchoredRoot, "source"), { bigint: true })) || diff --git a/test/cp06-credential-isolation.test.mjs b/test/cp06-credential-isolation.test.mjs index adeed26..dd14825 100644 --- a/test/cp06-credential-isolation.test.mjs +++ b/test/cp06-credential-isolation.test.mjs @@ -1,5 +1,6 @@ import { strict as assert } from "node:assert"; -import { lstat, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { closeSync, constants, openSync } from "node:fs"; +import { lstat, mkdir, mkdtemp, readFile, rename, rm, symlink, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; @@ -103,6 +104,86 @@ test("live credential mount pins one inode behind distinct read-only regular mou } }); +test("credential mount consumes retained parent descriptors after path substitution", async () => { + const root = await mkdtemp(join(tmpdir(), "factory-cp06-retained-mount-test-")); + const sourceDir = join(root, "source"); + const targetDir = join(root, "agent"); + const maliciousDir = join(root, "malicious"); + await mkdir(sourceDir); + await mkdir(targetDir); + await mkdir(maliciousDir); + const source = join(sourceDir, "auth.json"); + const target = join(targetDir, "auth.json"); + await writeFile(source, "DUMMY original"); + await writeFile(join(maliciousDir, "auth.json"), "DUMMY malicious"); + const sourceFd = openSync( + sourceDir, + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW, + ); + const targetFd = openSync( + targetDir, + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW, + ); + const sourceAnchor = `/proc/${process.pid}/fd/${sourceFd}`; + const targetAnchor = `/proc/${process.pid}/fd/${targetFd}`; + const sourceStat = await lstat(source, { bigint: true }); + const calls = []; + let mountId = 90; + const mountIds = new Map(); + const spawnSyncImpl = (command, args) => { + calls.push([command, ...args]); + if (command === "findmnt") { + const path = args.at(-1); + if (!mountIds.has(path)) mountIds.set(path, ++mountId); + return { + status: 0, + stdout: JSON.stringify({ + filesystems: [ + { id: mountIds.get(path), target: path, "vfs-options": "ro,nosuid,nodev,noexec" }, + ], + }), + stderr: "", + }; + } + return { status: 0, stdout: "", stderr: "" }; + }; + const lstatImpl = async (path, options) => + String(path).endsWith("/auth.json") ? sourceStat : lstat(path, options); + try { + await rename(sourceDir, join(root, "source-moved")); + await symlink(maliciousDir, sourceDir); + const mount = await mountLiveCredentialReadOnly({ + root, + source, + target, + sourceParentAnchor: sourceAnchor, + targetParentAnchor: targetAnchor, + sourceLeaf: "auth.json", + targetLeaf: "auth.json", + spawnSyncImpl, + lstatImpl, + }); + try { + const mountCallsOnly = calls.filter(([command]) => command === "mount"); + assert.match( + mountCallsOnly[4][3], + new RegExp(`^/proc/${process.pid}/fd/${sourceFd}/auth\\.json$`), + ); + assert.match( + mountCallsOnly[6][3], + new RegExp(`^/proc/${process.pid}/fd/${targetFd}/auth\\.json$`), + ); + assert.equal(await readFile(join(maliciousDir, "auth.json"), "utf8"), "DUMMY malicious"); + } finally { + await mount.cleanup(); + } + } finally { + closeSync(sourceFd); + closeSync(targetFd); + await rm(root, { recursive: true, force: true }); + } +}); + test("mount setup failure reports cleanup failure instead of discarding it", async () => { const root = await mkdtemp(join(tmpdir(), "factory-cp06-cleanup-test-")); const source = join(root, "auth-source.json"); diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index cfb8884..4e1e517 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -1,5 +1,6 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; +import { readFileSync, writeFileSync } from "node:fs"; import { mkdir, mkdtemp, @@ -13,7 +14,11 @@ import { import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import test from "node:test"; -import { runDummySdkProof, runIsolationProbe } from "../scripts/cp06-auth-security.mjs"; +import { + compileIsolationHelpers, + runDummySdkProof, + runIsolationProbe, +} from "../scripts/cp06-auth-security.mjs"; import { openAnchoredDirectory, openProbeOutput } from "../scripts/cp06-probe-fixture.mjs"; const runners = [ @@ -147,7 +152,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { refresh_callbacks: 0, credential_store: credentialAudit(), }, - sdk_dependency: { version: "0.85.1", package_sha256: "1".repeat(64) }, + sdk_dependency: sdkDependency(), actual_sdk_outcomes: { fixture_origin: true, semantic_acceptance: false, @@ -176,6 +181,14 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { { ...validBase, auth_cases: [] }, { ...validBase, auth_cases: [...validBase.auth_cases, authCase("expired", "blocked", 0)] }, { ...validBase, extra: "DUMMY-SECRET" }, + { ...validBase, sdk_dependency: { ...validBase.sdk_dependency, version: "0.99.2" } }, + { + ...validBase, + actual_sdk_outcomes: { + ...validBase.actual_sdk_outcomes, + cases: [{ ...validBase.actual_sdk_outcomes.cases[0], forged: "DUMMY-SECRET" }], + }, + }, { ...validBase, sdk_worker: { ...validBase.sdk_worker, active_tools: ["read", "write"] } }, ]) { assert.throws( @@ -202,6 +215,38 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { assert.equal(accepted.result, "pass"); }); +test("helper compilation ignores preexisting symlinked artifact leaves", async () => { + const root = process.cwd(); + const outputRoot = resolve(".factory/state/auth-security/DUMMY-compile"); + const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-compile-external-")); + const sentinel = join(external, "DUMMY-sentinel"); + const output = openProbeOutput({ root, outputRoot, create: true }); + await writeFile(sentinel, "DUMMY sentinel unchanged"); + await symlink(sentinel, join(output.anchor, "cp06-seccomp-exec")); + await symlink(sentinel, join(output.anchor, "cp06-isolation-syscalls")); + let binaries; + try { + binaries = compileIsolationHelpers({ + root, + outputRoot, + anchoredOutputRoot: output.anchor, + spawnSyncImpl(_command, args) { + const outputPath = args[args.indexOf("-o") + 1]; + writeFileSync(outputPath, "DUMMY compiled helper"); + return { status: 0, stdout: "", stderr: "" }; + }, + }); + assert.equal(await readFile(sentinel, "utf8"), "DUMMY sentinel unchanged"); + assert.equal(readFileSync(binaries.helper, "utf8"), "DUMMY compiled helper"); + assert.equal(readFileSync(binaries.syscallProbe, "utf8"), "DUMMY compiled helper"); + } finally { + binaries?.close?.(); + output.close(); + await rm(outputRoot, { recursive: true, force: true }); + await rm(external, { recursive: true, force: true }); + } +}); + test("anchored runner child directories do not follow substituted paths", async () => { const root = process.cwd(); const outputRoot = resolve(".factory/state/cp06-correction/DUMMY-child-anchor"); @@ -241,15 +286,38 @@ function credentialAudit() { return { reads: 1, lists: 0, modify_denials: 1, delete_denials: 1 }; } -function outcome(scenario, exitCode, extra = {}) { +function sdkDependency() { return { + entry: "/DUMMY/pi-ai/dist/index.js", + entry_sha256: "2".repeat(64), + manifest_sha256: "3".repeat(64), + name: "@earendil-works/pi-ai", + package_sha256: "1".repeat(64), + resolution: "esm-import-condition", + root: "/DUMMY/pi-ai", + tarball_integrity: "sha512-DUMMY", + version: "0.85.1", + }; +} + +function outcome(scenario, exitCode, extra = {}) { + const result = { scenario, exit_code: exitCode, expected_exit_code: exitCode, + exit: exitCode, + fixture_origin: true, + sdk_dependency: sdkDependency(), default_storage: 0, network: 0, refresh: 0, + to_auth: ["expired", "near-expiry"].includes(scenario) ? 0 : 1, + runtime_create: ["expired", "near-expiry"].includes(scenario) ? 0 : 1, source_unchanged: true, ...extra, }; + if (["valid", "fenced-valid"].includes(scenario)) { + result.read_audit = { exact_original: true, read_count: 1 }; + } + return result; } From 7b96ee6f88691b0a9afc9afb1cfbba4d0792f7a0 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Sat, 3 Oct 2026 12:43:01 +0000 Subject: [PATCH 30/45] no-mistakes(review): Harden CP-06 proof boundaries --- docs/probes/CP-06.md | 4 +- scripts/cp06-auth-security.mjs | 157 ++++++++++++++++++-------- scripts/prove-cp06-worker-cleanup.mjs | 35 ++++-- test/cp06-proof-boundary.test.mjs | 72 ++++++++---- 4 files changed, 181 insertions(+), 87 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 42a5c43..bfb5495 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,13 +21,13 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is anchored to that directory. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. This does not yet pin the actual mount helper's pathname opens/target creation through a concurrent parent replacement, nor the compiled helpers' pathname execution and final hashes after output replacement. Those transitions remain unverified even if the ordinary DUMMY controls pass. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created descriptor paths, and the retained helper/probe identities must still match their anchored leaves before execution and hashing. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup consumes retained source and target parent descriptors plus the pinned source leaf; replacement of the prepared source, target, parent, helper, or output child before mount, exec, write, or hash is rejected by the executable transition regressions without touching external DUMMY sentinels. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and complete published file manifests must agree with the independently pinned exact-version npm artifacts in `.factory/cp06-sdk-integrity.json`; inherited `PATH`, a global installation, an unrelated same-version package, and same-version file substitutions are rejected before SDK import without fallback. The pin records npm tarball SHA-512 SRI and sorted `[relative path, file SHA-256]` manifest SHA-256 and file count for both packages; it excludes nested installed `node_modules` from each package manifest, while separately verifying the resolved `pi-ai` package. It does not attest the registry itself, transitive packages other than `pi-ai`, or immutable files after this check. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths, trusted artifact pins, manifest digests, and SHA-256 entry digests. To reproduce a pin independently, download the named exact-version tarballs into ignored `.factory/state/`, compare each tarball's SHA-512 against the tracked SRI, extract into a disposable directory, sort all regular published files by relative path and compute SHA-256 of JSON-serialized `[path, SHA-256(file)]` pairs (with no extra files or symlinks); compare count and digest to the tracked pin. Never derive a new trusted pin from the selected installation under evaluation. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. -Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it must not be attempted until the mount and executable identity handoff gaps above are repaired and the final-head DUMMY gate and transition regressions pass. The existing `cp06-seccomp-exec` helper closes all inherited file descriptors; the current Node interface has no FD-relative mount API. Repair requires a bounded FD-based mount consumer and a seccomp-safe handoff that executes/hashes the same retained helper/probe identities without passing credential descriptors to the evaluated child. No unsupported lexical fallback is acceptable. No destructive operation is ever directed at the real credential source or mounted target. +Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is not genuine acceptance until the exact final-head DUMMY gate, worker-cleanup proof, transition regressions, and semantic worker command all pass and their manifests identify the same tested SHA. The existing `cp06-seccomp-exec` helper closes all inherited file descriptors before launching the evaluated child; retained helper/probe descriptors are used for supervisor execution and hashing, while credential descriptors are not passed to that child. No unsupported lexical fallback is acceptable. No destructive operation is ever directed at the real credential source or mounted target. ## Normative result diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index b390159..26acf89 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -1,7 +1,17 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { closeSync, constants, mkdtempSync, openSync, readFileSync } from "node:fs"; +import { + closeSync, + constants, + fchmodSync, + fstatSync, + lstatSync, + mkdtempSync, + openSync, + readFileSync, +} from "node:fs"; import { join, resolve } from "node:path"; +import { resolvePinnedPiAi } from "./cp06-pi-dependency.mjs"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; import { openProbeOutput } from "./cp06-probe-fixture.mjs"; @@ -43,7 +53,12 @@ export async function runCp06SecurityPreflight(options = {}) { binaries, }); assertHardenedProof(hardened); - const sdk = runDummySdkProof({ root, helper: binaries.helper, piRoot: pi.root }); + const sdk = runDummySdkProof({ + root, + helper: binaries.helper, + piRoot: pi.root, + expectedSdkDependency: pi.provenance.dependency, + }); const proof = { schema_version: 1, gate: "CP-06-read-only-auth-security", @@ -120,39 +135,22 @@ export function compileIsolationHelpers(options) { let helperFd; let syscallProbeFd; try { - checked( - "cc", - [ - "-std=c11", - "-O2", - "-Wall", - "-Wextra", - "-Werror", - join(root, "scripts/cp06-seccomp-exec.c"), - "-Wl,-l:libseccomp.so.2", - "-o", - helper, - ], + helperFd = compilePinnedArtifact({ root, + artifactAnchor, + name: "cp06-seccomp-exec", + source: join(root, "scripts/cp06-seccomp-exec.c"), + extraArgs: ["-Wl,-l:libseccomp.so.2"], spawn, - ); - checked( - "cc", - [ - "-std=c11", - "-O2", - "-Wall", - "-Wextra", - "-Werror", - join(root, "scripts/cp06-isolation-syscalls.c"), - "-o", - syscallProbe, - ], + }); + syscallProbeFd = compilePinnedArtifact({ root, + artifactAnchor, + name: "cp06-isolation-syscalls", + source: join(root, "scripts/cp06-isolation-syscalls.c"), + extraArgs: [], spawn, - ); - helperFd = openSync(helper, constants.O_RDONLY | constants.O_NOFOLLOW); - syscallProbeFd = openSync(syscallProbe, constants.O_RDONLY | constants.O_NOFOLLOW); + }); let closed = false; closeSync(artifactFd); return { @@ -416,7 +414,13 @@ function assertHardenedProof(proof) { } } -export function runDummySdkProof({ root, helper, piRoot, spawnSyncImpl = spawnSync }) { +export function runDummySdkProof({ + root, + helper, + piRoot, + spawnSyncImpl = spawnSync, + expectedSdkDependency, +}) { const result = spawnSyncImpl( "unshare", [ @@ -462,10 +466,11 @@ export function runDummySdkProof({ root, helper, piRoot, spawnSyncImpl = spawnSy : "CP06_DUMMY_SDK_CHILD_EXIT"; throw unsupported(`DUMMY SDK proof failed: ${code}`); } - return validateDummySdkProof(parseCompactJson(result.stdout, "DUMMY SDK proof")); + const trustedDependency = expectedSdkDependency ?? resolvePinnedPiAi(piRoot).provenance; + return validateDummySdkProof(parseCompactJson(result.stdout, "DUMMY SDK proof"), trustedDependency); } -function validateDummySdkProof(proof) { +function validateDummySdkProof(proof, trustedDependency) { assertExactKeys(proof, [ "actual_sdk_outcomes", "auth_cases", @@ -484,8 +489,8 @@ function validateDummySdkProof(proof) { assert(proof.network_calls === 0, "DUMMY SDK attempted network access"); validateAuthCases(proof.auth_cases); validateSdkWorker(proof.sdk_worker); - validateSdkDependency(proof.sdk_dependency); - validateOutcomeProof(proof.actual_sdk_outcomes); + validateSdkDependency(proof.sdk_dependency, trustedDependency); + validateOutcomeProof(proof.actual_sdk_outcomes, trustedDependency); return proof; } @@ -545,7 +550,7 @@ function validateSdkWorker(worker) { assert(worker.read_audit?.read_count === 1, "DUMMY SDK read count mismatch"); } -function validateSdkDependency(dependency) { +function validateSdkDependency(dependency, trustedDependency) { assertExactKeys(dependency, [ "entry", "entry_sha256", @@ -557,24 +562,26 @@ function validateSdkDependency(dependency) { "tarball_integrity", "version", ]); - assert(dependency.name === "@earendil-works/pi-ai", "DUMMY SDK dependency name mismatch"); - assert(dependency.version === "0.85.1", "DUMMY SDK dependency version mismatch"); - assert( - dependency.resolution === "esm-import-condition", - "DUMMY SDK dependency resolution mismatch", - ); - for (const key of ["entry", "root", "tarball_integrity"]) { - assert( - typeof dependency[key] === "string" && dependency[key].length > 0, - "DUMMY SDK dependency path missing", - ); + assertExactKeys(trustedDependency, [ + "entry", + "entry_sha256", + "manifest_sha256", + "name", + "package_sha256", + "resolution", + "root", + "tarball_integrity", + "version", + ]); + for (const [key, value] of Object.entries(trustedDependency)) { + assert(dependency[key] === value, `DUMMY SDK dependency ${key} mismatch`); } for (const key of ["entry_sha256", "manifest_sha256", "package_sha256"]) { assert(isSha256(dependency[key]), "DUMMY SDK dependency digest missing"); } } -function validateOutcomeProof(outcomes) { +function validateOutcomeProof(outcomes, trustedDependency) { assertExactKeys(outcomes, ["cases", "fixture_origin", "semantic_acceptance"]); assert(outcomes.fixture_origin === true, "DUMMY outcome fixture origin missing"); assert(outcomes.semantic_acceptance === false, "DUMMY outcome semantic acceptance was claimed"); @@ -630,7 +637,7 @@ function validateOutcomeProof(outcomes) { "DUMMY outcome exit mismatch", ); assert(entry.fixture_origin === true, "DUMMY outcome fixture origin missing"); - validateSdkDependency(entry.sdk_dependency); + validateSdkDependency(entry.sdk_dependency, trustedDependency); assert(entry.source_unchanged === true, "DUMMY outcome source changed"); assert( entry.default_storage === 0 && entry.network === 0 && entry.refresh === 0, @@ -690,6 +697,52 @@ function resolvePiPackage(root, options) { } } +function compilePinnedArtifact({ root, artifactAnchor, name, source, extraArgs, spawn }) { + const leaf = join(artifactAnchor, name); + const fd = openSync( + leaf, + constants.O_CREAT | constants.O_EXCL | constants.O_RDWR | constants.O_NOFOLLOW, + 0o700, + ); + try { + fchmodSync(fd, 0o700); + const pinned = fstatSync(fd, { bigint: true }); + const result = spawn( + "cc", + [ + "-std=c11", + "-O2", + "-Wall", + "-Wextra", + "-Werror", + source, + ...extraArgs, + "-o", + "/proc/self/fd/3", + ], + { + cwd: root, + encoding: "utf8", + timeout: 60_000, + stdio: ["ignore", "pipe", "pipe", fd], + }, + ); + if (result.error !== undefined || result.status !== 0) { + throw unsupported("required command failed: cc"); + } + fchmodSync(fd, 0o700); + const current = fstatSync(fd, { bigint: true }); + const named = lstatSync(leaf, { bigint: true }); + if (!current.isFile() || !sameInode(pinned, current) || !sameInode(pinned, named)) { + throw unsupported("compiled helper identity changed"); + } + return fd; + } catch (error) { + closeSync(fd); + throw error; + } +} + function checked(command, args, cwd, spawn = spawnSync) { const result = spawn(command, args, { cwd, encoding: "utf8", timeout: 60_000 }); if (result.error !== undefined || result.status !== 0) { @@ -698,6 +751,10 @@ function checked(command, args, cwd, spawn = spawnSync) { return result; } +function sameInode(a, b) { + return a.dev === b.dev && a.ino === b.ino; +} + function commandText(spawn, command, args, cwd) { const result = spawn(command, args, { cwd, encoding: "utf8", timeout: 30_000 }); if (result.error !== undefined || result.status !== 0) { diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index f290bb7..ece7eb7 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -5,16 +5,16 @@ import { spawnSync } from "node:child_process"; import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; +import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; const root = resolve(fileURLToPath(new URL("..", import.meta.url))); -const output = resolve( - root, - process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction", - "worker-cleanup", -); -await mkdir(output, { recursive: true }); -const scratch = await mkdtemp(join(output, "DUMMY-cleanup-")); +const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); +const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); +const proofDirectory = openAnchoredDirectory(output.anchor, "worker-cleanup", { reset: true }); +const proofRoot = proofDirectory.anchor; +const proofRootPath = join(outputRoot, "worker-cleanup"); +const scratch = await mkdtemp(join(proofRoot, "DUMMY-cleanup-")); const cases = await withCleanup( async () => { const cases = []; @@ -25,7 +25,13 @@ const cases = await withCleanup( ]) { const home = join(scratch, scenario); const temporary = join(home, "tmp"); - const proofOutput = join(home, "proofs"); + const proofOutputPath = join(proofRootPath, "proofs", scenario); + const proofOutputDirectory = openAnchoredDirectory( + proofDirectory.anchor, + `proofs/${scenario}`, + { reset: true }, + ); + const proofOutput = proofOutputDirectory.anchor; const source = join(home, "credential", "DUMMY-auth.json"); await mkdir(temporary, { recursive: true }); await mkdir(join(home, "credential"), { recursive: true }); @@ -66,19 +72,22 @@ const cases = await withCleanup( CP06_PI_AUTH_FILE: source, CP06_PI_PACKAGE_ROOT: process.env.CP06_PI_PACKAGE_ROOT, CP06_PI_BIN: process.env.CP06_PI_BIN, - CP06_OUTPUT: proofOutput, + CP06_OUTPUT: proofOutputPath, CP06_REVIEWER: "DUMMY cleanup regression", }, }); assert.equal(result.error, undefined); assert.equal(result.status, expectedExit, result.stderr); - const blocked = JSON.parse(await readFile(join(proofOutput, "worker/blocked.json"), "utf8")); + const blocked = JSON.parse( + await readFile(join(proofOutputPath, "worker/blocked.json"), "utf8"), + ); assert.equal(blocked.result, "blocked"); assert.equal(blocked.stage, "isolated-sdk-worker"); assert.deepEqual(await readdir(temporary), [], `${scenario}: evaluation home leaked`); if (scenario !== "missing-source-setup") assert.equal(await readFile(source, "utf8"), sourceBytes); - await assert.rejects(readFile(join(proofOutput, "worker/evidence.json")), { code: "ENOENT" }); + await assert.rejects(readFile(join(proofOutputPath, "worker/evidence.json")), { code: "ENOENT" }); + proofOutputDirectory.close(); cases.push({ scenario, command: [process.execPath, ...args], @@ -130,7 +139,9 @@ const manifest = { "Actual full-runner expired/near-expiry and missing-source setup cases use DUMMY auth only. Launch, timeout, audit and compound cleanup failures are additionally injected at the executable supervisor interface in the unit suite; SDK timeout is exercised by the DUMMY auth-security proof.", ], }; -await writeFile(join(output, "evidence.json"), `${JSON.stringify(manifest, null, 2)}\n`); +await writeFile(join(proofRoot, "evidence.json"), `${JSON.stringify(manifest, null, 2)}\n`); +proofDirectory.close(); +output.close(); console.log( JSON.stringify({ result: "pass", diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 4e1e517..dfa735b 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -1,6 +1,6 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { readFileSync, writeFileSync } from "node:fs"; +import { readlinkSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { mkdir, mkdtemp, @@ -25,6 +25,7 @@ const runners = [ "scripts/replay-cp06-worker.mjs", "scripts/replay-cp06-ten-pack.mjs", "scripts/replay-cp06-p07.mjs", + "scripts/prove-cp06-worker-cleanup.mjs", ]; test("all proof runners reject external and symlinked output before removing artifacts", async () => { @@ -32,7 +33,7 @@ test("all proof runners reject external and symlinked output before removing art const base = resolve(".factory/state/cp06-correction"); const alias = join(base, "DUMMY-output-alias"); await mkdir(base, { recursive: true }); - for (const name of ["worker", "ten-pack", "p07"]) { + for (const name of ["worker", "ten-pack", "p07", "worker-cleanup"]) { await mkdir(join(external, name)); await writeFile(join(external, name, "DUMMY-sentinel"), "DUMMY-SECRET-UNCHANGED"); } @@ -49,7 +50,7 @@ test("all proof runners reject external and symlinked output before removing art assert.notEqual(result.status, 0, `${runner} accepted ${output}`); assert.equal(result.stdout.includes("DUMMY-SECRET-UNCHANGED"), false); assert.equal(result.stderr.includes("DUMMY-SECRET-UNCHANGED"), false); - for (const name of ["worker", "ten-pack", "p07"]) { + for (const name of ["worker", "ten-pack", "p07", "worker-cleanup"]) { assert.deepEqual(await readdir(join(external, name)), ["DUMMY-sentinel"]); assert.equal( await readFile(join(external, name, "DUMMY-sentinel"), "utf8"), @@ -135,6 +136,7 @@ test("failed DUMMY probes never forward child output or spawn exceptions", async }); test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { + const trustedDependency = sdkDependency(); const validBase = { schema_version: 1, result: "pass", @@ -152,7 +154,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { refresh_callbacks: 0, credential_store: credentialAudit(), }, - sdk_dependency: sdkDependency(), + sdk_dependency: trustedDependency, actual_sdk_outcomes: { fixture_origin: true, semantic_acceptance: false, @@ -182,6 +184,24 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { { ...validBase, auth_cases: [...validBase.auth_cases, authCase("expired", "blocked", 0)] }, { ...validBase, extra: "DUMMY-SECRET" }, { ...validBase, sdk_dependency: { ...validBase.sdk_dependency, version: "0.99.2" } }, + { + ...validBase, + sdk_dependency: { + ...validBase.sdk_dependency, + root: resolve(".factory/state/cp06-sdk/node_modules/@earendil-works/pi-ai-forged"), + }, + }, + { + ...validBase, + sdk_dependency: { + ...validBase.sdk_dependency, + entry: resolve(".factory/state/cp06-sdk/node_modules/@earendil-works/pi-ai/dist/forged.js"), + }, + }, + { + ...validBase, + sdk_dependency: { ...validBase.sdk_dependency, manifest_sha256: "4".repeat(64) }, + }, { ...validBase, actual_sdk_outcomes: { @@ -200,6 +220,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { spawnSyncImpl() { return { status: 0, stdout: JSON.stringify(payload), stderr: "" }; }, + expectedSdkDependency: trustedDependency, }), (error) => !String(error.message).includes("DUMMY-SECRET"), ); @@ -211,36 +232,40 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { spawnSyncImpl() { return { status: 0, stdout: JSON.stringify(validBase), stderr: "" }; }, + expectedSdkDependency: trustedDependency, }); assert.equal(accepted.result, "pass"); }); -test("helper compilation ignores preexisting symlinked artifact leaves", async () => { +test("helper compilation rejects replaced artifact leaves", async () => { const root = process.cwd(); const outputRoot = resolve(".factory/state/auth-security/DUMMY-compile"); const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-compile-external-")); const sentinel = join(external, "DUMMY-sentinel"); const output = openProbeOutput({ root, outputRoot, create: true }); await writeFile(sentinel, "DUMMY sentinel unchanged"); - await symlink(sentinel, join(output.anchor, "cp06-seccomp-exec")); - await symlink(sentinel, join(output.anchor, "cp06-isolation-syscalls")); - let binaries; + let replaced = false; try { - binaries = compileIsolationHelpers({ - root, - outputRoot, - anchoredOutputRoot: output.anchor, - spawnSyncImpl(_command, args) { - const outputPath = args[args.indexOf("-o") + 1]; - writeFileSync(outputPath, "DUMMY compiled helper"); - return { status: 0, stdout: "", stderr: "" }; - }, - }); + assert.throws(() => + compileIsolationHelpers({ + root, + outputRoot, + anchoredOutputRoot: output.anchor, + spawnSyncImpl(_command, _args, options) { + const outputPath = `/proc/${process.pid}/fd/${options.stdio[3]}`; + writeFileSync(outputPath, "DUMMY compiled helper"); + if (!replaced) { + replaced = true; + const leaf = readlinkSync(outputPath); + rmSync(leaf, { force: true }); + symlinkSync(sentinel, leaf); + } + return { status: 0, stdout: "", stderr: "" }; + }, + }), + ); assert.equal(await readFile(sentinel, "utf8"), "DUMMY sentinel unchanged"); - assert.equal(readFileSync(binaries.helper, "utf8"), "DUMMY compiled helper"); - assert.equal(readFileSync(binaries.syscallProbe, "utf8"), "DUMMY compiled helper"); } finally { - binaries?.close?.(); output.close(); await rm(outputRoot, { recursive: true, force: true }); await rm(external, { recursive: true, force: true }); @@ -287,14 +312,15 @@ function credentialAudit() { } function sdkDependency() { + const root = resolve(".factory/state/cp06-sdk/node_modules/@earendil-works/pi-ai"); return { - entry: "/DUMMY/pi-ai/dist/index.js", + entry: join(root, "dist/index.js"), entry_sha256: "2".repeat(64), manifest_sha256: "3".repeat(64), name: "@earendil-works/pi-ai", package_sha256: "1".repeat(64), resolution: "esm-import-condition", - root: "/DUMMY/pi-ai", + root, tarball_integrity: "sha512-DUMMY", version: "0.85.1", }; From 19e785da1a2420062e0d13e612796bba33772a60 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Sat, 3 Oct 2026 18:08:14 +0000 Subject: [PATCH 31/45] no-mistakes(review): Fix helper execution, serializer imports, and nested proof resets --- scripts/cp06-auth-security.mjs | 18 ++- scripts/cp06-probe-fixture.mjs | 6 +- scripts/replay-cp06-worker.mjs | 23 ++-- test/cp06-selected-boundaries.test.mjs | 158 +++++++++++++++++++++++++ 4 files changed, 194 insertions(+), 11 deletions(-) create mode 100644 test/cp06-selected-boundaries.test.mjs diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 26acf89..b61950b 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -704,6 +704,7 @@ function compilePinnedArtifact({ root, artifactAnchor, name, source, extraArgs, constants.O_CREAT | constants.O_EXCL | constants.O_RDWR | constants.O_NOFOLLOW, 0o700, ); + let executionFd; try { fchmodSync(fd, 0o700); const pinned = fstatSync(fd, { bigint: true }); @@ -736,10 +737,23 @@ function compilePinnedArtifact({ root, artifactAnchor, name, source, extraArgs, if (!current.isFile() || !sameInode(pinned, current) || !sameInode(pinned, named)) { throw unsupported("compiled helper identity changed"); } - return fd; + executionFd = openSync(`/proc/${process.pid}/fd/${fd}`, constants.O_RDONLY); + const executable = fstatSync(executionFd, { bigint: true }); + const executionLeaf = lstatSync(leaf, { bigint: true }); + if ( + !executable.isFile() || + !executionLeaf.isFile() || + !sameInode(pinned, executable) || + !sameInode(pinned, executionLeaf) + ) { + throw unsupported("compiled helper identity changed"); + } + return executionFd; } catch (error) { - closeSync(fd); + if (executionFd !== undefined) closeSync(executionFd); throw error; + } finally { + closeSync(fd); } } diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index 6df932e..5ce7144 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -98,12 +98,14 @@ export function openAnchoredDirectory( ) { throw new Error("DUMMY proof output child path is invalid"); } - if (reset) rmSync(join(parentAnchor, components[0]), { recursive: true, force: true }); let currentFd; let currentAnchor = parentAnchor; try { - for (const component of components) { + for (const [index, component] of components.entries()) { const path = join(currentAnchor, component); + if (reset && index === components.length - 1) { + rmSync(path, { recursive: true, force: true }); + } if (create) { try { mkdirSync(path, { mode: 0o700 }); diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 81126c2..0354823 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -16,9 +16,6 @@ import { const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); -const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); -const workerDirectory = openAnchoredDirectory(output.anchor, "worker", { reset: true }); -const workerRoot = workerDirectory.anchor; const securityRoot = join(outputRoot, "auth-security"); const fixtureRoot = join(outputRoot, "ten-pack", "repo"); const initialPackPath = join(outputRoot, "ten-pack", "raw", "auth-worker-missing.pack.md"); @@ -40,6 +37,18 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur } async function main() { + const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); + let workerDirectory; + try { + workerDirectory = openAnchoredDirectory(output.anchor, "worker", { reset: true }); + await runWorker(workerDirectory.anchor); + } finally { + workerDirectory?.close(); + output.close(); + } +} + +async function runWorker(workerRoot) { let security; try { // This DUMMY-only gate includes the unsafe control, hardened evaluated-child @@ -52,7 +61,7 @@ async function main() { retainBinaries: true, }); } catch (error) { - await recordBlocked(error, "dummy-security-preflight"); + await recordBlocked(workerRoot, error, "dummy-security-preflight"); throw error; } @@ -92,7 +101,7 @@ async function main() { () => rm(evaluationHome, { recursive: true, force: true }), ); } catch (error) { - await recordBlocked(error, "isolated-sdk-worker"); + await recordBlocked(workerRoot, error, "isolated-sdk-worker"); throw error; } finally { binaries?.close?.(); @@ -106,7 +115,7 @@ async function main() { isolated.child?.result !== "pass" ) { const error = new Error("isolated SDK worker omitted a required safety assertion"); - await recordBlocked(error, "isolated-sdk-worker-audit"); + await recordBlocked(workerRoot, error, "isolated-sdk-worker-audit"); throw error; } @@ -181,7 +190,7 @@ async function main() { ); } -async function recordBlocked(error, stage) { +async function recordBlocked(workerRoot, error, stage) { const status = createWorkerBlockedStatus(error, stage); await writeFile(join(workerRoot, "blocked.json"), `${JSON.stringify(status, null, 2)}\n`); } diff --git a/test/cp06-selected-boundaries.test.mjs b/test/cp06-selected-boundaries.test.mjs new file mode 100644 index 0000000..6abefa8 --- /dev/null +++ b/test/cp06-selected-boundaries.test.mjs @@ -0,0 +1,158 @@ +import { strict as assert } from "node:assert"; +import { spawnSync } from "node:child_process"; +import { readlinkSync } from "node:fs"; +import { + mkdir, + mkdtemp, + readFile, + readdir, + rename, + rm, + symlink, + writeFile, +} from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { pathToFileURL } from "node:url"; +import test from "node:test"; +import { compileIsolationHelpers } from "../scripts/cp06-auth-security.mjs"; +import { openAnchoredDirectory, openProbeOutput } from "../scripts/cp06-probe-fixture.mjs"; + +async function dummyOutput() { + const base = resolve(".factory/state/cp06-correction"); + await mkdir(base, { recursive: true }); + return mkdtemp(join(base, "DUMMY-selected-boundaries-")); +} + +test("compiled helpers execute and hash through retained read-only inode descriptors", async () => { + const directory = await dummyOutput(); + const outputRoot = join(directory, "auth-security"); + const output = openProbeOutput({ root: process.cwd(), outputRoot, create: true }); + let binaries; + try { + binaries = compileIsolationHelpers({ + root: process.cwd(), + outputRoot, + anchoredOutputRoot: output.anchor, + }); + for (const descriptorPath of [binaries.helper, binaries.syscallProbe]) { + const original = await readFile(descriptorPath); + const leaf = readlinkSync(descriptorPath); + await rename(leaf, `${leaf}.original`); + await writeFile(leaf, "DUMMY replacement must not execute"); + assert.deepEqual(await readFile(descriptorPath), original); + } + const execution = spawnSync( + binaries.helper, + [process.execPath, "-e", 'process.stdout.write("DUMMY executable control")'], + { encoding: "utf8", timeout: 15_000 }, + ); + assert.equal(execution.error, undefined); + assert.equal(execution.status, 0, execution.stderr); + assert.equal(execution.stdout, "DUMMY executable control"); + const probe = spawnSync(binaries.syscallProbe, [], { encoding: "utf8", timeout: 15_000 }); + assert.equal(probe.error, undefined); + assert.equal(probe.status, 64); + binaries.close(); + binaries.close(); + await assert.rejects(readFile(binaries.helper)); + } finally { + binaries?.close(); + output.close(); + await rm(directory, { recursive: true, force: true }); + } +}); + +test("importing worker serializers preserves evidence and creates no output", async () => { + const directory = await dummyOutput(); + const workerRoot = join(directory, "worker"); + await mkdir(workerRoot); + const records = { + "evidence.json": '{"result":"DUMMY evidence"}\n', + "blocked.json": '{"result":"DUMMY blocked"}\n', + }; + for (const [name, bytes] of Object.entries(records)) { + await writeFile(join(workerRoot, name), bytes); + } + const moduleUrl = pathToFileURL(resolve("scripts/replay-cp06-worker.mjs")).href; + try { + for (const output of [directory, join(directory, "uncreated")]) { + const result = spawnSync( + process.execPath, + [ + "--input-type=module", + "-e", + `const {createWorkerBlockedStatus}=await import(${JSON.stringify(moduleUrl)});const result=createWorkerBlockedStatus(new Error("DUMMY"),"audit","DUMMY date");if(result.result!=="blocked")process.exit(1);`, + ], + { + cwd: process.cwd(), + encoding: "utf8", + timeout: 15_000, + env: { ...process.env, CP06_OUTPUT: output }, + }, + ); + assert.equal(result.status, 0, result.stderr); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, ""); + assert.deepEqual((await readdir(workerRoot)).sort(), Object.keys(records).sort()); + for (const [name, bytes] of Object.entries(records)) { + assert.equal(await readFile(join(workerRoot, name), "utf8"), bytes); + } + } + await assert.rejects(readdir(join(directory, "uncreated")), { code: "ENOENT" }); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + +test("nested directory reset preserves sibling proofs and rejects substituted parents", async () => { + const directory = await dummyOutput(); + const output = openProbeOutput({ + root: process.cwd(), + outputRoot: directory, + create: true, + runner: true, + }); + const scenarios = ["expired", "near-expiry", "missing-source-setup"]; + try { + for (const scenario of scenarios) { + const child = openAnchoredDirectory(output.anchor, `proofs/${scenario}`, { reset: true }); + try { + await writeFile(join(child.anchor, "evidence.json"), `DUMMY ${scenario}`); + } finally { + child.close(); + } + } + await writeFile(join(directory, "proofs/near-expiry/stale.json"), "DUMMY stale"); + const reset = openAnchoredDirectory(output.anchor, "proofs/near-expiry", { reset: true }); + try { + assert.deepEqual(await readdir(reset.anchor), []); + await writeFile(join(reset.anchor, "evidence.json"), "DUMMY near-expiry"); + } finally { + reset.close(); + } + for (const scenario of scenarios) { + assert.equal( + await readFile(join(directory, "proofs", scenario, "evidence.json"), "utf8"), + `DUMMY ${scenario}`, + ); + } + await rename(join(directory, "proofs"), join(directory, "proofs-original")); + await mkdir(join(directory, "DUMMY-sentinel/expired"), { recursive: true }); + await writeFile(join(directory, "DUMMY-sentinel/expired/evidence.json"), "DUMMY unchanged"); + await symlink(join(directory, "DUMMY-sentinel"), join(directory, "proofs")); + assert.throws(() => openAnchoredDirectory(output.anchor, "proofs/expired", { reset: true })); + assert.equal( + await readFile(join(directory, "DUMMY-sentinel/expired/evidence.json"), "utf8"), + "DUMMY unchanged", + ); + for (const scenario of scenarios) { + assert.equal( + await readFile(join(directory, "proofs-original", scenario, "evidence.json"), "utf8"), + `DUMMY ${scenario}`, + ); + } + } finally { + output.close(); + await rm(directory, { recursive: true, force: true }); + } +}); From 93f424c3b12c087840edae74e849e5624e4d560b Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Sat, 3 Oct 2026 18:23:15 +0000 Subject: [PATCH 32/45] no-mistakes(document): Refresh CP06 documentation and format proof scripts --- ARCHITECTURE.md | 2 +- BUILD_PLAN.md | 2 +- README.md | 18 ++++++++++++++---- docs/CONTRACTS.md | 2 +- docs/INTEGRATIONS.md | 2 +- docs/WORKFLOW.md | 2 +- docs/probes/CP-06.md | 13 ++++++++++++- scripts/cp06-auth-security.mjs | 5 ++++- scripts/cp06-dummy-worker-hook.mjs | 11 ++++++----- scripts/cp06-guarded-read.mjs | 4 +--- scripts/cp06-package-integrity.mjs | 5 ++++- scripts/cp06-sdk-worker.mjs | 6 +----- scripts/prove-cp06-worker-cleanup.mjs | 4 +++- skills/factory/SKILL.md | 12 +++--------- 14 files changed, 53 insertions(+), 35 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 09a02cc..b9ee8ef 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -43,7 +43,7 @@ Recommended exit codes: `0` success; `2` validation failure; `3` unavailable/sta ## CTX context pack -`factory context TASK-ID` first includes the exact tracked task contract and a concise global project constraint section. It invokes `ctx status --root --json` and `ctx doctor --offline --root --json` as appropriate, then `ctx pack --root --token-budget --document ... --json`. The query is derived from title, outcome, acceptance, and topic strings; it is never a command. Repeated `--document` filters come from the task's declared `context.required` selection, so CTX ranks within task-local authority instead of adding unrelated corpus excerpts. When that valid selection is empty, Factory passes the project's mandatory `documents.required` paths instead; it never invokes `ctx pack` without a document filter, and a global-only task remains viable without widening to the corpus. Canonical required documents are read from their original files, checked against root containment, and admitted before CTX excerpts; declaring a source required never substitutes an excerpt for the exact original. The adapter records CTX generation, file and line provenance, and available hashes. On stale, missing, or insufficient retrieval, it returns `CONTEXT_BLOCKED` with an actionable diagnostic. It does not fill the pack with guessed material or hide post-retrieval excerpts to improve a relevance score. +`factory context TASK-ID` first includes the exact tracked task contract and a concise global project constraint section. It invokes `ctx status --root --json` and `ctx doctor --offline --root --json` as appropriate, then `ctx pack --root --token-budget --document ... --json`. The query is derived from title, outcome, acceptance, and topic strings; it is never a command. Repeated `--document` filters implement the authority-selection contract in [docs/CONTRACTS.md](docs/CONTRACTS.md#task-factorytaskssave-001yaml). Factory verifies returned provenance against the selected canonical source identities and blocks with `RETRIEVAL_SCOPE_WIDENED` if CTX returns an out-of-scope original; it does not silently discard that excerpt and report success. Canonical required documents are read from their original files, checked against root containment, and admitted before CTX excerpts; declaring a source required never substitutes an excerpt for the exact original. The adapter records CTX generation, file and line provenance, and available hashes. On stale, missing, or insufficient retrieval, it returns `CONTEXT_BLOCKED` with an actionable diagnostic. It does not fill the pack with guessed material or hide post-retrieval excerpts to improve a relevance score. Default total task-context budgets (including contract and mandatory material) are 4,000 tokens for bounded, 8,000 for normal, and 15,000 for critical; these are design defaults to measure, not promises that CTX necessarily returns that many tokens. Reserve space for task contract and required sources before calling CTX with the remainder. The adapter enforces a byte ceiling as well as CTX's token budget, deduplicates overlapping excerpts, and never truncates an authoritative requirement mid-sentence. The pack is written atomically to `.factory/state/context/TASK-ID.md` with a sidecar receipt containing contract digest, source digests, CTX generation, timestamp, and retrieval mode. A worker verifies these before trusting the pack; if changed, regenerate it. diff --git a/BUILD_PLAN.md b/BUILD_PLAN.md index 034adb6..44d9035 100644 --- a/BUILD_PLAN.md +++ b/BUILD_PLAN.md @@ -89,7 +89,7 @@ - [x] Create tests/fixtures for critical auth isolation and normal API behavior, plus one simulated reviewer finding that causes repair and retest. - [x] Run an actual worker task with only task/pack/evidence sources in a fresh context; inspect whether it needs missing docs. -- [ ] **Gate:** P-06 and P-07 via the tracked ten-pack, protocol-negative/fresh-worker, and P-07 replays in `docs/probes/CP-06.md`; no change to Firstmate routing or shipping config by Factory. The PR12 audit's C1 malformed-output-plus-cleanup finding is repaired at the supervisor audit boundary with propagated replay diagnostics; exact-final-head proofs and another independent review remain required. DUMMY SDK controls are not live semantic acceptance, and strict-agent PARTIAL retrieval is not COMPLETE. +- [ ] **Gate:** P-06 and P-07 via the tracked replays in [docs/probes/CP-06.md](docs/probes/CP-06.md), which owns proof interpretation and residual limits; exact-final-head proofs and independent review remain required. No change to Firstmate routing or shipping config by Factory. ## CP-07 — Thin Pi command and status convenience diff --git a/README.md b/README.md index 453002a..36de660 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Factory: build package -**Status:** implementation specification, 26 September 2026. No Factory software is claimed to exist yet. +**Status:** development CLI with CP-06 corrections in progress. Product completion requires the independent gates in [COMPLETION.md](COMPLETION.md); checkpoint state is owned by [BUILD_PLAN.md](BUILD_PLAN.md). Factory is a thin layer for turning a software product brief into independently verifiable work for an existing Pi + Firstmate setup. Its purpose is to minimize the intelligence, context, and human attention required to transform intent into verified software. @@ -14,11 +14,21 @@ Factory is a thin layer for turning a software product brief into independently Read [docs/CONTRACTS.md](docs/CONTRACTS.md) when implementing validation, [docs/INTEGRATIONS.md](docs/INTEGRATIONS.md) when touching CTX/Pi/Firstmate, [docs/WORKFLOW.md](docs/WORKFLOW.md) when designing the Factory skill, and [docs/RESEARCH.md](docs/RESEARCH.md) for sources and decisions. [docs/BOOTSTRAP_PROMPTS.md](docs/BOOTSTRAP_PROMPTS.md) contains copyable Pi instructions for the initial checkpoints. -## Repository bootstrap +## Common usage -Create an empty Git repository for Factory and copy this directory's contents to its root. Do not copy an archived CTX repository or a Firstmate home into it. Review the files, resolve only genuine product decisions, then use your existing `/CTX startup` workflow to index the authoritative Markdown. Begin CP-00 from [BUILD_PLAN.md](BUILD_PLAN.md). The current workspace contains documents only; commands described here are target behavior, not commands that already run. +From this source checkout: -The baseline interaction is your current research → project truth → checkpoint → worker → independent review process. Once validation and task publication work, use Factory's own backlog to build later checkpoints. Do not assert that dogfooding happened until a real Firstmate worker receives, executes, and closes a Factory task. +```sh +npm ci +npm run build +node dist/src/cli.js --help +node dist/src/cli.js validate --root --json +node dist/src/cli.js context --root --json +``` + +Prepare reviewed contracts and a deliberate offline CTX authority set first; see [docs/CONTRACTS.md](docs/CONTRACTS.md) and the [Factory skill](skills/factory/SKILL.md) for the workflow and named-home publication safeguards. Do not copy an archived CTX repository or a Firstmate home into a target project. Factory does not dispatch workers or choose delivery policy. + +CP-06's development proof commands, exact SDK requirements, evidence interpretation, and platform limits are owned by [docs/probes/CP-06.md](docs/probes/CP-06.md). A DUMMY proof or closed backlog is not product completion. ## Document authority diff --git a/docs/CONTRACTS.md b/docs/CONTRACTS.md index c8dc595..840f463 100644 --- a/docs/CONTRACTS.md +++ b/docs/CONTRACTS.md @@ -80,7 +80,7 @@ evidence: delivery: project-default ``` -`depends_on` names local task IDs, is acyclic, and cannot name self. `advances` names completion IDs and may be empty only for a task explicitly marked `kind: enabling`; enabling tasks still have testable outcomes. `complexity` is `bounded | normal | critical`; `risk` is `bounded | normal | critical` and controls review policy, not a specific model name. `context.required` is the reviewed task-local authority selection: Factory reads every listed original exactly and passes the same paths as CTX document filters. An empty list explicitly means that no task-specific original is needed; Factory still filters CTX to the project's mandatory `documents.required` authority and never treats the empty selection as unrestricted corpus access. Include each known source that can decide an acceptance or risk condition; do not add broad directories or unrelated documents as insurance. A worker that discovers an omitted decisive source names it, performs one targeted original read, and proposes that exact path for the contract before regeneration. `delivery` defaults to `project-default` and cannot override the actual Firstmate project mode; if the project mode conflicts with a task request, validation blocks publication. All required evidence categories are verified by evidence receipts rather than assumed from task closure. +`depends_on` names local task IDs, is acyclic, and cannot name self. `advances` names completion IDs and may be empty only for a task explicitly marked `kind: enabling`; enabling tasks still have testable outcomes. `complexity` is `bounded | normal | critical`; `risk` is `bounded | normal | critical` and controls review policy, not a specific model name. `context.required` is the reviewed task-local authority selection: Factory reads every listed original exactly and passes the same paths as CTX document filters. An empty list explicitly means that no task-specific original is needed; Factory still filters CTX to the project's mandatory `documents.required` authority and never treats the empty selection as unrestricted corpus access. Include each known source that can decide an acceptance or risk condition; do not add broad directories or unrelated documents as insurance. The omitted-source correction procedure is owned by [WORKFLOW.md](WORKFLOW.md#task-local-worker-brief). `delivery` defaults to `project-default` and cannot override the actual Firstmate project mode; if the project mode conflicts with a task request, validation blocks publication. All required evidence categories are verified by evidence receipts rather than assumed from task closure. ## Task receipt: `.factory/state/evidence/SAVE-001.json` diff --git a/docs/INTEGRATIONS.md b/docs/INTEGRATIONS.md index d4bbb3b..acbeee0 100644 --- a/docs/INTEGRATIONS.md +++ b/docs/INTEGRATIONS.md @@ -8,7 +8,7 @@ The user's archived CTX README describes a Python 3.12+ local CLI, source-proven CP-00 probes: detect `ctx`; create a disposable Git fixture with two normative Markdown docs; initialize and add docs according to local `ctx --help`; index with `--no-embeddings` so the test does not download anything; run status, offline doctor, pack, and exact lines; record actual JSON fields and failure codes. Confirm stale-source detection by changing a source after indexing. A user's globally installed model may then be tested separately. Do not index the Factory repo until its initial documents are approved. -Context policy: required files are read exactly. The installed `ctx pack` accepts repeated `--document` filters; Factory supplies the task's reviewed `context.required` paths so retrieval ranks only within task-local authority while exact originals remain in the pack. If that valid selection is empty, Factory supplies the project's mandatory `documents.required` paths; it never sends an unfiltered pack request. Changed docs invalidate a pack; unavailable local embedding may allow the explicitly reported lexical mode if it retrieves sufficient authority. Query text is data, never shell code. `ctx pack` is one context source, not a substitute for Git, `rg`, or project checks, and Factory does not post-filter CTX output to falsify provenance. +The installed `ctx pack` accepts repeated `--document` filters. Factory's selection contract is owned by [CONTRACTS.md](CONTRACTS.md#task-factorytaskssave-001yaml); provenance verification and blocking behavior are owned by [ARCHITECTURE.md](../ARCHITECTURE.md#ctx-context-pack). Query text is data, never shell code. `ctx pack` is one context source, not a substitute for Git, `rg`, or project checks. ## Pi diff --git a/docs/WORKFLOW.md b/docs/WORKFLOW.md index f387a65..322b3c0 100644 --- a/docs/WORKFLOW.md +++ b/docs/WORKFLOW.md @@ -9,7 +9,7 @@ This document specifies the methodology. CP-04 turns it into a concise, discover 3. **Project truth:** author a concise `PROJECT.md` with goal, audience, user journeys, non-goals, constraints, and V1 limits. Draft `ARCHITECTURE.md` with component boundaries, data model, trust boundary, deployment shape, and failure handling. Keep technical choices proportionate to the first release. 4. **Independent oracle:** in a separate reasoning pass, author `.factory/completion.yaml` from product intent and risks before decomposing into tasks. Specify observable end-to-end behavior, isolation/security invariants, recovery, and any performance thresholds the project actually needs. Include one negative test for important trust boundaries. Record the current Git SHA of this first oracle so later changes are visible. 5. **Slices and DAG:** choose the first vertical slice that a user can exercise; add a minimal foundation only where it unlocks slices. Tasks should have one independently reviewable outcome, bounded scope, testable acceptance IDs, expected evidence, and dependencies. Prefer a serial backbone of slices with at most three genuinely independent ready tasks. Do not turn one feature into separate backend/frontend/UI megatasks that integrate only at the end. -6. **Validate and retrieve:** use `factory validate`; repair schema/graph gaps; declare each known decisive original in the task's `context.required` selection, then generate its context pack via CTX. Factory constrains CTX to those task-local documents while retaining exact global and required originals; an explicitly empty task-local selection is constrained to mandatory global originals rather than becoming an unscoped corpus query. Inspect the first few packs for a missing decisive source and irrelevant bulk; correct an omission with one targeted exact read and the narrowest contract update, not a broad corpus expansion or a post-retrieval provenance filter. +6. **Validate and retrieve:** use `factory validate`; repair schema/graph gaps; select authority according to [CONTRACTS.md](CONTRACTS.md#task-factorytaskssave-001yaml), then generate a context pack via CTX. Inspect the first few packs for a missing decisive source and irrelevant bulk. Use the targeted correction procedure below rather than widening the corpus or hiding provenance. 7. **Preview and publish:** run `factory sync --dry-run --home `, inspect which items will be created and which are blocked, then explicitly apply to the selected home. The engineer's direction to execute work authorizes normal publication into the intended Firstmate home; ask only if the home or project identity is genuinely ambiguous. 8. **Execute with Firstmate:** Firstmate chooses ready work and workers through its existing dispatch policy. Each worker receives the task contract, relevant source pack, and evidence expectations; worker tests and repairs locally. Firstmate handles the actual delivery path and approvals required by its configured project mode. 9. **Verify per task and per project:** run deterministic gates first; semantic review receives task, diff, architecture, and observed evidence rather than the worker's full transcript. Repair specific findings and rerun changed checks. Then execute the original project oracle at the release candidate SHA. Failed project behavior creates a reviewed follow-up task. The status report must show backlog and product completion separately. diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index bfb5495..3100ec1 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,7 +21,7 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created descriptor paths, and the retained helper/probe identities must still match their anchored leaves before execution and hashing. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup consumes retained source and target parent descriptors plus the pinned source leaf; replacement of the prepared source, target, parent, helper, or output child before mount, exec, write, or hash is rejected by the executable transition regressions without touching external DUMMY sentinels. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup consumes retained source and target parent descriptors plus the pinned source leaf; replacement of the prepared source, target, parent, or output child before mount, write, or hash is rejected by the executable transition regressions without touching external DUMMY sentinels. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. @@ -115,6 +115,17 @@ git diff --check The live worker command is omitted rather than passed when DUMMY protection, exact SDK selection, or non-refresh credential-validity preflight fails. +`test/cp06-selected-boundaries.test.mjs` owns the bounded helper-execution, +serializer-import, and nested-reset regressions. Its compiled-helper control +executes through retained read-only descriptors even after replacing the named +leaves, avoiding `ETXTBSY` without selecting replacement bytes. Importing +`createWorkerBlockedStatus` from `scripts/replay-cp06-worker.mjs` does not create +or reset output: only the executable's `main` initializes its worker directory. +`openAnchoredDirectory(..., { reset: true })` traverses parent components with +no-follow directory descriptors and resets only the requested leaf; successive +`proofs/` resets preserve sibling evidence. These are DUMMY development +controls, not final-head kernel or semantic acceptance. + The exact runtime manifests are: ```text diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index b61950b..a3ffd14 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -467,7 +467,10 @@ export function runDummySdkProof({ throw unsupported(`DUMMY SDK proof failed: ${code}`); } const trustedDependency = expectedSdkDependency ?? resolvePinnedPiAi(piRoot).provenance; - return validateDummySdkProof(parseCompactJson(result.stdout, "DUMMY SDK proof"), trustedDependency); + return validateDummySdkProof( + parseCompactJson(result.stdout, "DUMMY SDK proof"), + trustedDependency, + ); } function validateDummySdkProof(proof, trustedDependency) { diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index 65a5a38..ee5813b 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -91,11 +91,12 @@ sdk.ModelRuntime.create = async function (options) { ? input.symlink_path : input.original_path; const jsonResponse = JSON.stringify(response); - const responseText = scenario === "extra-before-fence" - ? `Creation verified and passed\n\x60\x60\x60json\n${jsonResponse}\n\x60\x60\x60` - : scenario === "fenced-valid" - ? `\x60\x60\x60json\n${jsonResponse}\n\x60\x60\x60` - : jsonResponse; + const responseText = + scenario === "extra-before-fence" + ? `Creation verified and passed\n\x60\x60\x60json\n${jsonResponse}\n\x60\x60\x60` + : scenario === "fenced-valid" + ? `\x60\x60\x60json\n${jsonResponse}\n\x60\x60\x60` + : jsonResponse; faux.setResponses([ ai.fauxAssistantMessage( ai.fauxToolCall("read", { path: readPath }, { id: "DUMMY-actual-worker-read" }), diff --git a/scripts/cp06-guarded-read.mjs b/scripts/cp06-guarded-read.mjs index be53ba6..5b6df3b 100644 --- a/scripts/cp06-guarded-read.mjs +++ b/scripts/cp06-guarded-read.mjs @@ -89,9 +89,7 @@ export function assertAllowedRead(args, expectedOriginalPath, completeLineLimit if (hasOffset !== hasLimit) throw new Error("read denied: incomplete explicit range"); if ( hasOffset && - (args.offset !== 1 || - !Number.isSafeInteger(args.limit) || - args.limit < completeLineLimit) + (args.offset !== 1 || !Number.isSafeInteger(args.limit) || args.limit < completeLineLimit) ) { throw new Error("read denied: explicit range must cover the complete original"); } diff --git a/scripts/cp06-package-integrity.mjs b/scripts/cp06-package-integrity.mjs index 6198a0f..542cdca 100644 --- a/scripts/cp06-package-integrity.mjs +++ b/scripts/cp06-package-integrity.mjs @@ -4,7 +4,10 @@ import { join, relative, resolve } from "node:path"; import { fileURLToPath } from "node:url"; const pins = JSON.parse( - readFileSync(fileURLToPath(new URL("../.factory/cp06-sdk-integrity.json", import.meta.url)), "utf8"), + readFileSync( + fileURLToPath(new URL("../.factory/cp06-sdk-integrity.json", import.meta.url)), + "utf8", + ), ).packages; export function verifyPinnedPackage(root, name) { diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs index 3ee6524..555be56 100644 --- a/scripts/cp06-sdk-worker.mjs +++ b/scripts/cp06-sdk-worker.mjs @@ -16,11 +16,7 @@ try { const credentialTarget = resolve(credentialTargetArg); const input = JSON.parse(await readFile(resolve(inputPathArg), "utf8")); validateInput(input); - const paths = canonicalWorkerPaths( - input, - credentialTarget, - process.env.CP06_CREDENTIAL_SOURCE, - ); + const paths = canonicalWorkerPaths(input, credentialTarget, process.env.CP06_CREDENTIAL_SOURCE); const pi = resolvePiPackage(); const sdk = await import(pathToFileURL(join(pi.root, "dist/index.js"))); assertSdkSurface(sdk); diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index ece7eb7..76ef066 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -86,7 +86,9 @@ const cases = await withCleanup( assert.deepEqual(await readdir(temporary), [], `${scenario}: evaluation home leaked`); if (scenario !== "missing-source-setup") assert.equal(await readFile(source, "utf8"), sourceBytes); - await assert.rejects(readFile(join(proofOutputPath, "worker/evidence.json")), { code: "ENOENT" }); + await assert.rejects(readFile(join(proofOutputPath, "worker/evidence.json")), { + code: "ENOENT", + }); proofOutputDirectory.close(); cases.push({ scenario, diff --git a/skills/factory/SKILL.md b/skills/factory/SKILL.md index 0f790f3..5ce0348 100644 --- a/skills/factory/SKILL.md +++ b/skills/factory/SKILL.md @@ -37,19 +37,13 @@ For each ready task, run: context --root --json ``` -Before generation, make `context.required` the reviewed task-local source selection: include every known original that can decide an acceptance or risk condition, but no unrelated documents as insurance. Factory reads those originals exactly and constrains CTX retrieval to the same document paths. Inspect the pack and receipt for the exact task contract, required originals, source and contract digests, CTX generation/retrieval mode, relevance, and token/byte bounds. Regenerate after any source, contract, or index change. Never place secrets, credentials, source archives, or agent transcripts in a pack. +Select `context.required` according to [`CONTRACTS.md`](../../docs/CONTRACTS.md#task-factorytaskssave-001yaml), including its empty-selection rule. Inspect the pack and receipt for source and contract digests, CTX generation/retrieval mode, relevance, and token/byte bounds. Regenerate after any source, contract, or index change. Never place secrets, credentials, source archives, or agent transcripts in a pack. -Give a fresh worker only the task-local brief, exact contract, bounded pack, and existing evidence—not a previous agent transcript. If a decisive source is absent, have the worker name it and make one targeted exact-original read, verify provenance, record the corrected assumption, and add that exact path to `context.required` before regeneration when it is durable authority. Do not widen every pack, hide raw retrieved excerpts, or omit a critical constraint to make retrieval appear complete. +Use [`WORKFLOW.md`'s task-local worker brief and omitted-source procedure](../../docs/WORKFLOW.md#task-local-worker-brief) for fresh handoffs and targeted correction reads. ## Risk and task review -Classify by failure consequence, not diff size, and use the higher class when observed risk exceeds the contract: - -- **bounded:** ordinary project checks and Firstmate's standard delivery policy; -- **normal:** targeted integration/e2e evidence, then a fresh semantic review when feasible; -- **critical:** explicit trust-boundary constraints, isolated negative tests, independent review, and human resolution of consequential policy. Use disposable identities only—never real accounts or secrets. - -Before semantic review, record the exact deterministic commands, exits, artifacts, and tested SHA. Give the reviewer the task contract, focused diff, relevant architecture and exact-original correction reads, and observed evidence—not the worker transcript. A finding must cite an acceptance ID, demonstrated scenario, or concrete risk. Repair it and rerun the changed behavior plus affected checks, record an evidence-backed disagreement, or block; unresolved critical findings cannot pass. +Follow the risk classes, check ordering, review packet, and finding dispositions in [`WORKFLOW.md`](../../docs/WORKFLOW.md#risk-and-review). Use disposable identities for critical negative tests, never real accounts or secrets; unresolved critical findings block delivery. This is task-local evidence policy, not another shipping gate. Do not call, embed, or rerun no-mistakes from Factory, and do not alter Firstmate routing or approvals. Firstmate's configured delivery invokes its normal gate once; Factory receipts may reference those resulting artifacts. From 75620d08135e437c9fa01de6925f48bbca1b3745 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Sat, 3 Oct 2026 18:30:47 +0000 Subject: [PATCH 33/45] no-mistakes(lint): Format CP06 regression tests --- test/cp06-guarded-read.test.mjs | 5 +--- test/cp06-pi-dependency.test.mjs | 48 +++++++++++++++++++------------- 2 files changed, 30 insertions(+), 23 deletions(-) diff --git a/test/cp06-guarded-read.test.mjs b/test/cp06-guarded-read.test.mjs index daf9170..c8acd73 100644 --- a/test/cp06-guarded-read.test.mjs +++ b/test/cp06-guarded-read.test.mjs @@ -3,10 +3,7 @@ import { mkdir, mkdtemp, rm, symlink, unlink, writeFile } from "node:fs/promises import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; -import { - canonicalWorkerPaths, - createGuardedReadTool, -} from "../scripts/cp06-guarded-read.mjs"; +import { canonicalWorkerPaths, createGuardedReadTool } from "../scripts/cp06-guarded-read.mjs"; test("guarded read denies noncanonical and unauthorized paths before tool access", async () => { const root = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-read-")); diff --git a/test/cp06-pi-dependency.test.mjs b/test/cp06-pi-dependency.test.mjs index ab833ee..aca6073 100644 --- a/test/cp06-pi-dependency.test.mjs +++ b/test/cp06-pi-dependency.test.mjs @@ -13,12 +13,7 @@ const resolverUrl = pathToFileURL(resolve("scripts/cp06-pi-dependency.mjs")).hre test("Pi selection requires the explicit task-local package and executable paths", async () => { const projectRoot = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-pi-install-")); const installRoot = join(projectRoot, ".factory/state/cp06-sdk"); - const packageRoot = join( - installRoot, - "node_modules", - "@earendil-works", - "pi-coding-agent", - ); + const packageRoot = join(installRoot, "node_modules", "@earendil-works", "pi-coding-agent"); const entry = join(packageRoot, "dist/cli.js"); const bin = join(installRoot, "node_modules/.bin/pi"); try { @@ -37,15 +32,19 @@ test("Pi selection requires the explicit task-local package and executable paths await chmod(entry, 0o755); await symlink("../@earendil-works/pi-coding-agent/dist/cli.js", bin); const invocations = []; - assert.throws(() => resolvePinnedPiInstall({ - projectRoot, - packageRoot, - executable: bin, - spawnSyncImpl(command, args) { - invocations.push([command, ...args]); - return { status: 0, stdout: "0.85.1\n", stderr: "" }; - }, - }), /pinned artifact/); + assert.throws( + () => + resolvePinnedPiInstall({ + projectRoot, + packageRoot, + executable: bin, + spawnSyncImpl(command, args) { + invocations.push([command, ...args]); + return { status: 0, stdout: "0.85.1\n", stderr: "" }; + }, + }), + /pinned artifact/, + ); assert.deepEqual(invocations, []); for (const substitution of [ @@ -78,11 +77,19 @@ test("exact artifact succeeds and same-version selected entry substitutions fail await cp(join(installed, name), join(packages, name), { recursive: true }); } const binPath = JSON.parse(await readFile(join(packageRoot, "package.json"))).bin; - await symlink(join(packageRoot, typeof binPath === "string" ? binPath : binPath.pi), executable); + await symlink( + join(packageRoot, typeof binPath === "string" ? binPath : binPath.pi), + executable, + ); let launches = 0; const options = { - projectRoot, packageRoot, executable, - spawnSyncImpl() { launches++; return { status: 0, stdout: "0.85.1\n" }; }, + projectRoot, + packageRoot, + executable, + spawnSyncImpl() { + launches++; + return { status: 0, stdout: "0.85.1\n" }; + }, }; const positive = resolvePinnedPiInstall(options); assert.equal(positive.provenance.dependency.version, "0.85.1"); @@ -154,7 +161,10 @@ for (const scenario of [ assert.match(output.provenance.package_sha256, /^[a-f0-9]{64}$/); assert.match(output.provenance.entry_sha256, /^[a-f0-9]{64}$/); } else { - assert.match(result.stderr, scenario.version === "0.85.1" ? /pinned artifact/ : /not exact version 0\.85\.1/); + assert.match( + result.stderr, + scenario.version === "0.85.1" ? /pinned artifact/ : /not exact version 0\.85\.1/, + ); } } finally { await rm(directory, { recursive: true, force: true }); From c12d24d4c424af62caa90469f9066af0baee9a39 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 5 Oct 2026 09:02:15 +0000 Subject: [PATCH 34/45] no-mistakes(review): Fix six CP06 findings; mount, CLI, kernel gaps remain --- docs/probes/CP-06.md | 50 +++++--- scripts/cp06-auth-security.mjs | 138 ++++++++++++++++------ scripts/cp06-dummy-outcome-proof.mjs | 9 ++ scripts/cp06-dummy-worker-hook.mjs | 19 ++- scripts/cp06-guarded-read.mjs | 31 ++++- scripts/cp06-namespace-supervisor.mjs | 53 +++++++++ scripts/cp06-probe-fixture.mjs | 39 +++++++ scripts/cp06-sdk-worker.mjs | 24 ++-- scripts/cp06-worker-audit.mjs | 107 +++++++++++++---- scripts/prove-cp06-auth-security.mjs | 11 +- scripts/prove-cp06-worker-cleanup.mjs | 35 ++++-- scripts/replay-cp06-worker.mjs | 20 +++- test/cp06-guarded-read.test.mjs | 31 ++++- test/cp06-proof-boundary.test.mjs | 159 +++++++++++++++++++++++--- test/cp06-worker-audit.test.mjs | 61 +++++++++- test/cp06-worker-lifecycle.test.mjs | 64 ++++++++++- 16 files changed, 713 insertions(+), 138 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 3100ec1..e897e32 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -19,9 +19,9 @@ daemon. ## Read-only Pi authentication correction -The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then proves the actual evaluated child has empty capability sets, locked root securebits, `no_new_privs`, and inherited seccomp denial of remount/unmount, alternate mount APIs, namespace creation/entry, handle-based inode reopening, UID change, ptrace/process-memory access, and child-process variants. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. +The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then observes that the actual evaluated child has empty capability sets, `no_new_privs`, and seccomp filter mode, and that its attempted remount/unmount, direct `setns`/`unshare`/`clone`/`clone3`/`mount`/`umount2` syscalls, nested namespace creation, `nsenter`, and UID change fail. The seccomp helper also installs denials for alternate mount APIs, handle-based inode reopening, ptrace and process-memory access, and the evaluated child is launched with locked root securebits requested through `setpriv`; the probe does not yet execute ptrace/process-memory or inherited-subprocess escape attempts or read back securebits, so those outcomes are configured but unproven (open finding `CP06-KERNEL-PROOF-INCOMPLETE`). On the WSL2 6.18 host used for the 2026-10-05 review round, the hardened probe blocked at credential mount setup (`CP06_ISOLATION_SETUP_FAILED`, mount identity check), so the full DUMMY gate did not pass there. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. The bind mounts are still issued through util-linux `mount` with `/proc//fd` anchor paths, which util-linux canonicalizes; a native descriptor-to-descriptor mount handoff is not implemented yet (open finding `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup consumes retained source and target parent descriptors plus the pinned source leaf; replacement of the prepared source, target, parent, or output child before mount, write, or hash is rejected by the executable transition regressions without touching external DUMMY sentinels. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof's DUMMY evaluation home and credential scratch directories are still created by path under the pinned proof directory. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. @@ -218,7 +218,10 @@ reports `MISSING_SOURCE`, performs exactly one read of the supplied creation ownership and the ignored request-owner rule, and identifies the missing creation-time integration evidence. -The exact original is opened with `O_NOFOLLOW`, identity-checked, and snapshotted +The exact original is opened by walking every ancestor from `/` with no-follow +directory descriptors and opening the leaf with `O_NOFOLLOW` relative to the +pinned parent, so a symlink substituted for any ancestor after canonicalization +is rejected before bytes are read; it is then identity-checked and snapshotted before provider execution. At the actual tool boundary, the lexical path and range are checked and the named path must still identify the pinned regular file; Pi's SDK read implementation receives only the immutable snapshot, so a @@ -238,9 +241,10 @@ The genuine Pi SDK process runs in a private Linux user/mount namespace only after the complete DUMMY auth and OS preflight passes. A supervisor pins the regular source inode with `O_NOFOLLOW`, creates distinct source/target bind mounts with read-only VFS flags, and closes the pin before launch. The evaluated -child has all capability sets empty, locked securebits, `no_new_privs`, and an -inherited seccomp filter blocking mount and namespace escapes, alternate inode -handles, ptrace/process-memory access, and namespace-bearing child calls. Its +child is observed with all capability sets empty, `no_new_privs`, and an +inherited seccomp filter whose mount and namespace denials are exercised; its +alternate-inode-handle and ptrace/process-memory denials and the requested +locked securebits are configured but not yet exercised or observed. Its credential store rejects refresh/mutation callbacks before entry; expired or near-expiry OAuth is therefore blocked before a provider request or persistence. A missing credential source is a sanitized `CP06_CREDENTIAL_SOURCE_MISSING` @@ -287,8 +291,20 @@ remain mandatory. `npm run proof:cp06:auth-security` additionally runs the actual SDK worker with a local faux provider in the inherited capability-free/seccomp context. It records -a valid structured positive, missing/empty/contradictory gap negatives, expired -and near-expiry preflight denial, and an SDK timeout. Every case is explicitly +a valid structured positive, missing/empty/contradictory gap negatives, an +earlier assistant message carrying text beside its read call, a final message +carrying a non-text channel, an unexpected tool named with a DUMMY sentinel +(whose name must not reach stdout or stderr), expired and near-expiry preflight +denial, and an SDK timeout. The worker validates every assistant message: earlier +messages may carry only tool calls, thinking blocks, and whitespace text with +`stopReason: "toolUse"`, and the final message only text and thinking blocks with +`stopReason: "stop"`. SDK-worker stderr carries only allowlisted fixed audit +reasons. The consumer validates the exact nested `read_audit` schema (one +correlated exact-original read, matching digests, ordered event indexes, fixed +tool-call ID), exact per-scenario exit, `runtime_create`, and `to_auth` counts +(valid runs observe three `toAuth` calls: the preflight plus two model turns), +and exact credential-store `modify`/`delete` denial counts (one modify denial for +expired and near-expiry auth, none otherwise). Every case is explicitly `fixture_origin: true`, `semantic_acceptance: false`; these are protocol and infrastructure proofs, never live-model acceptance. Default AuthStorage, refresh, and network entry are instrumented and must remain zero. The full read still @@ -303,12 +319,18 @@ output audit, and cleanup causes remain distinct without retaining child stderr or arbitrary exception payloads. PR12 still parsed successful-child output after cleanup, so malformed output plus cleanup failure lost the parse/audit cause. The correction keeps child status, source-integrity verification, compact JSON -parsing, and evidence assembly inside the cleanup action. The shipped -actual-boundary regression separately triggers successful-child malformed -output and an independent awaited cleanup error, then verifies that the -supervisor JSON, namespace wrapper, and replay `blocked.json` retain both cause -identities; child-exit+cleanup, cleanup-only, audit-only, and success remain -separate controls. `npm run proof:cp06:worker-cleanup` executes the full runner +parsing, the mode-specific child semantic contract (for workers: exact output +keys, `result: "pass"`, Pi `0.85.1`, in-memory read-only session, unrefreshed +OAuth, and one exact-original read), and evidence assembly inside the cleanup +action, so valid JSON with an invalid contract plus a cleanup failure retains +both the audit and cleanup causes. These compound cases (malformed output and +semantic audit each with cleanup, child-exit+cleanup, audit-only, cleanup-only, +success) are regression-tested at the in-process supervisor interface with +injected mount and spawn stubs, and the serialized record is checked through +`namespaceWorkerError` and `createWorkerBlockedStatus`. Executable CLI +regressions that assert serialized stderr, exit status, and a persisted +`blocked.json` for each compound case do not exist yet (open finding +`CP06-CLI-COMPOUND-REGRESSIONS-ABSENT`). `npm run proof:cp06:worker-cleanup` executes the full runner with task-local expired/near-expiry DUMMY auth and missing-source setup, asserts the blocked result, unchanged DUMMY source, absence of a pass manifest, and removal of the evaluation home. Unit tests also inject setup, launch, timeout, diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index a3ffd14..38eb0b3 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -473,7 +473,7 @@ export function runDummySdkProof({ ); } -function validateDummySdkProof(proof, trustedDependency) { +export function validateDummySdkProof(proof, trustedDependency) { assertExactKeys(proof, [ "actual_sdk_outcomes", "auth_cases", @@ -500,9 +500,9 @@ function validateDummySdkProof(proof, trustedDependency) { function validateAuthCases(cases) { assert(Array.isArray(cases) && cases.length === 3, "DUMMY SDK auth cases incomplete"); const expected = new Map([ - ["expired", { result: "blocked", toAuth: 0 }], - ["near_expiry", { result: "blocked", toAuth: 0 }], - ["unexpired", { result: "resolved", toAuth: 1 }], + ["expired", { result: "blocked", toAuth: 0, modifyDenials: 1 }], + ["near_expiry", { result: "blocked", toAuth: 0, modifyDenials: 1 }], + ["unexpired", { result: "resolved", toAuth: 1, modifyDenials: 0 }], ]); const seen = new Set(); for (const entry of cases) { @@ -523,7 +523,12 @@ function validateAuthCases(cases) { assert(entry.to_auth_calls === expectation.toAuth, "DUMMY SDK toAuth count mismatch"); assert(entry.persistence_operations === 0, "DUMMY SDK persistence occurred"); assert(entry.source_unchanged === true, "DUMMY SDK auth source changed"); - validateCredentialAudit(entry.credential_store); + validateCredentialAudit(entry.credential_store, { + reads: 1, + lists: 0, + modify_denials: expectation.modifyDenials, + delete_denials: 0, + }); } } @@ -548,9 +553,11 @@ function validateSdkWorker(worker) { ); assert(isSha256(worker.event_sha256), "DUMMY SDK event digest missing"); assert(worker.refresh_callbacks === 0, "DUMMY SDK worker refresh callback ran"); - validateCredentialAudit(worker.credential_store); - assert(worker.read_audit?.exact_original === true, "DUMMY SDK exact original read missing"); - assert(worker.read_audit?.read_count === 1, "DUMMY SDK read count mismatch"); + validateCredentialAudit(worker.credential_store, { modify_denials: 0, delete_denials: 0 }); + validateReadAudit(worker.read_audit, { + toolCallId: "DUMMY-read", + originalSuffix: "/DUMMY-original.md", + }); } function validateSdkDependency(dependency, trustedDependency) { @@ -590,20 +597,23 @@ function validateOutcomeProof(outcomes, trustedDependency) { assert(outcomes.semantic_acceptance === false, "DUMMY outcome semantic acceptance was claimed"); assert(Array.isArray(outcomes.cases), "DUMMY outcome cases missing"); const expected = new Map([ - ["valid", 0], - ["fenced-valid", 0], - ["extra-before-fence", 75], - ["empty-gap", 75], - ["missing-gap", 75], - ["contradictory-gap", 75], - ["extra-evidence", 75], - ["constraint-pass", 75], - ["missing-reads", 75], - ["credential-read", 75], - ["symlink-read", 75], - ["expired", 75], - ["near-expiry", 75], - ["timeout", 70], + ["valid", [0, 1, 3]], + ["fenced-valid", [0, 1, 3]], + ["extra-before-fence", [75, 1, 3]], + ["empty-gap", [75, 1, 3]], + ["missing-gap", [75, 1, 3]], + ["contradictory-gap", [75, 1, 3]], + ["extra-evidence", [75, 1, 3]], + ["constraint-pass", [75, 1, 3]], + ["missing-reads", [75, 1, 3]], + ["credential-read", [75, 1, 3]], + ["symlink-read", [75, 1, 3]], + ["earlier-assistant-claim", [75, 1, 3]], + ["final-extra-channel", [75, 1, 4]], + ["unexpected-tool-secret", [75, 1, 3]], + ["expired", [75, 0, 0]], + ["near-expiry", [75, 0, 0]], + ["timeout", [70, 1, 2]], ]); const seen = new Set(); for (const entry of outcomes.cases) { @@ -611,7 +621,8 @@ function validateOutcomeProof(outcomes, trustedDependency) { entry !== null && typeof entry === "object" && !Array.isArray(entry), "DUMMY outcome case malformed", ); - const expectedExit = expected.get(entry.scenario); + const [expectedExit, expectedRuntimeCreate, expectedToAuth] = + expected.get(entry.scenario) ?? []; assert( expectedExit !== undefined && !seen.has(entry.scenario), "DUMMY outcome scenario mismatch", @@ -646,29 +657,84 @@ function validateOutcomeProof(outcomes, trustedDependency) { entry.default_storage === 0 && entry.network === 0 && entry.refresh === 0, "DUMMY outcome side effect occurred", ); - if (["expired", "near-expiry"].includes(entry.scenario)) { - assert( - entry.runtime_create === 0 && entry.to_auth === 0, - "DUMMY expired auth reached runtime creation", - ); - } + assert( + entry.runtime_create === expectedRuntimeCreate && entry.to_auth === expectedToAuth, + "DUMMY outcome runtime/auth count mismatch", + ); if (["valid", "fenced-valid"].includes(entry.scenario)) { - assert( - entry.runtime_create === 1 && entry.to_auth === 1, - "DUMMY valid auth did not run exactly once", - ); - assert(entry.read_audit?.exact_original === true, "DUMMY outcome exact read missing"); - assert(entry.read_audit?.read_count === 1, "DUMMY outcome read count mismatch"); + validateReadAudit(entry.read_audit, { + toolCallId: "DUMMY-actual-worker-read", + originalSuffix: "/docs/AUTH.md", + }); } } assert(seen.size === expected.size, "DUMMY outcome cases incomplete"); } -function validateCredentialAudit(audit) { +function validateCredentialAudit(audit, expected) { assertExactKeys(audit, ["delete_denials", "lists", "modify_denials", "reads"]); for (const key of ["delete_denials", "lists", "modify_denials", "reads"]) { assert(Number.isSafeInteger(audit[key]) && audit[key] >= 0, "DUMMY credential audit malformed"); + if (Object.hasOwn(expected, key)) { + assert(audit[key] === expected[key], `DUMMY credential audit ${key} mismatch`); + } } + assert(audit.reads >= 1, "DUMMY credential audit omitted the credential read"); +} + +function validateReadAudit(audit, { toolCallId, originalSuffix }) { + assertExactKeys(audit, [ + "completed_successfully", + "end_event_index", + "exact_original", + "expected_sha256", + "other_tool_calls", + "other_tool_completions", + "project_read_count", + "project_read_paths", + "read_count", + "rejected_read_paths", + "response_event_index", + "returned_bytes", + "returned_sha256", + "start_event_index", + "tool_call_id", + ]); + const isEmptyArray = (value) => Array.isArray(value) && value.length === 0; + const isIndex = (value) => Number.isSafeInteger(value) && value >= 0; + assert( + audit.read_count === 1 && + audit.project_read_count === 1 && + Array.isArray(audit.project_read_paths) && + audit.project_read_paths.length === 1 && + typeof audit.project_read_paths[0] === "string" && + audit.project_read_paths[0].startsWith("/") && + audit.project_read_paths[0].endsWith(originalSuffix) && + isEmptyArray(audit.rejected_read_paths) && + isEmptyArray(audit.other_tool_calls) && + isEmptyArray(audit.other_tool_completions), + "DUMMY read audit did not record exactly one original read", + ); + assert( + audit.completed_successfully === true && audit.exact_original === true, + "DUMMY exact original read missing", + ); + assert( + isSha256(audit.expected_sha256) && + audit.returned_sha256 === audit.expected_sha256 && + Number.isSafeInteger(audit.returned_bytes) && + audit.returned_bytes > 0, + "DUMMY read audit bytes mismatch", + ); + assert(audit.tool_call_id === toolCallId, "DUMMY read audit tool call mismatch"); + assert( + isIndex(audit.start_event_index) && + isIndex(audit.end_event_index) && + isIndex(audit.response_event_index) && + audit.start_event_index < audit.end_event_index && + audit.end_event_index < audit.response_event_index, + "DUMMY read audit event order mismatch", + ); } function assertExactKeys(value, keys) { diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index 7d81594..0428ad1 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -42,6 +42,9 @@ export async function proveDummyWorkerOutcome({ piRoot }) { ["missing-reads", 75], ["credential-read", 75], ["symlink-read", 75], + ["earlier-assistant-claim", 75], + ["final-extra-channel", 75], + ["unexpected-tool-secret", 75], ["expired", 75], ["near-expiry", 75], ["timeout", 70], @@ -132,6 +135,12 @@ export async function proveDummyWorkerOutcome({ piRoot }) { assert.equal(result.stdout.includes("DUMMY-ACCESS"), false); assert.equal(result.stderr.includes("DUMMY-ACCESS"), false); } + if (["earlier-assistant-claim", "final-extra-channel"].includes(scenario)) + assert.match(result.stderr, /SDK assistant envelope rejected/); + if (scenario === "unexpected-tool-secret") { + assert.match(result.stderr, /SDK event audit failed: unexpected tool events\n$/); + assert.equal(`${result.stdout}${result.stderr}`.includes("DUMMY-SECRET"), false); + } if (scenario === "timeout") assert.match(result.stderr, /worker timed out and was aborted/); if (scenario === "extra-before-fence") assert.match(result.stderr, /SDK worker did not return the required JSON response/); diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index ee5813b..49702ae 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -97,12 +97,27 @@ sdk.ModelRuntime.create = async function (options) { : scenario === "fenced-valid" ? `\x60\x60\x60json\n${jsonResponse}\n\x60\x60\x60` : jsonResponse; + const readCall = ai.fauxToolCall( + scenario === "unexpected-tool-secret" ? "DUMMY-SECRET-TOOL" : "read", + { path: readPath }, + { id: "DUMMY-actual-worker-read" }, + ); faux.setResponses([ ai.fauxAssistantMessage( - ai.fauxToolCall("read", { path: readPath }, { id: "DUMMY-actual-worker-read" }), + scenario === "earlier-assistant-claim" + ? [ai.fauxText("Creation verified and passed"), readCall] + : readCall, { stopReason: "toolUse" }, ), - ai.fauxAssistantMessage(responseText, { stopReason: "stop" }), + ai.fauxAssistantMessage( + scenario === "final-extra-channel" + ? [ + ai.fauxText(responseText), + ai.fauxToolCall("read", { path: readPath }, { id: "DUMMY-final-extra" }), + ] + : responseText, + { stopReason: "stop" }, + ), ]); return runtime; }; diff --git a/scripts/cp06-guarded-read.mjs b/scripts/cp06-guarded-read.mjs index 5b6df3b..b10fd85 100644 --- a/scripts/cp06-guarded-read.mjs +++ b/scripts/cp06-guarded-read.mjs @@ -95,11 +95,8 @@ export function assertAllowedRead(args, expectedOriginalPath, completeLineLimit } } -function readPinnedRegularFile(path) { - const descriptor = openSync( - path, - constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, - ); +export function readPinnedRegularFile(path) { + const descriptor = openWithoutFollowingAncestors(path); try { const before = fstatSync(descriptor, { bigint: true }); const named = lstatSync(path, { bigint: true }); @@ -134,6 +131,30 @@ function readPinnedRegularFile(path) { } } +function openWithoutFollowingAncestors(path) { + if (typeof path !== "string" || !isAbsolute(path) || resolve(path) !== path) { + throw new Error("supplied original path must be absolute and normalized"); + } + const components = path.split("/").filter((component) => component.length > 0); + if (components.length === 0) throw new Error("supplied original path names no file"); + const directoryFlags = + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC; + let parentFd = openSync("/", directoryFlags); + try { + for (const component of components.slice(0, -1)) { + const nextFd = openSync(`/proc/self/fd/${parentFd}/${component}`, directoryFlags); + closeSync(parentFd); + parentFd = nextFd; + } + return openSync( + `/proc/self/fd/${parentFd}/${components.at(-1)}`, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); + } finally { + closeSync(parentFd); + } +} + function sameIdentity(expected, actual) { return ( expected.dev === actual.dev && diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 2e5c475..828ca06 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -143,6 +143,9 @@ export async function superviseCredentialChild(options, dependencies = {}) { } catch { throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); } + if (!validChildContract(mode, childEvidence)) { + throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); + } return { schema_version: 1, mode, @@ -159,6 +162,56 @@ export async function superviseCredentialChild(options, dependencies = {}) { ); } +const WORKER_CHILD_KEYS = [ + "credential_store", + "event_stream", + "model", + "oauth_preflight", + "pi_install", + "pi_version", + "provider", + "raw_transcript_retained", + "response", + "response_sha256", + "result", + "schema_version", + "session", +].join(","); + +function validChildContract(mode, child) { + if (!isPlainObject(child)) return false; + if (mode !== "worker") { + return ( + isPlainObject(child.identity) && + isPlainObject(child.identity.status) && + isPlainObject(child.source) && + typeof child.source.unchanged === "boolean" && + isPlainObject(child.direct_namespace_syscalls) + ); + } + return ( + Object.keys(child).sort().join(",") === WORKER_CHILD_KEYS && + child.schema_version === 1 && + child.result === "pass" && + child.pi_version === "0.85.1" && + child.raw_transcript_retained === false && + isPlainObject(child.pi_install) && + isPlainObject(child.credential_store) && + isPlainObject(child.response) && + typeof child.response_sha256 === "string" && + /^[0-9a-f]{64}$/u.test(child.response_sha256) && + child.oauth_preflight?.refreshed === false && + child.session?.in_memory === true && + JSON.stringify(child.session?.active_tools) === '["read"]' && + child.event_stream?.read_audit?.exact_original === true && + child.event_stream.read_audit.read_count === 1 + ); +} + +function isPlainObject(value) { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + function childCommand({ mode, source, diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index 5ce7144..ae27700 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -9,6 +9,7 @@ import { openSync, readSync, rmSync, + writeSync, } from "node:fs"; import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { join, relative, resolve } from "node:path"; @@ -132,6 +133,44 @@ export function openAnchoredDirectory( } } +export function writeAnchoredFile(parentAnchor, name, data) { + if ( + typeof name !== "string" || + name.length === 0 || + name === "." || + name === ".." || + name.includes("/") || + name.includes("\0") + ) { + throw new Error("DUMMY proof artifact name is invalid"); + } + const bytes = Buffer.from(data); + const path = join(parentAnchor, name); + const fd = openSync( + path, + constants.O_WRONLY | + constants.O_CREAT | + constants.O_EXCL | + constants.O_NOFOLLOW | + constants.O_CLOEXEC, + 0o600, + ); + try { + const created = fstatSync(fd, { bigint: true }); + if (!created.isFile() || created.nlink !== 1n) { + throw new Error("DUMMY proof artifact is not one fresh regular file"); + } + let offset = 0; + while (offset < bytes.length) offset += writeSync(fd, bytes, offset, bytes.length - offset); + if (!sameInode(created, lstatSync(path, { bigint: true }))) { + throw new Error("DUMMY proof artifact identity changed"); + } + return createHash("sha256").update(bytes).digest("hex"); + } finally { + closeSync(fd); + } +} + export async function createProbeFixture({ root, outputRoot, spawn = spawnSync }) { const output = openProbeOutput({ root, outputRoot, create: true }); let rootFd; diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs index 555be56..c198b0c 100644 --- a/scripts/cp06-sdk-worker.mjs +++ b/scripts/cp06-sdk-worker.mjs @@ -6,7 +6,11 @@ import { pathToFileURL } from "node:url"; import { Cp06AuthBlockedError, Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; import { canonicalWorkerPaths, createGuardedReadTool } from "./cp06-guarded-read.mjs"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; -import { auditReadEvents } from "./cp06-worker-audit.mjs"; +import { + assistantResponseText, + auditReadEvents, + sanitizedAuditReason, +} from "./cp06-worker-audit.mjs"; import { auditWorkerOutcome } from "./cp06-worker-outcome.mjs"; const [credentialTargetArg, inputPathArg] = process.argv.slice(2); @@ -110,7 +114,7 @@ try { decisiveText: ["request-supplied owner fields are ignored"], }); if (!readAudit.ok) { - throw blocked(`SDK event audit failed: ${readAudit.reason}`); + throw blocked(`SDK event audit failed: ${sanitizedAuditReason(readAudit.reason)}`); } const outcome = auditWorkerOutcome(response, { root: paths.root, @@ -217,19 +221,9 @@ function buildPrompt({ contract, pack, originalPath }) { } function finalAssistantText(events) { - const messages = events - .filter((event) => event.type === "message_end" && event.message?.role === "assistant") - .map((event) => event.message); - const assistant = messages.at(-1); - if (assistant === undefined) - throw blocked("SDK event stream omitted the final assistant message"); - if (typeof assistant.content === "string") return assistant.content; - return Array.isArray(assistant.content) - ? assistant.content - .filter((block) => block?.type === "text" && typeof block.text === "string") - .map((block) => block.text) - .join("") - : ""; + const envelope = assistantResponseText(events); + if (!envelope.ok) throw blocked(`SDK assistant envelope rejected: ${envelope.reason}`); + return envelope.text; } function parseResponse(output) { diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs index 411ceb7..22f8166 100644 --- a/scripts/cp06-worker-audit.mjs +++ b/scripts/cp06-worker-audit.mjs @@ -5,6 +5,26 @@ export function isExpectedOriginalReference(value, options) { return value === (options.expectedOriginalReference ?? "docs/AUTH.md"); } +export const AUDIT_REJECTION_REASONS = Object.freeze({ + unexpectedTools: "unexpected tool events", + readCount: "expected exactly one read of the supplied original", + missingCallId: "original read start omitted toolCallId", + range: "original read used a range other than offset 1 with an explicit limit", + matchingCompletion: "expected one matching read completion", + duplicateCompletion: "read event stream contains an unmatched or duplicate completion", + completionOrder: "read completion did not occur after its matching start", + completionFailed: "matching read completion was not explicitly successful", + completionContent: "successful read completion omitted one authoritative text result", + bytes: "read result bytes do not equal the supplied exact original", + decisive: "the supplied original does not contain every declared decisive constraint", + responseOrder: "final assistant response did not occur after the successful original read", +}); +const ALLOWLISTED_REASONS = new Set(Object.values(AUDIT_REJECTION_REASONS)); + +export function sanitizedAuditReason(reason) { + return ALLOWLISTED_REASONS.has(reason) ? reason : "SDK event audit rejected the event stream"; +} + export function auditReadEvents(events, options) { const expectedPath = resolve(options.root, options.expectedOriginalPath); const expectedBytes = Buffer.isBuffer(options.expectedOriginal) @@ -51,17 +71,10 @@ export function auditReadEvents(events, options) { const reject = (reason) => ({ ok: false, reason, summary }); if (otherToolCalls.length > 0 || otherToolCompletions.length > 0) { - return reject( - `unexpected tool events ${[ - ...summary.other_tool_calls, - ...summary.other_tool_completions, - ].join(", ")}`, - ); + return reject(AUDIT_REJECTION_REASONS.unexpectedTools); } if (readStarts.length !== 1 || originalReads.length !== 1 || rejectedReads.length > 0) { - return reject( - `expected exactly one read of the supplied original, saw ${originalReads.length} original and ${rejectedReads.length} other reads`, - ); + return reject(AUDIT_REJECTION_REASONS.readCount); } const start = originalReads[0]; @@ -69,31 +82,29 @@ export function auditReadEvents(events, options) { summary.tool_call_id = typeof callId === "string" ? callId : null; summary.start_event_index = start.index; if (typeof callId !== "string" || callId.length === 0) { - return reject("original read start omitted toolCallId"); + return reject(AUDIT_REJECTION_REASONS.missingCallId); } const offset = start.event.args?.offset; const limit = start.event.args?.limit; const usesExplicitRange = offset !== undefined || limit !== undefined; if (usesExplicitRange && (offset !== 1 || !Number.isSafeInteger(limit) || limit < 1)) { - return reject("original read used a range other than offset 1 with an explicit limit"); + return reject(AUDIT_REJECTION_REASONS.range); } const matchingEnds = readEnds.filter(({ event }) => event.toolCallId === callId); if (matchingEnds.length !== 1) { - return reject( - `expected one matching read completion for ${callId}, saw ${matchingEnds.length}`, - ); + return reject(AUDIT_REJECTION_REASONS.matchingCompletion); } if (readEnds.length !== 1) { - return reject("read event stream contains an unmatched or duplicate completion"); + return reject(AUDIT_REJECTION_REASONS.duplicateCompletion); } const end = matchingEnds[0]; summary.end_event_index = end.index; if (end.index <= start.index) { - return reject("read completion did not occur after its matching start"); + return reject(AUDIT_REJECTION_REASONS.completionOrder); } if (end.event.isError !== false) { - return reject("matching read completion was not explicitly successful"); + return reject(AUDIT_REJECTION_REASONS.completionFailed); } const content = end.event.result?.content; if ( @@ -102,16 +113,16 @@ export function auditReadEvents(events, options) { content[0]?.type !== "text" || typeof content[0].text !== "string" ) { - return reject("successful read completion omitted one authoritative text result"); + return reject(AUDIT_REJECTION_REASONS.completionContent); } const returnedBytes = Buffer.from(content[0].text, "utf8"); summary.returned_sha256 = sha256(returnedBytes); summary.returned_bytes = returnedBytes.length; if (!returnedBytes.equals(expectedBytes)) { - return reject("read result bytes do not equal the supplied exact original"); + return reject(AUDIT_REJECTION_REASONS.bytes); } if (decisiveText.some((text) => !expectedText.includes(text))) { - return reject("the supplied original does not contain every declared decisive constraint"); + return reject(AUDIT_REJECTION_REASONS.decisive); } const finalResponse = events @@ -120,7 +131,7 @@ export function auditReadEvents(events, options) { .at(-1); summary.response_event_index = finalResponse?.index ?? null; if (finalResponse === undefined || finalResponse.index <= end.index) { - return reject("final assistant response did not occur after the successful original read"); + return reject(AUDIT_REJECTION_REASONS.responseOrder); } summary.completed_successfully = true; @@ -131,3 +142,57 @@ export function auditReadEvents(events, options) { function sha256(value) { return createHash("sha256").update(value).digest("hex"); } + +export function assistantResponseText(events) { + const messages = events + .filter((event) => event?.type === "message_end" && event.message?.role === "assistant") + .map((event) => event.message); + const final = messages.at(-1); + if (final === undefined) { + return { ok: false, reason: "SDK event stream omitted the final assistant message" }; + } + for (const message of messages.slice(0, -1)) { + if (!isToolCallEnvelope(message)) { + return { ok: false, reason: "an earlier assistant message carried unsupported content" }; + } + } + if (final.stopReason !== "stop") { + return { ok: false, reason: "final assistant message did not stop normally" }; + } + if (typeof final.content === "string") return { ok: true, text: final.content }; + if ( + !Array.isArray(final.content) || + !final.content.some((block) => block?.type === "text") || + !final.content.every( + (block) => + (block?.type === "text" && typeof block.text === "string") || isThinkingBlock(block), + ) + ) { + return { ok: false, reason: "final assistant message carried an unsupported content channel" }; + } + return { + ok: true, + text: final.content + .filter((block) => block.type === "text") + .map((block) => block.text) + .join(""), + }; +} + +function isToolCallEnvelope(message) { + return ( + message.stopReason === "toolUse" && + Array.isArray(message.content) && + message.content.some((block) => block?.type === "toolCall") && + message.content.every( + (block) => + (block?.type === "toolCall" && typeof block.id === "string") || + isThinkingBlock(block) || + (block?.type === "text" && typeof block.text === "string" && block.text.trim() === ""), + ) + ); +} + +function isThinkingBlock(block) { + return block?.type === "thinking" && typeof block.thinking === "string"; +} diff --git a/scripts/prove-cp06-auth-security.mjs b/scripts/prove-cp06-auth-security.mjs index 4e8cd88..fc4388f 100644 --- a/scripts/prove-cp06-auth-security.mjs +++ b/scripts/prove-cp06-auth-security.mjs @@ -1,9 +1,9 @@ #!/usr/bin/env node -import { writeFile } from "node:fs/promises"; +import { unlinkSync } from "node:fs"; import { dirname, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { Cp06IsolationUnsupportedError, runCp06SecurityPreflight } from "./cp06-auth-security.mjs"; -import { openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { openProbeOutput, writeAnchoredFile } from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); @@ -16,7 +16,12 @@ try { const path = resolve(outputRoot, "auth-security", "evidence.json"); const output = openProbeOutput({ root, outputRoot: resolve(outputRoot, "auth-security") }); try { - await writeFile(resolve(output.anchor, "evidence.json"), `${JSON.stringify(proof, null, 2)}\n`); + try { + unlinkSync(resolve(output.anchor, "evidence.json")); + } catch (error) { + if (error?.code !== "ENOENT") throw error; + } + writeAnchoredFile(output.anchor, "evidence.json", `${JSON.stringify(proof, null, 2)}\n`); } finally { output.close(); } diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index 76ef066..8ee983a 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -5,7 +5,11 @@ import { spawnSync } from "node:child_process"; import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; -import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { + openAnchoredDirectory, + openProbeOutput, + writeAnchoredFile, +} from "./cp06-probe-fixture.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; const root = resolve(fileURLToPath(new URL("..", import.meta.url))); @@ -44,16 +48,23 @@ const cases = await withCleanup( }, }); if (scenario !== "missing-source-setup") await writeFile(source, sourceBytes); - await mkdir(join(proofOutput, "ten-pack/raw"), { recursive: true }); - await mkdir(join(proofOutput, "ten-pack/repo/docs"), { recursive: true }); - await writeFile( - join(proofOutput, "ten-pack/raw/auth-worker-missing.pack.md"), - "DUMMY cleanup-only handoff; no semantic acceptance claimed.\n", - ); - await writeFile( - join(proofOutput, "ten-pack/repo/docs/AUTH.md"), - await readFile(join(root, "test/fixtures/cp06-context/docs/AUTH.md")), - ); + const rawDirectory = openAnchoredDirectory(proofOutput, "ten-pack/raw"); + const docsDirectory = openAnchoredDirectory(proofOutput, "ten-pack/repo/docs"); + try { + writeAnchoredFile( + rawDirectory.anchor, + "auth-worker-missing.pack.md", + "DUMMY cleanup-only handoff; no semantic acceptance claimed.\n", + ); + writeAnchoredFile( + docsDirectory.anchor, + "AUTH.md", + await readFile(join(root, "test/fixtures/cp06-context/docs/AUTH.md")), + ); + } finally { + rawDirectory.close(); + docsDirectory.close(); + } const args = [ "--experimental-import-meta-resolve", join(root, "scripts/replay-cp06-worker.mjs"), @@ -141,7 +152,7 @@ const manifest = { "Actual full-runner expired/near-expiry and missing-source setup cases use DUMMY auth only. Launch, timeout, audit and compound cleanup failures are additionally injected at the executable supervisor interface in the unit suite; SDK timeout is exercised by the DUMMY auth-security proof.", ], }; -await writeFile(join(proofRoot, "evidence.json"), `${JSON.stringify(manifest, null, 2)}\n`); +writeAnchoredFile(proofRoot, "evidence.json", `${JSON.stringify(manifest, null, 2)}\n`); proofDirectory.close(); output.close(); console.log( diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 0354823..6252290 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -2,12 +2,16 @@ import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; import { realpathSync } from "node:fs"; -import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { mkdtemp, readFile, rm } from "node:fs/promises"; import { homedir, tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; -import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { + openAnchoredDirectory, + openProbeOutput, + writeAnchoredFile, +} from "./cp06-probe-fixture.mjs"; import { Cp06IsolationUnsupportedError, runCp06SecurityPreflight, @@ -78,7 +82,7 @@ async function runWorker(workerRoot) { system_prompt: systemPrompt, }; const inputPath = join(workerRoot, "input.json"); - await writeFile(inputPath, `${JSON.stringify(workerInput, null, 2)}\n`); + writeAnchoredFile(workerRoot, "input.json", `${JSON.stringify(workerInput, null, 2)}\n`); const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-sdk-worker-")); const credentialTarget = join(evaluationHome, "pi-agent", "auth.json"); const credentialSource = resolve( @@ -174,14 +178,18 @@ async function runWorker(workerRoot) { "Live model wording is nondeterministic. The proof runner is Linux x86-64 only and exits blocked before credential access/provider use when user/mount namespaces, libseccomp, exact Pi 0.85.1, or adequate unexpired OAuth validity are unavailable.", }; const manifestPath = join(workerRoot, "evidence.json"); - await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); + const manifestSha256 = writeAnchoredFile( + workerRoot, + "evidence.json", + `${JSON.stringify(manifest, null, 2)}\n`, + ); console.log( JSON.stringify({ ok: true, gate: manifest.gate, tested_sha: testedSha, manifest_path: relative(manifestPath), - manifest_sha256: await fileHash(manifestPath), + manifest_sha256: manifestSha256, exit_code: 0, status: manifest.response.status, reads: manifest.event_stream.read_audit.project_read_paths, @@ -192,7 +200,7 @@ async function runWorker(workerRoot) { async function recordBlocked(workerRoot, error, stage) { const status = createWorkerBlockedStatus(error, stage); - await writeFile(join(workerRoot, "blocked.json"), `${JSON.stringify(status, null, 2)}\n`); + writeAnchoredFile(workerRoot, "blocked.json", `${JSON.stringify(status, null, 2)}\n`); } export function createWorkerBlockedStatus(error, stage, recordedAt = new Date().toISOString()) { diff --git a/test/cp06-guarded-read.test.mjs b/test/cp06-guarded-read.test.mjs index c8acd73..ef3ccd8 100644 --- a/test/cp06-guarded-read.test.mjs +++ b/test/cp06-guarded-read.test.mjs @@ -1,9 +1,13 @@ import { strict as assert } from "node:assert"; -import { mkdir, mkdtemp, rm, symlink, unlink, writeFile } from "node:fs/promises"; +import { mkdir, mkdtemp, rename, rm, symlink, unlink, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; -import { canonicalWorkerPaths, createGuardedReadTool } from "../scripts/cp06-guarded-read.mjs"; +import { + canonicalWorkerPaths, + createGuardedReadTool, + readPinnedRegularFile, +} from "../scripts/cp06-guarded-read.mjs"; test("guarded read denies noncanonical and unauthorized paths before tool access", async () => { const root = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-read-")); @@ -98,3 +102,26 @@ test("worker paths reject symlink escapes and credential overlap", async () => { function credentialPath(root) { return join(root, "auth.json"); } + +test("initial original snapshot does not follow a substituted ancestor directory", async () => { + const root = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-ancestor-")); + const original = join(root, "fixture", "docs", "AUTH.md"); + const external = join(root, "DUMMY-external"); + await mkdir(join(root, "fixture", "docs"), { recursive: true }); + await mkdir(external); + await writeFile(original, "DUMMY exact original"); + await writeFile(join(external, "AUTH.md"), "DUMMY-EXTERNAL-SECRET"); + try { + assert.equal(readPinnedRegularFile(original).bytes.toString("utf8"), "DUMMY exact original"); + await rename(join(root, "fixture", "docs"), join(root, "fixture", "docs-moved")); + await symlink(external, join(root, "fixture", "docs")); + assert.throws( + () => readPinnedRegularFile(original), + (error) => + ["ELOOP", "ENOTDIR"].includes(error.code) && + !String(error.message).includes("DUMMY-EXTERNAL"), + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index dfa735b..0a6ec05 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -19,7 +19,11 @@ import { runDummySdkProof, runIsolationProbe, } from "../scripts/cp06-auth-security.mjs"; -import { openAnchoredDirectory, openProbeOutput } from "../scripts/cp06-probe-fixture.mjs"; +import { + openAnchoredDirectory, + openProbeOutput, + writeAnchoredFile, +} from "../scripts/cp06-probe-fixture.mjs"; const runners = [ "scripts/replay-cp06-worker.mjs", @@ -141,18 +145,18 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { schema_version: 1, result: "pass", auth_cases: [ - authCase("expired", "blocked", 0), - authCase("near_expiry", "blocked", 0), - authCase("unexpired", "resolved", 1), + authCase("expired", "blocked", 0, 1), + authCase("near_expiry", "blocked", 0, 1), + authCase("unexpired", "resolved", 1, 0), ], sdk_worker: { active_tools: ["read"], in_memory_session: true, event_count: 2, event_sha256: "0".repeat(64), - read_audit: { exact_original: true, read_count: 1 }, + read_audit: readAudit("DUMMY-read", "/tmp/DUMMY-worker/DUMMY-original.md"), refresh_callbacks: 0, - credential_store: credentialAudit(), + credential_store: { reads: 203, lists: 1, modify_denials: 0, delete_denials: 0 }, }, sdk_dependency: trustedDependency, actual_sdk_outcomes: { @@ -170,9 +174,12 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { outcome("missing-reads", 75), outcome("credential-read", 75), outcome("symlink-read", 75), - outcome("expired", 75, { runtime_create: 0 }), - outcome("near-expiry", 75, { runtime_create: 0 }), - outcome("timeout", 70), + outcome("earlier-assistant-claim", 75), + outcome("final-extra-channel", 75, { to_auth: 4 }), + outcome("unexpected-tool-secret", 75), + outcome("expired", 75), + outcome("near-expiry", 75), + outcome("timeout", 70, { to_auth: 2 }), ], }, fixture_origin: true, @@ -210,6 +217,79 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { }, }, { ...validBase, sdk_worker: { ...validBase.sdk_worker, active_tools: ["read", "write"] } }, + { + ...validBase, + sdk_worker: { + ...validBase.sdk_worker, + read_audit: { ...validBase.sdk_worker.read_audit, extra: "DUMMY-SECRET" }, + }, + }, + { + ...validBase, + sdk_worker: { ...validBase.sdk_worker, read_audit: { exact_original: true, read_count: 1 } }, + }, + { + ...validBase, + sdk_worker: { + ...validBase.sdk_worker, + read_audit: { ...validBase.sdk_worker.read_audit, other_tool_calls: ["DUMMY-SECRET"] }, + }, + }, + { + ...validBase, + sdk_worker: { + ...validBase.sdk_worker, + credential_store: { ...validBase.sdk_worker.credential_store, modify_denials: 7 }, + }, + }, + { + ...validBase, + auth_cases: [ + authCase("expired", "blocked", 0, 0), + validBase.auth_cases[1], + validBase.auth_cases[2], + ], + }, + { + ...validBase, + auth_cases: [ + validBase.auth_cases[0], + authCase("near_expiry", "blocked", 0, 9), + validBase.auth_cases[2], + ], + }, + ...[ + ["valid", { to_auth: 0 }], + ["missing-reads", { runtime_create: 0, to_auth: 0 }], + ["credential-read", { to_auth: 17 }], + ["expired", { runtime_create: 1 }], + [ + "valid", + { + read_audit: { + ...readAudit("DUMMY-actual-worker-read", "/tmp/DUMMY-outcome/docs/AUTH.md"), + extra: "DUMMY-SECRET", + }, + }, + ], + [ + "fenced-valid", + { + read_audit: { + ...readAudit("DUMMY-actual-worker-read", "/tmp/DUMMY-outcome/docs/AUTH.md"), + returned_sha256: "9".repeat(64), + }, + }, + ], + ].map(([scenario, changes]) => ({ + ...validBase, + actual_sdk_outcomes: { + ...validBase.actual_sdk_outcomes, + cases: validBase.actual_sdk_outcomes.cases.map((entry) => + entry.scenario === scenario ? { ...entry, ...changes } : entry, + ), + }, + })), ]) { assert.throws( () => @@ -295,20 +375,36 @@ test("anchored runner child directories do not follow substituted paths", async } }); -function authCase(scenario, result, toAuth) { +function authCase(scenario, result, toAuth, modifyDenials) { return { scenario, result, refresh_callbacks: 0, to_auth_calls: toAuth, - credential_store: credentialAudit(), + credential_store: { reads: 1, lists: 0, modify_denials: modifyDenials, delete_denials: 0 }, persistence_operations: 0, source_unchanged: true, }; } -function credentialAudit() { - return { reads: 1, lists: 0, modify_denials: 1, delete_denials: 1 }; +function readAudit(toolCallId, originalPath) { + return { + read_count: 1, + project_read_count: 1, + project_read_paths: [originalPath], + rejected_read_paths: [], + other_tool_calls: [], + other_tool_completions: [], + completed_successfully: true, + exact_original: true, + expected_sha256: "5".repeat(64), + returned_sha256: "5".repeat(64), + returned_bytes: 117, + tool_call_id: toolCallId, + start_event_index: 12, + end_event_index: 13, + response_event_index: 46, + }; } function sdkDependency() { @@ -337,13 +433,46 @@ function outcome(scenario, exitCode, extra = {}) { default_storage: 0, network: 0, refresh: 0, - to_auth: ["expired", "near-expiry"].includes(scenario) ? 0 : 1, + to_auth: ["expired", "near-expiry"].includes(scenario) ? 0 : 3, runtime_create: ["expired", "near-expiry"].includes(scenario) ? 0 : 1, source_unchanged: true, ...extra, }; if (["valid", "fenced-valid"].includes(scenario)) { - result.read_audit = { exact_original: true, read_count: 1 }; + result.read_audit = readAudit("DUMMY-actual-worker-read", "/tmp/DUMMY-outcome/docs/AUTH.md"); } return result; } + +test("anchored artifact writes refuse substituted leaves before external mutation", async () => { + const root = process.cwd(); + const outputRoot = resolve(".factory/state/cp06-correction/DUMMY-artifact-leaf"); + const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-leaf-external-")); + const sentinel = join(external, "DUMMY-sentinel"); + await writeFile(sentinel, "DUMMY-SENTINEL-UNCHANGED"); + const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); + const child = openAnchoredDirectory(output.anchor, "worker", { reset: true }); + try { + for (const name of ["input.json", "evidence.json", "blocked.json"]) { + await symlink(sentinel, join(outputRoot, "worker", name)); + assert.throws(() => writeAnchoredFile(child.anchor, name, "DUMMY redirected write"), { + code: "EEXIST", + }); + } + assert.equal(await readFile(sentinel, "utf8"), "DUMMY-SENTINEL-UNCHANGED"); + const digest = writeAnchoredFile(child.anchor, "worker-control.json", "DUMMY control"); + assert.equal( + await readFile(join(outputRoot, "worker", "worker-control.json"), "utf8"), + "DUMMY control", + ); + assert.match(digest, /^[0-9a-f]{64}$/u); + for (const name of ["../escape.json", "nested/blocked.json", ""]) { + assert.throws(() => writeAnchoredFile(child.anchor, name, "DUMMY")); + } + } finally { + child.close(); + output.close(); + await rm(outputRoot, { recursive: true, force: true }); + await rm(external, { recursive: true, force: true }); + } +}); diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs index 71698ec..ccc45eb 100644 --- a/test/cp06-worker-audit.test.mjs +++ b/test/cp06-worker-audit.test.mjs @@ -2,7 +2,12 @@ import { strict as assert } from "node:assert"; import { readFile } from "node:fs/promises"; import { resolve } from "node:path"; import test from "node:test"; -import { auditReadEvents, isExpectedOriginalReference } from "../scripts/cp06-worker-audit.mjs"; +import { + assistantResponseText, + auditReadEvents, + isExpectedOriginalReference, + sanitizedAuditReason, +} from "../scripts/cp06-worker-audit.mjs"; const root = process.cwd(); const originalPath = resolve(root, "test/fixtures/cp06-context/docs/AUTH.md"); @@ -192,3 +197,57 @@ for (const [name, acceptedStart] of [ assert.equal(audit.summary.expected_sha256, audit.summary.returned_sha256); }); } + +test("worker read audit reasons never carry event-controlled names or IDs", () => { + const secretTool = { + type: "tool_execution_start", + toolName: "DUMMY-SECRET-TOOL", + toolCallId: "DUMMY-SECRET-ID", + args: {}, + }; + const unmatched = successfulEnd({ toolCallId: "DUMMY-SECRET-ID" }); + for (const events of [ + [start, successfulEnd(), secretTool, finalMessage], + [{ ...start, toolCallId: "DUMMY-SECRET-ID" }, successfulEnd(), finalMessage], + [start, unmatched, finalMessage], + ]) { + const audit = auditReadEvents(events, options); + assert.equal(audit.ok, false); + assert.equal(audit.reason.includes("DUMMY-SECRET"), false); + assert.equal(sanitizedAuditReason(audit.reason), audit.reason); + } + assert.equal( + sanitizedAuditReason("unexpected tool DUMMY-SECRET-TOOL"), + "SDK event audit rejected the event stream", + ); +}); + +test("assistant envelope rejects competing evidence and unsupported final channels", () => { + const toolCall = { type: "toolCall", id: "read-auth", name: "read", arguments: {} }; + const toolUse = (content) => ({ + type: "message_end", + message: { role: "assistant", stopReason: "toolUse", content }, + }); + const final = (content) => ({ + type: "message_end", + message: { role: "assistant", stopReason: "stop", content }, + }); + const accepted = assistantResponseText([ + toolUse([{ type: "thinking", thinking: "DUMMY plan" }, toolCall]), + final([{ type: "text", text: '{"status":"MISSING_SOURCE"}' }]), + ]); + assert.deepEqual(accepted, { ok: true, text: '{"status":"MISSING_SOURCE"}' }); + for (const events of [ + [], + [toolUse([{ type: "text", text: "Creation verified and passed" }, toolCall]), final("{}")], + [final([{ type: "text", text: "Creation passed" }]), final([{ type: "text", text: "{}" }])], + [toolUse([toolCall]), final([{ type: "text", text: "{}" }, toolCall])], + [toolUse([toolCall]), final([{ type: "image", data: "DUMMY", mimeType: "image/png" }])], + [ + toolUse([toolCall]), + { ...final("{}"), message: { ...final("{}").message, stopReason: "length" } }, + ], + ]) { + assert.equal(assistantResponseText(events).ok, false); + } +}); diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 2b9c02e..a5b81b0 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -38,6 +38,8 @@ for (const scenario of [ "cleanup", "primary-and-cleanup", "audit-and-cleanup", + "semantic-audit", + "semantic-audit-and-cleanup", ]) { test(`supervisor awaits cleanup and preserves failures: ${scenario}`, async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-lifecycle-")); @@ -86,7 +88,14 @@ for (const scenario of [ await new Promise((done) => setTimeout(done, 5)); cleanups++; await rm(target); - if (["cleanup", "primary-and-cleanup", "audit-and-cleanup"].includes(scenario)) + if ( + [ + "cleanup", + "primary-and-cleanup", + "audit-and-cleanup", + "semantic-audit-and-cleanup", + ].includes(scenario) + ) throw cleanupError; }, }; @@ -106,7 +115,9 @@ for (const scenario of [ status: 0, stdout: ["audit", "audit-and-cleanup"].includes(scenario) ? "invalid JSON" - : '{"result":"pass"}', + : scenario.startsWith("semantic-audit") + ? JSON.stringify({ ...validWorkerChild(), result: "fail" }) + : JSON.stringify(validWorkerChild()), }; }, }, @@ -158,7 +169,18 @@ for (const scenario of [ assert.equal(error.code, "CP06_AUTH_BLOCKED"); } if (scenario === "timeout") assert.equal(error.cause.code, "ETIMEDOUT"); - if (["cleanup", "primary-and-cleanup", "audit-and-cleanup"].includes(scenario)) { + if (scenario === "semantic-audit") { + assert.equal(error.code, "CP06_CHILD_OUTPUT_INVALID"); + assert.equal(error.stage, "audit"); + } + if ( + [ + "cleanup", + "primary-and-cleanup", + "audit-and-cleanup", + "semantic-audit-and-cleanup", + ].includes(scenario) + ) { assert(error instanceof Cp06CleanupError); assert.equal(error.exitCode, 74); assert.equal(error.errors.at(-1), cleanupError); @@ -168,7 +190,7 @@ for (const scenario of [ assert.equal(error.errors[0].code, "CP06_AUTH_BLOCKED"); assert.equal(error.cause, error.errors[0]); } - if (scenario === "audit-and-cleanup") { + if (["audit-and-cleanup", "semantic-audit-and-cleanup"].includes(scenario)) { assert.equal(error.errors[0].exitCode, 70); assert.equal(error.errors[0].code, "CP06_CHILD_OUTPUT_INVALID"); assert.equal(error.cause, error.errors[0]); @@ -231,7 +253,7 @@ test("both probe modes require a supervisor-generated DUMMY fixture", async () = spawn() { launches++; if (mode === "unsafe-probe") writeFileSync(source, "DUMMY mutated"); - return { status: 0, stdout: '{"result":"DUMMY"}' }; + return { status: 0, stdout: JSON.stringify(validProbeChild()) }; }, }; await assert.rejects(superviseCredentialChild({ ...options, target: secret }, dependencies), { @@ -461,7 +483,7 @@ test("worker source integrity uses metadata without reading credential bytes", a }; }, spawn() { - return { status: 0, stdout: '{"result":"pass"}' }; + return { status: 0, stdout: JSON.stringify(validWorkerChild()) }; }, }, ); @@ -574,6 +596,36 @@ test("audit and cleanup causes survive supervisor output and replay blocked reco } }); +function validWorkerChild() { + return { + schema_version: 1, + result: "pass", + pi_version: "0.85.1", + pi_install: { version: "0.85.1" }, + provider: "openai-codex", + model: "DUMMY-model", + oauth_preflight: { minimum_validity_ms: 1, remaining_validity_ms: 2, refreshed: false }, + credential_store: { reads: 1, lists: 0, modify_denials: 0, delete_denials: 0 }, + session: { in_memory: true, active_tools: ["read"] }, + event_stream: { + sha256: "0".repeat(64), + event_count: 3, + read_audit: { exact_original: true, read_count: 1 }, + }, + response: { status: "MISSING_SOURCE" }, + response_sha256: "1".repeat(64), + raw_transcript_retained: false, + }; +} + +function validProbeChild() { + return { + identity: { uid: 0, gid: 0, status: {} }, + source: { unchanged: true }, + direct_namespace_syscalls: { exit: null, skipped: true }, + }; +} + function dummyStat() { return { dev: 1n, From 5953aea79cb9791637d11b1443d239edb8060afe Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 5 Oct 2026 09:11:17 +0000 Subject: [PATCH 35/45] no-mistakes(review): Pin cleanup scratch, exact read-audit paths, honest manifest --- docs/probes/CP-06.md | 9 ++-- scripts/cp06-auth-security.mjs | 59 ++++++++++++++++++++++----- scripts/cp06-dummy-outcome-proof.mjs | 10 ++++- scripts/cp06-dummy-sdk-proof.mjs | 20 +++++---- scripts/prove-cp06-worker-cleanup.mjs | 32 ++++++++++----- test/cp06-proof-boundary.test.mjs | 40 +++++++++++++----- 6 files changed, 126 insertions(+), 44 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index e897e32..35e960a 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -19,9 +19,9 @@ daemon. ## Read-only Pi authentication correction -The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then observes that the actual evaluated child has empty capability sets, `no_new_privs`, and seccomp filter mode, and that its attempted remount/unmount, direct `setns`/`unshare`/`clone`/`clone3`/`mount`/`umount2` syscalls, nested namespace creation, `nsenter`, and UID change fail. The seccomp helper also installs denials for alternate mount APIs, handle-based inode reopening, ptrace and process-memory access, and the evaluated child is launched with locked root securebits requested through `setpriv`; the probe does not yet execute ptrace/process-memory or inherited-subprocess escape attempts or read back securebits, so those outcomes are configured but unproven (open finding `CP06-KERNEL-PROOF-INCOMPLETE`). On the WSL2 6.18 host used for the 2026-10-05 review round, the hardened probe blocked at credential mount setup (`CP06_ISOLATION_SETUP_FAILED`, mount identity check), so the full DUMMY gate did not pass there. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. The bind mounts are still issued through util-linux `mount` with `/proc//fd` anchor paths, which util-linux canonicalizes; a native descriptor-to-descriptor mount handoff is not implemented yet (open finding `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. +The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then observes that the actual evaluated child has empty capability sets, `no_new_privs`, and seccomp filter mode, and that its attempted remount/unmount, direct `setns`/`unshare`/`clone`/`clone3`/`mount`/`umount2` syscalls, nested namespace creation, `nsenter`, and UID change fail. The seccomp helper also installs denials for alternate mount APIs, handle-based inode reopening, ptrace and process-memory access, and the evaluated child is launched with locked root securebits requested through `setpriv`; the probe does not yet execute ptrace/process-memory or inherited-subprocess escape attempts or read back securebits, so those outcomes are configured but unproven (open finding `CP06-KERNEL-PROOF-INCOMPLETE`). On the WSL2 6.18 host used for the 2026-10-05 review rounds, the hardened probe blocks at credential mount setup with `CP06_ISOLATION_SETUP_FAILED`. The failing check is the post-mount identity check in `mountLiveCredentialReadOnly` (`credential mount identity changed: /proc//fd//DUMMY-auth.json`), while the unsafe probe, which skips the parent bind mounts, passes. Diagnosis from the code and that error (not yet confirmed by a fix): util-linux canonicalizes the `/proc//fd/` parent anchor, so the read-only parent bind covers the canonical directory, and the later leaf bind, also canonicalized, lands inside that parent bind. The identity check walks from the retained parent descriptor, which still refers to the original mount, where the target leaf is the empty placeholder rather than the pinned inode. The source check passes only because the underlying source file is the pinned inode. This is a path-handoff design defect, not a missing kernel primitive, and nothing in it is WSL-specific, so the worker path with parent protection is expected to fail the same way on other Linux hosts. The check is not weakened; the gate fails closed until the native descriptor-bound mount handoff (`CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`) lands. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. The bind mounts are still issued through util-linux `mount` with `/proc//fd` anchor paths, which util-linux canonicalizes; a native descriptor-to-descriptor mount handoff is not implemented yet (open finding `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof's DUMMY evaluation home and credential scratch directories are still created by path under the pinned proof directory. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes those directories to the runner as paths under the retained descriptors. That proof cannot complete on a host where the DUMMY security preflight fails, as on the review host above. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. @@ -300,8 +300,9 @@ messages may carry only tool calls, thinking blocks, and whitespace text with `stopReason: "toolUse"`, and the final message only text and thinking blocks with `stopReason: "stop"`. SDK-worker stderr carries only allowlisted fixed audit reasons. The consumer validates the exact nested `read_audit` schema (one -correlated exact-original read, matching digests, ordered event indexes, fixed -tool-call ID), exact per-scenario exit, `runtime_create`, and `to_auth` counts +correlated read whose path must equal the exact canonical original the consumer +created under its own fresh proof root, matching digests, ordered event indexes, +fixed tool-call ID), exact per-scenario exit, `runtime_create`, and `to_auth` counts (valid runs observe three `toAuth` calls: the preflight plus two model turns), and exact credential-store `modify`/`delete` denial counts (one modify denial for expired and near-expiry auth, none otherwise). Every case is explicitly diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 38eb0b3..2b225a6 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -9,7 +9,10 @@ import { mkdtempSync, openSync, readFileSync, + realpathSync, + rmSync, } from "node:fs"; +import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import { resolvePinnedPiAi } from "./cp06-pi-dependency.mjs"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; @@ -420,6 +423,29 @@ export function runDummySdkProof({ piRoot, spawnSyncImpl = spawnSync, expectedSdkDependency, +}) { + const proofRoot = realpathSync(mkdtempSync(join(tmpdir(), "factory-cp06-DUMMY-sdk-"))); + try { + return runDummySdkProofIn({ + root, + helper, + piRoot, + spawnSyncImpl, + expectedSdkDependency, + proofRoot, + }); + } finally { + rmSync(proofRoot, { recursive: true, force: true }); + } +} + +function runDummySdkProofIn({ + root, + helper, + piRoot, + spawnSyncImpl, + expectedSdkDependency, + proofRoot, }) { const result = spawnSyncImpl( "unshare", @@ -442,6 +468,7 @@ export function runDummySdkProof({ "--experimental-import-meta-resolve", join(root, "scripts/cp06-dummy-sdk-proof.mjs"), piRoot, + proofRoot, ], { cwd: root, @@ -470,10 +497,22 @@ export function runDummySdkProof({ return validateDummySdkProof( parseCompactJson(result.stdout, "DUMMY SDK proof"), trustedDependency, + dummySdkOriginalPaths(proofRoot), ); } -export function validateDummySdkProof(proof, trustedDependency) { +export function dummySdkOriginalPaths(proofRoot) { + return { + sdkWorker: join(proofRoot, "DUMMY-worker", "DUMMY-original.md"), + outcome: join(proofRoot, "DUMMY-outcome", "docs", "AUTH.md"), + }; +} + +export function validateDummySdkProof(proof, trustedDependency, originalPaths) { + assert( + typeof originalPaths?.sdkWorker === "string" && typeof originalPaths?.outcome === "string", + "DUMMY SDK expected original paths missing", + ); assertExactKeys(proof, [ "actual_sdk_outcomes", "auth_cases", @@ -491,9 +530,9 @@ export function validateDummySdkProof(proof, trustedDependency) { assert(proof.semantic_acceptance === false, "DUMMY SDK semantic acceptance was claimed"); assert(proof.network_calls === 0, "DUMMY SDK attempted network access"); validateAuthCases(proof.auth_cases); - validateSdkWorker(proof.sdk_worker); + validateSdkWorker(proof.sdk_worker, originalPaths.sdkWorker); validateSdkDependency(proof.sdk_dependency, trustedDependency); - validateOutcomeProof(proof.actual_sdk_outcomes, trustedDependency); + validateOutcomeProof(proof.actual_sdk_outcomes, trustedDependency, originalPaths.outcome); return proof; } @@ -532,7 +571,7 @@ function validateAuthCases(cases) { } } -function validateSdkWorker(worker) { +function validateSdkWorker(worker, originalPath) { assertExactKeys(worker, [ "active_tools", "credential_store", @@ -556,7 +595,7 @@ function validateSdkWorker(worker) { validateCredentialAudit(worker.credential_store, { modify_denials: 0, delete_denials: 0 }); validateReadAudit(worker.read_audit, { toolCallId: "DUMMY-read", - originalSuffix: "/DUMMY-original.md", + originalPath, }); } @@ -591,7 +630,7 @@ function validateSdkDependency(dependency, trustedDependency) { } } -function validateOutcomeProof(outcomes, trustedDependency) { +function validateOutcomeProof(outcomes, trustedDependency, originalPath) { assertExactKeys(outcomes, ["cases", "fixture_origin", "semantic_acceptance"]); assert(outcomes.fixture_origin === true, "DUMMY outcome fixture origin missing"); assert(outcomes.semantic_acceptance === false, "DUMMY outcome semantic acceptance was claimed"); @@ -664,7 +703,7 @@ function validateOutcomeProof(outcomes, trustedDependency) { if (["valid", "fenced-valid"].includes(entry.scenario)) { validateReadAudit(entry.read_audit, { toolCallId: "DUMMY-actual-worker-read", - originalSuffix: "/docs/AUTH.md", + originalPath, }); } } @@ -682,7 +721,7 @@ function validateCredentialAudit(audit, expected) { assert(audit.reads >= 1, "DUMMY credential audit omitted the credential read"); } -function validateReadAudit(audit, { toolCallId, originalSuffix }) { +function validateReadAudit(audit, { toolCallId, originalPath }) { assertExactKeys(audit, [ "completed_successfully", "end_event_index", @@ -707,9 +746,7 @@ function validateReadAudit(audit, { toolCallId, originalSuffix }) { audit.project_read_count === 1 && Array.isArray(audit.project_read_paths) && audit.project_read_paths.length === 1 && - typeof audit.project_read_paths[0] === "string" && - audit.project_read_paths[0].startsWith("/") && - audit.project_read_paths[0].endsWith(originalSuffix) && + audit.project_read_paths[0] === originalPath && isEmptyArray(audit.rejected_read_paths) && isEmptyArray(audit.other_tool_calls) && isEmptyArray(audit.other_tool_completions), diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index 0428ad1..b015318 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -7,9 +7,15 @@ import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; /** Exercise the actual SDK worker executable, with only a local faux provider. */ -export async function proveDummyWorkerOutcome({ piRoot }) { +export async function proveDummyWorkerOutcome({ piRoot, directory: requested }) { const root = resolve(fileURLToPath(new URL("..", import.meta.url))); - const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-outcome-")); + let directory; + if (requested === undefined) { + directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-outcome-")); + } else { + await mkdir(requested, { mode: 0o700 }); + directory = requested; + } return withCleanup( async () => { await mkdir(join(directory, "docs")); diff --git a/scripts/cp06-dummy-sdk-proof.mjs b/scripts/cp06-dummy-sdk-proof.mjs index 5b56672..53f84c8 100644 --- a/scripts/cp06-dummy-sdk-proof.mjs +++ b/scripts/cp06-dummy-sdk-proof.mjs @@ -1,8 +1,8 @@ #!/usr/bin/env node import { strict as assert } from "node:assert"; import { createHash } from "node:crypto"; -import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; +import { readdirSync, realpathSync } from "node:fs"; +import { readFile, rm, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; @@ -10,8 +10,8 @@ import { auditReadEvents } from "./cp06-worker-audit.mjs"; import { proveDummyWorkerOutcome } from "./cp06-dummy-outcome-proof.mjs"; import { importPinnedPiAi } from "./cp06-pi-dependency.mjs"; -const [piRootArg] = process.argv.slice(2); -if (!piRootArg) process.exit(64); +const [piRootArg, proofRootArg] = process.argv.slice(2); +if (!piRootArg || !proofRootArg) process.exit(64); const piRoot = resolve(piRootArg); const sdk = await import(pathToFileURL(join(piRoot, "dist/index.js"))); const { module: ai, provenance: piAi } = await importPinnedPiAi(piRoot); @@ -21,7 +21,8 @@ globalThis.fetch = async () => { networkCalls += 1; throw new Error("DUMMY proof forbids network"); }; -const proofRoot = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-sdk-")); +const proofRoot = realpathSync(resolve(proofRootArg)); +if (proofRoot !== resolve(proofRootArg) || readdirSync(proofRoot).length !== 0) process.exit(64); try { const authCases = []; for (const scenario of [ @@ -32,7 +33,10 @@ try { authCases.push(await authCase(scenario)); } const worker = await fauxWorkerProof(); - const outcomeProof = await proveDummyWorkerOutcome({ piRoot }); + const outcomeProof = await proveDummyWorkerOutcome({ + piRoot, + directory: join(proofRoot, "DUMMY-outcome"), + }); assert.equal(networkCalls, 0); console.log( JSON.stringify({ @@ -49,7 +53,9 @@ try { ); } finally { globalThis.fetch = originalFetch; - await rm(proofRoot, { recursive: true, force: true }); + for (const entry of readdirSync(proofRoot)) { + await rm(join(proofRoot, entry), { recursive: true, force: true }); + } } async function authCase(scenario) { diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index 8ee983a..9d70be4 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -2,7 +2,7 @@ import { strict as assert } from "node:assert"; import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises"; +import { readFile, readdir, rm } from "node:fs/promises"; import { join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { @@ -18,7 +18,7 @@ const output = openProbeOutput({ root, outputRoot, create: true, runner: true }) const proofDirectory = openAnchoredDirectory(output.anchor, "worker-cleanup", { reset: true }); const proofRoot = proofDirectory.anchor; const proofRootPath = join(outputRoot, "worker-cleanup"); -const scratch = await mkdtemp(join(proofRoot, "DUMMY-cleanup-")); +const scratchDirectory = openAnchoredDirectory(proofRoot, "DUMMY-cleanup", { reset: true }); const cases = await withCleanup( async () => { const cases = []; @@ -27,8 +27,11 @@ const cases = await withCleanup( ["near-expiry", 75], ["missing-source-setup", 70], ]) { - const home = join(scratch, scenario); - const temporary = join(home, "tmp"); + const homeDirectory = openAnchoredDirectory(scratchDirectory.anchor, scenario); + const temporaryDirectory = openAnchoredDirectory(homeDirectory.anchor, "tmp"); + const credentialDirectory = openAnchoredDirectory(homeDirectory.anchor, "credential"); + const home = join(scratchDirectory.anchor, scenario); + const temporary = join(homeDirectory.anchor, "tmp"); const proofOutputPath = join(proofRootPath, "proofs", scenario); const proofOutputDirectory = openAnchoredDirectory( proofDirectory.anchor, @@ -36,9 +39,7 @@ const cases = await withCleanup( { reset: true }, ); const proofOutput = proofOutputDirectory.anchor; - const source = join(home, "credential", "DUMMY-auth.json"); - await mkdir(temporary, { recursive: true }); - await mkdir(join(home, "credential"), { recursive: true }); + const source = join(homeDirectory.anchor, "credential", "DUMMY-auth.json"); const sourceBytes = JSON.stringify({ "openai-codex": { type: "oauth", @@ -47,7 +48,9 @@ const cases = await withCleanup( expires: scenario === "expired" ? 0 : Date.now() + 60_000, }, }); - if (scenario !== "missing-source-setup") await writeFile(source, sourceBytes); + if (scenario !== "missing-source-setup") { + writeAnchoredFile(credentialDirectory.anchor, "DUMMY-auth.json", sourceBytes); + } const rawDirectory = openAnchoredDirectory(proofOutput, "ten-pack/raw"); const docsDirectory = openAnchoredDirectory(proofOutput, "ten-pack/repo/docs"); try { @@ -101,6 +104,9 @@ const cases = await withCleanup( code: "ENOENT", }); proofOutputDirectory.close(); + credentialDirectory.close(); + temporaryDirectory.close(); + homeDirectory.close(); cases.push({ scenario, command: [process.execPath, ...args], @@ -116,7 +122,13 @@ const cases = await withCleanup( } return cases; }, - () => rm(scratch, { recursive: true, force: true }), + async () => { + try { + await rm(join(proofRoot, "DUMMY-cleanup"), { recursive: true, force: true }); + } finally { + scratchDirectory.close(); + } + }, ); const git = spawnSync("git", ["rev-parse", "HEAD"], { cwd: root, encoding: "utf8" }); assert.equal(git.status, 0); @@ -149,7 +161,7 @@ const manifest = { ), ), limitations: [ - "Actual full-runner expired/near-expiry and missing-source setup cases use DUMMY auth only. Launch, timeout, audit and compound cleanup failures are additionally injected at the executable supervisor interface in the unit suite; SDK timeout is exercised by the DUMMY auth-security proof.", + "Actual full-runner expired/near-expiry and missing-source setup cases use DUMMY auth only. Launch, timeout, audit and compound cleanup failures are additionally injected through stubbed mount and spawn dependencies at the in-process supervisor interface in the unit suite, not through an executable CLI boundary; SDK timeout is exercised by the DUMMY auth-security proof.", ], }; writeAnchoredFile(proofRoot, "evidence.json", `${JSON.stringify(manifest, null, 2)}\n`); diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 0a6ec05..e4a8ee4 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -154,7 +154,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { in_memory_session: true, event_count: 2, event_sha256: "0".repeat(64), - read_audit: readAudit("DUMMY-read", "/tmp/DUMMY-worker/DUMMY-original.md"), + read_audit: readAudit("DUMMY-read", "@ROOT@/DUMMY-worker/DUMMY-original.md"), refresh_callbacks: 0, credential_store: { reads: 203, lists: 1, modify_denials: 0, delete_denials: 0 }, }, @@ -228,6 +228,13 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { ...validBase, sdk_worker: { ...validBase.sdk_worker, read_audit: { exact_original: true, read_count: 1 } }, }, + { + ...validBase, + sdk_worker: { + ...validBase.sdk_worker, + read_audit: readAudit("DUMMY-read", "/etc/DUMMY-worker/DUMMY-original.md"), + }, + }, { ...validBase, sdk_worker: { @@ -267,16 +274,25 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { "valid", { read_audit: { - ...readAudit("DUMMY-actual-worker-read", "/tmp/DUMMY-outcome/docs/AUTH.md"), + ...readAudit("DUMMY-actual-worker-read", "@ROOT@/DUMMY-outcome/docs/AUTH.md"), extra: "DUMMY-SECRET", }, }, ], + [ + "valid", + { + read_audit: readAudit( + "DUMMY-actual-worker-read", + "/home/DUMMY/.pi/agent/DUMMY-outcome/docs/AUTH.md", + ), + }, + ], [ "fenced-valid", { read_audit: { - ...readAudit("DUMMY-actual-worker-read", "/tmp/DUMMY-outcome/docs/AUTH.md"), + ...readAudit("DUMMY-actual-worker-read", "@ROOT@/DUMMY-outcome/docs/AUTH.md"), returned_sha256: "9".repeat(64), }, }, @@ -297,9 +313,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { root: process.cwd(), helper: "/DUMMY-helper", piRoot: "/DUMMY-pi", - spawnSyncImpl() { - return { status: 0, stdout: JSON.stringify(payload), stderr: "" }; - }, + spawnSyncImpl: stubbedSdkProof(payload), expectedSdkDependency: trustedDependency, }), (error) => !String(error.message).includes("DUMMY-SECRET"), @@ -309,9 +323,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { root: process.cwd(), helper: "/DUMMY-helper", piRoot: "/DUMMY-pi", - spawnSyncImpl() { - return { status: 0, stdout: JSON.stringify(validBase), stderr: "" }; - }, + spawnSyncImpl: stubbedSdkProof(validBase), expectedSdkDependency: trustedDependency, }); assert.equal(accepted.result, "pass"); @@ -375,6 +387,14 @@ test("anchored runner child directories do not follow substituted paths", async } }); +function stubbedSdkProof(payload) { + return (_command, args) => ({ + status: 0, + stdout: JSON.stringify(payload).replaceAll("@ROOT@", args.at(-1)), + stderr: "", + }); +} + function authCase(scenario, result, toAuth, modifyDenials) { return { scenario, @@ -439,7 +459,7 @@ function outcome(scenario, exitCode, extra = {}) { ...extra, }; if (["valid", "fenced-valid"].includes(scenario)) { - result.read_audit = readAudit("DUMMY-actual-worker-read", "/tmp/DUMMY-outcome/docs/AUTH.md"); + result.read_audit = readAudit("DUMMY-actual-worker-read", "@ROOT@/DUMMY-outcome/docs/AUTH.md"); } return result; } From 05c496028968fc10cf246d670f1a498ffe8835aa Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 5 Oct 2026 09:19:18 +0000 Subject: [PATCH 36/45] no-mistakes(review): Pin cleanup HOME and add descriptor-bound anchored removal --- docs/probes/CP-06.md | 2 +- scripts/cp06-probe-fixture.mjs | 42 +++++++++++++++++++++++++-- scripts/prove-cp06-worker-cleanup.mjs | 7 +++-- test/cp06-proof-boundary.test.mjs | 38 +++++++++++++++++++++++- 4 files changed, 81 insertions(+), 8 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 35e960a..17f2da1 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,7 +21,7 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then observes that the actual evaluated child has empty capability sets, `no_new_privs`, and seccomp filter mode, and that its attempted remount/unmount, direct `setns`/`unshare`/`clone`/`clone3`/`mount`/`umount2` syscalls, nested namespace creation, `nsenter`, and UID change fail. The seccomp helper also installs denials for alternate mount APIs, handle-based inode reopening, ptrace and process-memory access, and the evaluated child is launched with locked root securebits requested through `setpriv`; the probe does not yet execute ptrace/process-memory or inherited-subprocess escape attempts or read back securebits, so those outcomes are configured but unproven (open finding `CP06-KERNEL-PROOF-INCOMPLETE`). On the WSL2 6.18 host used for the 2026-10-05 review rounds, the hardened probe blocks at credential mount setup with `CP06_ISOLATION_SETUP_FAILED`. The failing check is the post-mount identity check in `mountLiveCredentialReadOnly` (`credential mount identity changed: /proc//fd//DUMMY-auth.json`), while the unsafe probe, which skips the parent bind mounts, passes. Diagnosis from the code and that error (not yet confirmed by a fix): util-linux canonicalizes the `/proc//fd/` parent anchor, so the read-only parent bind covers the canonical directory, and the later leaf bind, also canonicalized, lands inside that parent bind. The identity check walks from the retained parent descriptor, which still refers to the original mount, where the target leaf is the empty placeholder rather than the pinned inode. The source check passes only because the underlying source file is the pinned inode. This is a path-handoff design defect, not a missing kernel primitive, and nothing in it is WSL-specific, so the worker path with parent protection is expected to fail the same way on other Linux hosts. The check is not weakened; the gate fails closed until the native descriptor-bound mount handoff (`CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`) lands. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. The bind mounts are still issued through util-linux `mount` with `/proc//fd` anchor paths, which util-linux canonicalizes; a native descriptor-to-descriptor mount handoff is not implemented yet (open finding `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes those directories to the runner as paths under the retained descriptors. That proof cannot complete on a host where the DUMMY security preflight fails, as on the review host above. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal and `openAnchoredDirectory(..., { reset: true })` use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink. That proof cannot complete on a host where the DUMMY security preflight fails, as on the review host above. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index ae27700..d8a82a8 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -8,7 +8,9 @@ import { mkdirSync, openSync, readSync, - rmSync, + readdirSync, + rmdirSync, + unlinkSync, writeSync, } from "node:fs"; import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; @@ -105,7 +107,7 @@ export function openAnchoredDirectory( for (const [index, component] of components.entries()) { const path = join(currentAnchor, component); if (reset && index === components.length - 1) { - rmSync(path, { recursive: true, force: true }); + removeAnchoredEntry(currentAnchor, component); } if (create) { try { @@ -133,7 +135,37 @@ export function openAnchoredDirectory( } } -export function writeAnchoredFile(parentAnchor, name, data) { +export function removeAnchoredEntry(parentAnchor, name, { beforeDescend } = {}) { + assertArtifactName(name); + const path = join(parentAnchor, name); + let entry; + try { + entry = lstatSync(path, { bigint: true }); + } catch (error) { + if (error?.code === "ENOENT") return; + throw error; + } + if (!entry.isDirectory()) { + unlinkSync(path); + return; + } + beforeDescend?.(path); + const fd = openSync(path, directoryFlags); + try { + if (!sameInode(entry, fstatSync(fd, { bigint: true }))) { + throw new Error("DUMMY proof directory identity changed during removal"); + } + const anchor = `/proc/${process.pid}/fd/${fd}`; + for (const child of readdirSync(anchor)) { + removeAnchoredEntry(anchor, child, { beforeDescend }); + } + } finally { + closeSync(fd); + } + rmdirSync(path); +} + +function assertArtifactName(name) { if ( typeof name !== "string" || name.length === 0 || @@ -144,6 +176,10 @@ export function writeAnchoredFile(parentAnchor, name, data) { ) { throw new Error("DUMMY proof artifact name is invalid"); } +} + +export function writeAnchoredFile(parentAnchor, name, data) { + assertArtifactName(name); const bytes = Buffer.from(data); const path = join(parentAnchor, name); const fd = openSync( diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index 9d70be4..da22f2c 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -2,12 +2,13 @@ import { strict as assert } from "node:assert"; import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { readFile, readdir, rm } from "node:fs/promises"; +import { readFile, readdir } from "node:fs/promises"; import { join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { openAnchoredDirectory, openProbeOutput, + removeAnchoredEntry, writeAnchoredFile, } from "./cp06-probe-fixture.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; @@ -30,7 +31,7 @@ const cases = await withCleanup( const homeDirectory = openAnchoredDirectory(scratchDirectory.anchor, scenario); const temporaryDirectory = openAnchoredDirectory(homeDirectory.anchor, "tmp"); const credentialDirectory = openAnchoredDirectory(homeDirectory.anchor, "credential"); - const home = join(scratchDirectory.anchor, scenario); + const home = homeDirectory.anchor; const temporary = join(homeDirectory.anchor, "tmp"); const proofOutputPath = join(proofRootPath, "proofs", scenario); const proofOutputDirectory = openAnchoredDirectory( @@ -124,7 +125,7 @@ const cases = await withCleanup( }, async () => { try { - await rm(join(proofRoot, "DUMMY-cleanup"), { recursive: true, force: true }); + removeAnchoredEntry(proofRoot, "DUMMY-cleanup"); } finally { scratchDirectory.close(); } diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index e4a8ee4..37a8cf4 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -1,6 +1,6 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { readlinkSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { readlinkSync, renameSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { mkdir, mkdtemp, @@ -22,6 +22,7 @@ import { import { openAnchoredDirectory, openProbeOutput, + removeAnchoredEntry, writeAnchoredFile, } from "../scripts/cp06-probe-fixture.mjs"; @@ -496,3 +497,38 @@ test("anchored artifact writes refuse substituted leaves before external mutatio await rm(external, { recursive: true, force: true }); } }); + +test("anchored removal never descends into a directory swapped for a symlink", async () => { + const root = process.cwd(); + const outputRoot = resolve(".factory/state/cp06-correction/DUMMY-anchored-removal"); + const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-removal-external-")); + const sentinel = join(external, "DUMMY-sentinel"); + await writeFile(sentinel, "DUMMY-SENTINEL-UNCHANGED"); + const output = openProbeOutput({ root, outputRoot, create: true, runner: true }); + try { + await mkdir(join(outputRoot, "DUMMY-tree", "home", "credential"), { recursive: true }); + await writeFile(join(outputRoot, "DUMMY-tree", "home", "credential", "DUMMY-auth.json"), "x"); + await symlink(external, join(outputRoot, "DUMMY-tree", "home", "DUMMY-link")); + assert.throws(() => + removeAnchoredEntry(output.anchor, "DUMMY-tree", { + beforeDescend(path) { + if (!path.endsWith("/credential")) return; + renameSync(path, `${path}-moved`); + symlinkSync(external, path); + }, + }), + ); + assert.deepEqual(await readdir(external), ["DUMMY-sentinel"]); + assert.equal(await readFile(sentinel, "utf8"), "DUMMY-SENTINEL-UNCHANGED"); + + removeAnchoredEntry(output.anchor, "DUMMY-tree"); + await assert.rejects(readdir(join(outputRoot, "DUMMY-tree")), { code: "ENOENT" }); + assert.deepEqual(await readdir(external), ["DUMMY-sentinel"]); + assert.equal(await readFile(sentinel, "utf8"), "DUMMY-SENTINEL-UNCHANGED"); + removeAnchoredEntry(output.anchor, "DUMMY-absent"); + } finally { + output.close(); + await rm(outputRoot, { recursive: true, force: true }); + await rm(external, { recursive: true, force: true }); + } +}); From 948e785d2ab8494c1dc6b4c21fe28f3de3980833 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 5 Oct 2026 09:23:25 +0000 Subject: [PATCH 37/45] no-mistakes(review): Anchor evaluation-home removal and test identity-change rejection --- docs/probes/CP-06.md | 2 +- scripts/replay-cp06-worker.mjs | 30 +++++++++++++++++++++++++----- test/cp06-proof-boundary.test.mjs | 24 ++++++++++++++++++++++++ 3 files changed, 50 insertions(+), 6 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 17f2da1..10d8d48 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -21,7 +21,7 @@ daemon. The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then observes that the actual evaluated child has empty capability sets, `no_new_privs`, and seccomp filter mode, and that its attempted remount/unmount, direct `setns`/`unshare`/`clone`/`clone3`/`mount`/`umount2` syscalls, nested namespace creation, `nsenter`, and UID change fail. The seccomp helper also installs denials for alternate mount APIs, handle-based inode reopening, ptrace and process-memory access, and the evaluated child is launched with locked root securebits requested through `setpriv`; the probe does not yet execute ptrace/process-memory or inherited-subprocess escape attempts or read back securebits, so those outcomes are configured but unproven (open finding `CP06-KERNEL-PROOF-INCOMPLETE`). On the WSL2 6.18 host used for the 2026-10-05 review rounds, the hardened probe blocks at credential mount setup with `CP06_ISOLATION_SETUP_FAILED`. The failing check is the post-mount identity check in `mountLiveCredentialReadOnly` (`credential mount identity changed: /proc//fd//DUMMY-auth.json`), while the unsafe probe, which skips the parent bind mounts, passes. Diagnosis from the code and that error (not yet confirmed by a fix): util-linux canonicalizes the `/proc//fd/` parent anchor, so the read-only parent bind covers the canonical directory, and the later leaf bind, also canonicalized, lands inside that parent bind. The identity check walks from the retained parent descriptor, which still refers to the original mount, where the target leaf is the empty placeholder rather than the pinned inode. The source check passes only because the underlying source file is the pinned inode. This is a path-handoff design defect, not a missing kernel primitive, and nothing in it is WSL-specific, so the worker path with parent protection is expected to fail the same way on other Linux hosts. The check is not weakened; the gate fails closed until the native descriptor-bound mount handoff (`CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`) lands. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. The bind mounts are still issued through util-linux `mount` with `/proc//fd` anchor paths, which util-linux canonicalizes; a native descriptor-to-descriptor mount handoff is not implemented yet (open finding `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal and `openAnchoredDirectory(..., { reset: true })` use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink. That proof cannot complete on a host where the DUMMY security preflight fails, as on the review host above. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal, `openAnchoredDirectory(..., { reset: true })`, and the replay runner's removal of its evaluation home (created by `mkdtemp` relative to a retained no-follow `TMPDIR` descriptor) use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink or for a different real directory. That proof cannot complete on a host where the DUMMY security preflight fails, as on the review host above. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index 6252290..a3af530 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -1,15 +1,16 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; import { spawnSync } from "node:child_process"; -import { realpathSync } from "node:fs"; -import { mkdtemp, readFile, rm } from "node:fs/promises"; +import { closeSync, constants, openSync, realpathSync } from "node:fs"; +import { mkdtemp, readFile } from "node:fs/promises"; import { homedir, tmpdir } from "node:os"; -import { dirname, join, resolve } from "node:path"; +import { basename, dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; import { openAnchoredDirectory, openProbeOutput, + removeAnchoredEntry, writeAnchoredFile, } from "./cp06-probe-fixture.mjs"; import { @@ -83,7 +84,20 @@ async function runWorker(workerRoot) { }; const inputPath = join(workerRoot, "input.json"); writeAnchoredFile(workerRoot, "input.json", `${JSON.stringify(workerInput, null, 2)}\n`); - const evaluationHome = await mkdtemp(join(tmpdir(), "factory-cp06-sdk-worker-")); + const temporaryRoot = realpathSync(tmpdir()); + const temporaryFd = openSync( + temporaryRoot, + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); + const temporaryAnchor = `/proc/${process.pid}/fd/${temporaryFd}`; + let evaluationName; + try { + evaluationName = basename(await mkdtemp(join(temporaryAnchor, "factory-cp06-sdk-worker-"))); + } catch (error) { + closeSync(temporaryFd); + throw error; + } + const evaluationHome = join(temporaryRoot, evaluationName); const credentialTarget = join(evaluationHome, "pi-agent", "auth.json"); const credentialSource = resolve( process.env.CP06_PI_AUTH_FILE ?? @@ -102,7 +116,13 @@ async function runWorker(workerRoot) { inputPath, timeout: timeoutMs + 60_000, }), - () => rm(evaluationHome, { recursive: true, force: true }), + () => { + try { + removeAnchoredEntry(temporaryAnchor, evaluationName); + } finally { + closeSync(temporaryFd); + } + }, ); } catch (error) { await recordBlocked(workerRoot, error, "isolated-sdk-worker"); diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 37a8cf4..61f751d 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -521,6 +521,30 @@ test("anchored removal never descends into a directory swapped for a symlink", a assert.deepEqual(await readdir(external), ["DUMMY-sentinel"]); assert.equal(await readFile(sentinel, "utf8"), "DUMMY-SENTINEL-UNCHANGED"); + await rm(join(outputRoot, "DUMMY-tree", "home", "credential")); + await mkdir(join(outputRoot, "DUMMY-tree", "home", "credential")); + const substitute = join(outputRoot, "DUMMY-substitute"); + await mkdir(substitute); + await writeFile(join(substitute, "DUMMY-substitute-sentinel"), "DUMMY-SUBSTITUTE-UNCHANGED"); + assert.throws( + () => + removeAnchoredEntry(output.anchor, "DUMMY-tree", { + beforeDescend(path) { + if (!path.endsWith("/credential")) return; + renameSync(path, `${path}-replaced`); + renameSync(substitute, path); + }, + }), + /identity changed during removal/, + ); + assert.equal( + await readFile( + join(outputRoot, "DUMMY-tree", "home", "credential", "DUMMY-substitute-sentinel"), + "utf8", + ), + "DUMMY-SUBSTITUTE-UNCHANGED", + ); + removeAnchoredEntry(output.anchor, "DUMMY-tree"); await assert.rejects(readdir(join(outputRoot, "DUMMY-tree")), { code: "ENOENT" }); assert.deepEqual(await readdir(external), ["DUMMY-sentinel"]); From fdc83217b9216485b817511cddcb9846fa005d74 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 5 Oct 2026 09:57:06 +0000 Subject: [PATCH 38/45] Recover timed-out CP06 mount/CLI fix-agent edits Preserves uncommitted work from no-mistakes run 01M45P9Q8DNTVQ4A79YVW11S6K, whose review fix turn timed out at 25m before committing. Snapshot taken read-only from the run worktree shortly before the timeout (base 948e785). Partial: fd-based mount handoff in the native helper and CLI shims/tests. With it the hardened DUMMY probe passes mount setup (baseline 948e785 fails CP06_ISOLATION_SETUP_FAILED) but now fails "direct syscall probe failed to execute"; cp06 tests 77 pass / 0 fail / 1 skip (SDK-dependent). Needs pipeline review and completion; not a validated fix. Co-Authored-By: Claude Opus 5.5 --- scripts/cp06-auth-security.mjs | 151 +++++++++------ scripts/cp06-credential-isolation.mjs | 190 ++++++++++++------- scripts/cp06-dummy-cli-shims.mjs | 145 ++++++++++++++ scripts/cp06-namespace-supervisor.mjs | 9 +- scripts/cp06-seccomp-exec.c | 93 +++++++++ scripts/prove-cp06-worker-cleanup.mjs | 62 +++++- test/cp06-credential-isolation.test.mjs | 241 +++++++----------------- test/cp06-proof-boundary.test.mjs | 6 +- test/cp06-worker-lifecycle.test.mjs | 117 +++++++++++- test/fixtures/cp06-compiled-helper.mjs | 24 +++ test/fixtures/cp06-credential-mount.mjs | 108 +++++++++++ 11 files changed, 825 insertions(+), 321 deletions(-) create mode 100644 scripts/cp06-dummy-cli-shims.mjs create mode 100644 test/fixtures/cp06-compiled-helper.mjs create mode 100644 test/fixtures/cp06-credential-mount.mjs diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 2b225a6..072de13 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -260,29 +260,52 @@ export async function runIsolationProbe({ } function runNamespace(options) { - const args = [ - "--user", - "--map-root-user", - "--mount", - "--propagation", - "private", - "--", - process.execPath, - join(options.root, "scripts/cp06-namespace-supervisor.mjs"), - options.mode, - resolve(options.source), - options.mode === "worker" ? resolve(options.target) : "-", - resolve(options.helper), - resolve(options.syscallProbe), - ...options.childArgs.map((value) => resolve(value)), - ]; - return spawnSync("unshare", args, { - cwd: options.root, - encoding: "utf8", - timeout: options.timeout, - maxBuffer: 1024 * 1024, - env: options.environment ?? process.env, - }); + return withInheritedExecutables([options.helper, options.syscallProbe], ([helper, probe]) => + spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + join(options.root, "scripts/cp06-namespace-supervisor.mjs"), + options.mode, + resolve(options.source), + options.mode === "worker" ? resolve(options.target) : "-", + helper.path, + probe.path, + ...options.childArgs.map((value) => resolve(value)), + ], + { + cwd: options.root, + encoding: "utf8", + timeout: options.timeout, + maxBuffer: 1024 * 1024, + env: options.environment ?? process.env, + stdio: ["pipe", "pipe", "pipe", helper.fd, probe.fd], + }, + ), + ); +} + +// A new user namespace cannot open the runner's /proc//fd entries, so the +// retained execution descriptors are inherited as fds 3+ instead of named. +function withInheritedExecutables(paths, action) { + const descriptors = []; + try { + for (const [index, path] of paths.entries()) { + descriptors.push({ + fd: openSync(resolve(path), constants.O_RDONLY | constants.O_CLOEXEC), + path: `/proc/self/fd/${index + 3}`, + }); + } + return action(descriptors); + } finally { + for (const { fd } of descriptors) closeSync(fd); + } } function isSupervisorFailure(value, status) { @@ -447,43 +470,51 @@ function runDummySdkProofIn({ expectedSdkDependency, proofRoot, }) { - const result = spawnSyncImpl( - "unshare", - [ - "--user", - "--map-root-user", - "--mount", - "--propagation", - "private", - "--", - "setpriv", - "--no-new-privs", - "--bounding-set=-all", - "--inh-caps=-all", - "--ambient-caps=-all", - "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", - "--", - helper, - process.execPath, - "--experimental-import-meta-resolve", - join(root, "scripts/cp06-dummy-sdk-proof.mjs"), - piRoot, - proofRoot, - ], - { - cwd: root, - encoding: "utf8", - env: { - PATH: process.env.PATH ?? "/usr/bin:/bin", - LANG: process.env.LANG ?? "C.UTF-8", - PI_OFFLINE: "1", - PI_SKIP_VERSION_CHECK: "1", - PI_TELEMETRY: "0", - }, - timeout: 120_000, - maxBuffer: 1024 * 1024, - }, - ); + let result; + try { + result = withInheritedExecutables([helper], ([inherited]) => + spawnSyncImpl( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + "setpriv", + "--no-new-privs", + "--bounding-set=-all", + "--inh-caps=-all", + "--ambient-caps=-all", + "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", + "--", + inherited.path, + process.execPath, + "--experimental-import-meta-resolve", + join(root, "scripts/cp06-dummy-sdk-proof.mjs"), + piRoot, + proofRoot, + ], + { + cwd: root, + encoding: "utf8", + env: { + PATH: process.env.PATH ?? "/usr/bin:/bin", + LANG: process.env.LANG ?? "C.UTF-8", + PI_OFFLINE: "1", + PI_SKIP_VERSION_CHECK: "1", + PI_TELEMETRY: "0", + }, + timeout: 120_000, + maxBuffer: 1024 * 1024, + stdio: ["pipe", "pipe", "pipe", inherited.fd], + }, + ), + ); + } catch { + throw unsupported("DUMMY SDK proof failed: CP06_DUMMY_SDK_LAUNCH_FAILED"); + } if (result.error !== undefined || result.status !== 0) { const code = result.error?.code === "ETIMEDOUT" diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs index e09dcee..c607d59 100644 --- a/scripts/cp06-credential-isolation.mjs +++ b/scripts/cp06-credential-isolation.mjs @@ -35,6 +35,7 @@ export async function proveDummyCredentialIsolation(options = {}) { root, source, target, + mountHelper: options.mountHelper, spawnSyncImpl: options.spawnSyncImpl, }); await assertDestructiveCredentialWritesDenied([source, target]); @@ -58,39 +59,32 @@ export async function proveDummyCredentialIsolation(options = {}) { /** * Pin one regular credential inode, bind it over both its source path and a fresh * regular target, and make both mount instances read-only before closing the FD. + * The native helper consumes the retained source and destination descriptors. * This function must run only in the private setup namespace. */ export async function mountLiveCredentialReadOnly(options) { const root = resolve(options.root ?? process.cwd()); const source = resolveRequiredPath(options.source, "credential source"); const target = resolveRequiredPath(options.target, "credential target"); + const helper = resolveRequiredPath(options.mountHelper, "credential mount helper"); const maxBytes = options.maxBytes ?? MAX_CREDENTIAL_BYTES; const spawn = options.spawnSyncImpl ?? spawnSync; const statPath = options.lstatImpl ?? lstat; const protectParents = options.protectParents !== false; + const sourceLeaf = options.sourceLeaf ?? basename(source); + const targetLeaf = options.targetLeaf ?? basename(target); let handle; let sourceParentHandle; let targetParentHandle; - let closeSourceParentHandle = true; - let closeTargetParentHandle = true; - let sourceMounted = false; - let targetMounted = false; let sourceMountPath = source; let targetMountPath = target; - const mountedParents = []; + const mountStack = []; let parentHandlesTransferred = false; try { - let sourceParentPath = options.sourceParentAnchor; - if (sourceParentPath === undefined) { - sourceParentHandle = await open( - dirname(source), - constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, - ); - sourceParentPath = `/proc/${process.pid}/fd/${sourceParentHandle.fd}`; - } else { - closeSourceParentHandle = false; - } - sourceMountPath = join(sourceParentPath, options.sourceLeaf ?? basename(source)); + sourceParentHandle = await openParent(options.sourceParentAnchor, dirname(source)); + const sourceParentPath = + options.sourceParentAnchor ?? `/proc/${process.pid}/fd/${sourceParentHandle.fd}`; + sourceMountPath = join(sourceParentPath, sourceLeaf); handle = await open( sourceMountPath, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, @@ -99,45 +93,33 @@ export async function mountLiveCredentialReadOnly(options) { const named = await statPath(sourceMountPath, { bigint: true }); assertPinnedRegularFile(pinned, named, maxBytes); - let targetParentPath = options.targetParentAnchor; - if (targetParentPath === undefined) { + if (options.targetParentAnchor === undefined) { await mkdir(dirname(target), { recursive: true, mode: 0o700 }); - targetParentHandle = await open( - dirname(target), - constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, - ); - targetParentPath = `/proc/${process.pid}/fd/${targetParentHandle.fd}`; - } else { - closeTargetParentHandle = false; } - targetMountPath = join(targetParentPath, options.targetLeaf ?? basename(target)); - await writeFile(targetMountPath, "", { flag: "wx", mode: 0o600 }); + targetParentHandle = await openParent(options.targetParentAnchor, dirname(target)); + const targetParentPath = + options.targetParentAnchor ?? `/proc/${process.pid}/fd/${targetParentHandle.fd}`; + targetMountPath = join(targetParentPath, targetLeaf); + const placeholder = await createPlaceholder(targetMountPath); + await options.beforeMount?.({ sourceMountPath, targetMountPath }); + + const mountOptions = { helper, root, spawn }; + bindFile(mountOptions, handle, pinned, sourceParentHandle, sourceLeaf, pinned, source); + mountStack.push({ path: sourceMountPath, recursive: false }); + bindFile(mountOptions, handle, pinned, targetParentHandle, targetLeaf, placeholder, target); + mountStack.push({ path: targetMountPath, recursive: false }); const parentMounts = []; if (protectParents) { - for (const parent of [...new Set([sourceParentPath, targetParentPath])]) { - checkedSpawn("mount", ["--bind", parent, parent], { root, spawn }); - mountedParents.push(parent); - checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", parent], { - root, - spawn, - }); - parentMounts.push(inspectReadOnlyMount(parent, { spawn })); + const parents = await uniqueParents([ + { handle: sourceParentHandle, path: sourceParentPath, visible: dirname(source) }, + { handle: targetParentHandle, path: targetParentPath, visible: dirname(target) }, + ]); + for (const parent of parents) { + bindTree(mountOptions, parent); + mountStack.push({ path: parent.path, recursive: true }); + parentMounts.push(inspectReadOnlyMount(parent.path, { spawn })); } } - const pinnedPath = `/proc/${process.pid}/fd/${handle.fd}`; - - checkedSpawn("mount", ["--bind", pinnedPath, sourceMountPath], { root, spawn }); - sourceMounted = true; - checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", sourceMountPath], { - root, - spawn, - }); - checkedSpawn("mount", ["--bind", pinnedPath, targetMountPath], { root, spawn }); - targetMounted = true; - checkedSpawn("mount", ["-o", "remount,bind,ro,nosuid,nodev,noexec", targetMountPath], { - root, - spawn, - }); const sourceMount = inspectReadOnlyMount(sourceMountPath, { spawn }); const targetMount = inspectReadOnlyMount(targetMountPath, { spawn }); @@ -181,25 +163,19 @@ export async function mountLiveCredentialReadOnly(options) { if (cleaned) return; cleaned = true; const failures = []; - if (targetMounted) unmount(targetMountPath, { root, spawn, failures }); - if (sourceMounted) unmount(sourceMountPath, { root, spawn, failures }); - for (const parent of [...mountedParents].reverse()) { - unmount(parent, { root, spawn, failures }); - } - const closeFailures = []; - try { - if (closeSourceParentHandle) await sourceParentHandle?.close(); - } catch (error) { - closeFailures.push(error?.message ?? "source parent close failed"); - } - try { - if (closeTargetParentHandle) await targetParentHandle?.close(); - } catch (error) { - closeFailures.push(error?.message ?? "target parent close failed"); + unmountAll(mountStack, { root, spawn, failures }); + for (const [label, parentHandle] of [ + ["source", sourceParentHandle], + ["target", targetParentHandle], + ]) { + try { + await parentHandle?.close(); + } catch (error) { + failures.push(error?.message ?? `${label} parent close failed`); + } } sourceParentHandle = undefined; targetParentHandle = undefined; - failures.push(...closeFailures); if (failures.length > 0) { throw new Error(`credential mount cleanup failed: ${failures.join("; ")}`); } @@ -207,11 +183,7 @@ export async function mountLiveCredentialReadOnly(options) { }; } catch (error) { const cleanupFailures = []; - if (targetMounted) unmount(targetMountPath, { root, spawn, failures: cleanupFailures }); - if (sourceMounted) unmount(sourceMountPath, { root, spawn, failures: cleanupFailures }); - for (const parent of [...mountedParents].reverse()) { - unmount(parent, { root, spawn, failures: cleanupFailures }); - } + unmountAll(mountStack, { root, spawn, failures: cleanupFailures }); if (cleanupFailures.length > 0) { throw new AggregateError( [error, ...cleanupFailures.map((message) => new Error(message))], @@ -222,12 +194,83 @@ export async function mountLiveCredentialReadOnly(options) { } finally { await handle?.close(); if (!parentHandlesTransferred) { - if (closeSourceParentHandle) await sourceParentHandle?.close(); - if (closeTargetParentHandle) await targetParentHandle?.close(); + await sourceParentHandle?.close(); + await targetParentHandle?.close(); + } + } +} + +async function openParent(anchor, path) { + return open( + anchor ?? path, + constants.O_RDONLY | + constants.O_DIRECTORY | + constants.O_CLOEXEC | + (anchor === undefined ? constants.O_NOFOLLOW : 0), + ); +} + +async function createPlaceholder(path) { + const placeholder = await open( + path, + constants.O_WRONLY | + constants.O_CREAT | + constants.O_EXCL | + constants.O_NOFOLLOW | + constants.O_CLOEXEC, + 0o600, + ); + try { + const identity = await placeholder.stat({ bigint: true }); + if (!identity.isFile() || identity.nlink !== 1n) { + throw new Error("credential target placeholder is not one fresh regular file"); } + return identity; + } finally { + await placeholder.close(); } } +async function uniqueParents(parents) { + const unique = new Map(); + for (const parent of parents) { + const identity = await parent.handle.stat({ bigint: true }); + const key = `${identity.dev}:${identity.ino}`; + if (!unique.has(key)) unique.set(key, { ...parent, identity }); + } + return [...unique.values()].sort( + (left, right) => right.visible.split("/").length - left.visible.split("/").length, + ); +} + +function bindFile(options, sourceHandle, sourceIdentity, parentHandle, leaf, expected, visible) { + checkedSpawn( + options.helper, + [ + "--cp06-bind-file", + String(sourceIdentity.dev), + String(sourceIdentity.ino), + leaf, + String(expected.dev), + String(expected.ino), + visible, + ], + { ...options, fds: [sourceHandle.fd, parentHandle.fd] }, + ); +} + +function bindTree(options, parent) { + checkedSpawn( + options.helper, + ["--cp06-bind-tree", String(parent.identity.dev), String(parent.identity.ino), parent.visible], + { ...options, fds: [parent.handle.fd] }, + ); +} + +function unmountAll(mounted, options) { + for (const entry of [...mounted].reverse()) unmount(entry.path, { ...options, ...entry }); +} + export async function assertDestructiveCredentialWritesDenied(paths, operations = {}) { const write = operations.writeFile ?? writeFile; const move = operations.rename ?? rename; @@ -333,6 +376,7 @@ function checkedSpawn(command, args, options) { const result = options.spawn(command, args, { cwd: options.root, encoding: "utf8", + stdio: ["ignore", "pipe", "pipe", ...(options.fds ?? [])], }); if (result.error !== undefined) throw result.error; if (result.status !== 0) { @@ -344,7 +388,7 @@ function checkedSpawn(command, args, options) { } function unmount(path, options) { - const result = options.spawn("umount", [path], { + const result = options.spawn("umount", options.recursive ? ["--recursive", path] : [path], { cwd: options.root, encoding: "utf8", }); diff --git a/scripts/cp06-dummy-cli-shims.mjs b/scripts/cp06-dummy-cli-shims.mjs new file mode 100644 index 0000000..be28947 --- /dev/null +++ b/scripts/cp06-dummy-cli-shims.mjs @@ -0,0 +1,145 @@ +// Development-only DUMMY executable shims for CLI failure-composition proofs. +// Only the evaluated worker launch and the credential-target unmount are +// replaced; every other setpriv/umount invocation runs the real command. +import { accessSync, closeSync, constants, openSync, writeSync } from "node:fs"; +import { delimiter, join } from "node:path"; + +const AUDIT = { + stage: "audit", + code: "CP06_CHILD_OUTPUT_INVALID", + exit_code: 70, + description: "isolated child output was invalid", +}; +const CHILD_EXIT = { + stage: "child-exit", + code: "CP06_CHILD_EXIT_FAILED", + exit_code: 1, + description: "isolated child exited unsuccessfully", +}; +const CLEANUP = { + stage: "cleanup", + code: "CP06_CREDENTIAL_CLEANUP_FAILED", + exit_code: null, + description: "credential mount cleanup failed", +}; +const compound = (causes) => ({ + schema_version: 1, + stage: "cleanup", + code: "CP06_CLEANUP_FAILED", + exit_code: 74, + description: "credential namespace cleanup failed", + causes, +}); + +/** Child behavior, injected cleanup failure and the expected supervisor record. */ +export const DUMMY_CLI_SCENARIOS = { + "malformed-output-and-cleanup": { + child: "malformed", + cleanupFails: true, + record: compound([AUDIT, CLEANUP]), + }, + "semantic-audit-and-cleanup": { + child: "semantic", + cleanupFails: true, + record: compound([AUDIT, CLEANUP]), + }, + "child-exit-and-cleanup": { + child: "exit", + cleanupFails: true, + record: compound([CHILD_EXIT, CLEANUP]), + }, + "audit-only": { + child: "malformed", + cleanupFails: false, + record: { schema_version: 1, ...AUDIT, causes: [] }, + }, + "cleanup-only": { child: "valid", cleanupFails: true, record: compound([CLEANUP]) }, + success: { child: "valid", cleanupFails: false, record: null }, +}; + +export const DUMMY_CLI_SECRET = "DUMMY-SECRET-CLI-MARKER"; + +const VALID_WORKER_CHILD = { + schema_version: 1, + result: "pass", + pi_version: "0.85.1", + pi_install: { fixture_origin: true }, + provider: "DUMMY", + model: "DUMMY", + credential_store: { fixture_origin: true }, + oauth_preflight: { refreshed: false }, + session: { in_memory: true, active_tools: ["read"] }, + event_stream: { read_audit: { exact_original: true, read_count: 1 } }, + response: { fixture_origin: true }, + response_sha256: "0".repeat(64), + raw_transcript_retained: false, +}; + +/** Write scenario shims into `anchor`; returns a PATH that resolves them first. */ +export function writeDummyCliShims({ anchor, path, scenario, inheritedPath = process.env.PATH }) { + const selected = DUMMY_CLI_SCENARIOS[scenario]; + if (selected === undefined) throw new Error("unsupported DUMMY CLI scenario"); + const realSetpriv = locate("setpriv", inheritedPath); + const realUmount = locate("umount", inheritedPath); + const child = { + malformed: `printf '%s' '${DUMMY_CLI_SECRET} malformed'; printf '%s' '${DUMMY_CLI_SECRET}' >&2; exit 0`, + semantic: `printf '%s' '{"result":"fail","detail":"${DUMMY_CLI_SECRET}"}'; exit 0`, + exit: `printf '%s' '${DUMMY_CLI_SECRET}' >&2; exit 1`, + valid: `printf '%s' '${JSON.stringify(VALID_WORKER_CHILD)}'; exit 0`, + }[selected.child]; + writeExecutable( + anchor, + "setpriv", + `#!/bin/sh +for argument do + case "$argument" in + */scripts/cp06-sdk-worker.mjs) ${child} ;; + esac +done +exec '${realSetpriv}' "$@" +`, + ); + writeExecutable( + anchor, + "umount", + `#!/bin/sh +for argument do last="$argument"; done +case "$last" in + */auth.json) ${selected.cleanupFails ? `printf '%s' '${DUMMY_CLI_SECRET}' >&2; exit 32` : ":"} ;; +esac +exec '${realUmount}' "$@" +`, + ); + return `${path}${delimiter}${inheritedPath}`; +} + +function locate(command, searchPath = "") { + for (const directory of searchPath.split(delimiter)) { + if (!directory.startsWith("/")) continue; + const candidate = join(directory, command); + try { + accessSync(candidate, constants.X_OK); + return candidate; + } catch {} + } + throw new Error(`DUMMY CLI shim cannot locate ${command}`); +} + +function writeExecutable(anchor, name, text) { + const fd = openSync( + join(anchor, name), + constants.O_WRONLY | + constants.O_CREAT | + constants.O_EXCL | + constants.O_NOFOLLOW | + constants.O_CLOEXEC, + 0o700, + ); + try { + const bytes = Buffer.from(text); + let offset = 0; + while (offset < bytes.length) offset += writeSync(fd, bytes, offset, bytes.length - offset); + } finally { + closeSync(fd); + } +} diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 828ca06..a05ee3f 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -59,6 +59,7 @@ export async function superviseCredentialChild(options, dependencies = {}) { root, source, target, + mountHelper: helper, protectParents: mode !== "unsafe-probe", sourceParentAnchor: probe?.credentialAnchors?.sourceParent, targetParentAnchor: probe?.credentialAnchors?.targetParent, @@ -403,7 +404,13 @@ const FAILURE_DESCRIPTIONS = { }; if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { - const [mode, source, target, helper, syscallProbe, ...childArgs] = process.argv.slice(2); + const [mode, source, target, inheritedHelper, inheritedProbe, ...childArgs] = + process.argv.slice(2); + const [helper, syscallProbe] = [inheritedHelper, inheritedProbe].map((path) => + /^\/proc\/self\/fd\/\d+$/u.test(path ?? "") + ? `/proc/${process.pid}/fd/${path.slice("/proc/self/fd/".length)}` + : path, + ); try { console.log( JSON.stringify( diff --git a/scripts/cp06-seccomp-exec.c b/scripts/cp06-seccomp-exec.c index 5ead15a..b77a02f 100644 --- a/scripts/cp06-seccomp-exec.c +++ b/scripts/cp06-seccomp-exec.c @@ -1,11 +1,15 @@ #define _GNU_SOURCE #include +#include #include #include #include #include #include +#include +#include #include +#include #include #include @@ -55,7 +59,96 @@ static void close_inherited_fds(void) { for (int fd = 3; fd < limit; ++fd) close(fd); } +/* + * Descriptor-bound read-only bind mounts for the private setup namespace. The + * source tree comes from a retained descriptor and the destination is an + * identity-checked O_PATH descriptor, so no mount input is resolved by name. + */ +static void refuse(const char *message) { + fprintf(stderr, "%s\n", message); + _exit(65); +} + +static int parse_u64(const char *text, uint64_t *value) { + char *end = NULL; + if (text == NULL || text[0] < '0' || text[0] > '9') return -1; + errno = 0; + unsigned long long parsed = strtoull(text, &end, 10); + if (errno != 0 || end == NULL || *end != '\0') return -1; + *value = (uint64_t)parsed; + return 0; +} + +static int has_identity(int fd, mode_t type, const char *dev, const char *ino) { + struct stat st; + uint64_t expected_dev; + uint64_t expected_ino; + return parse_u64(dev, &expected_dev) == 0 && parse_u64(ino, &expected_ino) == 0 && + fstat(fd, &st) == 0 && (st.st_mode & S_IFMT) == type && + (uint64_t)st.st_dev == expected_dev && (uint64_t)st.st_ino == expected_ino; +} + +static void require_visible(const char *path, mode_t type, const char *dev, const char *ino) { + int fd = open(path, O_PATH | O_NOFOLLOW | O_CLOEXEC); + if (fd < 0 || !has_identity(fd, type, dev, ino)) { + refuse("CP-06 mount destination path identity changed"); + } + close(fd); +} + +static void attach_read_only(int source_fd, unsigned int recursive, int destination_fd) { + int tree = (int)syscall(SYS_open_tree, source_fd, "", + OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC | AT_EMPTY_PATH | recursive); + if (tree < 0) fail("open_tree"); + struct mount_attr attributes = { + .attr_set = MOUNT_ATTR_RDONLY | MOUNT_ATTR_NOSUID | MOUNT_ATTR_NODEV | MOUNT_ATTR_NOEXEC, + }; + if (syscall(SYS_mount_setattr, tree, "", AT_EMPTY_PATH | recursive, &attributes, + sizeof(attributes)) != 0) { + fail("mount_setattr"); + } + if (syscall(SYS_move_mount, tree, "", destination_fd, "", + MOVE_MOUNT_F_EMPTY_PATH | MOVE_MOUNT_T_EMPTY_PATH) != 0) { + fail("move_mount"); + } + close(tree); +} + +/* fd 3: pinned regular source; fd 4: retained destination parent directory. */ +static int bind_file(int argc, char **argv) { + if (argc != 8 || argv[4][0] == '\0' || strchr(argv[4], '/') != NULL || + strcmp(argv[4], ".") == 0 || strcmp(argv[4], "..") == 0) { + fprintf(stderr, "usage: cp06-seccomp-exec --cp06-bind-file SRC_DEV SRC_INO LEAF DEV INO PATH\n"); + return 64; + } + if (!has_identity(3, S_IFREG, argv[2], argv[3])) refuse("CP-06 mount source identity changed"); + int destination = openat(4, argv[4], O_PATH | O_NOFOLLOW | O_CLOEXEC); + if (destination < 0 || !has_identity(destination, S_IFREG, argv[5], argv[6])) { + refuse("CP-06 mount destination identity changed"); + } + require_visible(argv[7], S_IFREG, argv[5], argv[6]); + attach_read_only(3, 0, destination); + close(destination); + return 0; +} + +/* fd 3: retained directory, bound read-only over itself with its submounts. */ +static int bind_tree(int argc, char **argv) { + if (argc != 5) { + fprintf(stderr, "usage: cp06-seccomp-exec --cp06-bind-tree DEV INO PATH\n"); + return 64; + } + if (!has_identity(3, S_IFDIR, argv[2], argv[3])) { + refuse("CP-06 mount directory identity changed"); + } + require_visible(argv[4], S_IFDIR, argv[2], argv[3]); + attach_read_only(3, AT_RECURSIVE, 3); + return 0; +} + int main(int argc, char **argv) { + if (argc >= 2 && strcmp(argv[1], "--cp06-bind-file") == 0) return bind_file(argc, argv); + if (argc >= 2 && strcmp(argv[1], "--cp06-bind-tree") == 0) return bind_tree(argc, argv); if (argc < 2) { fprintf(stderr, "usage: cp06-seccomp-exec COMMAND [ARG...]\n"); return 64; diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index da22f2c..c6744ba 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -12,6 +12,11 @@ import { writeAnchoredFile, } from "./cp06-probe-fixture.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; +import { + DUMMY_CLI_SCENARIOS, + DUMMY_CLI_SECRET, + writeDummyCliShims, +} from "./cp06-dummy-cli-shims.mjs"; const root = resolve(fileURLToPath(new URL("..", import.meta.url))); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); @@ -23,10 +28,14 @@ const scratchDirectory = openAnchoredDirectory(proofRoot, "DUMMY-cleanup", { res const cases = await withCleanup( async () => { const cases = []; - for (const [scenario, expectedExit] of [ + const compoundScenarios = Object.entries(DUMMY_CLI_SCENARIOS) + .filter(([, { record }]) => record !== null) + .map(([scenario, { record }]) => [scenario, record.exit_code, record]); + for (const [scenario, expectedExit, expectedRecord] of [ ["expired", 75], ["near-expiry", 75], ["missing-source-setup", 70], + ...compoundScenarios, ]) { const homeDirectory = openAnchoredDirectory(scratchDirectory.anchor, scenario); const temporaryDirectory = openAnchoredDirectory(homeDirectory.anchor, "tmp"); @@ -41,12 +50,22 @@ const cases = await withCleanup( ); const proofOutput = proofOutputDirectory.anchor; const source = join(homeDirectory.anchor, "credential", "DUMMY-auth.json"); + const namespaceSource = join( + proofRootPath, + "DUMMY-cleanup", + scenario, + "credential", + "DUMMY-auth.json", + ); const sourceBytes = JSON.stringify({ "openai-codex": { type: "oauth", access: "DUMMY-ACCESS", refresh: "DUMMY-REFRESH", - expires: scenario === "expired" ? 0 : Date.now() + 60_000, + expires: + scenario === "expired" + ? 0 + : Date.now() + (expectedRecord === undefined ? 60_000 : 3_600_000), }, }); if (scenario !== "missing-source-setup") { @@ -69,6 +88,19 @@ const cases = await withCleanup( rawDirectory.close(); docsDirectory.close(); } + let path = process.env.PATH; + if (expectedRecord !== undefined) { + const shimDirectory = openAnchoredDirectory(homeDirectory.anchor, "shims"); + try { + path = writeDummyCliShims({ + anchor: shimDirectory.anchor, + path: join(proofRootPath, "DUMMY-cleanup", scenario, "shims"), + scenario, + }); + } finally { + shimDirectory.close(); + } + } const args = [ "--experimental-import-meta-resolve", join(root, "scripts/replay-cp06-worker.mjs"), @@ -79,12 +111,12 @@ const cases = await withCleanup( timeout: 180_000, maxBuffer: 1024 * 1024, env: { - PATH: process.env.PATH, + PATH: path, HOME: home, TMPDIR: temporary, PI_OFFLINE: "1", PI_SKIP_VERSION_CHECK: "1", - CP06_PI_AUTH_FILE: source, + CP06_PI_AUTH_FILE: namespaceSource, CP06_PI_PACKAGE_ROOT: process.env.CP06_PI_PACKAGE_ROOT, CP06_PI_BIN: process.env.CP06_PI_BIN, CP06_OUTPUT: proofOutputPath, @@ -93,11 +125,21 @@ const cases = await withCleanup( }); assert.equal(result.error, undefined); assert.equal(result.status, expectedExit, result.stderr); - const blocked = JSON.parse( - await readFile(join(proofOutputPath, "worker/blocked.json"), "utf8"), - ); + const blockedText = await readFile(join(proofOutputPath, "worker/blocked.json"), "utf8"); + const blocked = JSON.parse(blockedText); assert.equal(blocked.result, "blocked"); assert.equal(blocked.stage, "isolated-sdk-worker"); + if (expectedRecord !== undefined) { + assert.equal(result.stdout, ""); + assert.equal( + result.stderr, + `${expectedRecord.code}: CP-06 isolated SDK worker exited ${expectedExit}\n`, + ); + assert.equal(`${result.stderr}${blockedText}`.includes(DUMMY_CLI_SECRET), false); + assert.deepEqual(blocked.worker_failure, expectedRecord); + assert.equal(blocked.code, expectedRecord.code); + assert.equal(blocked.cleanup_failed, expectedRecord.code === "CP06_CLEANUP_FAILED"); + } assert.deepEqual(await readdir(temporary), [], `${scenario}: evaluation home leaked`); if (scenario !== "missing-source-setup") assert.equal(await readFile(source, "utf8"), sourceBytes); @@ -115,6 +157,8 @@ const cases = await withCleanup( expected_exit_code: expectedExit, stage: blocked.stage, worker_failure: blocked.worker_failure, + child_and_cleanup_injection: + expectedRecord === undefined ? null : "DUMMY setpriv/umount PATH shims", evaluation_home_removed: true, source_unchanged: scenario === "missing-source-setup" ? null : true, fixture_origin: true, @@ -136,7 +180,9 @@ assert.equal(git.status, 0); const sources = [ "scripts/replay-cp06-worker.mjs", "scripts/cp06-namespace-supervisor.mjs", + "scripts/cp06-credential-isolation.mjs", "scripts/cp06-worker-lifecycle.mjs", + "scripts/cp06-dummy-cli-shims.mjs", "scripts/prove-cp06-worker-cleanup.mjs", ]; const manifest = { @@ -162,7 +208,7 @@ const manifest = { ), ), limitations: [ - "Actual full-runner expired/near-expiry and missing-source setup cases use DUMMY auth only. Launch, timeout, audit and compound cleanup failures are additionally injected through stubbed mount and spawn dependencies at the in-process supervisor interface in the unit suite, not through an executable CLI boundary; SDK timeout is exercised by the DUMMY auth-security proof.", + "Actual full-runner expired/near-expiry and missing-source setup cases use DUMMY auth only. Malformed-output, semantic-audit and child-exit plus cleanup failure, audit-only and cleanup-only cases run through the replay CLI, namespace supervisor and real descriptor-bound mounts, but the evaluated child is replaced by a DUMMY setpriv shim and the credential-target unmount by a failing DUMMY umount shim; they prove failure composition, not SDK behavior. Launch and timeout failures are injected only in-process in the unit suite; SDK timeout is exercised by the DUMMY auth-security proof.", ], }; writeAnchoredFile(proofRoot, "evidence.json", `${JSON.stringify(manifest, null, 2)}\n`); diff --git a/test/cp06-credential-isolation.test.mjs b/test/cp06-credential-isolation.test.mjs index dd14825..3e53b30 100644 --- a/test/cp06-credential-isolation.test.mjs +++ b/test/cp06-credential-isolation.test.mjs @@ -1,187 +1,72 @@ import { strict as assert } from "node:assert"; -import { closeSync, constants, openSync } from "node:fs"; -import { lstat, mkdir, mkdtemp, readFile, rename, rm, symlink, writeFile } from "node:fs/promises"; +import { spawnSync } from "node:child_process"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { join, resolve } from "node:path"; import test from "node:test"; import { assertDestructiveCredentialWritesDenied, mountLiveCredentialReadOnly, } from "../scripts/cp06-credential-isolation.mjs"; +import { withCompiledHelper } from "./fixtures/cp06-compiled-helper.mjs"; -test("live credential mount pins one inode behind distinct read-only regular mounts", async () => { - const root = await mkdtemp(join(tmpdir(), "factory-cp06-live-mount-test-")); - const source = join(root, "auth-source.json"); - const target = join(root, "agent", "auth.json"); - const sourceBytes = '{"live":true}\n'; - await writeFile(source, sourceBytes); - const sourceStat = await lstat(source, { bigint: true }); - const calls = []; - let mountId = 40; - const mountIds = new Map(); - const spawnSyncImpl = (command, args) => { - calls.push([command, ...args]); - if (command === "findmnt") { - const path = args.at(-1); - if (!mountIds.has(path)) mountIds.set(path, ++mountId); - return { - status: 0, - stdout: JSON.stringify({ - filesystems: [ - { - id: mountIds.get(path), - target: path, - "vfs-options": "ro,nosuid,nodev,noexec,relatime", - }, - ], - }), - stderr: "", - }; - } - return { status: 0, stdout: "", stderr: "" }; - }; - const lstatImpl = async (path, options) => - String(path).endsWith("/auth.json") ? sourceStat : lstat(path, options); - try { - const mount = await mountLiveCredentialReadOnly({ - root, - source, - target, - spawnSyncImpl, - lstatImpl, +function runMountScenario(helperFd, scenario) { + const result = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + resolve("test/fixtures/cp06-credential-mount.mjs"), + scenario, + ], + { encoding: "utf8", timeout: 30_000, stdio: ["pipe", "pipe", "pipe", helperFd] }, + ); + assert.equal(result.status, 0, `${scenario}: ${result.stderr}`); + return JSON.parse(result.stdout); +} + +test("descriptor-bound credential mount exposes one read-only inode at both paths", async () => { + await withCompiledHelper("mount-control", async ({ helperFd }) => { + assert.deepEqual(runMountScenario(helperFd, "control"), { + scenario: "control", + mounted: true, + target_bytes: '{"fixture":"DUMMY","token":"DUMMY-NOT-REAL"}', + target_write: "EROFS", + parent_write: "EROFS", + integrity: true, + cleanup: "pass", + target_after_cleanup: "", + marker_unchanged: true, + source_unchanged: true, + remaining_mounts: 0, }); - try { - assert.equal(await readFile(source, "utf8"), sourceBytes); - assert.notEqual(mount.mountIds.source, mount.mountIds.target); - const mountCallsOnly = calls.filter(([command]) => command === "mount"); - assert.match(mountCallsOnly[0][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.equal(mountCallsOnly[0][2], mountCallsOnly[0][3]); - assert.deepEqual(mountCallsOnly[1].slice(0, 3), [ - "mount", - "-o", - "remount,bind,ro,nosuid,nodev,noexec", - ]); - assert.match(mountCallsOnly[2][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.equal(mountCallsOnly[2][2], mountCallsOnly[2][3]); - assert.deepEqual(mountCallsOnly[3].slice(0, 3), [ - "mount", - "-o", - "remount,bind,ro,nosuid,nodev,noexec", - ]); - assert.equal(mount.parentMounts.length, 2); - assert.equal(await mount.verifyIntegrity(), true); - assert.match(mountCallsOnly[4][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.match( - mountCallsOnly[4][3], - new RegExp(`^/proc/${process.pid}/fd/\\d+/auth-source\\.json$`), - ); - assert.deepEqual(mountCallsOnly[5].slice(0, 3), [ - "mount", - "-o", - "remount,bind,ro,nosuid,nodev,noexec", - ]); - assert.match( - mountCallsOnly[5][3], - new RegExp(`^/proc/${process.pid}/fd/\\d+/auth-source\\.json$`), - ); - assert.match(mountCallsOnly[6][2], new RegExp(`^/proc/${process.pid}/fd/\\d+$`)); - assert.match(mountCallsOnly[6][3], new RegExp(`^/proc/${process.pid}/fd/\\d+/auth\\.json$`)); - assert.deepEqual(mountCallsOnly[7].slice(0, 3), [ - "mount", - "-o", - "remount,bind,ro,nosuid,nodev,noexec", - ]); - assert.match(mountCallsOnly[7][3], new RegExp(`^/proc/${process.pid}/fd/\\d+/auth\\.json$`)); - } finally { - await mount.cleanup(); - } - for (const [command, path] of calls.slice(-4)) { - assert.equal(command, "umount"); - assert.match(path, new RegExp(`^/proc/${process.pid}/fd/\\d+`)); - } - } finally { - await rm(root, { recursive: true, force: true }); - } + }); }); -test("credential mount consumes retained parent descriptors after path substitution", async () => { - const root = await mkdtemp(join(tmpdir(), "factory-cp06-retained-mount-test-")); - const sourceDir = join(root, "source"); - const targetDir = join(root, "agent"); - const maliciousDir = join(root, "malicious"); - await mkdir(sourceDir); - await mkdir(targetDir); - await mkdir(maliciousDir); - const source = join(sourceDir, "auth.json"); - const target = join(targetDir, "auth.json"); - await writeFile(source, "DUMMY original"); - await writeFile(join(maliciousDir, "auth.json"), "DUMMY malicious"); - const sourceFd = openSync( - sourceDir, - constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW, - ); - const targetFd = openSync( - targetDir, - constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW, - ); - const sourceAnchor = `/proc/${process.pid}/fd/${sourceFd}`; - const targetAnchor = `/proc/${process.pid}/fd/${targetFd}`; - const sourceStat = await lstat(source, { bigint: true }); - const calls = []; - let mountId = 90; - const mountIds = new Map(); - const spawnSyncImpl = (command, args) => { - calls.push([command, ...args]); - if (command === "findmnt") { - const path = args.at(-1); - if (!mountIds.has(path)) mountIds.set(path, ++mountId); - return { - status: 0, - stdout: JSON.stringify({ - filesystems: [ - { id: mountIds.get(path), target: path, "vfs-options": "ro,nosuid,nodev,noexec" }, - ], - }), - stderr: "", - }; - } - return { status: 0, stdout: "", stderr: "" }; - }; - const lstatImpl = async (path, options) => - String(path).endsWith("/auth.json") ? sourceStat : lstat(path, options); - try { - await rename(sourceDir, join(root, "source-moved")); - await symlink(maliciousDir, sourceDir); - const mount = await mountLiveCredentialReadOnly({ - root, - source, - target, - sourceParentAnchor: sourceAnchor, - targetParentAnchor: targetAnchor, - sourceLeaf: "auth.json", - targetLeaf: "auth.json", - spawnSyncImpl, - lstatImpl, - }); - try { - const mountCallsOnly = calls.filter(([command]) => command === "mount"); - assert.match( - mountCallsOnly[4][3], - new RegExp(`^/proc/${process.pid}/fd/${sourceFd}/auth\\.json$`), - ); - assert.match( - mountCallsOnly[6][3], - new RegExp(`^/proc/${process.pid}/fd/${targetFd}/auth\\.json$`), - ); - assert.equal(await readFile(join(maliciousDir, "auth.json"), "utf8"), "DUMMY malicious"); - } finally { - await mount.cleanup(); +test("leaf or parent substitution between preparation and mount is refused", async () => { + await withCompiledHelper("mount-substitution", async ({ helperFd }) => { + for (const [scenario, error] of [ + ["target-leaf-symlink", "CP-06 mount destination identity changed"], + ["target-leaf-replaced", "CP-06 mount destination identity changed"], + ["source-leaf-symlink", "CP-06 mount destination identity changed"], + ["target-parent-replaced", "CP-06 mount destination path identity changed"], + ["source-parent-replaced", "CP-06 mount destination path identity changed"], + ]) { + assert.deepEqual(runMountScenario(helperFd, scenario), { + scenario, + mounted: false, + error: `read-only credential isolation unavailable: ${error}\n`, + marker_unchanged: true, + source_unchanged: true, + remaining_mounts: 0, + }); } - } finally { - closeSync(sourceFd); - closeSync(targetFd); - await rm(root, { recursive: true, force: true }); - } + }); }); test("mount setup failure reports cleanup failure instead of discarding it", async () => { @@ -191,8 +76,8 @@ test("mount setup failure reports cleanup failure instead of discarding it", asy await writeFile(source, "DUMMY"); let mountCalls = 0; const spawnSyncImpl = (command) => { - if (command === "mount" && ++mountCalls === 2) { - return { status: 32, stdout: "", stderr: "DUMMY remount failure" }; + if (command === "/DUMMY-helper" && ++mountCalls === 2) { + return { status: 65, stdout: "", stderr: "DUMMY bind failure" }; } if (command === "umount") { return { status: 32, stdout: "", stderr: "DUMMY cleanup failure" }; @@ -201,7 +86,13 @@ test("mount setup failure reports cleanup failure instead of discarding it", asy }; try { await assert.rejects( - mountLiveCredentialReadOnly({ root, source, target, spawnSyncImpl }), + mountLiveCredentialReadOnly({ + root, + source, + target, + mountHelper: "/DUMMY-helper", + spawnSyncImpl, + }), AggregateError, ); } finally { diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 61f751d..7515bd1 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -129,7 +129,7 @@ test("failed DUMMY probes never forward child output or spawn exceptions", async () => runDummySdkProof({ root: process.cwd(), - helper: "/DUMMY-helper", + helper: process.execPath, piRoot: "/DUMMY-pi", spawnSyncImpl() { return result; @@ -312,7 +312,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { () => runDummySdkProof({ root: process.cwd(), - helper: "/DUMMY-helper", + helper: process.execPath, piRoot: "/DUMMY-pi", spawnSyncImpl: stubbedSdkProof(payload), expectedSdkDependency: trustedDependency, @@ -322,7 +322,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { } const accepted = runDummySdkProof({ root: process.cwd(), - helper: "/DUMMY-helper", + helper: process.execPath, piRoot: "/DUMMY-pi", spawnSyncImpl: stubbedSdkProof(validBase), expectedSdkDependency: trustedDependency, diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index a5b81b0..cb77f46 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -10,7 +10,7 @@ import { symlink, writeFile, } from "node:fs/promises"; -import { writeFileSync } from "node:fs"; +import { closeSync, constants, openSync, realpathSync, writeFileSync } from "node:fs"; import { spawnSync } from "node:child_process"; import { resolve } from "node:path"; import { tmpdir } from "node:os"; @@ -23,6 +23,12 @@ import { import { namespaceWorkerError } from "../scripts/cp06-auth-security.mjs"; import { Cp06CleanupError, withCleanup } from "../scripts/cp06-worker-lifecycle.mjs"; import { createWorkerBlockedStatus } from "../scripts/replay-cp06-worker.mjs"; +import { + DUMMY_CLI_SCENARIOS, + DUMMY_CLI_SECRET, + writeDummyCliShims, +} from "../scripts/cp06-dummy-cli-shims.mjs"; +import { withCompiledHelper } from "./fixtures/cp06-compiled-helper.mjs"; for (const scenario of [ "success", @@ -446,6 +452,115 @@ test("nested audit, mount and fixture cleanup failures retain typed causes", asy } }); +test("supervisor executable serializes compound worker failures as typed causes", async () => { + const audit = { + stage: "audit", + code: "CP06_CHILD_OUTPUT_INVALID", + exit_code: 70, + description: "isolated child output was invalid", + }; + const childExit = { + stage: "child-exit", + code: "CP06_CHILD_EXIT_FAILED", + exit_code: 1, + description: "isolated child exited unsuccessfully", + }; + const cleanup = { + stage: "cleanup", + code: "CP06_CREDENTIAL_CLEANUP_FAILED", + exit_code: null, + description: "credential mount cleanup failed", + }; + const compound = (causes) => ({ + schema_version: 1, + stage: "cleanup", + code: "CP06_CLEANUP_FAILED", + exit_code: 74, + description: "credential namespace cleanup failed", + causes, + }); + const expected = { + "malformed-output-and-cleanup": compound([audit, cleanup]), + "semantic-audit-and-cleanup": compound([audit, cleanup]), + "child-exit-and-cleanup": compound([childExit, cleanup]), + "audit-only": { schema_version: 1, ...audit, causes: [] }, + "cleanup-only": compound([cleanup]), + success: null, + }; + assert.deepEqual(Object.keys(expected).sort(), Object.keys(DUMMY_CLI_SCENARIOS).sort()); + await withCompiledHelper("cli-compound", async ({ helperFd, binaries }) => { + const probeFd = openSync(binaries.syscallProbe, constants.O_RDONLY | constants.O_CLOEXEC); + try { + for (const [scenario, record] of Object.entries(expected)) { + const directory = realpathSync(await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-cli-"))); + const shims = join(directory, "shims"); + const source = join(directory, "credential", "DUMMY-source.json"); + const sourceBytes = '{"DUMMY":"credential fixture only"}'; + await mkdir(shims); + await mkdir(join(directory, "credential")); + await writeFile(source, sourceBytes); + try { + const result = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + resolve("scripts/cp06-namespace-supervisor.mjs"), + "worker", + source, + join(directory, "home", "pi-agent", "auth.json"), + "/proc/self/fd/3", + "/proc/self/fd/4", + join(directory, "DUMMY-input.json"), + ], + { + encoding: "utf8", + timeout: 60_000, + stdio: ["pipe", "pipe", "pipe", helperFd, probeFd], + env: { + ...process.env, + PATH: writeDummyCliShims({ anchor: shims, path: shims, scenario }), + }, + }, + ); + assert.equal(`${result.stdout}${result.stderr}`.includes(DUMMY_CLI_SECRET), false); + assert.equal(await readFile(source, "utf8"), sourceBytes); + if (record === null) { + assert.equal(result.status, 0, result.stderr); + assert.equal(result.stderr, ""); + const success = JSON.parse(result.stdout); + assert.equal(success.cleanup, "pass"); + assert.equal(success.source_unchanged, true); + assert.equal(success.mount_ids_distinct, true); + assert.equal(success.parent_roots_read_only, true); + continue; + } + assert.equal(result.status, record.exit_code, scenario); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, `${JSON.stringify(record)}\n`); + const blocked = createWorkerBlockedStatus( + namespaceWorkerError(result), + "isolated-sdk-worker", + "2026-01-01T00:00:00.000Z", + ); + assert.deepEqual(blocked.worker_failure, record); + assert.equal(blocked.code, record.code); + assert.equal(blocked.cleanup_failed, record.code === "CP06_CLEANUP_FAILED"); + } finally { + await rm(directory, { recursive: true, force: true }); + } + } + } finally { + closeSync(probeFd); + } + }); +}); + test("worker source integrity uses metadata without reading credential bytes", async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-metadata-")); const source = join(directory, "DUMMY-unreadable-source"); diff --git a/test/fixtures/cp06-compiled-helper.mjs b/test/fixtures/cp06-compiled-helper.mjs new file mode 100644 index 0000000..15acf09 --- /dev/null +++ b/test/fixtures/cp06-compiled-helper.mjs @@ -0,0 +1,24 @@ +import { closeSync, constants, openSync } from "node:fs"; +import { rm } from "node:fs/promises"; +import { resolve } from "node:path"; +import { compileIsolationHelpers } from "../../scripts/cp06-auth-security.mjs"; +import { openProbeOutput } from "../../scripts/cp06-probe-fixture.mjs"; + +/** Compile the real CP-06 helpers into ignored state and expose an inheritable helper fd. */ +export async function withCompiledHelper(name, action) { + const root = process.cwd(); + const outputRoot = resolve(`.factory/state/auth-security/DUMMY-${name}-${process.pid}`); + const output = openProbeOutput({ root, outputRoot, create: true }); + let binaries; + let helperFd; + try { + binaries = compileIsolationHelpers({ root, outputRoot, anchoredOutputRoot: output.anchor }); + helperFd = openSync(binaries.helper, constants.O_RDONLY | constants.O_CLOEXEC); + return await action({ helperFd, binaries }); + } finally { + if (helperFd !== undefined) closeSync(helperFd); + binaries?.close(); + output.close(); + await rm(outputRoot, { recursive: true, force: true }); + } +} diff --git a/test/fixtures/cp06-credential-mount.mjs b/test/fixtures/cp06-credential-mount.mjs new file mode 100644 index 0000000..f010917 --- /dev/null +++ b/test/fixtures/cp06-credential-mount.mjs @@ -0,0 +1,108 @@ +// Runs inside a disposable private user/mount namespace with the compiled +// helper inherited as fd 3. Uses literal DUMMY files only. +import { spawnSync } from "node:child_process"; +import { realpathSync } from "node:fs"; +import { + lstat, + mkdir, + mkdtemp, + readFile, + rename, + rm, + symlink, + unlink, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { mountLiveCredentialReadOnly } from "../../scripts/cp06-credential-isolation.mjs"; + +const [scenario] = process.argv.slice(2); +const marker = "DUMMY-EXTERNAL-MARKER"; +const credential = '{"fixture":"DUMMY","token":"DUMMY-NOT-REAL"}'; +const directory = realpathSync(await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-mount-"))); +const source = join(directory, "source", "DUMMY-auth.json"); +const target = join(directory, "agent", "DUMMY-auth.json"); +const external = join(directory, "external"); +const externalLeaf = join(external, "DUMMY-auth.json"); +await mkdir(dirname(source)); +await mkdir(dirname(target)); +await mkdir(external); +await writeFile(source, credential); +await writeFile(externalLeaf, marker); + +const markerReads = []; +async function substitute(leafPath) { + if (scenario === "target-leaf-symlink") { + await unlink(leafPath); + await symlink(externalLeaf, leafPath); + } else if (scenario === "target-leaf-replaced") { + await rename(leafPath, join(directory, "DUMMY-placeholder-moved")); + await writeFile(leafPath, "DUMMY substitute"); + } else if (scenario === "source-leaf-symlink") { + await rename(source, join(directory, "DUMMY-source-moved")); + await symlink(externalLeaf, source); + } else if (scenario === "target-parent-replaced" || scenario === "source-parent-replaced") { + const parent = dirname(scenario === "target-parent-replaced" ? target : source); + await rename(parent, `${parent}-moved`); + await symlink(external, parent); + } else if (scenario !== "control") { + throw new Error("unsupported DUMMY scenario"); + } +} + +const result = { scenario }; +let mount; +try { + mount = await mountLiveCredentialReadOnly({ + root: process.cwd(), + source, + target, + mountHelper: `/proc/${process.pid}/fd/3`, + async beforeMount({ targetMountPath }) { + await substitute(targetMountPath); + }, + async lstatImpl(path, options) { + markerReads.push(await readFile(externalLeaf, "utf8")); + return lstat(path, options); + }, + }); + result.mounted = true; + result.target_bytes = await readFile(target, "utf8"); + try { + await writeFile(target, "DUMMY write"); + result.target_write = "allowed"; + } catch (error) { + result.target_write = error.code; + } + try { + await writeFile(join(dirname(target), "DUMMY-entry"), "DUMMY entry"); + result.parent_write = "allowed"; + } catch (error) { + result.parent_write = error.code; + } + result.integrity = await mount.verifyIntegrity(); + await mount.cleanup(); + result.cleanup = "pass"; + result.target_after_cleanup = await readFile(target, "utf8"); +} catch (error) { + result.mounted = mount !== undefined; + result.error = error instanceof AggregateError ? "aggregate" : error.message; +} +markerReads.push(await readFile(externalLeaf, "utf8")); +result.marker_unchanged = markerReads.every((value) => value === marker); +const originalSource = + scenario === "source-leaf-symlink" + ? join(directory, "DUMMY-source-moved") + : scenario === "source-parent-replaced" + ? join(directory, "source-moved", "DUMMY-auth.json") + : source; +result.source_unchanged = (await readFile(originalSource, "utf8")) === credential; +const listed = spawnSync("findmnt", ["--json", "--list", "--output", "TARGET"], { + encoding: "utf8", +}); +result.remaining_mounts = JSON.parse(listed.stdout).filesystems.filter(({ target: path }) => + path.startsWith(`${directory}/`), +).length; +await rm(directory, { recursive: true, force: true }); +console.log(JSON.stringify(result)); From cc86a0e3bc66eccc51556456eb0fbd34aa7797b2 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 5 Oct 2026 13:02:46 +0000 Subject: [PATCH 39/45] no-mistakes(review): Harden CP06 descriptor handoff, evidence validation, and artifact boundaries --- docs/probes/CP-06.md | 62 +++++++++--- scripts/cp06-auth-security.mjs | 68 ++++++++----- scripts/cp06-dummy-cli-shims.mjs | 22 +---- scripts/cp06-dummy-originals.mjs | 2 + scripts/cp06-dummy-outcome-proof.mjs | 44 +++++++-- scripts/cp06-dummy-sdk-proof.mjs | 38 ++++++-- scripts/cp06-dummy-worker-evidence.mjs | 61 ++++++++++++ scripts/cp06-dummy-worker-hook.mjs | 15 ++- scripts/cp06-guarded-read.mjs | 8 +- scripts/cp06-isolation-adversary.mjs | 13 ++- scripts/cp06-isolation-syscalls.c | 130 ++++++++++++++++++++++--- scripts/cp06-namespace-supervisor.mjs | 40 ++++++-- scripts/cp06-probe-fixture.mjs | 55 ++++++++++- scripts/cp06-sdk-worker.mjs | 5 +- scripts/cp06-seccomp-exec.c | 20 +++- scripts/cp06-worker-evidence.mjs | 76 +++++++++++++++ scripts/replay-cp06-p07.mjs | 14 +-- scripts/replay-cp06-ten-pack.mjs | 53 +++++++--- test/cp06-proof-boundary.test.mjs | 15 ++- test/cp06-review-boundaries.test.mjs | 118 ++++++++++++++++++++++ test/cp06-worker-lifecycle.test.mjs | 34 +++---- 21 files changed, 739 insertions(+), 154 deletions(-) create mode 100644 scripts/cp06-dummy-originals.mjs create mode 100644 scripts/cp06-dummy-worker-evidence.mjs create mode 100644 scripts/cp06-worker-evidence.mjs create mode 100644 test/cp06-review-boundaries.test.mjs diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 10d8d48..b14e834 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -19,15 +19,43 @@ daemon. ## Read-only Pi authentication correction -The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` now runs a DUMMY-only gate before opening any configured credential path or making a provider request. That gate proves the retained-capability counterexample, then observes that the actual evaluated child has empty capability sets, `no_new_privs`, and seccomp filter mode, and that its attempted remount/unmount, direct `setns`/`unshare`/`clone`/`clone3`/`mount`/`umount2` syscalls, nested namespace creation, `nsenter`, and UID change fail. The seccomp helper also installs denials for alternate mount APIs, handle-based inode reopening, ptrace and process-memory access, and the evaluated child is launched with locked root securebits requested through `setpriv`; the probe does not yet execute ptrace/process-memory or inherited-subprocess escape attempts or read back securebits, so those outcomes are configured but unproven (open finding `CP06-KERNEL-PROOF-INCOMPLETE`). On the WSL2 6.18 host used for the 2026-10-05 review rounds, the hardened probe blocks at credential mount setup with `CP06_ISOLATION_SETUP_FAILED`. The failing check is the post-mount identity check in `mountLiveCredentialReadOnly` (`credential mount identity changed: /proc//fd//DUMMY-auth.json`), while the unsafe probe, which skips the parent bind mounts, passes. Diagnosis from the code and that error (not yet confirmed by a fix): util-linux canonicalizes the `/proc//fd/` parent anchor, so the read-only parent bind covers the canonical directory, and the later leaf bind, also canonicalized, lands inside that parent bind. The identity check walks from the retained parent descriptor, which still refers to the original mount, where the target leaf is the empty placeholder rather than the pinned inode. The source check passes only because the underlying source file is the pinned inode. This is a path-handoff design defect, not a missing kernel primitive, and nothing in it is WSL-specific, so the worker path with parent protection is expected to fail the same way on other Linux hosts. The check is not weakened; the gate fails closed until the native descriptor-bound mount handoff (`CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`) lands. Source and target are distinct `ro,nosuid,nodev,noexec` mount instances of one pinned `O_NOFOLLOW` regular inode. The bind mounts are still issued through util-linux `mount` with `/proc//fd` anchor paths, which util-linux canonicalizes; a native descriptor-to-descriptor mount handoff is not implemented yet (open finding `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Their containing source and target directories are separate read-only bind mounts inside the private namespace, while the DUMMY proof root used as scratch remains writable. The host namespace and host parent permissions are unchanged. Proof support is limited to Linux x86-64 and WSL2 Linux guests with the same required kernel namespace, mount, seccomp, and capability behavior; unsupported Linux x86-64 or WSL2 environments, libseccomp, helper compilation, Pi version, or a cleanup failure block the proof with no unsandboxed fallback. WSL2 coverage is only for the Linux guest proof environment, not Windows host credential or filesystem semantics. Windows and macOS are not claimed. +The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK path. `scripts/replay-cp06-worker.mjs` runs a DUMMY-only gate before opening any configured credential path or making a provider request. Passing requires the retained-capability counterexample and an evaluated child with empty capability sets, `no_new_privs`, seccomp filter mode, failed remount/unmount, namespace entry/creation and UID change. The native syscall probe now creates its own bounded literal-DUMMY target process, attempts `ptrace` attach/seize, `process_vm_readv/writev`, `pidfd_open` plus denied `pidfd_getfd`, and denied `/proc//mem` opening, in addition to `setns`, `unshare`, namespace `clone/clone3`, `mount` and `umount2`. An inherited subprocess repeats those denials and reads capability sets, NoNewPrivs, seccomp and `PR_GET_SECUREBITS`; both processes must observe locked root securebits 15. The target is non-dumpable, checks its DUMMY buffer unchanged, and is awaited, with parent-death signals and a probe alarm bounding failure cleanup. It accepts no external process target or address. These are required executed outcomes, not proof from the seccomp deny list. Alternate mount and inode-handle APIs remain configured rather than individually exercised. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup derives its leaf paths from retained source and target parent descriptors and the pinned source leaf, and the executable transition regressions reject a substituted source ancestor or pre-planted target slot before mount without touching external DUMMY sentinels; the mount commands themselves still receive canonicalized paths rather than retained source and destination inode descriptors (see `CP06-MOUNT-INODE-HANDOFF-INCOMPLETE`). Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal, `openAnchoredDirectory(..., { reset: true })`, and the replay runner's removal of its evaluation home (created by `mkdtemp` relative to a retained no-follow `TMPDIR` descriptor) use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink or for a different real directory. That proof cannot complete on a host where the DUMMY security preflight fails, as on the review host above. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The historical 2026-10-05 WSL2 mount identity failure belonged to the util-linux path handoff. Preserved head `fdc8321` replaces credential bind/remount commands with native `open_tree`, `mount_setattr` and `move_mount`, retaining the source inode and an identity-checked `O_PATH | O_NOFOLLOW` destination. Leaf and parent substitution regressions retain outside DUMMY sentinels unchanged. The post-mount identity check is unchanged. At that head mount setup passed, but preflight still failed with "direct syscall probe failed to execute": the zero-capability adversary could not reopen the supervisor's descriptor and the helper closed inherited descriptors. The current correction supplies bounded read-only descriptors through each actual launch instead. Final-head full SDK/security, worker-cleanup and genuine semantic acceptance must still be run by the outer executor; this source correction alone does not close `CP06-KERNEL-PROOF-INCOMPLETE` as an acceptance blocker. + +Source and target remain distinct `ro,nosuid,nodev,noexec` mount instances of one pinned inode, with containing directories separately read-only inside the private namespace and writable DUMMY scratch. Host namespace and permissions are unchanged. Support is Linux x86-64, including WSL2 Linux guests with the required kernel primitives; unsupported namespaces, mount APIs, libseccomp, compilation, SDK versions or cleanup fail closed, without fallback. This is not Windows host filesystem/credential coverage; Windows and macOS are not claimed. + +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup retains source and target parent descriptors, checks the prepared target identity, and consumes inode descriptors in the native mount helper; executable transition regressions reject replaced ancestors, leaves and parents without touching external DUMMY sentinels. Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal, `openAnchoredDirectory(..., { reset: true })`, and the replay runner's removal of its evaluation home (created by `mkdtemp` relative to a retained no-follow `TMPDIR` descriptor) use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink or for a different real directory. Sibling ten-pack/P-07 manifests, raw outputs, destination copies and hashes now also consume descriptor-relative no-follow artifact operations. Replaced task contracts are opened without truncation, checked for a single regular inode, then truncated through that retained descriptor. DUMMY SDK/outcome fixture writes and exit counters use the same bounded artifact operations. A security preflight failure still blocks worker-cleanup proof before any configured credential use. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and complete published file manifests must agree with the independently pinned exact-version npm artifacts in `.factory/cp06-sdk-integrity.json`; inherited `PATH`, a global installation, an unrelated same-version package, and same-version file substitutions are rejected before SDK import without fallback. The pin records npm tarball SHA-512 SRI and sorted `[relative path, file SHA-256]` manifest SHA-256 and file count for both packages; it excludes nested installed `node_modules` from each package manifest, while separately verifying the resolved `pi-ai` package. It does not attest the registry itself, transitive packages other than `pi-ai`, or immutable files after this check. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths, trusted artifact pins, manifest digests, and SHA-256 entry digests. To reproduce a pin independently, download the named exact-version tarballs into ignored `.factory/state/`, compare each tarball's SHA-512 against the tracked SRI, extract into a disposable directory, sort all regular published files by relative path and compute SHA-256 of JSON-serialized `[path, SHA-256(file)]` pairs (with no extra files or symlinks); compare count and digest to the tracked pin. Never derive a new trusted pin from the selected installation under evaluation. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. -Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is not genuine acceptance until the exact final-head DUMMY gate, worker-cleanup proof, transition regressions, and semantic worker command all pass and their manifests identify the same tested SHA. The existing `cp06-seccomp-exec` helper closes all inherited file descriptors before launching the evaluated child; retained helper/probe descriptors are used for supervisor execution and hashing, while credential descriptors are not passed to that child. No unsupported lexical fallback is acceptable. No destructive operation is ever directed at the real credential source or mounted target. +Run the standalone DUMMY gate with `npm run proof:cp06:auth-security`. A positive real semantic replay remains `npm run proof:cp06:worker`, but it is not genuine acceptance until the exact final-head DUMMY gate, worker-cleanup proof, transition regressions, and semantic worker command all pass and their manifests identify the same tested SHA. The helper closes all inherited descriptors at least 3, except an explicitly selected read-only regular fd 4. For hardened probes, fd 4 belongs only to the compiled probe: the adversary transfers a read-only copy as fd 3 to the native probe and closes its copies before other subprocesses. For workers, fd 4 belongs only to the compact input file and the SDK worker closes it after parsing. The outer runner transfers helper/probe/input as supervisor fds 3/4/5; the supervisor retains them only until its process ends, and per-launch copies are closed immediately after the synchronous child returns. Writable helper descriptors and credential descriptors are never passed to the evaluated child. No unsupported lexical fallback is acceptable. No destructive operation is ever directed at the real credential source or mounted target. + +## Source-review correction verification + +Review phase only, on parent SHA +`fdc83217b9216485b817511cddcb9846fa005d74` plus the uncommitted correction: +Node `24.20.0`, npm `12.0.2`, GCC `13.3.0`. + +- `node --experimental-import-meta-resolve --test test/cp06-proof-boundary.test.mjs test/cp06-worker-lifecycle.test.mjs test/cp06-review-boundaries.test.mjs` + exited 1: 35 passed, one failed. The real hardened DUMMY probe passed; the + closed-descriptor negative fixture launched outside a disposable user namespace + and exited 127 instead of reaching the helper's expected exit 70. +- After putting that negative inside the same private namespace and tightening + DUMMY counter ancestor pinning, + `node --experimental-import-meta-resolve --test test/cp06-review-boundaries.test.mjs test/cp06-guarded-read.test.mjs` + exited 0: eight passed, none failed or skipped. This executes unsafe/hardened + fixture controls, native memory-access denials and inherited observations, + the closed-descriptor counterfactual, nested semantic/cleanup rejection, + artifact redirection negatives and the real ten-pack/P-07 raw-write boundaries. + +These focused checks are not exact-final-head SDK, worker-cleanup, STANDARD +10-pack/P-07 reproduction or genuine semantic acceptance. Clean install, +format/lint/typecheck, full tests, complete SDK proofs, final-head independent +review and CI remain owned by the outer executor. No real credential or model +was used during this source-review phase. ## Normative result @@ -242,9 +270,10 @@ after the complete DUMMY auth and OS preflight passes. A supervisor pins the regular source inode with `O_NOFOLLOW`, creates distinct source/target bind mounts with read-only VFS flags, and closes the pin before launch. The evaluated child is observed with all capability sets empty, `no_new_privs`, and an -inherited seccomp filter whose mount and namespace denials are exercised; its -alternate-inode-handle and ptrace/process-memory denials and the requested -locked securebits are configured but not yet exercised or observed. Its +inherited seccomp filter. Passing DUMMY preflight now requires native mount, +namespace, ptrace/process-memory denials, inherited-subprocess repetition and +locked-securebits read-back. Alternate inode-handle denials remain configured +rather than individually exercised; full final-head acceptance remains required. Its credential store rejects refresh/mutation callbacks before entry; expired or near-expiry OAuth is therefore blocked before a provider request or persistence. A missing credential source is a sanitized `CP06_CREDENTIAL_SOURCE_MISSING` @@ -321,18 +350,21 @@ or arbitrary exception payloads. PR12 still parsed successful-child output after cleanup, so malformed output plus cleanup failure lost the parse/audit cause. The correction keeps child status, source-integrity verification, compact JSON parsing, the mode-specific child semantic contract (for workers: exact output -keys, `result: "pass"`, Pi `0.85.1`, in-memory read-only session, unrefreshed -OAuth, and one exact-original read), and evidence assembly inside the cleanup +keys and nested evidence schemas, `result: "pass"`, Pi `0.85.1`, in-memory +read-only session, unrefreshed OAuth, the complete canonical A1 outcome, and +one exact-original read tied to independently pinned bytes and bounded event +indexes), and evidence assembly inside the cleanup action, so valid JSON with an invalid contract plus a cleanup failure retains both the audit and cleanup causes. These compound cases (malformed output and semantic audit each with cleanup, child-exit+cleanup, audit-only, cleanup-only, -success) are regression-tested at the in-process supervisor interface with -injected mount and spawn stubs, and the serialized record is checked through -`namespaceWorkerError` and `createWorkerBlockedStatus`. Executable CLI -regressions that assert serialized stderr, exit status, and a persisted -`blocked.json` for each compound case do not exist yet (open finding -`CP06-CLI-COMPOUND-REGRESSIONS-ABSENT`). `npm run proof:cp06:worker-cleanup` executes the full runner -with task-local expired/near-expiry DUMMY auth and missing-source setup, asserts +success) have both injected supervisor tests and actual supervisor CLI +regressions checking sanitized serialized stderr and exact exits. The DUMMY +CLI control now returns the complete canonical A1 contract; its semantic +negative changes only the response to an invalid PASS outcome. The full-runner +compound scenarios in `npm run proof:cp06:worker-cleanup` assert persisted +`blocked.json` causes, in addition to expired/near-expiry DUMMY auth and +missing-source setup. They remain subject to a working complete DUMMY SDK +preflight and are not claimed as final-head passing evidence here. The proof asserts the blocked result, unchanged DUMMY source, absence of a pass manifest, and removal of the evaluation home. Unit tests also inject setup, launch, timeout, audit and cleanup failures at the executable supervisor boundary; they are diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 072de13..7091163 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -17,6 +17,7 @@ import { join, resolve } from "node:path"; import { resolvePinnedPiAi } from "./cp06-pi-dependency.mjs"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; import { openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { DUMMY_SDK_ORIGINAL, DUMMY_OUTCOME_ORIGINAL } from "./cp06-dummy-originals.mjs"; export class Cp06IsolationUnsupportedError extends Error { constructor(message) { @@ -260,7 +261,8 @@ export async function runIsolationProbe({ } function runNamespace(options) { - return withInheritedExecutables([options.helper, options.syscallProbe], ([helper, probe]) => + const inputs = options.mode === "worker" ? [options.helper, options.syscallProbe, options.childArgs[0]] : [options.helper, options.syscallProbe]; + return withInheritedExecutables(inputs, ([helper, probe, input]) => spawnSync( "unshare", [ @@ -277,7 +279,7 @@ function runNamespace(options) { options.mode === "worker" ? resolve(options.target) : "-", helper.path, probe.path, - ...options.childArgs.map((value) => resolve(value)), + ...(input ? [input.path] : options.childArgs.map((value) => resolve(value))), ], { cwd: options.root, @@ -285,7 +287,7 @@ function runNamespace(options) { timeout: options.timeout, maxBuffer: 1024 * 1024, env: options.environment ?? process.env, - stdio: ["pipe", "pipe", "pipe", helper.fd, probe.fd], + stdio: ["pipe", "pipe", "pipe", helper.fd, probe.fd, ...(input ? [input.fd] : [])], }, ), ); @@ -298,7 +300,7 @@ function withInheritedExecutables(paths, action) { try { for (const [index, path] of paths.entries()) { descriptors.push({ - fd: openSync(resolve(path), constants.O_RDONLY | constants.O_CLOEXEC), + fd: openSync(resolve(path), constants.O_RDONLY | constants.O_CLOEXEC | (index >= 2 ? constants.O_NOFOLLOW : 0)), path: `/proc/self/fd/${index + 3}`, }); } @@ -360,7 +362,7 @@ const SUPERVISOR_FAILURE_CODES = new Map([ ["CP06_CREDENTIAL_CLEANUP_FAILED", "cleanup"], ]); -function assertUnsafeControl(proof) { +export function assertUnsafeControl(proof) { assert(proof.mode === "unsafe-probe", "unsafe control mode mismatch"); assert(proof.mount_ids_distinct === true, "unsafe control lacked distinct mounts"); assert(proof.source_unchanged === false, "unsafe control did not mutate DUMMY source"); @@ -376,7 +378,7 @@ function assertUnsafeControl(proof) { ); } -function assertHardenedProof(proof) { +export function assertHardenedProof(proof) { assert(proof.mode === "hardened-probe", "hardened proof mode mismatch"); assert(proof.mount_ids_distinct === true, "hardened proof lacked distinct mounts"); assert(proof.source_unchanged === true, "hardened child changed DUMMY source"); @@ -428,16 +430,27 @@ function assertHardenedProof(proof) { ); const syscalls = proof.child.direct_namespace_syscalls; assert(syscalls.exit === 0, "direct syscall probe failed to execute"); - for (const expected of [ - "setns=-1:1", - "unshare=-1:1", - "clone=-1:1", - "clone3=-1:38", - "mount=-1:1", - "umount2=-1:1", - ]) { - assert(syscalls.stdout.includes(expected), `direct syscall was not filtered: ${expected}`); + const observations = new Map(); + for (const line of syscalls.stdout.split("\n")) { + const [key, value, extra] = line.split("="); + assert(key && value && extra === undefined && !observations.has(key), "syscall observations malformed"); + observations.set(key, value); + } + for (const prefix of ["", "inherited_"]) { + for (const name of ["setns", "unshare", "clone", "mount", "umount2", "ptrace_attach", "ptrace_seize", "process_vm_readv", "process_vm_writev", "pidfd_getfd"]) { + assert(observations.get(`${prefix}${name}`) === "-1:1", "required DUMMY syscall denial missing"); + } + assert(observations.get(`${prefix}clone3`) === "-1:38", "clone3 denial missing"); + assert(observations.get(`${prefix}pidfd_open`) === "opened", "DUMMY pidfd target unavailable"); + assert(observations.get(`${prefix}proc_mem`) === "-1:13", "DUMMY proc memory denial missing"); + assert(observations.get(`${prefix}securebits`) === "15", "locked root securebits missing"); + assert(observations.get(`${prefix}NoNewPrivs`) === "1", "inherited no_new_privs missing"); + assert(observations.get(`${prefix}Seccomp`) === "2", "inherited seccomp missing"); + for (const name of ["CapInh", "CapPrm", "CapEff", "CapBnd", "CapAmb"]) { + assert(observations.get(`${prefix}${name}`) === "0000000000000000", "inherited capabilities not empty"); + } } + assert(observations.get("dummy_target_unchanged") === "true", "DUMMY target integrity missing"); } export function runDummySdkProof({ @@ -627,6 +640,8 @@ function validateSdkWorker(worker, originalPath) { validateReadAudit(worker.read_audit, { toolCallId: "DUMMY-read", originalPath, + original: DUMMY_SDK_ORIGINAL, + eventCount: worker.event_count, }); } @@ -711,7 +726,7 @@ function validateOutcomeProof(outcomes, trustedDependency, originalPath) { "source_unchanged", "to_auth", ]; - if (["valid", "fenced-valid"].includes(entry.scenario)) expectedKeys.push("read_audit"); + if (["valid", "fenced-valid"].includes(entry.scenario)) expectedKeys.push("read_audit", "event_count"); assertExactKeys(entry, expectedKeys); seen.add(entry.scenario); assert( @@ -735,6 +750,8 @@ function validateOutcomeProof(outcomes, trustedDependency, originalPath) { validateReadAudit(entry.read_audit, { toolCallId: "DUMMY-actual-worker-read", originalPath, + original: DUMMY_OUTCOME_ORIGINAL, + eventCount: entry.event_count, }); } } @@ -752,7 +769,7 @@ function validateCredentialAudit(audit, expected) { assert(audit.reads >= 1, "DUMMY credential audit omitted the credential read"); } -function validateReadAudit(audit, { toolCallId, originalPath }) { +export function validateReadAudit(audit, { toolCallId, originalPath, original, eventCount }) { assertExactKeys(audit, [ "completed_successfully", "end_event_index", @@ -771,7 +788,11 @@ function validateReadAudit(audit, { toolCallId, originalPath }) { "tool_call_id", ]); const isEmptyArray = (value) => Array.isArray(value) && value.length === 0; - const isIndex = (value) => Number.isSafeInteger(value) && value >= 0; + assert(Number.isSafeInteger(eventCount) && eventCount > 0, "read audit event count missing"); + assert(typeof original === "string" || Buffer.isBuffer(original), "read audit trusted bytes missing"); + const expectedBytes = Buffer.from(original); + const expectedHash = sha256(expectedBytes); + const isIndex = (value) => Number.isSafeInteger(value) && value >= 0 && value < eventCount; assert( audit.read_count === 1 && audit.project_read_count === 1 && @@ -788,10 +809,9 @@ function validateReadAudit(audit, { toolCallId, originalPath }) { "DUMMY exact original read missing", ); assert( - isSha256(audit.expected_sha256) && - audit.returned_sha256 === audit.expected_sha256 && - Number.isSafeInteger(audit.returned_bytes) && - audit.returned_bytes > 0, + audit.expected_sha256 === expectedHash && + audit.returned_sha256 === expectedHash && + audit.returned_bytes === expectedBytes.length, "DUMMY read audit bytes mismatch", ); assert(audit.tool_call_id === toolCallId, "DUMMY read audit tool call mismatch"); @@ -937,6 +957,10 @@ function summarizeHardened(proof) { capability_sets_zero: true, no_new_privs: true, seccomp_filter: true, + locked_root_securebits: 15, + dummy_process_memory_denials: true, + inherited_subprocess_denials: true, + dummy_process_target_unchanged: true, ordinary_child_allowed: true, direct_writes_denied: true, replacement_denied: true, diff --git a/scripts/cp06-dummy-cli-shims.mjs b/scripts/cp06-dummy-cli-shims.mjs index be28947..6043638 100644 --- a/scripts/cp06-dummy-cli-shims.mjs +++ b/scripts/cp06-dummy-cli-shims.mjs @@ -3,6 +3,7 @@ // replaced; every other setpriv/umount invocation runs the real command. import { accessSync, closeSync, constants, openSync, writeSync } from "node:fs"; import { delimiter, join } from "node:path"; +import { fileURLToPath } from "node:url"; const AUDIT = { stage: "audit", @@ -59,21 +60,7 @@ export const DUMMY_CLI_SCENARIOS = { export const DUMMY_CLI_SECRET = "DUMMY-SECRET-CLI-MARKER"; -const VALID_WORKER_CHILD = { - schema_version: 1, - result: "pass", - pi_version: "0.85.1", - pi_install: { fixture_origin: true }, - provider: "DUMMY", - model: "DUMMY", - credential_store: { fixture_origin: true }, - oauth_preflight: { refreshed: false }, - session: { in_memory: true, active_tools: ["read"] }, - event_stream: { read_audit: { exact_original: true, read_count: 1 } }, - response: { fixture_origin: true }, - response_sha256: "0".repeat(64), - raw_transcript_retained: false, -}; +const dummyEvidence = fileURLToPath(new URL("./cp06-dummy-worker-evidence.mjs", import.meta.url)); /** Write scenario shims into `anchor`; returns a PATH that resolves them first. */ export function writeDummyCliShims({ anchor, path, scenario, inheritedPath = process.env.PATH }) { @@ -83,14 +70,15 @@ export function writeDummyCliShims({ anchor, path, scenario, inheritedPath = pro const realUmount = locate("umount", inheritedPath); const child = { malformed: `printf '%s' '${DUMMY_CLI_SECRET} malformed'; printf '%s' '${DUMMY_CLI_SECRET}' >&2; exit 0`, - semantic: `printf '%s' '{"result":"fail","detail":"${DUMMY_CLI_SECRET}"}'; exit 0`, + semantic: `exec '${process.execPath}' '${dummyEvidence}' "$last" invalid`, exit: `printf '%s' '${DUMMY_CLI_SECRET}' >&2; exit 1`, - valid: `printf '%s' '${JSON.stringify(VALID_WORKER_CHILD)}'; exit 0`, + valid: `exec '${process.execPath}' '${dummyEvidence}' "$last"`, }[selected.child]; writeExecutable( anchor, "setpriv", `#!/bin/sh +for argument do last="$argument"; done for argument do case "$argument" in */scripts/cp06-sdk-worker.mjs) ${child} ;; diff --git a/scripts/cp06-dummy-originals.mjs b/scripts/cp06-dummy-originals.mjs new file mode 100644 index 0000000..c4e0586 --- /dev/null +++ b/scripts/cp06-dummy-originals.mjs @@ -0,0 +1,2 @@ +export const DUMMY_SDK_ORIGINAL = "# DUMMY authority\n\nrequest-supplied owner fields are ignored; ownership comes only from the authenticated principal.\n"; +export const DUMMY_OUTCOME_ORIGINAL = "# DUMMY authority\n\nOwnership comes only from the authenticated principal; request-supplied owner fields are ignored.\n"; diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index b015318..2edc54c 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -1,28 +1,45 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { mkdtemp, symlink } from "node:fs/promises"; +import { closeSync, constants, fstatSync, lstatSync, openSync, readdirSync } from "node:fs"; import { tmpdir } from "node:os"; -import { dirname, join, resolve } from "node:path"; +import { basename, dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; +import { openAnchoredDirectory, readArtifactFile, writeArtifactFile, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { DUMMY_OUTCOME_ORIGINAL } from "./cp06-dummy-originals.mjs"; +import { openPinnedDirectory } from "./cp06-guarded-read.mjs"; /** Exercise the actual SDK worker executable, with only a local faux provider. */ -export async function proveDummyWorkerOutcome({ piRoot, directory: requested }) { +export async function proveDummyWorkerOutcome({ piRoot, directory: requested, parentAnchor }) { const root = resolve(fileURLToPath(new URL("..", import.meta.url))); let directory; + let prepared; if (requested === undefined) { directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-outcome-")); } else { - await mkdir(requested, { mode: 0o700 }); + prepared = openAnchoredDirectory(parentAnchor, "DUMMY-outcome"); directory = requested; } + const fd = prepared ? openSync(prepared.anchor, constants.O_RDONLY | constants.O_DIRECTORY) : openPinnedDirectory(directory); + prepared?.close(); + const anchor = `/proc/${process.pid}/fd/${fd}`; + const identity = fstatSync(fd, { bigint: true }); + const parentFd = parentAnchor ? openSync(parentAnchor, constants.O_RDONLY | constants.O_DIRECTORY) : openPinnedDirectory(dirname(directory)); + const retainedParent = `/proc/${process.pid}/fd/${parentFd}`; + const anchored = (path) => { + if (!path.startsWith(`${directory}/`)) throw new Error("DUMMY artifact escaped fixture"); + return join(anchor, path.slice(directory.length + 1)); + }; + const writeFile = async (path, data) => writeArtifactFile(anchored(path), data); + const readFile = async (path, encoding) => readArtifactFile(anchored(path), encoding); return withCleanup( async () => { - await mkdir(join(directory, "docs")); + openAnchoredDirectory(anchor, "docs").close(); const originalPath = join(directory, "docs/AUTH.md"); await writeFile( originalPath, - "# DUMMY authority\n\nOwnership comes only from the authenticated principal; request-supplied owner fields are ignored.\n", + DUMMY_OUTCOME_ORIGINAL, ); const credentialAlias = join(directory, "DUMMY-credential-alias.json"); const contractPath = join(directory, "DUMMY-contract.yaml"); @@ -70,7 +87,7 @@ export async function proveDummyWorkerOutcome({ piRoot, directory: requested }) }, }); await writeFile(auth, authBytes); - if (scenario === "symlink-read") await symlink(auth, credentialAlias); + if (scenario === "symlink-read") await symlink(auth, anchored(credentialAlias)); await writeFile( inputPath, JSON.stringify({ @@ -159,18 +176,25 @@ export async function proveDummyWorkerOutcome({ piRoot, directory: requested }) assert.equal(output.event_stream.read_audit.read_count, 1); assert.equal(output.response.evidence_gaps[0].status, "unverified"); } else assert.equal(result.stdout, ""); - await rm(credentialAlias, { force: true }); + removeAnchoredEntry(anchor, "DUMMY-credential-alias.json"); cases.push({ scenario, exit_code: result.status, expected_exit_code: expectedExit, ...counts, source_unchanged: true, - read_audit: output?.event_stream.read_audit, + ...(output ? { read_audit: output.event_stream.read_audit, event_count: output.event_stream.event_count } : {}), }); } return { fixture_origin: true, semantic_acceptance: false, cases }; }, - () => rm(directory, { recursive: true, force: true }), + () => { + try { + for (const name of readdirSync(anchor)) removeAnchoredEntry(anchor, name); + const visible = lstatSync(join(retainedParent, basename(directory)), { bigint: true }); + if (!visible.isDirectory() || visible.dev !== identity.dev || visible.ino !== identity.ino) throw new Error("DUMMY outcome directory identity changed"); + removeAnchoredEntry(retainedParent, basename(directory)); + } finally { closeSync(fd); closeSync(parentFd); } + }, ); } diff --git a/scripts/cp06-dummy-sdk-proof.mjs b/scripts/cp06-dummy-sdk-proof.mjs index 53f84c8..3321d15 100644 --- a/scripts/cp06-dummy-sdk-proof.mjs +++ b/scripts/cp06-dummy-sdk-proof.mjs @@ -1,14 +1,16 @@ #!/usr/bin/env node import { strict as assert } from "node:assert"; import { createHash } from "node:crypto"; -import { readdirSync, realpathSync } from "node:fs"; -import { readFile, rm, writeFile } from "node:fs/promises"; +import { closeSync, readdirSync, realpathSync } from "node:fs"; import { join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; import { auditReadEvents } from "./cp06-worker-audit.mjs"; import { proveDummyWorkerOutcome } from "./cp06-dummy-outcome-proof.mjs"; import { importPinnedPiAi } from "./cp06-pi-dependency.mjs"; +import { openAnchoredDirectory, readArtifactFile, writeArtifactFile, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { DUMMY_SDK_ORIGINAL } from "./cp06-dummy-originals.mjs"; +import { createGuardedReadTool, openPinnedDirectory } from "./cp06-guarded-read.mjs"; const [piRootArg, proofRootArg] = process.argv.slice(2); if (!piRootArg || !proofRootArg) process.exit(64); @@ -23,6 +25,8 @@ globalThis.fetch = async () => { }; const proofRoot = realpathSync(resolve(proofRootArg)); if (proofRoot !== resolve(proofRootArg) || readdirSync(proofRoot).length !== 0) process.exit(64); +const proofFd = openPinnedDirectory(proofRoot); +const proofAnchor = `/proc/${process.pid}/fd/${proofFd}`; try { const authCases = []; for (const scenario of [ @@ -36,6 +40,7 @@ try { const outcomeProof = await proveDummyWorkerOutcome({ piRoot, directory: join(proofRoot, "DUMMY-outcome"), + parentAnchor: proofAnchor, }); assert.equal(networkCalls, 0); console.log( @@ -53,14 +58,16 @@ try { ); } finally { globalThis.fetch = originalFetch; - for (const entry of readdirSync(proofRoot)) { - await rm(join(proofRoot, entry), { recursive: true, force: true }); + try { + for (const entry of readdirSync(proofAnchor)) removeAnchoredEntry(proofAnchor, entry); + } finally { + closeSync(proofFd); } } async function authCase(scenario) { const providerId = "openai-codex"; - const path = join(proofRoot, `DUMMY-${scenario.name}.json`); + const path = join(proofAnchor, `DUMMY-${scenario.name}.json`); await writeFile( path, JSON.stringify({ @@ -125,13 +132,14 @@ async function authCase(scenario) { } async function fauxWorkerProof() { + const workerDirectory = openAnchoredDirectory(proofAnchor, "DUMMY-worker"); + try { const workerRoot = join(proofRoot, "DUMMY-worker"); const originalPath = join(workerRoot, "DUMMY-original.md"); - const authPath = join(workerRoot, "DUMMY-auth.json"); - await import("node:fs/promises").then(({ mkdir }) => mkdir(workerRoot)); - const original = - "# DUMMY authority\n\nrequest-supplied owner fields are ignored; ownership comes only from the authenticated principal.\n"; - await writeFile(originalPath, original); + const authPath = join(workerDirectory.anchor, "DUMMY-auth.json"); + const original = DUMMY_SDK_ORIGINAL; + writeArtifactFile(join(workerDirectory.anchor, "DUMMY-original.md"), original); + const guardedRead = createGuardedReadTool({ sdk, root: workerRoot, expectedOriginalPath: originalPath }); const faux = ai.fauxProvider({ provider: "DUMMY-readonly", models: [{ id: "DUMMY-model" }], @@ -197,6 +205,7 @@ async function fauxWorkerProof() { modelRuntime: runtime, resourceLoader, tools: ["read"], + customTools: [guardedRead.tool], sessionManager: sdk.SessionManager.inMemory(workerRoot), settingsManager: sdk.SettingsManager.inMemory({ compaction: { enabled: false }, @@ -230,6 +239,15 @@ async function fauxWorkerProof() { refresh_callbacks: refreshCallbacks, credential_store: store.audit(), }; + } finally { workerDirectory.close(); } +} + +async function writeFile(path, bytes) { + writeArtifactFile(path, bytes); +} + +async function readFile(path) { + return readArtifactFile(path); } function sha256(value) { diff --git a/scripts/cp06-dummy-worker-evidence.mjs b/scripts/cp06-dummy-worker-evidence.mjs new file mode 100644 index 0000000..5bdf019 --- /dev/null +++ b/scripts/cp06-dummy-worker-evidence.mjs @@ -0,0 +1,61 @@ +import { createHash } from "node:crypto"; +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { workerEvidenceContext } from "./cp06-worker-evidence.mjs"; +import { DUMMY_OUTCOME_ORIGINAL } from "./cp06-dummy-originals.mjs"; + +export function dummyWorkerContext(directory = "/DUMMY") { + return { + input: { provider: "openai-codex", model: "DUMMY-model", timeout_ms: 1_000 }, + paths: { root: directory, fixtureRoot: directory, originalPath: `${directory}/docs/AUTH.md` }, + original: Buffer.from(DUMMY_OUTCOME_ORIGINAL), + }; +} + +export function dummyWorkerEvidence({ input, paths, original } = dummyWorkerContext()) { + const hash = createHash("sha256").update(original).digest("hex"); + return { + schema_version: 1, + result: "pass", + pi_version: "0.85.1", + pi_install: { + install_root: "/DUMMY/sdk", package_root: "/DUMMY/sdk/package", executable: "/DUMMY/sdk/pi", executable_entry: "/DUMMY/sdk/package/cli.js", sdk_entry: "/DUMMY/sdk/package/index.js", + package_sha256: "0".repeat(64), executable_sha256: "0".repeat(64), sdk_entry_sha256: "0".repeat(64), + package_artifact: { manifest_sha256: "0".repeat(64), tarball_integrity: "sha512-DUMMY" }, + dependency: { entry: "/DUMMY/sdk/ai/index.js", root: "/DUMMY/sdk/ai", entry_sha256: "0".repeat(64), manifest_sha256: "0".repeat(64), package_sha256: "0".repeat(64), name: "@earendil-works/pi-ai", version: "0.85.1", resolution: "esm-import-condition", tarball_integrity: "sha512-DUMMY" }, + }, + provider: input.provider, + model: input.model, + oauth_preflight: { minimum_validity_ms: input.timeout_ms + 300_000, remaining_validity_ms: input.timeout_ms + 600_000, refreshed: false }, + credential_store: { reads: 1, lists: 0, modify_denials: 0, delete_denials: 0 }, + session: { in_memory: true, active_tools: ["read"] }, + event_stream: { + sha256: "0".repeat(64), event_count: 3, + read_audit: { + read_count: 1, project_read_count: 1, project_read_paths: [paths.originalPath], + rejected_read_paths: [], other_tool_calls: [], other_tool_completions: [], + completed_successfully: true, exact_original: true, + expected_sha256: hash, returned_sha256: hash, returned_bytes: Buffer.byteLength(original), + tool_call_id: "DUMMY-read", start_event_index: 0, end_event_index: 1, response_event_index: 2, + }, + }, + response: { + status: "MISSING_SOURCE", missing_source: "docs/AUTH.md", reads: [paths.originalPath], + corrected_constraints: ["ownership comes only from the authenticated principal", "request-supplied owner fields are ignored"], + evidence_gaps: [{ id: "creation-time-ownership", status: "unverified", missing_checks: ["principal-derived-owner", "request-owner-ignored"] }], broad_scan: false, + }, + response_sha256: "1".repeat(64), raw_transcript_retained: false, + }; +} + +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + try { + const context = workerEvidenceContext(process.argv[2]); + const evidence = dummyWorkerEvidence(context); + if (process.argv[3] === "invalid") evidence.response = { status: "PASS", evidence_gaps: [] }; + console.log(JSON.stringify(evidence)); + } catch { + process.stderr.write("CP06_DUMMY_EVIDENCE_FAILED\n"); + process.exitCode = 70; + } +} diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index 49702ae..5b642c2 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -1,8 +1,10 @@ // Development-only local provider instrumentation. Never a live semantic proof. -import { readFileSync, writeFileSync } from "node:fs"; -import { join } from "node:path"; +import { closeSync } from "node:fs"; +import { basename, dirname, join } from "node:path"; import { pathToFileURL } from "node:url"; import { importPinnedPiAi } from "./cp06-pi-dependency.mjs"; +import { writeAnchoredFile } from "./cp06-probe-fixture.mjs"; +import { openPinnedDirectory, readPinnedRegularFile } from "./cp06-guarded-read.mjs"; const piRoot = process.env.CP06_DUMMY_PI_ROOT; const countersPath = process.env.CP06_DUMMY_COUNTERS; @@ -11,7 +13,8 @@ if (!piRoot || !countersPath || !scenario) throw new Error("DUMMY proof configur const sdk = await import(pathToFileURL(join(piRoot, "dist/index.js"))); const { AuthStorage } = await import(pathToFileURL(join(piRoot, "dist/core/auth-storage.js"))); const { module: ai, provenance: piAi } = await importPinnedPiAi(piRoot); -const input = JSON.parse(readFileSync(process.argv[3], "utf8")); +const input = JSON.parse(readPinnedRegularFile(process.argv[3]).bytes.toString("utf8")); +const countersFd = openPinnedDirectory(dirname(countersPath)); const counts = { fixture_origin: true, sdk_dependency: piAi, @@ -121,4 +124,8 @@ sdk.ModelRuntime.create = async function (options) { ]); return runtime; }; -process.on("exit", (exit) => writeFileSync(countersPath, JSON.stringify({ ...counts, exit }))); +process.on("exit", (exit) => { + try { + writeAnchoredFile(`/proc/${process.pid}/fd/${countersFd}`, basename(countersPath), JSON.stringify({ ...counts, exit })); + } finally { closeSync(countersFd); } +}); diff --git a/scripts/cp06-guarded-read.mjs b/scripts/cp06-guarded-read.mjs index b10fd85..1a1b6ed 100644 --- a/scripts/cp06-guarded-read.mjs +++ b/scripts/cp06-guarded-read.mjs @@ -131,7 +131,11 @@ export function readPinnedRegularFile(path) { } } -function openWithoutFollowingAncestors(path) { +export function openPinnedDirectory(path) { + return openWithoutFollowingAncestors(path, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC); +} + +function openWithoutFollowingAncestors(path, leafFlags = constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC) { if (typeof path !== "string" || !isAbsolute(path) || resolve(path) !== path) { throw new Error("supplied original path must be absolute and normalized"); } @@ -148,7 +152,7 @@ function openWithoutFollowingAncestors(path) { } return openSync( `/proc/self/fd/${parentFd}/${components.at(-1)}`, - constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + leafFlags, ); } finally { closeSync(parentFd); diff --git a/scripts/cp06-isolation-adversary.mjs b/scripts/cp06-isolation-adversary.mjs index dfbf41d..160c1f8 100644 --- a/scripts/cp06-isolation-adversary.mjs +++ b/scripts/cp06-isolation-adversary.mjs @@ -4,6 +4,9 @@ import { spawnSync } from "node:child_process"; import { linkSync, mkdirSync, + closeSync, + constants, + openSync, readFileSync, renameSync, rmdirSync, @@ -99,7 +102,15 @@ if (securityMode === "hardened") { jsAttempt("hardlink_into_source_parent", () => linkSync(source, join(dirname(source), "DUMMY-parent-hardlink")), ); - commandAttempt("direct_namespace_syscalls", syscallProbe, [target]); + const probeFd = openSync(syscallProbe, constants.O_RDONLY | constants.O_CLOEXEC); + try { + commandAttempt("direct_namespace_syscalls", "/proc/self/fd/3", [target], { + stdio: ["ignore", "pipe", "pipe", probeFd], + }); + } finally { + closeSync(probeFd); + closeSync(Number(syscallProbe.split("/").at(-1))); + } } else { results.direct_namespace_syscalls = { exit: null, skipped: true }; } diff --git a/scripts/cp06-isolation-syscalls.c b/scripts/cp06-isolation-syscalls.c index 85019d9..2db5656 100644 --- a/scripts/cp06-isolation-syscalls.c +++ b/scripts/cp06-isolation-syscalls.c @@ -2,45 +2,145 @@ #include #include #include +#include #include +#include #include #include #include +#include +#include #include +#include +#include #include #if !defined(__linux__) || !defined(__x86_64__) #error "CP-06 isolation syscall probe currently supports Linux x86-64 only" #endif -static void result(const char *name, long value) { - printf("%s=%ld:%d\n", name, value, errno); +static void result(const char *prefix, const char *name, long value) { + int error = errno; + printf("%s%s=%ld:%d\n", prefix, name, value, error); } -int main(int argc, char **argv) { - if (argc != 2) return 64; - int namespace_fd = open("/proc/self/ns/mnt", O_RDONLY | O_CLOEXEC); - if (namespace_fd < 0) return 70; +static void observations(const char *prefix) { + printf("%ssecurebits=%d\n", prefix, prctl(PR_GET_SECUREBITS)); + printf("%sNoNewPrivs=%d\n", prefix, prctl(PR_GET_NO_NEW_PRIVS, 0, 0, 0, 0)); + printf("%sSeccomp=%d\n", prefix, prctl(PR_GET_SECCOMP, 0, 0, 0, 0)); + FILE *status = fopen("/proc/self/status", "r"); + if (!status) _exit(70); + char line[256], name[32], value[64]; + while (fgets(line, sizeof(line), status)) { + if (sscanf(line, "%31[^:]: %63s", name, value) == 2 && + (!strcmp(name, "CapInh") || !strcmp(name, "CapPrm") || + !strcmp(name, "CapEff") || !strcmp(name, "CapBnd") || !strcmp(name, "CapAmb"))) { + printf("%s%s=%s\n", prefix, name, value); + } + } + fclose(status); +} +static void attempts(const char *prefix, const char *mount_path, pid_t target, + uintptr_t address, int target_fd) { + int namespace_fd = open("/proc/self/ns/mnt", O_RDONLY | O_CLOEXEC); + if (namespace_fd < 0) _exit(70); errno = 0; - result("setns", syscall(SYS_setns, namespace_fd, 0)); + result(prefix, "setns", syscall(SYS_setns, namespace_fd, 0)); errno = 0; - result("unshare", syscall(SYS_unshare, CLONE_NEWUSER)); + result(prefix, "unshare", syscall(SYS_unshare, CLONE_NEWUSER)); errno = 0; long child = syscall(SYS_clone, (unsigned long)(CLONE_NEWUSER | SIGCHLD), 0, 0, 0, 0); if (child == 0) _exit(0); - result("clone", child); -#ifdef SYS_clone3 + result(prefix, "clone", child); + if (child > 0) waitpid((pid_t)child, NULL, 0); struct clone_args arguments = {.flags = CLONE_NEWUSER, .exit_signal = SIGCHLD}; errno = 0; child = syscall(SYS_clone3, &arguments, sizeof(arguments)); if (child == 0) _exit(0); - result("clone3", child); -#endif + result(prefix, "clone3", child); + if (child > 0) waitpid((pid_t)child, NULL, 0); errno = 0; - result("mount", syscall(SYS_mount, argv[1], argv[1], 0, MS_REMOUNT | MS_BIND, 0)); + result(prefix, "mount", syscall(SYS_mount, mount_path, mount_path, 0, MS_REMOUNT | MS_BIND, 0)); errno = 0; - result("umount2", syscall(SYS_umount2, argv[1], 0)); + result(prefix, "umount2", syscall(SYS_umount2, mount_path, 0)); close(namespace_fd); - return 0; + + errno = 0; + long attached = ptrace(PTRACE_ATTACH, target, NULL, NULL); + result(prefix, "ptrace_attach", attached); + if (attached == 0) { waitpid(target, NULL, 0); ptrace(PTRACE_DETACH, target, NULL, NULL); } + errno = 0; + long seized = ptrace(PTRACE_SEIZE, target, NULL, NULL); + result(prefix, "ptrace_seize", seized); + if (seized == 0) { kill(target, SIGSTOP); waitpid(target, NULL, WUNTRACED); ptrace(PTRACE_DETACH, target, NULL, NULL); } + char dummy[] = "DUMMY FIXTURE ONLY"; + struct iovec local = {.iov_base = dummy, .iov_len = sizeof(dummy)}; + struct iovec remote = {.iov_base = (void *)address, .iov_len = sizeof(dummy)}; + errno = 0; + result(prefix, "process_vm_readv", process_vm_readv(target, &local, 1, &remote, 1, 0)); + errno = 0; + result(prefix, "process_vm_writev", process_vm_writev(target, &local, 1, &remote, 1, 0)); + int pidfd = (int)syscall(SYS_pidfd_open, target, 0); + printf("%spidfd_open=%s\n", prefix, pidfd >= 0 ? "opened" : "failed"); + if (pidfd >= 0) { + errno = 0; + int copied = (int)syscall(SYS_pidfd_getfd, pidfd, target_fd, 0); + result(prefix, "pidfd_getfd", copied); + if (copied >= 0) close(copied); + close(pidfd); + } + char path[64]; + snprintf(path, sizeof(path), "/proc/%ld/mem", (long)target); + errno = 0; + int memory = open(path, O_RDWR | O_CLOEXEC); + result(prefix, "proc_mem", memory); + if (memory >= 0) close(memory); + observations(prefix); +} + +int main(int argc, char **argv) { + if (argc != 2) return 64; + alarm(10); + int ready[2], stop[2]; + if (pipe2(ready, O_CLOEXEC) || pipe2(stop, O_CLOEXEC)) return 70; + pid_t owner = getpid(); + pid_t target = fork(); + if (target < 0) return 70; + if (target == 0) { + close(ready[0]); close(stop[1]); + if (prctl(PR_SET_PDEATHSIG, SIGKILL) || getppid() != owner || + prctl(PR_SET_DUMPABLE, 0)) _exit(70); + volatile char dummy[] = "DUMMY FIXTURE ONLY"; + uintptr_t address = (uintptr_t)dummy; + if (write(ready[1], &address, sizeof(address)) != (ssize_t)sizeof(address)) _exit(70); + close(ready[1]); + char done; + if (read(stop[0], &done, 1) != 1) _exit(70); + const char expected[] = "DUMMY FIXTURE ONLY"; + for (size_t index = 0; index < sizeof(expected); ++index) if (dummy[index] != expected[index]) _exit(71); + _exit(0); + } + close(ready[1]); close(stop[0]); + uintptr_t address; + if (read(ready[0], &address, sizeof(address)) != (ssize_t)sizeof(address)) { + kill(target, SIGKILL); waitpid(target, NULL, 0); return 70; + } + close(ready[0]); + attempts("", argv[1], target, address, stop[0]); + fflush(stdout); + pid_t inherited = fork(); + if (inherited == 0) { + if (prctl(PR_SET_PDEATHSIG, SIGKILL) || getppid() != owner) _exit(70); + attempts("inherited_", argv[1], target, address, stop[0]); + fflush(stdout); + _exit(0); + } + int status = 0; + int ok = inherited > 0 && waitpid(inherited, &status, 0) == inherited && WIFEXITED(status) && WEXITSTATUS(status) == 0; + if (write(stop[1], "D", 1) != 1) ok = 0; + close(stop[1]); + if (waitpid(target, &status, 0) != target || !WIFEXITED(status) || WEXITSTATUS(status) != 0) ok = 0; + printf("dummy_target_unchanged=%s\n", ok ? "true" : "false"); + return ok ? 0 : 70; } diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index a05ee3f..3eb9ec4 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -1,10 +1,12 @@ #!/usr/bin/env node import { spawnSync } from "node:child_process"; +import { closeSync, constants, openSync } from "node:fs"; import { resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { mountLiveCredentialReadOnly } from "./cp06-credential-isolation.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; import { createProbeFixture } from "./cp06-probe-fixture.mjs"; +import { validateWorkerEvidence, workerEvidenceContext } from "./cp06-worker-evidence.mjs"; const root = resolve(fileURLToPath(new URL("..", import.meta.url))); @@ -89,6 +91,11 @@ export async function superviseCredentialChild(options, dependencies = {}) { throw failure("source-integrity", "CP06_SOURCE_IDENTITY_CHANGED", 74, cause); } } + let workerContext = dependencies.workerContext; + if (mode === "worker" && !dependencies.spawn) { + try { workerContext = workerEvidenceContext(childArgs[0], target, source); } + catch { throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); } + } const command = childCommand({ mode, source, @@ -101,13 +108,25 @@ export async function superviseCredentialChild(options, dependencies = {}) { }); let result; try { - result = spawn(command.command, command.args, { + const launch = (stdio) => spawn(command.command, command.args, { cwd: probe?.cwd ?? root, encoding: "utf8", env: childEnvironment(mode, source, target), timeout: mode === "worker" ? 600_000 : 60_000, maxBuffer: 1024 * 1024, + stdio, }); + if (dependencies.spawn) result = launch(undefined); + else { + const paths = mode === "unsafe-probe" ? [syscallProbe] : mode === "hardened-probe" ? [helper, syscallProbe] : [helper, childArgs[0]]; + const fds = []; + try { + for (const path of paths) fds.push(openSync(path, constants.O_RDONLY | constants.O_CLOEXEC)); + result = launch(["pipe", "pipe", "pipe", ...fds]); + } finally { + for (const fd of fds) closeSync(fd); + } + } } catch (cause) { throw failure("launch", "CP06_CHILD_LAUNCH_FAILED", 70, cause); } @@ -144,7 +163,12 @@ export async function superviseCredentialChild(options, dependencies = {}) { } catch { throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); } - if (!validChildContract(mode, childEvidence)) { + try { + if (!validChildContract(mode, childEvidence)) throw new Error(); + if (mode === "worker") { + validateWorkerEvidence(childEvidence, workerContext); + } + } catch { throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); } return { @@ -233,7 +257,7 @@ function childCommand({ "unsafe", childSource, childTarget, - syscallProbe, + "/proc/self/fd/3", beforeHash, String(process.pid), ], @@ -247,7 +271,7 @@ function childCommand({ "hardened", childSource, childTarget, - syscallProbe, + "/proc/self/fd/4", beforeHash, String(process.pid), ] @@ -256,7 +280,8 @@ function childCommand({ "--experimental-import-meta-resolve", resolve(root, "scripts/cp06-sdk-worker.mjs"), target, - ...childArgs, + "/proc/self/fd/4", + ...childArgs.slice(1), ]; return { command: "setpriv", @@ -267,7 +292,8 @@ function childCommand({ "--ambient-caps=-all", "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", "--", - helper, + "/proc/self/fd/3", + "--cp06-readonly-fd=4", ...program, ], }; @@ -414,7 +440,7 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur try { console.log( JSON.stringify( - await superviseCredentialChild({ mode, source, target, helper, syscallProbe, childArgs }), + await superviseCredentialChild({ mode, source, target, helper, syscallProbe, childArgs: childArgs.map((path) => /^\/proc\/self\/fd\/\d+$/u.test(path) ? `/proc/${process.pid}/fd/${path.slice("/proc/self/fd/".length)}` : path) }), ), ); } catch (error) { diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index d8a82a8..65908cc 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -8,6 +8,8 @@ import { mkdirSync, openSync, readSync, + readFileSync, + ftruncateSync, readdirSync, rmdirSync, unlinkSync, @@ -178,7 +180,7 @@ function assertArtifactName(name) { } } -export function writeAnchoredFile(parentAnchor, name, data) { +export function writeAnchoredFile(parentAnchor, name, data, { replace = false } = {}) { assertArtifactName(name); const bytes = Buffer.from(data); const path = join(parentAnchor, name); @@ -186,7 +188,7 @@ export function writeAnchoredFile(parentAnchor, name, data) { path, constants.O_WRONLY | constants.O_CREAT | - constants.O_EXCL | + (replace ? 0 : constants.O_EXCL) | constants.O_NOFOLLOW | constants.O_CLOEXEC, 0o600, @@ -196,6 +198,10 @@ export function writeAnchoredFile(parentAnchor, name, data) { if (!created.isFile() || created.nlink !== 1n) { throw new Error("DUMMY proof artifact is not one fresh regular file"); } + if (!sameInode(created, lstatSync(path, { bigint: true }))) { + throw new Error("DUMMY proof artifact identity changed"); + } + if (replace) ftruncateSync(fd, 0); let offset = 0; while (offset < bytes.length) offset += writeSync(fd, bytes, offset, bytes.length - offset); if (!sameInode(created, lstatSync(path, { bigint: true }))) { @@ -207,6 +213,51 @@ export function writeAnchoredFile(parentAnchor, name, data) { } } +export function readAnchoredFile(parentAnchor, name) { + assertArtifactName(name); + const path = join(parentAnchor, name); + const fd = openSync(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC); + try { + const pinned = fstatSync(fd, { bigint: true }); + if (!pinned.isFile() || pinned.nlink !== 1n || !sameInode(pinned, lstatSync(path, { bigint: true }))) { + throw new Error("DUMMY proof artifact identity changed"); + } + const bytes = readFileSync(fd); + const current = fstatSync(fd, { bigint: true }); + if (!sameInode(pinned, lstatSync(path, { bigint: true })) || + current.size !== pinned.size || current.mtimeNs !== pinned.mtimeNs || + current.ctimeNs !== pinned.ctimeNs) { + throw new Error("DUMMY proof artifact identity changed"); + } + return bytes; + } finally { + closeSync(fd); + } +} + +export function withArtifactParent(path, action) { + const match = String(path).match(new RegExp(`^(/proc/${process.pid}/fd/\\d+)/(.+)$`, "u")); + if (!match) throw new Error("DUMMY proof artifact must use a retained directory"); + const components = match[2].split("/"); + const name = components.pop(); + assertArtifactName(name); + const parent = components.length ? openAnchoredDirectory(match[1], components.join("/"), { create: false }) : null; + try { + return action(parent?.anchor ?? match[1], name); + } finally { + parent?.close(); + } +} + +export function writeArtifactFile(path, data, options) { + return withArtifactParent(path, (anchor, name) => writeAnchoredFile(anchor, name, data, options)); +} + +export function readArtifactFile(path, encoding) { + const bytes = withArtifactParent(path, readAnchoredFile); + return encoding ? bytes.toString(encoding) : bytes; +} + export async function createProbeFixture({ root, outputRoot, spawn = spawnSync }) { const output = openProbeOutput({ root, outputRoot, create: true }); let rootFd; diff --git a/scripts/cp06-sdk-worker.mjs b/scripts/cp06-sdk-worker.mjs index c198b0c..90be6c9 100644 --- a/scripts/cp06-sdk-worker.mjs +++ b/scripts/cp06-sdk-worker.mjs @@ -1,5 +1,7 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; +import { closeSync } from "node:fs"; +import { readWorkerInput } from "./cp06-worker-evidence.mjs"; import { readFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; @@ -18,7 +20,8 @@ if (!credentialTargetArg || !inputPathArg) process.exit(64); try { const credentialTarget = resolve(credentialTargetArg); - const input = JSON.parse(await readFile(resolve(inputPathArg), "utf8")); + const input = readWorkerInput(resolve(inputPathArg)); + if (inputPathArg === "/proc/self/fd/4") closeSync(4); validateInput(input); const paths = canonicalWorkerPaths(input, credentialTarget, process.env.CP06_CREDENTIAL_SOURCE); const pi = resolvePiPackage(); diff --git a/scripts/cp06-seccomp-exec.c b/scripts/cp06-seccomp-exec.c index b77a02f..cedb22d 100644 --- a/scripts/cp06-seccomp-exec.c +++ b/scripts/cp06-seccomp-exec.c @@ -49,14 +49,17 @@ static void deny(scmp_filter_ctx ctx, const char *name, int error) { } } -static void close_inherited_fds(void) { +static void close_inherited_fds(int retained) { #ifdef SYS_close_range - if (syscall(SYS_close_range, 3U, ~0U, 0U) == 0) return; + if (retained == 4) { + close(3); + if (syscall(SYS_close_range, 5U, ~0U, 0U) == 0) return; + } else if (syscall(SYS_close_range, 3U, ~0U, 0U) == 0) return; if (errno != ENOSYS) fail("close_range"); #endif long limit = sysconf(_SC_OPEN_MAX); if (limit < 0 || limit > 1048576) limit = 1048576; - for (int fd = 3; fd < limit; ++fd) close(fd); + for (int fd = 3; fd < limit; ++fd) if (fd != retained) close(fd); } /* @@ -160,7 +163,16 @@ int main(int argc, char **argv) { fprintf(stderr, "CP-06 supervisor disappeared before isolation\n"); return 70; } - close_inherited_fds(); + int retained = -1; + if (strcmp(argv[1], "--cp06-readonly-fd=4") == 0) { + struct stat st; + if (argc < 3 || fstat(4, &st) != 0 || !S_ISREG(st.st_mode) || + (fcntl(4, F_GETFL) & O_ACCMODE) != O_RDONLY) return 64; + retained = 4; + ++argv; + --argc; + } + close_inherited_fds(retained); if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0) fail("PR_SET_NO_NEW_PRIVS"); scmp_filter_ctx ctx = seccomp_init(SCMP_ACT_ALLOW); diff --git a/scripts/cp06-worker-evidence.mjs b/scripts/cp06-worker-evidence.mjs new file mode 100644 index 0000000..f8cc704 --- /dev/null +++ b/scripts/cp06-worker-evidence.mjs @@ -0,0 +1,76 @@ +import { closeSync, constants, fstatSync, openSync, readFileSync } from "node:fs"; +import { auditWorkerOutcome } from "./cp06-worker-outcome.mjs"; +import { validateReadAudit } from "./cp06-auth-security.mjs"; +import { canonicalWorkerPaths, readPinnedRegularFile } from "./cp06-guarded-read.mjs"; + +export function workerEvidenceContext(inputPath, target, source) { + const input = readWorkerInput(inputPath); + const paths = canonicalWorkerPaths(input, target, source); + const snapshot = readPinnedRegularFile(paths.originalPath); + return { input, paths, original: snapshot.bytes, originalIdentity: snapshot.identity }; +} + +export function readWorkerInput(path) { + if (path === `/proc/${process.pid}/fd/5` || path === "/proc/self/fd/4") { + const fd = openSync(path, constants.O_RDONLY | constants.O_CLOEXEC); + try { + const identity = fstatSync(fd); + if (!identity.isFile() || identity.nlink !== 1 || identity.size < 1 || identity.size > 50 * 1024) throw new Error("CP06_CHILD_OUTPUT_INVALID"); + return JSON.parse(readFileSync(fd, "utf8")); + } finally { closeSync(fd); } + } + return JSON.parse(readPinnedRegularFile(path).bytes.toString("utf8")); +} + +export function validateWorkerEvidence(child, { input, paths, original, originalIdentity }) { + if (originalIdentity) { + const current = readPinnedRegularFile(paths.originalPath); + for (const key of ["dev", "ino", "size", "nlink", "mode", "mtimeNs", "ctimeNs"]) requireValue(current.identity[key] === originalIdentity[key]); + requireValue(current.bytes.equals(original)); + } + exact(child, ["credential_store", "event_stream", "model", "oauth_preflight", "pi_install", "pi_version", "provider", "raw_transcript_retained", "response", "response_sha256", "result", "schema_version", "session"]); + requireValue(child.schema_version === 1 && child.result === "pass" && child.pi_version === "0.85.1" && child.raw_transcript_retained === false); + requireValue(child.provider === input.provider && child.model === input.model); + exact(child.pi_install, ["install_root", "package_root", "package_artifact", "dependency", "executable", "executable_entry", "sdk_entry", "package_sha256", "executable_sha256", "sdk_entry_sha256"]); + for (const key of ["install_root", "package_root", "executable", "executable_entry", "sdk_entry"]) requireValue(typeof child.pi_install[key] === "string" && child.pi_install[key].startsWith("/")); + for (const key of ["package_sha256", "executable_sha256", "sdk_entry_sha256"]) requireValue(digest(child.pi_install[key])); + exact(child.pi_install.package_artifact, ["manifest_sha256", "tarball_integrity"]); + requireValue(digest(child.pi_install.package_artifact.manifest_sha256) && typeof child.pi_install.package_artifact.tarball_integrity === "string" && child.pi_install.package_artifact.tarball_integrity.startsWith("sha512-")); + exact(child.pi_install.dependency, ["entry", "entry_sha256", "manifest_sha256", "name", "package_sha256", "resolution", "root", "tarball_integrity", "version"]); + const dependency = child.pi_install.dependency; + requireValue(dependency.version === "0.85.1" && dependency.name === "@earendil-works/pi-ai" && dependency.resolution === "esm-import-condition"); + for (const key of ["entry_sha256", "manifest_sha256", "package_sha256"]) requireValue(digest(dependency[key])); + for (const key of ["root", "entry"]) requireValue(typeof dependency[key] === "string" && dependency[key].startsWith("/")); + requireValue(typeof dependency.tarball_integrity === "string" && dependency.tarball_integrity.startsWith("sha512-")); + exact(child.oauth_preflight, ["minimum_validity_ms", "remaining_validity_ms", "refreshed"]); + requireValue(child.oauth_preflight.refreshed === false && child.oauth_preflight.minimum_validity_ms === input.timeout_ms + 300_000 && Number.isSafeInteger(child.oauth_preflight.remaining_validity_ms) && child.oauth_preflight.remaining_validity_ms >= child.oauth_preflight.minimum_validity_ms); + exact(child.credential_store, ["reads", "lists", "modify_denials", "delete_denials"]); + requireValue(Number.isSafeInteger(child.credential_store.reads) && child.credential_store.reads >= 1 && Number.isSafeInteger(child.credential_store.lists) && child.credential_store.lists >= 0 && child.credential_store.modify_denials === 0 && child.credential_store.delete_denials === 0); + exact(child.session, ["in_memory", "active_tools"]); + requireValue(child.session.in_memory === true && JSON.stringify(child.session.active_tools) === '["read"]'); + exact(child.event_stream, ["sha256", "event_count", "read_audit"]); + requireValue(digest(child.event_stream.sha256) && digest(child.response_sha256)); + validateReadAudit(child.event_stream.read_audit, { + toolCallId: child.event_stream.read_audit?.tool_call_id, + originalPath: paths.originalPath, + original, + eventCount: child.event_stream.event_count, + }); + requireValue(typeof child.event_stream.read_audit.tool_call_id === "string" && child.event_stream.read_audit.tool_call_id.length > 0); + requireValue(auditWorkerOutcome(child.response, { + root: paths.root, + referenceRoot: paths.fixtureRoot, + expectedOriginalPath: paths.originalPath, + }).ok); + return child; +} + +function exact(value, keys) { + requireValue(value !== null && typeof value === "object" && !Array.isArray(value) && Object.keys(value).sort().join(",") === keys.sort().join(",")); +} +function requireValue(value) { + if (!value) throw new Error("CP06_CHILD_OUTPUT_INVALID"); +} +function digest(value) { + return typeof value === "string" && /^[0-9a-f]{64}$/u.test(value); +} diff --git a/scripts/replay-cp06-p07.mjs b/scripts/replay-cp06-p07.mjs index fa1fecc..1b6ab9e 100644 --- a/scripts/replay-cp06-p07.mjs +++ b/scripts/replay-cp06-p07.mjs @@ -1,11 +1,11 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; -import { realpathSync, writeFileSync } from "node:fs"; -import { mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import { realpathSync } from "node:fs"; +import { readFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { openAnchoredDirectory, openProbeOutput, writeArtifactFile, readArtifactFile } from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); @@ -68,14 +68,14 @@ const manifest = { ], }; const manifestPath = join(p07Root.anchor, "evidence.json"); -await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); +const manifestHash = writeArtifactFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); console.log( JSON.stringify({ ok: true, gate: "P-07", tested_sha: testedSha, manifest_path: relative(manifestPath), - manifest_sha256: await fileHash(manifestPath), + manifest_sha256: manifestHash, sequence: steps.map(({ id, command, exit_code, output_path, output_sha256 }) => ({ id, command, @@ -91,7 +91,7 @@ function run(id, command, args, expectedExit, expectedOutputFragments = []) { const exitCode = result.status; const output = `command: ${quote([command, ...args])}\nexit: ${exitCode}\n--- stdout ---\n${result.stdout ?? ""}\n--- stderr ---\n${result.stderr ?? ""}`; const outputPath = join(rawRoot, `${id}.txt`); - writeFileSync(outputPath, output); + writeArtifactFile(outputPath, output); const missingFragments = expectedOutputFragments.filter((fragment) => !output.includes(fragment)); if (result.error !== undefined || exitCode !== expectedExit || missingFragments.length > 0) { process.stderr.write(output); @@ -136,6 +136,6 @@ function sha256(text) { async function fileHash(path) { return createHash("sha256") - .update(await readFile(path)) + .update(path.startsWith(`/proc/${process.pid}/fd/`) ? readArtifactFile(path) : await readFile(path)) .digest("hex"); } diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index d50b225..d025927 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -1,11 +1,12 @@ #!/usr/bin/env node import { createHash } from "node:crypto"; -import { existsSync, readFileSync, realpathSync, writeFileSync } from "node:fs"; -import { cp, mkdir, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { existsSync, readFileSync, realpathSync, readdirSync } from "node:fs"; +import { stat } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -import { openAnchoredDirectory, openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { openAnchoredDirectory, openProbeOutput, writeArtifactFile, readArtifactFile, withArtifactParent, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { readPinnedRegularFile } from "./cp06-guarded-read.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const fixtureSource = join(root, "test/fixtures/cp06-context"); @@ -14,7 +15,8 @@ const output = openProbeOutput({ root, outputRoot, create: true, runner: true }) const proofDirectory = openAnchoredDirectory(output.anchor, "ten-pack", { reset: true }); const rawDirectory = openAnchoredDirectory(proofDirectory.anchor, "raw"); const proofRoot = proofDirectory.anchor; -const fixtureRoot = join(proofRoot, "repo"); +const fixtureDirectory = openAnchoredDirectory(proofRoot, "repo"); +const fixtureRoot = fixtureDirectory.anchor; const rawRoot = rawDirectory.anchor; const cli = join(root, "dist/src/cli.js"); const ctx = resolveTool(process.env.CTX_BIN ?? "ctx"); @@ -22,8 +24,8 @@ configureOfflineCtxModelDir(); const reviewer = process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker"; const commands = []; -await cp(fixtureSource, fixtureRoot, { recursive: true }); -await mkdir(join(fixtureRoot, ".factory/state"), { recursive: true }); +copyFixtureTree(fixtureSource, fixtureRoot); +openAnchoredDirectory(fixtureRoot, ".factory/state").close(); const oracle = JSON.parse(await readFile(join(fixtureRoot, "oracle.json"), "utf8")); const fixedAuth = await readFile(join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), "utf8"); const beforeAuth = await readFile(join(fixtureRoot, "cases/AUTH-before.yaml"), "utf8"); @@ -325,8 +327,8 @@ for (const task of oracle.tasks) { } const packCopy = join(rawRoot, `${task.id}.pack.md`); const receiptCopy = join(rawRoot, `${task.id}.receipt.json`); - await cp(packAbsolute, packCopy); - await cp(receiptAbsolute, receiptCopy); + writeArtifactFile(packCopy, readArtifactFile(packAbsolute)); + writeArtifactFile(receiptCopy, readArtifactFile(receiptAbsolute)); packs.push({ task_id: task.id, risk: task.risk, @@ -647,7 +649,7 @@ function assertSemantic(status, label) { } async function copyArtifact(path, name) { - await cp(join(fixtureRoot, path), join(rawRoot, name)); + writeArtifactFile(join(rawRoot, name), readArtifactFile(join(fixtureRoot, path))); } function fileEvidence(path) { @@ -699,15 +701,38 @@ function sha256(value) { } async function fileHash(path) { - return createHash("sha256") - .update(await readFile(path)) - .digest("hex"); + return sha256(requireRead(path)); } function requireRead(path) { - return readFileSync(path); + return path.startsWith(`/proc/${process.pid}/fd/`) ? readArtifactFile(path) : readPinnedRegularFile(path).bytes; } function requireWrite(path, value) { - writeFileSync(path, value); + writeArtifactFile(path, value); +} + +async function readFile(path, encoding) { + const bytes = requireRead(path); + return encoding ? bytes.toString(encoding) : bytes; +} + +async function writeFile(path, value) { + return writeArtifactFile(path, value, { replace: path === join(fixtureRoot, ".factory/tasks/AUTH-001.yaml") }); +} + +async function rm(path) { + withArtifactParent(path, removeAnchoredEntry); +} + +function copyFixtureTree(source, destination) { + for (const entry of readdirSync(source, { withFileTypes: true })) { + if (entry.isDirectory()) { + const child = openAnchoredDirectory(destination, entry.name); + try { copyFixtureTree(join(source, entry.name), child.anchor); } + finally { child.close(); } + } else if (entry.isFile()) { + writeArtifactFile(join(destination, entry.name), readPinnedRegularFile(join(source, entry.name)).bytes); + } else throw new Error("DUMMY fixture contains a nonregular entry"); + } } diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 7515bd1..9ba8797 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -1,4 +1,6 @@ import { strict as assert } from "node:assert"; +import { createHash } from "node:crypto"; +import { DUMMY_SDK_ORIGINAL, DUMMY_OUTCOME_ORIGINAL } from "../scripts/cp06-dummy-originals.mjs"; import { spawnSync } from "node:child_process"; import { readlinkSync, renameSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { @@ -153,7 +155,7 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { sdk_worker: { active_tools: ["read"], in_memory_session: true, - event_count: 2, + event_count: 47, event_sha256: "0".repeat(64), read_audit: readAudit("DUMMY-read", "@ROOT@/DUMMY-worker/DUMMY-original.md"), refresh_callbacks: 0, @@ -189,6 +191,10 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { }; for (const payload of [ { ...validBase, auth_cases: [] }, + { ...validBase, sdk_worker: { ...validBase.sdk_worker, event_count: 2 } }, + { ...validBase, sdk_worker: { ...validBase.sdk_worker, read_audit: { ...validBase.sdk_worker.read_audit, expected_sha256: "a".repeat(64), returned_sha256: "a".repeat(64), returned_bytes: 1 } } }, + { ...validBase, actual_sdk_outcomes: { ...validBase.actual_sdk_outcomes, cases: validBase.actual_sdk_outcomes.cases.map((entry) => entry.scenario === "valid" ? { ...entry, event_count: 2 } : entry) } }, + { ...validBase, actual_sdk_outcomes: { ...validBase.actual_sdk_outcomes, cases: validBase.actual_sdk_outcomes.cases.map((entry) => entry.scenario === "valid" ? { ...entry, read_audit: { ...entry.read_audit, expected_sha256: "a".repeat(64), returned_sha256: "a".repeat(64), returned_bytes: 1 } } : entry) } }, { ...validBase, auth_cases: [...validBase.auth_cases, authCase("expired", "blocked", 0)] }, { ...validBase, extra: "DUMMY-SECRET" }, { ...validBase, sdk_dependency: { ...validBase.sdk_dependency, version: "0.99.2" } }, @@ -418,9 +424,9 @@ function readAudit(toolCallId, originalPath) { other_tool_completions: [], completed_successfully: true, exact_original: true, - expected_sha256: "5".repeat(64), - returned_sha256: "5".repeat(64), - returned_bytes: 117, + expected_sha256: createHash("sha256").update(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL).digest("hex"), + returned_sha256: createHash("sha256").update(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL).digest("hex"), + returned_bytes: Buffer.byteLength(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL), tool_call_id: toolCallId, start_event_index: 12, end_event_index: 13, @@ -461,6 +467,7 @@ function outcome(scenario, exitCode, extra = {}) { }; if (["valid", "fenced-valid"].includes(scenario)) { result.read_audit = readAudit("DUMMY-actual-worker-read", "@ROOT@/DUMMY-outcome/docs/AUTH.md"); + result.event_count = 47; } return result; } diff --git a/test/cp06-review-boundaries.test.mjs b/test/cp06-review-boundaries.test.mjs new file mode 100644 index 0000000..137f599 --- /dev/null +++ b/test/cp06-review-boundaries.test.mjs @@ -0,0 +1,118 @@ +import { strict as assert } from "node:assert"; +import { spawnSync } from "node:child_process"; +import { closeSync, constants, openSync } from "node:fs"; +import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import test from "node:test"; +import { assertHardenedProof, assertUnsafeControl, runIsolationProbe } from "../scripts/cp06-auth-security.mjs"; +import { openProbeOutput, openAnchoredDirectory, readArtifactFile, writeArtifactFile } from "../scripts/cp06-probe-fixture.mjs"; +import { superviseCredentialChild, supervisorFailureRecord } from "../scripts/cp06-namespace-supervisor.mjs"; +import { dummyWorkerContext, dummyWorkerEvidence } from "../scripts/cp06-dummy-worker-evidence.mjs"; +import { withCompiledHelper } from "./fixtures/cp06-compiled-helper.mjs"; +import { openPinnedDirectory } from "../scripts/cp06-guarded-read.mjs"; + +const state = resolve(".factory/state/cp06-correction"); + +test("real hardened probe executes retained read-only descriptors and inherited DUMMY denials", async () => { + await withCompiledHelper("review-kernel", async ({ binaries, helperFd }) => { + const outputRoot = join(state, "DUMMY-review-kernel", "auth-security"); + const options = { root: process.cwd(), outputRoot, binaries }; + try { + assertUnsafeControl(await runIsolationProbe({ ...options, mode: "unsafe-probe" })); + const proof = await runIsolationProbe({ ...options, mode: "hardened-probe" }); + assertHardenedProof(proof); + const missing = structuredClone(proof); + missing.child.direct_namespace_syscalls.stdout = missing.child.direct_namespace_syscalls.stdout.split("\n").filter((line) => !line.startsWith("inherited_ptrace_attach=")).join("\n"); + assert.throws(() => assertHardenedProof(missing)); + const probeFd = openSync(binaries.syscallProbe, constants.O_RDONLY | constants.O_CLOEXEC); + try { + const closed = spawnSync("unshare", ["--user", "--map-root-user", "--mount", "--propagation", "private", "--", "setpriv", "--no-new-privs", "--bounding-set=-all", "--inh-caps=-all", "--ambient-caps=-all", "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", "--", "/proc/self/fd/3", "/proc/self/fd/4", "/DUMMY-no-mount"], { + stdio: ["ignore", "pipe", "pipe", helperFd, probeFd], encoding: "utf8", timeout: 20_000, + }); + assert.equal(closed.status, 70); + assert.equal(closed.stdout, ""); + } finally { closeSync(probeFd); } + } finally { await rm(join(state, "DUMMY-review-kernel"), { recursive: true, force: true }); } + }); +}); + +test("full semantic contracts reject forged nested evidence inside independently awaited cleanup", async () => { + const context = dummyWorkerContext(); + const good = dummyWorkerEvidence(context); + const variants = [ + { ...good, response: { status: "PASS", evidence_gaps: [] } }, + { ...good, response: { ...good.response, evidence_gaps: [] } }, + { ...good, event_stream: { ...good.event_stream, extra: "DUMMY-SECRET" } }, + { ...good, event_stream: { ...good.event_stream, event_count: 2 } }, + { ...good, event_stream: { ...good.event_stream, read_audit: { ...good.event_stream.read_audit, expected_sha256: "a".repeat(64), returned_sha256: "a".repeat(64), returned_bytes: 1 } } }, + { ...good, pi_install: { ...good.pi_install, extra: "DUMMY-SECRET" } }, + ]; + for (const child of [good, ...variants]) { + for (const cleanupFails of [false, true]) { + let cleaned = 0; + let caught; + try { + await superviseCredentialChild({ mode: "worker", source: "/DUMMY-source", target: "/DUMMY-target", helper: "/DUMMY-helper", syscallProbe: "/DUMMY-probe" }, { + workerContext: context, + mount: async () => ({ mountIds: { source: "1", target: "2" }, parentMounts: ["DUMMY"], verifyIntegrity: async () => true, cleanup: async () => { + await new Promise((done) => setTimeout(done, 1)); cleaned++; + if (cleanupFails) throw new Error("DUMMY-SECRET"); + } }), + spawn: () => ({ status: 0, stdout: JSON.stringify(child) }), + }); + } catch (error) { caught = error; } + assert.equal(cleaned, 1); + if (child === good && !cleanupFails) assert.equal(caught, undefined); + else { + const record = supervisorFailureRecord(caught); + assert.equal(JSON.stringify(record).includes("DUMMY-SECRET"), false); + const codes = record.causes.length ? record.causes.map((cause) => cause.code) : [record.code]; + assert.deepEqual(codes, [...(child === good ? [] : ["CP06_CHILD_OUTPUT_INVALID"]), ...(cleanupFails ? ["CP06_CREDENTIAL_CLEANUP_FAILED"] : [])]); + } + } + } +}); + +test("artifact writes, replacements, copies and reads reject redirected leaves and intermediate directories", async () => { + const outputRoot = join(state, `DUMMY-review-artifacts-${process.pid}`); + const output = openProbeOutput({ root: process.cwd(), outputRoot, create: true, runner: true }); + const child = openAnchoredDirectory(output.anchor, "raw"); + const sentinel = join(outputRoot, "DUMMY-sentinel"); + await writeFile(sentinel, "DUMMY SENTINEL UNCHANGED"); + try { + for (const name of ["evidence.json", "DUMMY-counters.json", "raw-copy.txt"]) { + const path = join(child.anchor, name); + await symlink(sentinel, join(outputRoot, "raw", name)); + assert.throws(() => writeArtifactFile(path, "DUMMY write")); + assert.throws(() => writeArtifactFile(path, "DUMMY replace", { replace: true })); + assert.throws(() => readArtifactFile(path)); + assert.equal(await readFile(sentinel, "utf8"), "DUMMY SENTINEL UNCHANGED"); + } + await symlink(outputRoot, join(outputRoot, "raw", "redirected")); + assert.throws(() => writeArtifactFile(join(child.anchor, "redirected", "DUMMY-sentinel"), "DUMMY write", { replace: true })); + assert.throws(() => openPinnedDirectory(join(outputRoot, "raw", "redirected", "raw"))); + const source = join(child.anchor, "DUMMY-original.txt"); + writeArtifactFile(source, "DUMMY bytes"); + writeArtifactFile(join(child.anchor, "DUMMY-copy.txt"), readArtifactFile(source)); + assert.equal(readArtifactFile(join(child.anchor, "DUMMY-copy.txt"), "utf8"), "DUMMY bytes"); + } finally { + child.close(); output.close(); await rm(outputRoot, { recursive: true, force: true }); + } +}); + +for (const [runner, directory, rawName] of [["replay-cp06-ten-pack.mjs", "ten-pack", "git-init.txt"], ["replay-cp06-p07.mjs", "p07", "build.txt"]]) { + test(`${runner} refuses a substituted raw artifact at its executable boundary`, async () => { + await mkdir(state, { recursive: true }); + const directoryRoot = await mkdtemp(join(state, "DUMMY-runner-")); + const sentinel = join(directoryRoot, "DUMMY-sentinel"); + const hook = join(directoryRoot, "DUMMY-hook.mjs"); + await writeFile(sentinel, "DUMMY SENTINEL UNCHANGED"); + await writeFile(hook, `import child from 'node:child_process'; import {symlinkSync} from 'node:fs'; import {syncBuiltinESMExports} from 'node:module'; const real = child.spawnSync; child.spawnSync = (command,args,options) => { if (${JSON.stringify(directory)} === 'ten-pack' ? command === 'git' && args[0] === 'init' : command === 'npm') { symlinkSync(${JSON.stringify(sentinel)}, ${JSON.stringify(join(directoryRoot, directory, "raw", rawName))}); return {status:0, stdout:'DUMMY', stderr:''}; } return real(command,args,options); }; syncBuiltinESMExports();`); + try { + const result = spawnSync(process.execPath, ["--import", hook, resolve("scripts", runner)], { env: { ...process.env, CP06_OUTPUT: directoryRoot }, encoding: "utf8", timeout: 30_000 }); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /EEXIST/); + assert.equal(await readFile(sentinel, "utf8"), "DUMMY SENTINEL UNCHANGED"); + } finally { await rm(directoryRoot, { recursive: true, force: true }); } + }); +} diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index cb77f46..413dd7d 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -29,6 +29,8 @@ import { writeDummyCliShims, } from "../scripts/cp06-dummy-cli-shims.mjs"; import { withCompiledHelper } from "./fixtures/cp06-compiled-helper.mjs"; +import { dummyWorkerContext, dummyWorkerEvidence } from "../scripts/cp06-dummy-worker-evidence.mjs"; +import { DUMMY_OUTCOME_ORIGINAL } from "../scripts/cp06-dummy-originals.mjs"; for (const scenario of [ "success", @@ -70,6 +72,7 @@ for (const scenario of [ childArgs: ["/DUMMY-input"], }, { + workerContext: dummyWorkerContext(), async mount() { if (scenario === "setup") throw setupError; if (scenario === "setup-and-cleanup") { @@ -499,6 +502,16 @@ test("supervisor executable serializes compound worker failures as typed causes" await mkdir(shims); await mkdir(join(directory, "credential")); await writeFile(source, sourceBytes); + await mkdir(join(directory, "docs")); + await writeFile(join(directory, "docs/AUTH.md"), DUMMY_OUTCOME_ORIGINAL); + await writeFile(join(directory, "DUMMY-contract.yaml"), "DUMMY contract"); + await writeFile(join(directory, "DUMMY-pack.md"), "DUMMY pack"); + await writeFile(join(directory, "DUMMY-input.json"), JSON.stringify({ + schema_version: 1, root: directory, fixture_root: directory, + original_path: join(directory, "docs/AUTH.md"), + contract_path: join(directory, "DUMMY-contract.yaml"), pack_path: join(directory, "DUMMY-pack.md"), + provider: "openai-codex", model: "DUMMY-model", timeout_ms: 1_000, + })); try { const result = spawnSync( "unshare", @@ -585,6 +598,7 @@ test("worker source integrity uses metadata without reading credential bytes", a syscallProbe: "/DUMMY-probe", }, { + workerContext: dummyWorkerContext(), async mount() { return { sourceIdentity, @@ -712,25 +726,7 @@ test("audit and cleanup causes survive supervisor output and replay blocked reco }); function validWorkerChild() { - return { - schema_version: 1, - result: "pass", - pi_version: "0.85.1", - pi_install: { version: "0.85.1" }, - provider: "openai-codex", - model: "DUMMY-model", - oauth_preflight: { minimum_validity_ms: 1, remaining_validity_ms: 2, refreshed: false }, - credential_store: { reads: 1, lists: 0, modify_denials: 0, delete_denials: 0 }, - session: { in_memory: true, active_tools: ["read"] }, - event_stream: { - sha256: "0".repeat(64), - event_count: 3, - read_audit: { exact_original: true, read_count: 1 }, - }, - response: { status: "MISSING_SOURCE" }, - response_sha256: "1".repeat(64), - raw_transcript_retained: false, - }; + return dummyWorkerEvidence(); } function validProbeChild() { From 76e478347f4428e761611cad4aaa28f921342070 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Mon, 5 Oct 2026 14:13:47 +0000 Subject: [PATCH 40/45] no-mistakes(review): Bind CP06 ancestry, artifact identity, provenance, and cleanup causes --- docs/probes/CP-06.md | 30 ++++-- scripts/cp06-auth-security.mjs | 54 ++++++++-- scripts/cp06-credential-isolation.mjs | 42 +++++--- scripts/cp06-dummy-cli-shims.mjs | 6 ++ scripts/cp06-dummy-originals.mjs | 6 ++ scripts/cp06-dummy-worker-evidence.mjs | 13 ++- scripts/cp06-namespace-supervisor.mjs | 25 ++++- scripts/cp06-probe-fixture.mjs | 109 ++++++++++++++----- scripts/cp06-worker-evidence.mjs | 27 ++++- scripts/prove-cp06-worker-cleanup.mjs | 1 + scripts/replay-cp06-ten-pack.mjs | 41 +++---- scripts/replay-cp06-worker.mjs | 3 +- test/cp06-credential-isolation.test.mjs | 19 ++++ test/cp06-proof-boundary.test.mjs | 55 +++++++++- test/cp06-review-boundaries.test.mjs | 138 +++++++++++++++++++++++- test/cp06-worker-lifecycle.test.mjs | 2 + test/fixtures/cp06-credential-mount.mjs | 38 +++++-- 17 files changed, 507 insertions(+), 102 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index b14e834..5c21529 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -25,7 +25,7 @@ The historical 2026-10-05 WSL2 mount identity failure belonged to the util-linux Source and target remain distinct `ro,nosuid,nodev,noexec` mount instances of one pinned inode, with containing directories separately read-only inside the private namespace and writable DUMMY scratch. Host namespace and permissions are unchanged. Support is Linux x86-64, including WSL2 Linux guests with the required kernel primitives; unsupported namespaces, mount APIs, libseccomp, compilation, SDK versions or cleanup fail closed, without fallback. This is not Windows host filesystem/credential coverage; Windows and macOS are not claimed. -The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup retains source and target parent descriptors, checks the prepared target identity, and consumes inode descriptors in the native mount helper; executable transition regressions reject replaced ancestors, leaves and parents without touching external DUMMY sentinels. Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal, `openAnchoredDirectory(..., { reset: true })`, and the replay runner's removal of its evaluation home (created by `mkdtemp` relative to a retained no-follow `TMPDIR` descriptor) use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink or for a different real directory. Sibling ten-pack/P-07 manifests, raw outputs, destination copies and hashes now also consume descriptor-relative no-follow artifact operations. Replaced task contracts are opened without truncation, checked for a single regular inode, then truncated through that retained descriptor. DUMMY SDK/outcome fixture writes and exit counters use the same bounded artifact operations. A security preflight failure still blocks worker-cleanup proof before any configured credential use. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. +The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup retains source and target parent descriptors, checks the prepared target identity, and consumes inode descriptors in the native mount helper; executable transition regressions reject replaced ancestors, leaves and parents without touching external DUMMY sentinels. Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal, `openAnchoredDirectory(..., { reset: true })`, and the replay runner's removal of its evaluation home (created by `mkdtemp` relative to a retained no-follow `TMPDIR` descriptor) use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink or for a different real directory. Sibling ten-pack/P-07 manifests, raw outputs, destination copies and hashes now also consume descriptor-relative no-follow artifact operations. Replaced task contracts are opened without truncation, checked for a single regular inode equal to the inode the runner itself created when preparing the fixture, then truncated through that retained descriptor; reads of runner-created artifacts require the same recorded inode, and the ten-pack manifest, pack and receipt digests are the digests returned by their writes rather than a rehash of the path. The DUMMY SDK proof root is created relative to a retained no-follow `TMPDIR` descriptor and, after success, failure or timeout, removed with `removeAnchoredEntry` only while its name still identifies the retained root descriptor; otherwise the cleanup failure is reported beside the initiating cause. DUMMY SDK/outcome fixture writes and exit counters use the same bounded artifact operations. A security preflight failure still blocks worker-cleanup proof before any configured credential use. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. @@ -266,9 +266,13 @@ malformed-orphan, extra-project, credential-path, symlink-alias, and post-construction replacement-race cases. The genuine Pi SDK process runs in a private Linux user/mount namespace only -after the complete DUMMY auth and OS preflight passes. A supervisor pins the -regular source inode with `O_NOFOLLOW`, creates distinct source/target bind -mounts with read-only VFS flags, and closes the pin before launch. The evaluated +after the complete DUMMY auth and OS preflight passes. A supervisor walks the +source and target parent directories from `/` with no-follow directory +descriptors (creating missing target components relative to the retained +parent), so a symlink substituted for any ancestor is rejected before a target +directory, placeholder, or mount is created; it pins the regular source inode +with `O_NOFOLLOW`, creates distinct source/target bind mounts with read-only VFS +flags, and closes the pin before launch. The evaluated child is observed with all capability sets empty, `no_new_privs`, and an inherited seccomp filter. Passing DUMMY preflight now requires native mount, namespace, ptrace/process-memory denials, inherited-subprocess repetition and @@ -353,12 +357,22 @@ parsing, the mode-specific child semantic contract (for workers: exact output keys and nested evidence schemas, `result: "pass"`, Pi `0.85.1`, in-memory read-only session, unrefreshed OAuth, the complete canonical A1 outcome, and one exact-original read tied to independently pinned bytes and bounded event -indexes), and evidence assembly inside the cleanup +indexes, and `pi_install` provenance exactly equal to the selected installation +the supervisor resolves and verifies itself against the tracked integrity +pins), and evidence assembly inside the cleanup action, so valid JSON with an invalid contract plus a cleanup failure retains both the audit and cleanup causes. These compound cases (malformed output and -semantic audit each with cleanup, child-exit+cleanup, audit-only, cleanup-only, -success) have both injected supervisor tests and actual supervisor CLI -regressions checking sanitized serialized stderr and exact exits. The DUMMY +semantic audit each with cleanup, child-exit+cleanup, audit-only, forged +provenance, cleanup-only, success) have both injected supervisor tests and +actual supervisor CLI regressions checking sanitized serialized stderr and exact +exits. DUMMY CLI fixtures report a fixed literal `/DUMMY` provenance; the +supervisor accepts it only when its caller sets +`CP06_DUMMY_TRUSTED_PROVENANCE=1`, which no genuine installation can satisfy, +and the replay runner refuses to persist evidence whose install root is not the +task-local `.factory/state/cp06-sdk`. A DUMMY probe-fixture setup failure plus an +independent fixture-cleanup failure likewise serializes both causes; fixture +cleanup attempts every step, closes every retained descriptor, and removes the +fixture directory only through the retained output descriptor. The DUMMY CLI control now returns the complete canonical A1 contract; its semantic negative changes only the response to an invalid PASS outcome. The full-runner compound scenarios in `npm run proof:cp06:worker-cleanup` assert persisted diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 7091163..f3a004e 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -10,13 +10,13 @@ import { openSync, readFileSync, realpathSync, - rmSync, } from "node:fs"; import { tmpdir } from "node:os"; -import { join, resolve } from "node:path"; +import { basename, join, resolve } from "node:path"; import { resolvePinnedPiAi } from "./cp06-pi-dependency.mjs"; import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; -import { openProbeOutput } from "./cp06-probe-fixture.mjs"; +import { openProbeOutput, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { Cp06CleanupError } from "./cp06-worker-lifecycle.mjs"; import { DUMMY_SDK_ORIGINAL, DUMMY_OUTCOME_ORIGINAL } from "./cp06-dummy-originals.mjs"; export class Cp06IsolationUnsupportedError extends Error { @@ -460,19 +460,59 @@ export function runDummySdkProof({ spawnSyncImpl = spawnSync, expectedSdkDependency, }) { - const proofRoot = realpathSync(mkdtempSync(join(tmpdir(), "factory-cp06-DUMMY-sdk-"))); + const temporaryRoot = realpathSync(tmpdir()); + const directoryFlags = + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC; + const temporaryFd = openSync(temporaryRoot, directoryFlags); + const temporaryAnchor = `/proc/${process.pid}/fd/${temporaryFd}`; + let name; + let proofFd; + let result; + let primary; + let failed = false; try { - return runDummySdkProofIn({ + name = basename(mkdtempSync(join(temporaryAnchor, "factory-cp06-DUMMY-sdk-"))); + proofFd = openSync(join(temporaryAnchor, name), directoryFlags); + result = runDummySdkProofIn({ root, helper, piRoot, spawnSyncImpl, expectedSdkDependency, - proofRoot, + proofRoot: join(temporaryRoot, name), }); + } catch (error) { + failed = true; + primary = error; + } + try { + if (name !== undefined) removeRetainedDirectory(temporaryAnchor, name, proofFd); + } catch { + throw new Cp06CleanupError( + failed ? primary : undefined, + unsupported("DUMMY SDK proof failed: CP06_DUMMY_SDK_CLEANUP_FAILED"), + ); } finally { - rmSync(proofRoot, { recursive: true, force: true }); + if (proofFd !== undefined) closeSync(proofFd); + closeSync(temporaryFd); + } + if (failed) throw primary; + return result; +} + +function removeRetainedDirectory(parentAnchor, name, fd) { + let named; + try { + named = lstatSync(join(parentAnchor, name), { bigint: true }); + } catch (error) { + if (error?.code === "ENOENT") return; + throw error; + } + const pinned = fd === undefined ? undefined : fstatSync(fd, { bigint: true }); + if (pinned === undefined || named.dev !== pinned.dev || named.ino !== pinned.ino) { + throw new Error("DUMMY SDK proof root identity changed"); } + removeAnchoredEntry(parentAnchor, name); } function runDummySdkProofIn({ diff --git a/scripts/cp06-credential-isolation.mjs b/scripts/cp06-credential-isolation.mjs index c607d59..4fe2515 100644 --- a/scripts/cp06-credential-isolation.mjs +++ b/scripts/cp06-credential-isolation.mjs @@ -93,10 +93,9 @@ export async function mountLiveCredentialReadOnly(options) { const named = await statPath(sourceMountPath, { bigint: true }); assertPinnedRegularFile(pinned, named, maxBytes); - if (options.targetParentAnchor === undefined) { - await mkdir(dirname(target), { recursive: true, mode: 0o700 }); - } - targetParentHandle = await openParent(options.targetParentAnchor, dirname(target)); + targetParentHandle = await openParent(options.targetParentAnchor, dirname(target), { + create: true, + }); const targetParentPath = options.targetParentAnchor ?? `/proc/${process.pid}/fd/${targetParentHandle.fd}`; targetMountPath = join(targetParentPath, targetLeaf); @@ -200,14 +199,33 @@ export async function mountLiveCredentialReadOnly(options) { } } -async function openParent(anchor, path) { - return open( - anchor ?? path, - constants.O_RDONLY | - constants.O_DIRECTORY | - constants.O_CLOEXEC | - (anchor === undefined ? constants.O_NOFOLLOW : 0), - ); +async function openParent(anchor, path, { create = false } = {}) { + if (anchor !== undefined) { + return open(anchor, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_CLOEXEC); + } + const directoryFlags = + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC; + let handle = await open("/", directoryFlags); + try { + for (const component of resolve(path).split("/").filter(Boolean)) { + const child = join(`/proc/${process.pid}/fd/${handle.fd}`, component); + if (create) { + try { + await mkdir(child, { mode: 0o700 }); + } catch (error) { + if (error?.code !== "EEXIST") throw error; + } + } + const next = await open(child, directoryFlags); + await handle.close(); + handle = next; + } + const parent = handle; + handle = undefined; + return parent; + } finally { + await handle?.close(); + } } async function createPlaceholder(path) { diff --git a/scripts/cp06-dummy-cli-shims.mjs b/scripts/cp06-dummy-cli-shims.mjs index 6043638..fd9bb01 100644 --- a/scripts/cp06-dummy-cli-shims.mjs +++ b/scripts/cp06-dummy-cli-shims.mjs @@ -54,6 +54,11 @@ export const DUMMY_CLI_SCENARIOS = { cleanupFails: false, record: { schema_version: 1, ...AUDIT, causes: [] }, }, + "forged-provenance": { + child: "forged-provenance", + cleanupFails: false, + record: { schema_version: 1, ...AUDIT, causes: [] }, + }, "cleanup-only": { child: "valid", cleanupFails: true, record: compound([CLEANUP]) }, success: { child: "valid", cleanupFails: false, record: null }, }; @@ -73,6 +78,7 @@ export function writeDummyCliShims({ anchor, path, scenario, inheritedPath = pro semantic: `exec '${process.execPath}' '${dummyEvidence}' "$last" invalid`, exit: `printf '%s' '${DUMMY_CLI_SECRET}' >&2; exit 1`, valid: `exec '${process.execPath}' '${dummyEvidence}' "$last"`, + "forged-provenance": `exec '${process.execPath}' '${dummyEvidence}' "$last" forged-provenance`, }[selected.child]; writeExecutable( anchor, diff --git a/scripts/cp06-dummy-originals.mjs b/scripts/cp06-dummy-originals.mjs index c4e0586..84a6d25 100644 --- a/scripts/cp06-dummy-originals.mjs +++ b/scripts/cp06-dummy-originals.mjs @@ -1,2 +1,8 @@ export const DUMMY_SDK_ORIGINAL = "# DUMMY authority\n\nrequest-supplied owner fields are ignored; ownership comes only from the authenticated principal.\n"; export const DUMMY_OUTCOME_ORIGINAL = "# DUMMY authority\n\nOwnership comes only from the authenticated principal; request-supplied owner fields are ignored.\n"; +export const DUMMY_PI_PROVENANCE = Object.freeze({ + install_root: "/DUMMY/sdk", package_root: "/DUMMY/sdk/package", executable: "/DUMMY/sdk/pi", executable_entry: "/DUMMY/sdk/package/cli.js", sdk_entry: "/DUMMY/sdk/package/index.js", + package_sha256: "0".repeat(64), executable_sha256: "0".repeat(64), sdk_entry_sha256: "0".repeat(64), + package_artifact: Object.freeze({ manifest_sha256: "0".repeat(64), tarball_integrity: "sha512-DUMMY" }), + dependency: Object.freeze({ entry: "/DUMMY/sdk/ai/index.js", root: "/DUMMY/sdk/ai", entry_sha256: "0".repeat(64), manifest_sha256: "0".repeat(64), package_sha256: "0".repeat(64), name: "@earendil-works/pi-ai", version: "0.85.1", resolution: "esm-import-condition", tarball_integrity: "sha512-DUMMY" }), +}); diff --git a/scripts/cp06-dummy-worker-evidence.mjs b/scripts/cp06-dummy-worker-evidence.mjs index 5bdf019..32f227a 100644 --- a/scripts/cp06-dummy-worker-evidence.mjs +++ b/scripts/cp06-dummy-worker-evidence.mjs @@ -2,13 +2,14 @@ import { createHash } from "node:crypto"; import { resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { workerEvidenceContext } from "./cp06-worker-evidence.mjs"; -import { DUMMY_OUTCOME_ORIGINAL } from "./cp06-dummy-originals.mjs"; +import { DUMMY_OUTCOME_ORIGINAL, DUMMY_PI_PROVENANCE } from "./cp06-dummy-originals.mjs"; export function dummyWorkerContext(directory = "/DUMMY") { return { input: { provider: "openai-codex", model: "DUMMY-model", timeout_ms: 1_000 }, paths: { root: directory, fixtureRoot: directory, originalPath: `${directory}/docs/AUTH.md` }, original: Buffer.from(DUMMY_OUTCOME_ORIGINAL), + provenance: DUMMY_PI_PROVENANCE, }; } @@ -18,12 +19,7 @@ export function dummyWorkerEvidence({ input, paths, original } = dummyWorkerCont schema_version: 1, result: "pass", pi_version: "0.85.1", - pi_install: { - install_root: "/DUMMY/sdk", package_root: "/DUMMY/sdk/package", executable: "/DUMMY/sdk/pi", executable_entry: "/DUMMY/sdk/package/cli.js", sdk_entry: "/DUMMY/sdk/package/index.js", - package_sha256: "0".repeat(64), executable_sha256: "0".repeat(64), sdk_entry_sha256: "0".repeat(64), - package_artifact: { manifest_sha256: "0".repeat(64), tarball_integrity: "sha512-DUMMY" }, - dependency: { entry: "/DUMMY/sdk/ai/index.js", root: "/DUMMY/sdk/ai", entry_sha256: "0".repeat(64), manifest_sha256: "0".repeat(64), package_sha256: "0".repeat(64), name: "@earendil-works/pi-ai", version: "0.85.1", resolution: "esm-import-condition", tarball_integrity: "sha512-DUMMY" }, - }, + pi_install: structuredClone(DUMMY_PI_PROVENANCE), provider: input.provider, model: input.model, oauth_preflight: { minimum_validity_ms: input.timeout_ms + 300_000, remaining_validity_ms: input.timeout_ms + 600_000, refreshed: false }, @@ -53,6 +49,9 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur const context = workerEvidenceContext(process.argv[2]); const evidence = dummyWorkerEvidence(context); if (process.argv[3] === "invalid") evidence.response = { status: "PASS", evidence_gaps: [] }; + if (process.argv[3] === "forged-provenance") { + evidence.pi_install.package_artifact.tarball_integrity = "sha512-DUMMY-SECRET-CLI-MARKER"; + } console.log(JSON.stringify(evidence)); } catch { process.stderr.write("CP06_DUMMY_EVIDENCE_FAILED\n"); diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 3eb9ec4..5035df7 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -6,7 +6,11 @@ import { fileURLToPath } from "node:url"; import { mountLiveCredentialReadOnly } from "./cp06-credential-isolation.mjs"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; import { createProbeFixture } from "./cp06-probe-fixture.mjs"; -import { validateWorkerEvidence, workerEvidenceContext } from "./cp06-worker-evidence.mjs"; +import { + trustedWorkerProvenance, + validateWorkerEvidence, + workerEvidenceContext, +} from "./cp06-worker-evidence.mjs"; const root = resolve(fileURLToPath(new URL("..", import.meta.url))); @@ -44,7 +48,11 @@ export async function superviseCredentialChild(options, dependencies = {}) { outputRoot: requestedSource, }); } catch (cause) { - throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73, cause); + const setup = failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73, cause); + if (cause instanceof AggregateError) { + throw compoundFailure(setup, failure("cleanup", "CP06_CREDENTIAL_CLEANUP_FAILED", 74)); + } + throw setup; } } const source = probe?.source ?? requestedSource; @@ -93,8 +101,17 @@ export async function superviseCredentialChild(options, dependencies = {}) { } let workerContext = dependencies.workerContext; if (mode === "worker" && !dependencies.spawn) { - try { workerContext = workerEvidenceContext(childArgs[0], target, source); } - catch { throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); } + try { + workerContext = workerEvidenceContext(childArgs[0], target, source, { + provenance: trustedWorkerProvenance({ + root, + environment: childEnvironment(mode, source, target), + dummy: process.env.CP06_DUMMY_TRUSTED_PROVENANCE === "1", + }), + }); + } catch { + throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); + } } const command = childCommand({ mode, diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index 65908cc..41b1575 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -15,8 +15,8 @@ import { unlinkSync, writeSync, } from "node:fs"; -import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; -import { join, relative, resolve } from "node:path"; +import { mkdir, mkdtemp, writeFile } from "node:fs/promises"; +import { basename, join, relative, resolve } from "node:path"; const original = Buffer.from("DUMMY ORIGINAL"); const directoryFlags = constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW; @@ -180,7 +180,11 @@ function assertArtifactName(name) { } } -export function writeAnchoredFile(parentAnchor, name, data, { replace = false } = {}) { +export function writeAnchoredFile(parentAnchor, name, data, options) { + return writeAnchoredEntry(parentAnchor, name, data, options).sha256; +} + +export function writeAnchoredEntry(parentAnchor, name, data, { replace = false, expected } = {}) { assertArtifactName(name); const bytes = Buffer.from(data); const path = join(parentAnchor, name); @@ -198,7 +202,10 @@ export function writeAnchoredFile(parentAnchor, name, data, { replace = false } if (!created.isFile() || created.nlink !== 1n) { throw new Error("DUMMY proof artifact is not one fresh regular file"); } - if (!sameInode(created, lstatSync(path, { bigint: true }))) { + if ( + !sameInode(created, lstatSync(path, { bigint: true })) || + (replace && (expected === undefined || !sameInode(expected, created))) + ) { throw new Error("DUMMY proof artifact identity changed"); } if (replace) ftruncateSync(fd, 0); @@ -207,19 +214,27 @@ export function writeAnchoredFile(parentAnchor, name, data, { replace = false } if (!sameInode(created, lstatSync(path, { bigint: true }))) { throw new Error("DUMMY proof artifact identity changed"); } - return createHash("sha256").update(bytes).digest("hex"); + return { + sha256: createHash("sha256").update(bytes).digest("hex"), + identity: { dev: created.dev, ino: created.ino }, + }; } finally { closeSync(fd); } } -export function readAnchoredFile(parentAnchor, name) { +export function readAnchoredFile(parentAnchor, name, { expected } = {}) { assertArtifactName(name); const path = join(parentAnchor, name); const fd = openSync(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC); try { const pinned = fstatSync(fd, { bigint: true }); - if (!pinned.isFile() || pinned.nlink !== 1n || !sameInode(pinned, lstatSync(path, { bigint: true }))) { + if ( + !pinned.isFile() || + pinned.nlink !== 1n || + !sameInode(pinned, lstatSync(path, { bigint: true })) || + (expected !== undefined && !sameInode(expected, pinned)) + ) { throw new Error("DUMMY proof artifact identity changed"); } const bytes = readFileSync(fd); @@ -250,11 +265,15 @@ export function withArtifactParent(path, action) { } export function writeArtifactFile(path, data, options) { - return withArtifactParent(path, (anchor, name) => writeAnchoredFile(anchor, name, data, options)); + return writeArtifactEntry(path, data, options).sha256; } -export function readArtifactFile(path, encoding) { - const bytes = withArtifactParent(path, readAnchoredFile); +export function writeArtifactEntry(path, data, options) { + return withArtifactParent(path, (anchor, name) => writeAnchoredEntry(anchor, name, data, options)); +} + +export function readArtifactFile(path, encoding, options) { + const bytes = withArtifactParent(path, (anchor, name) => readAnchoredFile(anchor, name, options)); return encoding ? bytes.toString(encoding) : bytes; } @@ -339,34 +358,66 @@ export async function createProbeFixture({ root, outputRoot, spawn = spawnSync } } }, async cleanup() { - const alias = join(anchoredRoot, "DUMMY-alias"); - const mountedAlias = spawn("findmnt", ["--mountpoint", alias, "--noheadings"], { - encoding: "utf8", - timeout: 10_000, + const failures = releaseFixture({ + spawn, + output, + descriptors: [sourceFd, sourceDirFd, agentDirFd, rootFd], + directory, + mounted, + alias: join(anchoredRoot, "DUMMY-alias"), }); - if (mountedAlias.status === 0) checkedMount(spawn, ["--", alias], "umount"); - closeSync(sourceFd); - closeSync(sourceDirFd); - closeSync(agentDirFd); - closeSync(rootFd); - checkedMount(spawn, ["--", directory], "umount"); mounted = false; - await rm(directory, { recursive: true, force: true }); - output.close(); + if (failures.length > 0) { + throw new AggregateError(failures, "DUMMY fixture cleanup failed"); + } }, }; } catch (error) { - if (sourceFd !== undefined) closeSync(sourceFd); - if (sourceDirFd !== undefined) closeSync(sourceDirFd); - if (agentDirFd !== undefined) closeSync(agentDirFd); - if (rootFd !== undefined) closeSync(rootFd); - if (mounted) checkedMount(spawn, ["--", directory], "umount"); - if (directory !== undefined) await rm(directory, { recursive: true, force: true }); - output.close(); + const failures = releaseFixture({ + spawn, + output, + descriptors: [sourceFd, sourceDirFd, agentDirFd, rootFd], + directory, + mounted, + }); + if (failures.length > 0) { + throw new AggregateError([error, ...failures], "DUMMY fixture setup and cleanup failed"); + } throw error; } } +function releaseFixture({ spawn, output, descriptors, directory, mounted, alias }) { + const failures = []; + const attempt = (action) => { + try { + action(); + return true; + } catch (error) { + failures.push(error); + return false; + } + }; + if (alias !== undefined) { + attempt(() => { + const mountedAlias = spawn("findmnt", ["--mountpoint", alias, "--noheadings"], { + encoding: "utf8", + timeout: 10_000, + }); + if (mountedAlias.status === 0) checkedMount(spawn, ["--", alias], "umount"); + }); + } + for (const fd of descriptors) { + if (fd !== undefined) attempt(() => closeSync(fd)); + } + const unmounted = !mounted || attempt(() => checkedMount(spawn, ["--", directory], "umount")); + if (directory !== undefined && unmounted) { + attempt(() => removeAnchoredEntry(output.anchor, basename(directory))); + } + attempt(() => output.close()); + return failures; +} + function hashPinned(fd, length) { const buffer = Buffer.alloc(length); if (readSync(fd, buffer, 0, length, 0) !== length) return null; diff --git a/scripts/cp06-worker-evidence.mjs b/scripts/cp06-worker-evidence.mjs index f8cc704..c325d64 100644 --- a/scripts/cp06-worker-evidence.mjs +++ b/scripts/cp06-worker-evidence.mjs @@ -2,12 +2,26 @@ import { closeSync, constants, fstatSync, openSync, readFileSync } from "node:fs import { auditWorkerOutcome } from "./cp06-worker-outcome.mjs"; import { validateReadAudit } from "./cp06-auth-security.mjs"; import { canonicalWorkerPaths, readPinnedRegularFile } from "./cp06-guarded-read.mjs"; +import { DUMMY_PI_PROVENANCE } from "./cp06-dummy-originals.mjs"; +import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; -export function workerEvidenceContext(inputPath, target, source) { +export function workerEvidenceContext(inputPath, target, source, { provenance } = {}) { const input = readWorkerInput(inputPath); const paths = canonicalWorkerPaths(input, target, source); const snapshot = readPinnedRegularFile(paths.originalPath); - return { input, paths, original: snapshot.bytes, originalIdentity: snapshot.identity }; + return { input, paths, original: snapshot.bytes, originalIdentity: snapshot.identity, provenance }; +} + +// The supervisor resolves the selected installation itself; the DUMMY +// expectation is a fixed literal that no genuine installation can report. +export function trustedWorkerProvenance({ root, environment, dummy = false }) { + if (dummy) return DUMMY_PI_PROVENANCE; + return resolvePinnedPiInstall({ + projectRoot: root, + packageRoot: environment.CP06_PI_PACKAGE_ROOT, + executable: environment.CP06_PI_BIN, + environment, + }).provenance; } export function readWorkerInput(path) { @@ -22,7 +36,7 @@ export function readWorkerInput(path) { return JSON.parse(readPinnedRegularFile(path).bytes.toString("utf8")); } -export function validateWorkerEvidence(child, { input, paths, original, originalIdentity }) { +export function validateWorkerEvidence(child, { input, paths, original, originalIdentity, provenance }) { if (originalIdentity) { const current = readPinnedRegularFile(paths.originalPath); for (const key of ["dev", "ino", "size", "nlink", "mode", "mtimeNs", "ctimeNs"]) requireValue(current.identity[key] === originalIdentity[key]); @@ -42,6 +56,7 @@ export function validateWorkerEvidence(child, { input, paths, original, original for (const key of ["entry_sha256", "manifest_sha256", "package_sha256"]) requireValue(digest(dependency[key])); for (const key of ["root", "entry"]) requireValue(typeof dependency[key] === "string" && dependency[key].startsWith("/")); requireValue(typeof dependency.tarball_integrity === "string" && dependency.tarball_integrity.startsWith("sha512-")); + requireValue(provenance !== undefined && sameJson(child.pi_install, provenance)); exact(child.oauth_preflight, ["minimum_validity_ms", "remaining_validity_ms", "refreshed"]); requireValue(child.oauth_preflight.refreshed === false && child.oauth_preflight.minimum_validity_ms === input.timeout_ms + 300_000 && Number.isSafeInteger(child.oauth_preflight.remaining_validity_ms) && child.oauth_preflight.remaining_validity_ms >= child.oauth_preflight.minimum_validity_ms); exact(child.credential_store, ["reads", "lists", "modify_denials", "delete_denials"]); @@ -65,6 +80,12 @@ export function validateWorkerEvidence(child, { input, paths, original, original return child; } +function sameJson(actual, expected) { + if (expected === null || typeof expected !== "object") return actual === expected; + if (actual === null || typeof actual !== "object" || Array.isArray(actual) !== Array.isArray(expected)) return false; + const keys = Object.keys(expected).sort(); + return Object.keys(actual).sort().join(",") === keys.join(",") && keys.every((key) => sameJson(actual[key], expected[key])); +} function exact(value, keys) { requireValue(value !== null && typeof value === "object" && !Array.isArray(value) && Object.keys(value).sort().join(",") === keys.sort().join(",")); } diff --git a/scripts/prove-cp06-worker-cleanup.mjs b/scripts/prove-cp06-worker-cleanup.mjs index c6744ba..c6f6ac2 100644 --- a/scripts/prove-cp06-worker-cleanup.mjs +++ b/scripts/prove-cp06-worker-cleanup.mjs @@ -121,6 +121,7 @@ const cases = await withCleanup( CP06_PI_BIN: process.env.CP06_PI_BIN, CP06_OUTPUT: proofOutputPath, CP06_REVIEWER: "DUMMY cleanup regression", + ...(expectedRecord === undefined ? {} : { CP06_DUMMY_TRUSTED_PROVENANCE: "1" }), }, }); assert.equal(result.error, undefined); diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index d025927..3d4c926 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -5,7 +5,7 @@ import { stat } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -import { openAnchoredDirectory, openProbeOutput, writeArtifactFile, readArtifactFile, withArtifactParent, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { openAnchoredDirectory, openProbeOutput, writeArtifactEntry, readArtifactFile, withArtifactParent, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; import { readPinnedRegularFile } from "./cp06-guarded-read.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); @@ -23,6 +23,7 @@ const ctx = resolveTool(process.env.CTX_BIN ?? "ctx"); configureOfflineCtxModelDir(); const reviewer = process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker"; const commands = []; +const preparedArtifacts = new Map(); copyFixtureTree(fixtureSource, fixtureRoot); openAnchoredDirectory(fixtureRoot, ".factory/state").close(); @@ -327,15 +328,15 @@ for (const task of oracle.tasks) { } const packCopy = join(rawRoot, `${task.id}.pack.md`); const receiptCopy = join(rawRoot, `${task.id}.receipt.json`); - writeArtifactFile(packCopy, readArtifactFile(packAbsolute)); - writeArtifactFile(receiptCopy, readArtifactFile(receiptAbsolute)); + const packSha256 = await writeFile(packCopy, await readFile(packAbsolute)); + const receiptSha256 = await writeFile(receiptCopy, await readFile(receiptAbsolute)); packs.push({ task_id: task.id, risk: task.risk, contract: fileEvidence(join(fixtureRoot, `.factory/tasks/${task.id}.yaml`)), target_source: fileEvidence(join(fixtureRoot, task.target)), - pack: await fileEvidenceAsync(packCopy), - receipt: await fileEvidenceAsync(receiptCopy), + pack: { path: relative(packCopy), sha256: packSha256 }, + receipt: { path: relative(receiptCopy), sha256: receiptSha256 }, token_budget: payload.token_budget, estimated_tokens: payload.estimated_tokens, byte_budget: payload.byte_budget, @@ -486,14 +487,14 @@ const manifest = { ], }; const manifestPath = join(proofRoot, "evidence.json"); -await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); +const manifestSha256 = await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); console.log( JSON.stringify({ ok: true, gate: "P-06", tested_sha: testedSha, manifest_path: relative(manifestPath), - manifest_sha256: await fileHash(manifestPath), + manifest_sha256: manifestSha256, fixed_pack_count: packs.length, relevant_pack_count: manifest.fixed_sample.relevant_pack_count, irrelevant_topical_excerpts: manifest.fixed_sample.irrelevant_topical_excerpts, @@ -649,7 +650,7 @@ function assertSemantic(status, label) { } async function copyArtifact(path, name) { - writeArtifactFile(join(rawRoot, name), readArtifactFile(join(fixtureRoot, path))); + await writeFile(join(rawRoot, name), await readFile(join(fixtureRoot, path))); } function fileEvidence(path) { @@ -658,10 +659,6 @@ function fileEvidence(path) { return { path: relativePath, sha256: sha256(bytes) }; } -async function fileEvidenceAsync(path) { - return { path: relative(path), sha256: await fileHash(path) }; -} - function gitText(cwd, args) { return commandText("git", args, cwd); } @@ -705,11 +702,15 @@ async function fileHash(path) { } function requireRead(path) { - return path.startsWith(`/proc/${process.pid}/fd/`) ? readArtifactFile(path) : readPinnedRegularFile(path).bytes; + return path.startsWith(`/proc/${process.pid}/fd/`) + ? readArtifactFile(path, undefined, { expected: preparedArtifacts.get(path) }) + : readPinnedRegularFile(path).bytes; } -function requireWrite(path, value) { - writeArtifactFile(path, value); +function requireWrite(path, value, options) { + const written = writeArtifactEntry(path, value, { ...options, expected: preparedArtifacts.get(path) }); + preparedArtifacts.set(path, written.identity); + return written.sha256; } async function readFile(path, encoding) { @@ -718,21 +719,23 @@ async function readFile(path, encoding) { } async function writeFile(path, value) { - return writeArtifactFile(path, value, { replace: path === join(fixtureRoot, ".factory/tasks/AUTH-001.yaml") }); + return requireWrite(path, value, { replace: path === join(fixtureRoot, ".factory/tasks/AUTH-001.yaml") }); } async function rm(path) { withArtifactParent(path, removeAnchoredEntry); + preparedArtifacts.delete(path); } -function copyFixtureTree(source, destination) { +function copyFixtureTree(source, destination, logical = destination) { for (const entry of readdirSync(source, { withFileTypes: true })) { if (entry.isDirectory()) { const child = openAnchoredDirectory(destination, entry.name); - try { copyFixtureTree(join(source, entry.name), child.anchor); } + try { copyFixtureTree(join(source, entry.name), child.anchor, join(logical, entry.name)); } finally { child.close(); } } else if (entry.isFile()) { - writeArtifactFile(join(destination, entry.name), readPinnedRegularFile(join(source, entry.name)).bytes); + const written = writeArtifactEntry(join(destination, entry.name), readPinnedRegularFile(join(source, entry.name)).bytes); + preparedArtifacts.set(join(logical, entry.name), written.identity); } else throw new Error("DUMMY fixture contains a nonregular entry"); } } diff --git a/scripts/replay-cp06-worker.mjs b/scripts/replay-cp06-worker.mjs index a3af530..027c9c6 100644 --- a/scripts/replay-cp06-worker.mjs +++ b/scripts/replay-cp06-worker.mjs @@ -136,7 +136,8 @@ async function runWorker(workerRoot) { isolated.mount_ids_distinct !== true || isolated.parent_roots_read_only !== true || isolated.cleanup !== "pass" || - isolated.child?.result !== "pass" + isolated.child?.result !== "pass" || + isolated.child.pi_install?.install_root !== join(realpathSync(root), ".factory/state/cp06-sdk") ) { const error = new Error("isolated SDK worker omitted a required safety assertion"); await recordBlocked(workerRoot, error, "isolated-sdk-worker-audit"); diff --git a/test/cp06-credential-isolation.test.mjs b/test/cp06-credential-isolation.test.mjs index 3e53b30..3718666 100644 --- a/test/cp06-credential-isolation.test.mjs +++ b/test/cp06-credential-isolation.test.mjs @@ -69,6 +69,25 @@ test("leaf or parent substitution between preparation and mount is refused", asy }); }); +test("symlinked target or source ancestors are refused before external mutation or mount", async () => { + await withCompiledHelper("mount-ancestor", async ({ helperFd }) => { + for (const scenario of ["target-ancestor-symlink", "source-ancestor-symlink"]) { + assert.deepEqual(runMountScenario(helperFd, scenario), { + scenario, + mounted: false, + error: "no-follow-refused", + external_entries: + scenario === "source-ancestor-symlink" + ? ["DUMMY-auth.json", "source"] + : ["DUMMY-auth.json"], + marker_unchanged: true, + source_unchanged: true, + remaining_mounts: 0, + }); + } + }); +}); + test("mount setup failure reports cleanup failure instead of discarding it", async () => { const root = await mkdtemp(join(tmpdir(), "factory-cp06-cleanup-test-")); const source = join(root, "auth-source.json"); diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 9ba8797..6d27f03 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -2,7 +2,7 @@ import { strict as assert } from "node:assert"; import { createHash } from "node:crypto"; import { DUMMY_SDK_ORIGINAL, DUMMY_OUTCOME_ORIGINAL } from "../scripts/cp06-dummy-originals.mjs"; import { spawnSync } from "node:child_process"; -import { readlinkSync, renameSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { mkdirSync, readlinkSync, renameSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { mkdir, mkdtemp, @@ -142,6 +142,59 @@ test("failed DUMMY probes never forward child output or spawn exceptions", async } }); +test("DUMMY SDK proof outer cleanup removes child leftovers only through the retained root", async () => { + const external = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-sdk-external-")); + const sentinel = join(external, "DUMMY-sentinel"); + await writeFile(sentinel, "DUMMY SENTINEL UNCHANGED"); + const leftovers = []; + const options = { root: process.cwd(), helper: process.execPath, piRoot: "/DUMMY-pi" }; + try { + assert.throws( + () => + runDummySdkProof({ + ...options, + spawnSyncImpl(command, args) { + const proofRoot = args.at(-1); + leftovers.push(proofRoot); + mkdirSync(join(proofRoot, "DUMMY-worker", "nested"), { recursive: true }); + writeFileSync(join(proofRoot, "DUMMY-worker", "nested", "DUMMY-partial"), "DUMMY"); + symlinkSync(external, join(proofRoot, "DUMMY-worker", "DUMMY-escape")); + return { status: null, error: Object.assign(new Error("DUMMY"), { code: "ETIMEDOUT" }) }; + }, + }), + (error) => + error.code === "CP06_ISOLATION_UNSUPPORTED" && /CP06_DUMMY_SDK_TIMEOUT/u.test(error.message), + ); + await assert.rejects(readdir(leftovers[0]), { code: "ENOENT" }); + assert.equal(await readFile(sentinel, "utf8"), "DUMMY SENTINEL UNCHANGED"); + + assert.throws( + () => + runDummySdkProof({ + ...options, + spawnSyncImpl(command, args) { + const proofRoot = args.at(-1); + leftovers.push(proofRoot, `${proofRoot}-DUMMY-moved`); + renameSync(proofRoot, `${proofRoot}-DUMMY-moved`); + mkdirSync(proofRoot); + writeFileSync(join(proofRoot, "DUMMY-substitute"), "DUMMY SUBSTITUTE UNCHANGED"); + return { status: 1, stdout: "", stderr: "" }; + }, + }), + (error) => + error.code === "CP06_CLEANUP_FAILED" && + /CP06_DUMMY_SDK_CHILD_EXIT/u.test(error.errors[0].message) && + /CP06_DUMMY_SDK_CLEANUP_FAILED/u.test(error.errors[1].message), + ); + assert.equal( + await readFile(join(leftovers[1], "DUMMY-substitute"), "utf8"), + "DUMMY SUBSTITUTE UNCHANGED", + ); + } finally { + for (const path of [external, ...leftovers]) await rm(path, { recursive: true, force: true }); + } +}); + test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { const trustedDependency = sdkDependency(); const validBase = { diff --git a/test/cp06-review-boundaries.test.mjs b/test/cp06-review-boundaries.test.mjs index 137f599..7039487 100644 --- a/test/cp06-review-boundaries.test.mjs +++ b/test/cp06-review-boundaries.test.mjs @@ -1,11 +1,11 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; -import { closeSync, constants, openSync } from "node:fs"; -import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { chmodSync, closeSync, constants, openSync, readdirSync, writeFileSync } from "node:fs"; +import { mkdir, mkdtemp, readdir, readFile, rename, rm, symlink, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import test from "node:test"; import { assertHardenedProof, assertUnsafeControl, runIsolationProbe } from "../scripts/cp06-auth-security.mjs"; -import { openProbeOutput, openAnchoredDirectory, readArtifactFile, writeArtifactFile } from "../scripts/cp06-probe-fixture.mjs"; +import { createProbeFixture, openProbeOutput, openAnchoredDirectory, readArtifactFile, writeArtifactEntry, writeArtifactFile } from "../scripts/cp06-probe-fixture.mjs"; import { superviseCredentialChild, supervisorFailureRecord } from "../scripts/cp06-namespace-supervisor.mjs"; import { dummyWorkerContext, dummyWorkerEvidence } from "../scripts/cp06-dummy-worker-evidence.mjs"; import { withCompiledHelper } from "./fixtures/cp06-compiled-helper.mjs"; @@ -46,6 +46,11 @@ test("full semantic contracts reject forged nested evidence inside independently { ...good, event_stream: { ...good.event_stream, event_count: 2 } }, { ...good, event_stream: { ...good.event_stream, read_audit: { ...good.event_stream.read_audit, expected_sha256: "a".repeat(64), returned_sha256: "a".repeat(64), returned_bytes: 1 } } }, { ...good, pi_install: { ...good.pi_install, extra: "DUMMY-SECRET" } }, + { ...good, pi_install: { ...good.pi_install, package_root: "/usr/lib/node_modules/@earendil-works/pi-coding-agent" } }, + { ...good, pi_install: { ...good.pi_install, sdk_entry_sha256: "a".repeat(64) } }, + { ...good, pi_install: { ...good.pi_install, package_artifact: { ...good.pi_install.package_artifact, tarball_integrity: "sha512-DUMMY-SECRET" } } }, + { ...good, pi_install: { ...good.pi_install, dependency: { ...good.pi_install.dependency, root: "/usr/lib/node_modules/@earendil-works/pi-ai" } } }, + { ...good, pi_install: { ...good.pi_install, dependency: { ...good.pi_install.dependency, tarball_integrity: "sha512-DUMMY-SECRET" } } }, ]; for (const child of [good, ...variants]) { for (const cleanupFails of [false, true]) { @@ -73,6 +78,133 @@ test("full semantic contracts reject forged nested evidence inside independently } }); +test("worker provenance without an independently trusted expectation is rejected", async () => { + const { provenance, ...untrusted } = dummyWorkerContext(); + assert.ok(provenance); + let cleaned = 0; + await assert.rejects( + superviseCredentialChild({ mode: "worker", source: "/DUMMY-source", target: "/DUMMY-target", helper: "/DUMMY-helper", syscallProbe: "/DUMMY-probe" }, { + workerContext: untrusted, + mount: async () => ({ mountIds: { source: "1", target: "2" }, parentMounts: ["DUMMY"], verifyIntegrity: async () => true, cleanup: async () => { cleaned++; } }), + spawn: () => ({ status: 0, stdout: JSON.stringify(dummyWorkerEvidence(dummyWorkerContext())) }), + }), + { code: "CP06_CHILD_OUTPUT_INVALID" }, + ); + assert.equal(cleaned, 1); +}); + +test("artifact replacement and reads are bound to the prepared inode", async () => { + const outputRoot = join(state, `DUMMY-review-identity-${process.pid}`); + const output = openProbeOutput({ root: process.cwd(), outputRoot, create: true, runner: true }); + const child = openAnchoredDirectory(output.anchor, "raw"); + try { + const path = join(child.anchor, "DUMMY-task.yaml"); + const prepared = writeArtifactEntry(path, "DUMMY prepared"); + assert.equal(readArtifactFile(path, "utf8", { expected: prepared.identity }), "DUMMY prepared"); + assert.equal(writeArtifactEntry(path, "DUMMY replaced", { replace: true, expected: prepared.identity }).identity.ino, prepared.identity.ino); + assert.throws(() => writeArtifactFile(path, "DUMMY unbound", { replace: true }), /identity changed/); + await writeFile(join(outputRoot, "raw", "DUMMY-substitute"), "DUMMY SUBSTITUTE UNCHANGED"); + await rename(join(outputRoot, "raw", "DUMMY-substitute"), join(outputRoot, "raw", "DUMMY-task.yaml")); + assert.throws(() => writeArtifactFile(path, "DUMMY write", { replace: true, expected: prepared.identity }), /identity changed/); + assert.throws(() => readArtifactFile(path, "utf8", { expected: prepared.identity }), /identity changed/); + assert.equal(await readFile(join(outputRoot, "raw", "DUMMY-task.yaml"), "utf8"), "DUMMY SUBSTITUTE UNCHANGED"); + } finally { + child.close(); output.close(); await rm(outputRoot, { recursive: true, force: true }); + } +}); + +test("ten-pack refuses a prepared contract replaced by another regular inode", async () => { + await mkdir(state, { recursive: true }); + const directoryRoot = await mkdtemp(join(state, "DUMMY-runner-")); + const contract = join(directoryRoot, "ten-pack/repo/.factory/tasks/AUTH-001.yaml"); + const hook = join(directoryRoot, "DUMMY-hook.mjs"); + await writeFile(hook, `import fs from 'node:fs'; import {syncBuiltinESMExports} from 'node:module'; const real = fs.openSync; let done = false; fs.openSync = (path, ...rest) => { if (!done && String(path).endsWith('/AUTH-before.yaml')) { done = true; fs.writeFileSync(${JSON.stringify(`${contract}.DUMMY`)}, 'DUMMY SUBSTITUTE UNCHANGED'); fs.renameSync(${JSON.stringify(`${contract}.DUMMY`)}, ${JSON.stringify(contract)}); } return real(path, ...rest); }; syncBuiltinESMExports();`); + try { + const result = spawnSync(process.execPath, ["--import", hook, resolve("scripts/replay-cp06-ten-pack.mjs")], { env: { ...process.env, CP06_OUTPUT: directoryRoot }, encoding: "utf8", timeout: 30_000 }); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /DUMMY proof artifact identity changed/); + assert.equal(await readFile(contract, "utf8"), "DUMMY SUBSTITUTE UNCHANGED"); + } finally { await rm(directoryRoot, { recursive: true, force: true }); } +}); + +test("DUMMY fixture cleanup closes every descriptor and removes through its anchor despite failures", async () => { + const outputRoot = resolve(state, `DUMMY-fixture-${process.pid}`, "auth-security"); + const openDescriptors = () => readdirSync("/proc/self/fd").length; + try { + for (const scenario of ["alias-umount-fails", "setup-and-umount-fail", "setup-fails"]) { + const before = openDescriptors(); + const umounts = []; + const spawn = (command, args) => { + const path = args.at(-1); + if (command === "mount") { + if (scenario !== "alias-umount-fails") writeFileSync(join(path, "source"), "DUMMY"); + } else if (command === "umount") { + umounts.push(path.split("/").at(-1)); + if (scenario === "alias-umount-fails" ? path.endsWith("/DUMMY-alias") : scenario === "setup-and-umount-fail") return { status: 32 }; + } + return { status: 0 }; + }; + let caught; + try { + const fixture = await createProbeFixture({ root: process.cwd(), outputRoot, spawn }); + await fixture.cleanup(); + } catch (error) { caught = error; } + assert.equal(openDescriptors(), before, scenario); + assert.deepEqual(umounts.map((name) => name.replace(/^DUMMY-probe-.*/u, "DUMMY-probe")), scenario === "alias-umount-fails" ? ["DUMMY-alias", "DUMMY-probe"] : ["DUMMY-probe"]); + const remaining = (await readdir(outputRoot)).filter((name) => name.startsWith("DUMMY-probe-")); + if (scenario === "alias-umount-fails") { + assert.ok(caught instanceof AggregateError); + assert.deepEqual(caught.errors.map((error) => error.message), ["private DUMMY fixture mount unavailable"]); + assert.deepEqual(remaining, []); + } else if (scenario === "setup-and-umount-fail") { + assert.ok(caught instanceof AggregateError); + assert.deepEqual(caught.errors.map((error) => error.code ?? error.message), ["EEXIST", "private DUMMY fixture mount unavailable"]); + assert.equal(remaining.length, 1); + await rm(join(outputRoot, remaining[0]), { recursive: true }); + } else { + assert.equal(caught?.code, "EEXIST"); + assert.deepEqual(remaining, []); + } + } + } finally { await rm(resolve(state, `DUMMY-fixture-${process.pid}`), { recursive: true, force: true }); } +}); + +test("supervisor CLI serializes fixture setup and independent fixture cleanup causes", async () => { + const locate = (command) => spawnSync("sh", ["-c", `command -v ${command}`], { encoding: "utf8" }).stdout.trim(); + const [realMount, realUmount] = [locate("mount"), locate("umount")]; + const secret = "DUMMY-SECRET-FIXTURE-MARKER"; + const base = resolve(state, `DUMMY-fixture-cli-${process.pid}`); + const shims = join(base, "shims"); + const outputRoot = join(base, "auth-security"); + const setup = { stage: "setup", code: "CP06_ISOLATION_SETUP_FAILED", exit_code: 73, description: "credential namespace setup failed" }; + const cleanup = { stage: "cleanup", code: "CP06_CREDENTIAL_CLEANUP_FAILED", exit_code: 74, description: "credential mount cleanup failed" }; + try { + for (const umountFails of [true, false]) { + await rm(base, { recursive: true, force: true }); + await mkdir(shims, { recursive: true }); + for (const [name, body] of [ + ["mount", `"${realMount}" "$@" || exit $?\ncase "$last" in */DUMMY-probe-*) : > "$last/source" ;; esac`], + ["umount", `case "$last" in */DUMMY-probe-*) ${umountFails ? `printf '%s' '${secret}' >&2; exit 32` : ":"} ;; esac\nexec "${realUmount}" "$@"`], + ]) { + writeFileSync(join(shims, name), `#!/bin/sh\nfor argument do last="$argument"; done\n${body}\n`); + chmodSync(join(shims, name), 0o700); + } + const result = spawnSync("unshare", ["--user", "--map-root-user", "--mount", "--propagation", "private", "--", process.execPath, resolve("scripts/cp06-namespace-supervisor.mjs"), "hardened-probe", outputRoot, "-", "/DUMMY-helper", "/DUMMY-probe"], { + encoding: "utf8", timeout: 30_000, env: { ...process.env, PATH: `${shims}:${process.env.PATH}` }, + }); + const record = umountFails + ? { schema_version: 1, stage: "cleanup", code: "CP06_CLEANUP_FAILED", exit_code: 74, description: "credential namespace cleanup failed", causes: [setup, cleanup] } + : { schema_version: 1, ...setup, causes: [] }; + assert.equal(result.status, record.exit_code, result.stderr); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, `${JSON.stringify(record)}\n`); + assert.equal(`${result.stdout}${result.stderr}`.includes(secret), false); + const remaining = (await readdir(outputRoot)).filter((name) => name.startsWith("DUMMY-probe-")); + assert.equal(remaining.length, umountFails ? 1 : 0); + } + } finally { await rm(base, { recursive: true, force: true }); } +}); + test("artifact writes, replacements, copies and reads reject redirected leaves and intermediate directories", async () => { const outputRoot = join(state, `DUMMY-review-artifacts-${process.pid}`); const output = openProbeOutput({ root: process.cwd(), outputRoot, create: true, runner: true }); diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 413dd7d..1e2757c 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -487,6 +487,7 @@ test("supervisor executable serializes compound worker failures as typed causes" "semantic-audit-and-cleanup": compound([audit, cleanup]), "child-exit-and-cleanup": compound([childExit, cleanup]), "audit-only": { schema_version: 1, ...audit, causes: [] }, + "forged-provenance": { schema_version: 1, ...audit, causes: [] }, "cleanup-only": compound([cleanup]), success: null, }; @@ -537,6 +538,7 @@ test("supervisor executable serializes compound worker failures as typed causes" stdio: ["pipe", "pipe", "pipe", helperFd, probeFd], env: { ...process.env, + CP06_DUMMY_TRUSTED_PROVENANCE: "1", PATH: writeDummyCliShims({ anchor: shims, path: shims, scenario }), }, }, diff --git a/test/fixtures/cp06-credential-mount.mjs b/test/fixtures/cp06-credential-mount.mjs index f010917..9316e2e 100644 --- a/test/fixtures/cp06-credential-mount.mjs +++ b/test/fixtures/cp06-credential-mount.mjs @@ -6,6 +6,7 @@ import { lstat, mkdir, mkdtemp, + readdir, readFile, rename, rm, @@ -21,15 +22,28 @@ const [scenario] = process.argv.slice(2); const marker = "DUMMY-EXTERNAL-MARKER"; const credential = '{"fixture":"DUMMY","token":"DUMMY-NOT-REAL"}'; const directory = realpathSync(await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-mount-"))); -const source = join(directory, "source", "DUMMY-auth.json"); -const target = join(directory, "agent", "DUMMY-auth.json"); +const realSource = join(directory, "source", "DUMMY-auth.json"); +const source = + scenario === "source-ancestor-symlink" + ? join(directory, "linked", "source", "DUMMY-auth.json") + : realSource; +const target = + scenario === "target-ancestor-symlink" + ? join(directory, "home", "pi-agent", "DUMMY-auth.json") + : join(directory, "agent", "DUMMY-auth.json"); const external = join(directory, "external"); const externalLeaf = join(external, "DUMMY-auth.json"); -await mkdir(dirname(source)); -await mkdir(dirname(target)); +await mkdir(dirname(realSource)); +await mkdir(join(directory, "agent")); await mkdir(external); -await writeFile(source, credential); +await writeFile(realSource, credential); await writeFile(externalLeaf, marker); +if (scenario === "target-ancestor-symlink") await symlink(external, join(directory, "home")); +if (scenario === "source-ancestor-symlink") { + await mkdir(join(external, "source")); + await writeFile(join(external, "source", "DUMMY-auth.json"), marker); + await symlink(external, join(directory, "linked")); +} const markerReads = []; async function substitute(leafPath) { @@ -46,7 +60,9 @@ async function substitute(leafPath) { const parent = dirname(scenario === "target-parent-replaced" ? target : source); await rename(parent, `${parent}-moved`); await symlink(external, parent); - } else if (scenario !== "control") { + } else if ( + !["control", "target-ancestor-symlink", "source-ancestor-symlink"].includes(scenario) + ) { throw new Error("unsupported DUMMY scenario"); } } @@ -87,8 +103,14 @@ try { result.target_after_cleanup = await readFile(target, "utf8"); } catch (error) { result.mounted = mount !== undefined; - result.error = error instanceof AggregateError ? "aggregate" : error.message; + result.error = + error instanceof AggregateError + ? "aggregate" + : ["ELOOP", "ENOTDIR"].includes(error.code) + ? "no-follow-refused" + : error.message; } +if (scenario.endsWith("-ancestor-symlink")) result.external_entries = (await readdir(external)).sort(); markerReads.push(await readFile(externalLeaf, "utf8")); result.marker_unchanged = markerReads.every((value) => value === marker); const originalSource = @@ -96,7 +118,7 @@ const originalSource = ? join(directory, "DUMMY-source-moved") : scenario === "source-parent-replaced" ? join(directory, "source-moved", "DUMMY-auth.json") - : source; + : realSource; result.source_unchanged = (await readFile(originalSource, "utf8")) === credential; const listed = spawnSync("findmnt", ["--json", "--list", "--output", "TARGET"], { encoding: "utf8", From 00321e0936f29163c918885b6a0dfe32511f84b7 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 6 Oct 2026 05:49:56 +0000 Subject: [PATCH 41/45] no-mistakes(review): Resolve CP06 supervisor provenance safely without executing Pi --- docs/probes/CP-06.md | 22 ++++- scripts/cp06-auth-security.mjs | 1 + scripts/cp06-dummy-cli-shims.mjs | 13 ++- scripts/cp06-dummy-worker-evidence.mjs | 8 ++ scripts/cp06-namespace-supervisor.mjs | 20 +++-- scripts/cp06-pi-install.mjs | 42 +++++---- scripts/cp06-worker-evidence.mjs | 11 ++- test/cp06-pi-dependency.test.mjs | 42 +++++++++ test/cp06-review-boundaries.test.mjs | 2 +- test/cp06-worker-lifecycle.test.mjs | 119 ++++++++++++++++++++++++- 10 files changed, 245 insertions(+), 35 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 5c21529..652d3c3 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -270,7 +270,14 @@ after the complete DUMMY auth and OS preflight passes. A supervisor walks the source and target parent directories from `/` with no-follow directory descriptors (creating missing target components relative to the retained parent), so a symlink substituted for any ancestor is rejected before a target -directory, placeholder, or mount is created; it pins the regular source inode +directory, placeholder, or mount is created. Because every source ancestor is +opened `O_RDONLY | O_DIRECTORY | O_NOFOLLOW` without canonicalization, a live +credential path (`CP06_PI_AUTH_FILE`, `PI_CODING_AGENT_DIR` or `~/.pi/agent`) +with a symlinked ancestor (for example a dotfile-managed `~/.pi` link) or a +search-only (`--x`) ancestor is unsupported: setup fails closed with +`CP06_ISOLATION_SETUP_FAILED` before any credential byte access, mount or +mutation. Genuine acceptance requires a credential path whose ancestors are real, +readable directories. It pins the regular source inode with `O_NOFOLLOW`, creates distinct source/target bind mounts with read-only VFS flags, and closes the pin before launch. The evaluated child is observed with all capability sets empty, `no_new_privs`, and an @@ -359,7 +366,12 @@ read-only session, unrefreshed OAuth, the complete canonical A1 outcome, and one exact-original read tied to independently pinned bytes and bounded event indexes, and `pi_install` provenance exactly equal to the selected installation the supervisor resolves and verifies itself against the tracked integrity -pins), and evidence assembly inside the cleanup +pins before mounting the credential, from selected paths, package metadata and +file manifests only, without executing Pi in its privileged namespace; the +supervisor is launched with `--experimental-import-meta-resolve` so the pi-ai +dependency resolves from the selected package, and a resolution failure is a +`setup`/`CP06_ISOLATION_SETUP_FAILED` exit 73 rather than an audit failure; +the `pi --version` check remains in the hardened child), and evidence assembly inside the cleanup action, so valid JSON with an invalid contract plus a cleanup failure retains both the audit and cleanup causes. These compound cases (malformed output and semantic audit each with cleanup, child-exit+cleanup, audit-only, forged @@ -369,7 +381,11 @@ exits. DUMMY CLI fixtures report a fixed literal `/DUMMY` provenance; the supervisor accepts it only when its caller sets `CP06_DUMMY_TRUSTED_PROVENANCE=1`, which no genuine installation can satisfy, and the replay runner refuses to persist evidence whose install root is not the -task-local `.factory/state/cp06-sdk`. A DUMMY probe-fixture setup failure plus an +task-local `.factory/state/cp06-sdk`. Without that flag, executable supervisor +regressions launched exactly as the runner launches it reject a missing selected +installation as a pre-mount setup failure, reject the literal `/DUMMY` provenance +as an audit failure, and (when the task-local SDK is installed) accept a DUMMY +child reporting the selected installation's provenance. A DUMMY probe-fixture setup failure plus an independent fixture-cleanup failure likewise serializes both causes; fixture cleanup attempts every step, closes every retained descriptor, and removes the fixture directory only through the retained output descriptor. The DUMMY diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index f3a004e..07641ab 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -273,6 +273,7 @@ function runNamespace(options) { "private", "--", process.execPath, + "--experimental-import-meta-resolve", join(options.root, "scripts/cp06-namespace-supervisor.mjs"), options.mode, resolve(options.source), diff --git a/scripts/cp06-dummy-cli-shims.mjs b/scripts/cp06-dummy-cli-shims.mjs index fd9bb01..b7c5ec7 100644 --- a/scripts/cp06-dummy-cli-shims.mjs +++ b/scripts/cp06-dummy-cli-shims.mjs @@ -63,13 +63,23 @@ export const DUMMY_CLI_SCENARIOS = { success: { child: "valid", cleanupFails: false, record: null }, }; +/** Scenarios run without the DUMMY trust flag against the task-local installation. */ +export const DUMMY_CLI_INSTALLED_SCENARIOS = { + "installed-provenance": { child: "installed", cleanupFails: false, record: null }, + "dummy-provenance-untrusted": { + child: "valid", + cleanupFails: false, + record: { schema_version: 1, ...AUDIT, causes: [] }, + }, +}; + export const DUMMY_CLI_SECRET = "DUMMY-SECRET-CLI-MARKER"; const dummyEvidence = fileURLToPath(new URL("./cp06-dummy-worker-evidence.mjs", import.meta.url)); /** Write scenario shims into `anchor`; returns a PATH that resolves them first. */ export function writeDummyCliShims({ anchor, path, scenario, inheritedPath = process.env.PATH }) { - const selected = DUMMY_CLI_SCENARIOS[scenario]; + const selected = DUMMY_CLI_SCENARIOS[scenario] ?? DUMMY_CLI_INSTALLED_SCENARIOS[scenario]; if (selected === undefined) throw new Error("unsupported DUMMY CLI scenario"); const realSetpriv = locate("setpriv", inheritedPath); const realUmount = locate("umount", inheritedPath); @@ -79,6 +89,7 @@ export function writeDummyCliShims({ anchor, path, scenario, inheritedPath = pro exit: `printf '%s' '${DUMMY_CLI_SECRET}' >&2; exit 1`, valid: `exec '${process.execPath}' '${dummyEvidence}' "$last"`, "forged-provenance": `exec '${process.execPath}' '${dummyEvidence}' "$last" forged-provenance`, + installed: `exec '${process.execPath}' --experimental-import-meta-resolve '${dummyEvidence}' "$last" installed`, }[selected.child]; writeExecutable( anchor, diff --git a/scripts/cp06-dummy-worker-evidence.mjs b/scripts/cp06-dummy-worker-evidence.mjs index 32f227a..b296dee 100644 --- a/scripts/cp06-dummy-worker-evidence.mjs +++ b/scripts/cp06-dummy-worker-evidence.mjs @@ -3,6 +3,7 @@ import { resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { workerEvidenceContext } from "./cp06-worker-evidence.mjs"; import { DUMMY_OUTCOME_ORIGINAL, DUMMY_PI_PROVENANCE } from "./cp06-dummy-originals.mjs"; +import { resolvePinnedPiProvenance } from "./cp06-pi-install.mjs"; export function dummyWorkerContext(directory = "/DUMMY") { return { @@ -49,6 +50,13 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur const context = workerEvidenceContext(process.argv[2]); const evidence = dummyWorkerEvidence(context); if (process.argv[3] === "invalid") evidence.response = { status: "PASS", evidence_gaps: [] }; + if (process.argv[3] === "installed") { + evidence.pi_install = resolvePinnedPiProvenance({ + projectRoot: process.env.CP06_PROJECT_ROOT, + packageRoot: process.env.CP06_PI_PACKAGE_ROOT, + executable: process.env.CP06_PI_BIN, + }); + } if (process.argv[3] === "forged-provenance") { evidence.pi_install.package_artifact.tarball_integrity = "sha512-DUMMY-SECRET-CLI-MARKER"; } diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index 5035df7..ed859f5 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -55,6 +55,18 @@ export async function superviseCredentialChild(options, dependencies = {}) { throw setup; } } + let provenance; + if (mode === "worker" && !dependencies.spawn) { + try { + provenance = trustedWorkerProvenance({ + root, + environment: process.env, + dummy: process.env.CP06_DUMMY_TRUSTED_PROVENANCE === "1", + }); + } catch (cause) { + throw failure("setup", "CP06_ISOLATION_SETUP_FAILED", 73, cause); + } + } const source = probe?.source ?? requestedSource; const target = probe?.target ?? requestedTarget; const beforeHash = probe?.beforeHash ?? null; @@ -102,13 +114,7 @@ export async function superviseCredentialChild(options, dependencies = {}) { let workerContext = dependencies.workerContext; if (mode === "worker" && !dependencies.spawn) { try { - workerContext = workerEvidenceContext(childArgs[0], target, source, { - provenance: trustedWorkerProvenance({ - root, - environment: childEnvironment(mode, source, target), - dummy: process.env.CP06_DUMMY_TRUSTED_PROVENANCE === "1", - }), - }); + workerContext = workerEvidenceContext(childArgs[0], target, source, { provenance }); } catch { throw failure("audit", "CP06_CHILD_OUTPUT_INVALID", 70); } diff --git a/scripts/cp06-pi-install.mjs b/scripts/cp06-pi-install.mjs index 41035c2..8e4739a 100644 --- a/scripts/cp06-pi-install.mjs +++ b/scripts/cp06-pi-install.mjs @@ -9,6 +9,30 @@ const PACKAGE_NAME = "@earendil-works/pi-coding-agent"; const SUPPORTED_VERSION = "0.85.1"; export function resolvePinnedPiInstall(options) { + const { projectRoot, packageRoot, executable, provenance } = selectPinnedPi(options); + const spawn = options.spawnSyncImpl ?? spawnSync; + const version = spawn(executable, ["--version"], { + cwd: projectRoot, + encoding: "utf8", + timeout: 10_000, + env: options.environment ?? process.env, + }); + if ( + version.error !== undefined || + version.status !== 0 || + version.stdout.trim() !== SUPPORTED_VERSION + ) { + throw new Error("selected task-local Pi executable version is unsupported"); + } + return { root: packageRoot, executable, version: SUPPORTED_VERSION, provenance }; +} + +/** Selected-installation provenance from paths, metadata and pins only; never executes Pi. */ +export function resolvePinnedPiProvenance(options) { + return selectPinnedPi(options).provenance; +} + +function selectPinnedPi(options) { const projectRoot = realpathSync(resolve(options.projectRoot)); const selectedRoot = resolve(projectRoot, ".factory/state/cp06-sdk"); const expectedPackageRoot = join( @@ -49,25 +73,11 @@ export function resolvePinnedPiInstall(options) { const packageIntegrity = verifyPinnedPackage(packageRoot, PACKAGE_NAME); const piAi = resolvePinnedPiAi(packageRoot); - const spawn = options.spawnSyncImpl ?? spawnSync; - const version = spawn(expectedExecutable, ["--version"], { - cwd: projectRoot, - encoding: "utf8", - timeout: 10_000, - env: options.environment ?? process.env, - }); - if ( - version.error !== undefined || - version.status !== 0 || - version.stdout.trim() !== SUPPORTED_VERSION - ) { - throw new Error("selected task-local Pi executable version is unsupported"); - } return { - root: packageRoot, + projectRoot, + packageRoot, executable: expectedExecutable, - version: SUPPORTED_VERSION, provenance: { install_root: selectedRoot, package_root: packageRoot, diff --git a/scripts/cp06-worker-evidence.mjs b/scripts/cp06-worker-evidence.mjs index c325d64..c0a2f71 100644 --- a/scripts/cp06-worker-evidence.mjs +++ b/scripts/cp06-worker-evidence.mjs @@ -3,7 +3,7 @@ import { auditWorkerOutcome } from "./cp06-worker-outcome.mjs"; import { validateReadAudit } from "./cp06-auth-security.mjs"; import { canonicalWorkerPaths, readPinnedRegularFile } from "./cp06-guarded-read.mjs"; import { DUMMY_PI_PROVENANCE } from "./cp06-dummy-originals.mjs"; -import { resolvePinnedPiInstall } from "./cp06-pi-install.mjs"; +import { resolvePinnedPiProvenance } from "./cp06-pi-install.mjs"; export function workerEvidenceContext(inputPath, target, source, { provenance } = {}) { const input = readWorkerInput(inputPath); @@ -12,16 +12,15 @@ export function workerEvidenceContext(inputPath, target, source, { provenance } return { input, paths, original: snapshot.bytes, originalIdentity: snapshot.identity, provenance }; } -// The supervisor resolves the selected installation itself; the DUMMY -// expectation is a fixed literal that no genuine installation can report. +// The supervisor resolves the selected installation itself without executing +// Pi; the DUMMY expectation is a fixed literal no genuine installation reports. export function trustedWorkerProvenance({ root, environment, dummy = false }) { if (dummy) return DUMMY_PI_PROVENANCE; - return resolvePinnedPiInstall({ + return resolvePinnedPiProvenance({ projectRoot: root, packageRoot: environment.CP06_PI_PACKAGE_ROOT, executable: environment.CP06_PI_BIN, - environment, - }).provenance; + }); } export function readWorkerInput(path) { diff --git a/test/cp06-pi-dependency.test.mjs b/test/cp06-pi-dependency.test.mjs index aca6073..7df343c 100644 --- a/test/cp06-pi-dependency.test.mjs +++ b/test/cp06-pi-dependency.test.mjs @@ -7,6 +7,7 @@ import { join, resolve } from "node:path"; import test from "node:test"; import { pathToFileURL } from "node:url"; import { resolvePinnedPiInstall } from "../scripts/cp06-pi-install.mjs"; +import { trustedWorkerProvenance } from "../scripts/cp06-worker-evidence.mjs"; const resolverUrl = pathToFileURL(resolve("scripts/cp06-pi-dependency.mjs")).href; @@ -110,6 +111,47 @@ test("exact artifact succeeds and same-version selected entry substitutions fail } }); +test("supervisor trusted provenance verifies the selected installation without executing Pi", async (t) => { + const installed = resolve(".factory/state/cp06-sdk/node_modules/@earendil-works"); + if (!existsSync(join(installed, "pi-ai"))) return t.skip("task-local CP-06 SDK is not installed"); + const projectRoot = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-trusted-provenance-")); + const selected = join(projectRoot, ".factory/state/cp06-sdk/node_modules"); + const packages = join(selected, "@earendil-works"); + const packageRoot = join(packages, "pi-coding-agent"); + const executable = join(selected, ".bin/pi"); + try { + await mkdir(packages, { recursive: true }); + await mkdir(join(selected, ".bin")); + for (const name of ["pi-coding-agent", "pi-ai"]) { + await cp(join(installed, name), join(packages, name), { recursive: true }); + } + const binPath = JSON.parse(await readFile(join(packageRoot, "package.json"))).bin; + await symlink( + join(packageRoot, typeof binPath === "string" ? binPath : binPath.pi), + executable, + ); + const environment = { + CP06_PI_PACKAGE_ROOT: packageRoot, + CP06_PI_BIN: executable, + PATH: join(projectRoot, "DUMMY-no-node"), + HOME: join(projectRoot, "DUMMY-home"), + }; + const trusted = trustedWorkerProvenance({ root: projectRoot, environment }); + const reported = resolvePinnedPiInstall({ + projectRoot, + packageRoot, + executable, + spawnSyncImpl: () => ({ status: 0, stdout: "0.85.1\n" }), + }).provenance; + assert.deepEqual(trusted, reported); + assert.equal(existsSync(join(projectRoot, "DUMMY-home")), false); + await writeFile(join(packages, "pi-ai/dist/index.js"), "DUMMY same-version dependency replacement"); + assert.throws(() => trustedWorkerProvenance({ root: projectRoot, environment }), /pinned artifact/); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + for (const scenario of [ { name: "substituted same-version import-only dependency", version: "0.85.1", expectedExit: 1 }, { name: "newer hoisted import-only dependency", version: "0.99.2", expectedExit: 1 }, diff --git a/test/cp06-review-boundaries.test.mjs b/test/cp06-review-boundaries.test.mjs index 7039487..52ff037 100644 --- a/test/cp06-review-boundaries.test.mjs +++ b/test/cp06-review-boundaries.test.mjs @@ -189,7 +189,7 @@ test("supervisor CLI serializes fixture setup and independent fixture cleanup ca writeFileSync(join(shims, name), `#!/bin/sh\nfor argument do last="$argument"; done\n${body}\n`); chmodSync(join(shims, name), 0o700); } - const result = spawnSync("unshare", ["--user", "--map-root-user", "--mount", "--propagation", "private", "--", process.execPath, resolve("scripts/cp06-namespace-supervisor.mjs"), "hardened-probe", outputRoot, "-", "/DUMMY-helper", "/DUMMY-probe"], { + const result = spawnSync("unshare", ["--user", "--map-root-user", "--mount", "--propagation", "private", "--", process.execPath, "--experimental-import-meta-resolve", resolve("scripts/cp06-namespace-supervisor.mjs"), "hardened-probe", outputRoot, "-", "/DUMMY-helper", "/DUMMY-probe"], { encoding: "utf8", timeout: 30_000, env: { ...process.env, PATH: `${shims}:${process.env.PATH}` }, }); const record = umountFails diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 1e2757c..140c8df 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -10,7 +10,7 @@ import { symlink, writeFile, } from "node:fs/promises"; -import { closeSync, constants, openSync, realpathSync, writeFileSync } from "node:fs"; +import { closeSync, constants, existsSync, openSync, realpathSync, writeFileSync } from "node:fs"; import { spawnSync } from "node:child_process"; import { resolve } from "node:path"; import { tmpdir } from "node:os"; @@ -24,10 +24,12 @@ import { namespaceWorkerError } from "../scripts/cp06-auth-security.mjs"; import { Cp06CleanupError, withCleanup } from "../scripts/cp06-worker-lifecycle.mjs"; import { createWorkerBlockedStatus } from "../scripts/replay-cp06-worker.mjs"; import { + DUMMY_CLI_INSTALLED_SCENARIOS, DUMMY_CLI_SCENARIOS, DUMMY_CLI_SECRET, writeDummyCliShims, } from "../scripts/cp06-dummy-cli-shims.mjs"; +import { resolvePinnedPiProvenance } from "../scripts/cp06-pi-install.mjs"; import { withCompiledHelper } from "./fixtures/cp06-compiled-helper.mjs"; import { dummyWorkerContext, dummyWorkerEvidence } from "../scripts/cp06-dummy-worker-evidence.mjs"; import { DUMMY_OUTCOME_ORIGINAL } from "../scripts/cp06-dummy-originals.mjs"; @@ -524,6 +526,7 @@ test("supervisor executable serializes compound worker failures as typed causes" "private", "--", process.execPath, + "--experimental-import-meta-resolve", resolve("scripts/cp06-namespace-supervisor.mjs"), "worker", source, @@ -576,6 +579,120 @@ test("supervisor executable serializes compound worker failures as typed causes" }); }); +test("supervisor executable resolves the selected installation without DUMMY trust", async () => { + const installRoot = resolve(".factory/state/cp06-sdk/node_modules"); + const install = { + CP06_PI_PACKAGE_ROOT: join(installRoot, "@earendil-works/pi-coding-agent"), + CP06_PI_BIN: join(installRoot, ".bin/pi"), + }; + const installed = existsSync(install.CP06_PI_PACKAGE_ROOT); + const setup = { + schema_version: 1, + stage: "setup", + code: "CP06_ISOLATION_SETUP_FAILED", + exit_code: 73, + description: "credential namespace setup failed", + causes: [], + }; + const cases = [ + { name: "installation-missing", scenario: "dummy-provenance-untrusted", install: {}, record: setup }, + ...(installed + ? Object.entries(DUMMY_CLI_INSTALLED_SCENARIOS).map(([scenario, { record }]) => ({ + name: scenario, + scenario, + install, + record, + })) + : []), + ]; + await withCompiledHelper("cli-installed", async ({ helperFd, binaries }) => { + const probeFd = openSync(binaries.syscallProbe, constants.O_RDONLY | constants.O_CLOEXEC); + try { + for (const { name, scenario, install: selected, record } of cases) { + const directory = realpathSync(await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-installed-"))); + const shims = join(directory, "shims"); + const source = join(directory, "credential", "DUMMY-source.json"); + const sourceBytes = '{"DUMMY":"credential fixture only"}'; + await mkdir(shims); + await mkdir(join(directory, "credential")); + await writeFile(source, sourceBytes); + await mkdir(join(directory, "docs")); + await writeFile(join(directory, "docs/AUTH.md"), DUMMY_OUTCOME_ORIGINAL); + await writeFile(join(directory, "DUMMY-contract.yaml"), "DUMMY contract"); + await writeFile(join(directory, "DUMMY-pack.md"), "DUMMY pack"); + await writeFile(join(directory, "DUMMY-input.json"), JSON.stringify({ + schema_version: 1, root: directory, fixture_root: directory, + original_path: join(directory, "docs/AUTH.md"), + contract_path: join(directory, "DUMMY-contract.yaml"), pack_path: join(directory, "DUMMY-pack.md"), + provider: "openai-codex", model: "DUMMY-model", timeout_ms: 1_000, + })); + const environment = { ...process.env, ...selected, HOME: directory }; + delete environment.CP06_DUMMY_TRUSTED_PROVENANCE; + if (selected.CP06_PI_PACKAGE_ROOT === undefined) { + delete environment.CP06_PI_PACKAGE_ROOT; + delete environment.CP06_PI_BIN; + } + environment.PATH = writeDummyCliShims({ anchor: shims, path: shims, scenario }); + try { + const result = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + "--experimental-import-meta-resolve", + resolve("scripts/cp06-namespace-supervisor.mjs"), + "worker", + source, + join(directory, "home", "pi-agent", "auth.json"), + "/proc/self/fd/3", + "/proc/self/fd/4", + join(directory, "DUMMY-input.json"), + ], + { + encoding: "utf8", + timeout: 60_000, + stdio: ["pipe", "pipe", "pipe", helperFd, probeFd], + env: environment, + }, + ); + assert.equal(`${result.stdout}${result.stderr}`.includes(DUMMY_CLI_SECRET), false); + assert.equal(await readFile(source, "utf8"), sourceBytes); + if (record === null) { + assert.equal(result.status, 0, result.stderr); + assert.equal(result.stderr, ""); + const success = JSON.parse(result.stdout); + assert.equal(success.cleanup, "pass"); + assert.deepEqual( + success.child.pi_install, + resolvePinnedPiProvenance({ + projectRoot: process.cwd(), + packageRoot: install.CP06_PI_PACKAGE_ROOT, + executable: install.CP06_PI_BIN, + }), + ); + continue; + } + assert.equal(result.status, record.exit_code, name); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, `${JSON.stringify(record)}\n`); + if (name === "installation-missing") { + await assert.rejects(stat(join(directory, "home")), { code: "ENOENT" }); + } + } finally { + await rm(directory, { recursive: true, force: true }); + } + } + } finally { + closeSync(probeFd); + } + }); +}); + test("worker source integrity uses metadata without reading credential bytes", async () => { const directory = await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-metadata-")); const source = join(directory, "DUMMY-unreadable-source"); From b21a39b5a51d28ea06831bc911ad883390812d3c Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 6 Oct 2026 06:00:30 +0000 Subject: [PATCH 42/45] no-mistakes(document): Apply Prettier formatting to CP06 scripts and tests --- scripts/cp06-auth-security.mjs | 46 ++- scripts/cp06-dummy-originals.mjs | 33 +- scripts/cp06-dummy-outcome-proof.mjs | 35 ++- scripts/cp06-dummy-sdk-proof.mjs | 222 +++++++------- scripts/cp06-dummy-worker-evidence.mjs | 50 ++- scripts/cp06-dummy-worker-hook.mjs | 10 +- scripts/cp06-guarded-read.mjs | 15 +- scripts/cp06-namespace-supervisor.mjs | 40 ++- scripts/cp06-probe-fixture.mjs | 17 +- scripts/cp06-worker-evidence.mjs | 163 ++++++++-- scripts/replay-cp06-p07.mjs | 11 +- scripts/replay-cp06-ten-pack.mjs | 30 +- test/cp06-pi-dependency.test.mjs | 10 +- test/cp06-proof-boundary.test.mjs | 63 +++- test/cp06-review-boundaries.test.mjs | 388 ++++++++++++++++++++---- test/cp06-worker-lifecycle.test.mjs | 51 +++- test/fixtures/cp06-credential-mount.mjs | 3 +- 17 files changed, 897 insertions(+), 290 deletions(-) diff --git a/scripts/cp06-auth-security.mjs b/scripts/cp06-auth-security.mjs index 07641ab..e4dbc88 100644 --- a/scripts/cp06-auth-security.mjs +++ b/scripts/cp06-auth-security.mjs @@ -261,7 +261,10 @@ export async function runIsolationProbe({ } function runNamespace(options) { - const inputs = options.mode === "worker" ? [options.helper, options.syscallProbe, options.childArgs[0]] : [options.helper, options.syscallProbe]; + const inputs = + options.mode === "worker" + ? [options.helper, options.syscallProbe, options.childArgs[0]] + : [options.helper, options.syscallProbe]; return withInheritedExecutables(inputs, ([helper, probe, input]) => spawnSync( "unshare", @@ -301,7 +304,10 @@ function withInheritedExecutables(paths, action) { try { for (const [index, path] of paths.entries()) { descriptors.push({ - fd: openSync(resolve(path), constants.O_RDONLY | constants.O_CLOEXEC | (index >= 2 ? constants.O_NOFOLLOW : 0)), + fd: openSync( + resolve(path), + constants.O_RDONLY | constants.O_CLOEXEC | (index >= 2 ? constants.O_NOFOLLOW : 0), + ), path: `/proc/self/fd/${index + 3}`, }); } @@ -434,12 +440,29 @@ export function assertHardenedProof(proof) { const observations = new Map(); for (const line of syscalls.stdout.split("\n")) { const [key, value, extra] = line.split("="); - assert(key && value && extra === undefined && !observations.has(key), "syscall observations malformed"); + assert( + key && value && extra === undefined && !observations.has(key), + "syscall observations malformed", + ); observations.set(key, value); } for (const prefix of ["", "inherited_"]) { - for (const name of ["setns", "unshare", "clone", "mount", "umount2", "ptrace_attach", "ptrace_seize", "process_vm_readv", "process_vm_writev", "pidfd_getfd"]) { - assert(observations.get(`${prefix}${name}`) === "-1:1", "required DUMMY syscall denial missing"); + for (const name of [ + "setns", + "unshare", + "clone", + "mount", + "umount2", + "ptrace_attach", + "ptrace_seize", + "process_vm_readv", + "process_vm_writev", + "pidfd_getfd", + ]) { + assert( + observations.get(`${prefix}${name}`) === "-1:1", + "required DUMMY syscall denial missing", + ); } assert(observations.get(`${prefix}clone3`) === "-1:38", "clone3 denial missing"); assert(observations.get(`${prefix}pidfd_open`) === "opened", "DUMMY pidfd target unavailable"); @@ -448,7 +471,10 @@ export function assertHardenedProof(proof) { assert(observations.get(`${prefix}NoNewPrivs`) === "1", "inherited no_new_privs missing"); assert(observations.get(`${prefix}Seccomp`) === "2", "inherited seccomp missing"); for (const name of ["CapInh", "CapPrm", "CapEff", "CapBnd", "CapAmb"]) { - assert(observations.get(`${prefix}${name}`) === "0000000000000000", "inherited capabilities not empty"); + assert( + observations.get(`${prefix}${name}`) === "0000000000000000", + "inherited capabilities not empty", + ); } } assert(observations.get("dummy_target_unchanged") === "true", "DUMMY target integrity missing"); @@ -767,7 +793,8 @@ function validateOutcomeProof(outcomes, trustedDependency, originalPath) { "source_unchanged", "to_auth", ]; - if (["valid", "fenced-valid"].includes(entry.scenario)) expectedKeys.push("read_audit", "event_count"); + if (["valid", "fenced-valid"].includes(entry.scenario)) + expectedKeys.push("read_audit", "event_count"); assertExactKeys(entry, expectedKeys); seen.add(entry.scenario); assert( @@ -830,7 +857,10 @@ export function validateReadAudit(audit, { toolCallId, originalPath, original, e ]); const isEmptyArray = (value) => Array.isArray(value) && value.length === 0; assert(Number.isSafeInteger(eventCount) && eventCount > 0, "read audit event count missing"); - assert(typeof original === "string" || Buffer.isBuffer(original), "read audit trusted bytes missing"); + assert( + typeof original === "string" || Buffer.isBuffer(original), + "read audit trusted bytes missing", + ); const expectedBytes = Buffer.from(original); const expectedHash = sha256(expectedBytes); const isIndex = (value) => Number.isSafeInteger(value) && value >= 0 && value < eventCount; diff --git a/scripts/cp06-dummy-originals.mjs b/scripts/cp06-dummy-originals.mjs index 84a6d25..bd5f658 100644 --- a/scripts/cp06-dummy-originals.mjs +++ b/scripts/cp06-dummy-originals.mjs @@ -1,8 +1,29 @@ -export const DUMMY_SDK_ORIGINAL = "# DUMMY authority\n\nrequest-supplied owner fields are ignored; ownership comes only from the authenticated principal.\n"; -export const DUMMY_OUTCOME_ORIGINAL = "# DUMMY authority\n\nOwnership comes only from the authenticated principal; request-supplied owner fields are ignored.\n"; +export const DUMMY_SDK_ORIGINAL = + "# DUMMY authority\n\nrequest-supplied owner fields are ignored; ownership comes only from the authenticated principal.\n"; +export const DUMMY_OUTCOME_ORIGINAL = + "# DUMMY authority\n\nOwnership comes only from the authenticated principal; request-supplied owner fields are ignored.\n"; export const DUMMY_PI_PROVENANCE = Object.freeze({ - install_root: "/DUMMY/sdk", package_root: "/DUMMY/sdk/package", executable: "/DUMMY/sdk/pi", executable_entry: "/DUMMY/sdk/package/cli.js", sdk_entry: "/DUMMY/sdk/package/index.js", - package_sha256: "0".repeat(64), executable_sha256: "0".repeat(64), sdk_entry_sha256: "0".repeat(64), - package_artifact: Object.freeze({ manifest_sha256: "0".repeat(64), tarball_integrity: "sha512-DUMMY" }), - dependency: Object.freeze({ entry: "/DUMMY/sdk/ai/index.js", root: "/DUMMY/sdk/ai", entry_sha256: "0".repeat(64), manifest_sha256: "0".repeat(64), package_sha256: "0".repeat(64), name: "@earendil-works/pi-ai", version: "0.85.1", resolution: "esm-import-condition", tarball_integrity: "sha512-DUMMY" }), + install_root: "/DUMMY/sdk", + package_root: "/DUMMY/sdk/package", + executable: "/DUMMY/sdk/pi", + executable_entry: "/DUMMY/sdk/package/cli.js", + sdk_entry: "/DUMMY/sdk/package/index.js", + package_sha256: "0".repeat(64), + executable_sha256: "0".repeat(64), + sdk_entry_sha256: "0".repeat(64), + package_artifact: Object.freeze({ + manifest_sha256: "0".repeat(64), + tarball_integrity: "sha512-DUMMY", + }), + dependency: Object.freeze({ + entry: "/DUMMY/sdk/ai/index.js", + root: "/DUMMY/sdk/ai", + entry_sha256: "0".repeat(64), + manifest_sha256: "0".repeat(64), + package_sha256: "0".repeat(64), + name: "@earendil-works/pi-ai", + version: "0.85.1", + resolution: "esm-import-condition", + tarball_integrity: "sha512-DUMMY", + }), }); diff --git a/scripts/cp06-dummy-outcome-proof.mjs b/scripts/cp06-dummy-outcome-proof.mjs index 2edc54c..553d40c 100644 --- a/scripts/cp06-dummy-outcome-proof.mjs +++ b/scripts/cp06-dummy-outcome-proof.mjs @@ -6,7 +6,12 @@ import { tmpdir } from "node:os"; import { basename, dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { withCleanup } from "./cp06-worker-lifecycle.mjs"; -import { openAnchoredDirectory, readArtifactFile, writeArtifactFile, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { + openAnchoredDirectory, + readArtifactFile, + writeArtifactFile, + removeAnchoredEntry, +} from "./cp06-probe-fixture.mjs"; import { DUMMY_OUTCOME_ORIGINAL } from "./cp06-dummy-originals.mjs"; import { openPinnedDirectory } from "./cp06-guarded-read.mjs"; @@ -21,11 +26,15 @@ export async function proveDummyWorkerOutcome({ piRoot, directory: requested, pa prepared = openAnchoredDirectory(parentAnchor, "DUMMY-outcome"); directory = requested; } - const fd = prepared ? openSync(prepared.anchor, constants.O_RDONLY | constants.O_DIRECTORY) : openPinnedDirectory(directory); + const fd = prepared + ? openSync(prepared.anchor, constants.O_RDONLY | constants.O_DIRECTORY) + : openPinnedDirectory(directory); prepared?.close(); const anchor = `/proc/${process.pid}/fd/${fd}`; const identity = fstatSync(fd, { bigint: true }); - const parentFd = parentAnchor ? openSync(parentAnchor, constants.O_RDONLY | constants.O_DIRECTORY) : openPinnedDirectory(dirname(directory)); + const parentFd = parentAnchor + ? openSync(parentAnchor, constants.O_RDONLY | constants.O_DIRECTORY) + : openPinnedDirectory(dirname(directory)); const retainedParent = `/proc/${process.pid}/fd/${parentFd}`; const anchored = (path) => { if (!path.startsWith(`${directory}/`)) throw new Error("DUMMY artifact escaped fixture"); @@ -37,10 +46,7 @@ export async function proveDummyWorkerOutcome({ piRoot, directory: requested, pa async () => { openAnchoredDirectory(anchor, "docs").close(); const originalPath = join(directory, "docs/AUTH.md"); - await writeFile( - originalPath, - DUMMY_OUTCOME_ORIGINAL, - ); + await writeFile(originalPath, DUMMY_OUTCOME_ORIGINAL); const credentialAlias = join(directory, "DUMMY-credential-alias.json"); const contractPath = join(directory, "DUMMY-contract.yaml"); const packPath = join(directory, "DUMMY-pack.md"); @@ -183,7 +189,12 @@ export async function proveDummyWorkerOutcome({ piRoot, directory: requested, pa expected_exit_code: expectedExit, ...counts, source_unchanged: true, - ...(output ? { read_audit: output.event_stream.read_audit, event_count: output.event_stream.event_count } : {}), + ...(output + ? { + read_audit: output.event_stream.read_audit, + event_count: output.event_stream.event_count, + } + : {}), }); } return { fixture_origin: true, semantic_acceptance: false, cases }; @@ -192,9 +203,13 @@ export async function proveDummyWorkerOutcome({ piRoot, directory: requested, pa try { for (const name of readdirSync(anchor)) removeAnchoredEntry(anchor, name); const visible = lstatSync(join(retainedParent, basename(directory)), { bigint: true }); - if (!visible.isDirectory() || visible.dev !== identity.dev || visible.ino !== identity.ino) throw new Error("DUMMY outcome directory identity changed"); + if (!visible.isDirectory() || visible.dev !== identity.dev || visible.ino !== identity.ino) + throw new Error("DUMMY outcome directory identity changed"); removeAnchoredEntry(retainedParent, basename(directory)); - } finally { closeSync(fd); closeSync(parentFd); } + } finally { + closeSync(fd); + closeSync(parentFd); + } }, ); } diff --git a/scripts/cp06-dummy-sdk-proof.mjs b/scripts/cp06-dummy-sdk-proof.mjs index 3321d15..0e8fa3a 100644 --- a/scripts/cp06-dummy-sdk-proof.mjs +++ b/scripts/cp06-dummy-sdk-proof.mjs @@ -8,7 +8,12 @@ import { Cp06ReadOnlyCredentialStore } from "./cp06-readonly-credentials.mjs"; import { auditReadEvents } from "./cp06-worker-audit.mjs"; import { proveDummyWorkerOutcome } from "./cp06-dummy-outcome-proof.mjs"; import { importPinnedPiAi } from "./cp06-pi-dependency.mjs"; -import { openAnchoredDirectory, readArtifactFile, writeArtifactFile, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { + openAnchoredDirectory, + readArtifactFile, + writeArtifactFile, + removeAnchoredEntry, +} from "./cp06-probe-fixture.mjs"; import { DUMMY_SDK_ORIGINAL } from "./cp06-dummy-originals.mjs"; import { createGuardedReadTool, openPinnedDirectory } from "./cp06-guarded-read.mjs"; @@ -134,112 +139,121 @@ async function authCase(scenario) { async function fauxWorkerProof() { const workerDirectory = openAnchoredDirectory(proofAnchor, "DUMMY-worker"); try { - const workerRoot = join(proofRoot, "DUMMY-worker"); - const originalPath = join(workerRoot, "DUMMY-original.md"); - const authPath = join(workerDirectory.anchor, "DUMMY-auth.json"); - const original = DUMMY_SDK_ORIGINAL; - writeArtifactFile(join(workerDirectory.anchor, "DUMMY-original.md"), original); - const guardedRead = createGuardedReadTool({ sdk, root: workerRoot, expectedOriginalPath: originalPath }); - const faux = ai.fauxProvider({ - provider: "DUMMY-readonly", - models: [{ id: "DUMMY-model" }], - tokensPerSecond: 100_000, - }); - const providerId = faux.provider.id; - await writeFile( - authPath, - JSON.stringify({ - [providerId]: { - type: "oauth", - access: "DUMMY-ACCESS", - refresh: "DUMMY-REFRESH", - expires: Date.now() + 600_000, - }, - }), - ); - const store = await Cp06ReadOnlyCredentialStore.load({ path: authPath, providerId }); - let refreshCallbacks = 0; - faux.provider.auth = { - oauth: { - async refresh() { - refreshCallbacks += 1; - throw new Error("DUMMY refresh must not run"); - }, - async toAuth() { - return { apiKey: "DUMMY-RESOLVED" }; + const workerRoot = join(proofRoot, "DUMMY-worker"); + const originalPath = join(workerRoot, "DUMMY-original.md"); + const authPath = join(workerDirectory.anchor, "DUMMY-auth.json"); + const original = DUMMY_SDK_ORIGINAL; + writeArtifactFile(join(workerDirectory.anchor, "DUMMY-original.md"), original); + const guardedRead = createGuardedReadTool({ + sdk, + root: workerRoot, + expectedOriginalPath: originalPath, + }); + const faux = ai.fauxProvider({ + provider: "DUMMY-readonly", + models: [{ id: "DUMMY-model" }], + tokensPerSecond: 100_000, + }); + const providerId = faux.provider.id; + await writeFile( + authPath, + JSON.stringify({ + [providerId]: { + type: "oauth", + access: "DUMMY-ACCESS", + refresh: "DUMMY-REFRESH", + expires: Date.now() + 600_000, + }, + }), + ); + const store = await Cp06ReadOnlyCredentialStore.load({ path: authPath, providerId }); + let refreshCallbacks = 0; + faux.provider.auth = { + oauth: { + async refresh() { + refreshCallbacks += 1; + throw new Error("DUMMY refresh must not run"); + }, + async toAuth() { + return { apiKey: "DUMMY-RESOLVED" }; + }, }, - }, - }; - const runtime = await sdk.ModelRuntime.create({ - credentials: store, - modelsPath: null, - refreshOnCreate: false, - allowModelNetwork: false, - }); - runtime.registerNativeProvider(faux.provider); - const model = runtime.getModel(providerId, "DUMMY-model"); - assert.ok(model); - faux.setResponses([ - ai.fauxAssistantMessage(ai.fauxToolCall("read", { path: originalPath }, { id: "DUMMY-read" }), { - stopReason: "toolUse", - }), - ai.fauxAssistantMessage("DUMMY complete", { stopReason: "stop" }), - ]); - const resourceLoader = { - getExtensions: () => ({ extensions: [], errors: [], runtime: sdk.createExtensionRuntime() }), - getSkills: () => ({ skills: [], diagnostics: [] }), - getPrompts: () => ({ prompts: [], diagnostics: [] }), - getThemes: () => ({ themes: [], diagnostics: [] }), - getAgentsFiles: () => ({ agentsFiles: [] }), - getSystemPrompt: () => "Use only read and read the supplied DUMMY original exactly once.", - getSystemPromptSource: () => undefined, - getAppendSystemPrompt: () => [], - getAppendSystemPromptSources: () => [], - extendResources: () => {}, - reload: async () => {}, - }; - const { session } = await sdk.createAgentSession({ - cwd: workerRoot, - agentDir: workerRoot, - model, - modelRuntime: runtime, - resourceLoader, - tools: ["read"], - customTools: [guardedRead.tool], - sessionManager: sdk.SessionManager.inMemory(workerRoot), - settingsManager: sdk.SettingsManager.inMemory({ - compaction: { enabled: false }, - retry: { enabled: false }, - }), - }); - const events = []; - const unsubscribe = session.subscribe((event) => events.push(structuredClone(event))); - try { - await session.prompt(`Read ${originalPath} exactly once.`, { expandPromptTemplates: false }); + }; + const runtime = await sdk.ModelRuntime.create({ + credentials: store, + modelsPath: null, + refreshOnCreate: false, + allowModelNetwork: false, + }); + runtime.registerNativeProvider(faux.provider); + const model = runtime.getModel(providerId, "DUMMY-model"); + assert.ok(model); + faux.setResponses([ + ai.fauxAssistantMessage( + ai.fauxToolCall("read", { path: originalPath }, { id: "DUMMY-read" }), + { + stopReason: "toolUse", + }, + ), + ai.fauxAssistantMessage("DUMMY complete", { stopReason: "stop" }), + ]); + const resourceLoader = { + getExtensions: () => ({ extensions: [], errors: [], runtime: sdk.createExtensionRuntime() }), + getSkills: () => ({ skills: [], diagnostics: [] }), + getPrompts: () => ({ prompts: [], diagnostics: [] }), + getThemes: () => ({ themes: [], diagnostics: [] }), + getAgentsFiles: () => ({ agentsFiles: [] }), + getSystemPrompt: () => "Use only read and read the supplied DUMMY original exactly once.", + getSystemPromptSource: () => undefined, + getAppendSystemPrompt: () => [], + getAppendSystemPromptSources: () => [], + extendResources: () => {}, + reload: async () => {}, + }; + const { session } = await sdk.createAgentSession({ + cwd: workerRoot, + agentDir: workerRoot, + model, + modelRuntime: runtime, + resourceLoader, + tools: ["read"], + customTools: [guardedRead.tool], + sessionManager: sdk.SessionManager.inMemory(workerRoot), + settingsManager: sdk.SettingsManager.inMemory({ + compaction: { enabled: false }, + retry: { enabled: false }, + }), + }); + const events = []; + const unsubscribe = session.subscribe((event) => events.push(structuredClone(event))); + try { + await session.prompt(`Read ${originalPath} exactly once.`, { expandPromptTemplates: false }); + } finally { + unsubscribe(); + session.dispose(); + } + const audit = auditReadEvents(events, { + root: workerRoot, + expectedOriginalPath: originalPath, + expectedOriginal: original, + decisiveText: ["request-supplied owner fields are ignored"], + }); + assert.equal(audit.ok, true); + assert.deepEqual(session.getActiveToolNames(), ["read"]); + assert.equal(session.sessionFile, undefined); + assert.equal(refreshCallbacks, 0); + return { + active_tools: session.getActiveToolNames(), + in_memory_session: session.sessionFile === undefined, + event_count: events.length, + event_sha256: sha256(JSON.stringify(events)), + read_audit: audit.summary, + refresh_callbacks: refreshCallbacks, + credential_store: store.audit(), + }; } finally { - unsubscribe(); - session.dispose(); + workerDirectory.close(); } - const audit = auditReadEvents(events, { - root: workerRoot, - expectedOriginalPath: originalPath, - expectedOriginal: original, - decisiveText: ["request-supplied owner fields are ignored"], - }); - assert.equal(audit.ok, true); - assert.deepEqual(session.getActiveToolNames(), ["read"]); - assert.equal(session.sessionFile, undefined); - assert.equal(refreshCallbacks, 0); - return { - active_tools: session.getActiveToolNames(), - in_memory_session: session.sessionFile === undefined, - event_count: events.length, - event_sha256: sha256(JSON.stringify(events)), - read_audit: audit.summary, - refresh_callbacks: refreshCallbacks, - credential_store: store.audit(), - }; - } finally { workerDirectory.close(); } } async function writeFile(path, bytes) { diff --git a/scripts/cp06-dummy-worker-evidence.mjs b/scripts/cp06-dummy-worker-evidence.mjs index b296dee..fad0680 100644 --- a/scripts/cp06-dummy-worker-evidence.mjs +++ b/scripts/cp06-dummy-worker-evidence.mjs @@ -23,25 +23,53 @@ export function dummyWorkerEvidence({ input, paths, original } = dummyWorkerCont pi_install: structuredClone(DUMMY_PI_PROVENANCE), provider: input.provider, model: input.model, - oauth_preflight: { minimum_validity_ms: input.timeout_ms + 300_000, remaining_validity_ms: input.timeout_ms + 600_000, refreshed: false }, + oauth_preflight: { + minimum_validity_ms: input.timeout_ms + 300_000, + remaining_validity_ms: input.timeout_ms + 600_000, + refreshed: false, + }, credential_store: { reads: 1, lists: 0, modify_denials: 0, delete_denials: 0 }, session: { in_memory: true, active_tools: ["read"] }, event_stream: { - sha256: "0".repeat(64), event_count: 3, + sha256: "0".repeat(64), + event_count: 3, read_audit: { - read_count: 1, project_read_count: 1, project_read_paths: [paths.originalPath], - rejected_read_paths: [], other_tool_calls: [], other_tool_completions: [], - completed_successfully: true, exact_original: true, - expected_sha256: hash, returned_sha256: hash, returned_bytes: Buffer.byteLength(original), - tool_call_id: "DUMMY-read", start_event_index: 0, end_event_index: 1, response_event_index: 2, + read_count: 1, + project_read_count: 1, + project_read_paths: [paths.originalPath], + rejected_read_paths: [], + other_tool_calls: [], + other_tool_completions: [], + completed_successfully: true, + exact_original: true, + expected_sha256: hash, + returned_sha256: hash, + returned_bytes: Buffer.byteLength(original), + tool_call_id: "DUMMY-read", + start_event_index: 0, + end_event_index: 1, + response_event_index: 2, }, }, response: { - status: "MISSING_SOURCE", missing_source: "docs/AUTH.md", reads: [paths.originalPath], - corrected_constraints: ["ownership comes only from the authenticated principal", "request-supplied owner fields are ignored"], - evidence_gaps: [{ id: "creation-time-ownership", status: "unverified", missing_checks: ["principal-derived-owner", "request-owner-ignored"] }], broad_scan: false, + status: "MISSING_SOURCE", + missing_source: "docs/AUTH.md", + reads: [paths.originalPath], + corrected_constraints: [ + "ownership comes only from the authenticated principal", + "request-supplied owner fields are ignored", + ], + evidence_gaps: [ + { + id: "creation-time-ownership", + status: "unverified", + missing_checks: ["principal-derived-owner", "request-owner-ignored"], + }, + ], + broad_scan: false, }, - response_sha256: "1".repeat(64), raw_transcript_retained: false, + response_sha256: "1".repeat(64), + raw_transcript_retained: false, }; } diff --git a/scripts/cp06-dummy-worker-hook.mjs b/scripts/cp06-dummy-worker-hook.mjs index 5b642c2..0e4e4af 100644 --- a/scripts/cp06-dummy-worker-hook.mjs +++ b/scripts/cp06-dummy-worker-hook.mjs @@ -126,6 +126,12 @@ sdk.ModelRuntime.create = async function (options) { }; process.on("exit", (exit) => { try { - writeAnchoredFile(`/proc/${process.pid}/fd/${countersFd}`, basename(countersPath), JSON.stringify({ ...counts, exit })); - } finally { closeSync(countersFd); } + writeAnchoredFile( + `/proc/${process.pid}/fd/${countersFd}`, + basename(countersPath), + JSON.stringify({ ...counts, exit }), + ); + } finally { + closeSync(countersFd); + } }); diff --git a/scripts/cp06-guarded-read.mjs b/scripts/cp06-guarded-read.mjs index 1a1b6ed..c4c9542 100644 --- a/scripts/cp06-guarded-read.mjs +++ b/scripts/cp06-guarded-read.mjs @@ -132,10 +132,16 @@ export function readPinnedRegularFile(path) { } export function openPinnedDirectory(path) { - return openWithoutFollowingAncestors(path, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC); + return openWithoutFollowingAncestors( + path, + constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | constants.O_CLOEXEC, + ); } -function openWithoutFollowingAncestors(path, leafFlags = constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC) { +function openWithoutFollowingAncestors( + path, + leafFlags = constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_CLOEXEC, +) { if (typeof path !== "string" || !isAbsolute(path) || resolve(path) !== path) { throw new Error("supplied original path must be absolute and normalized"); } @@ -150,10 +156,7 @@ function openWithoutFollowingAncestors(path, leafFlags = constants.O_RDONLY | co closeSync(parentFd); parentFd = nextFd; } - return openSync( - `/proc/self/fd/${parentFd}/${components.at(-1)}`, - leafFlags, - ); + return openSync(`/proc/self/fd/${parentFd}/${components.at(-1)}`, leafFlags); } finally { closeSync(parentFd); } diff --git a/scripts/cp06-namespace-supervisor.mjs b/scripts/cp06-namespace-supervisor.mjs index ed859f5..2119a1e 100644 --- a/scripts/cp06-namespace-supervisor.mjs +++ b/scripts/cp06-namespace-supervisor.mjs @@ -131,20 +131,27 @@ export async function superviseCredentialChild(options, dependencies = {}) { }); let result; try { - const launch = (stdio) => spawn(command.command, command.args, { - cwd: probe?.cwd ?? root, - encoding: "utf8", - env: childEnvironment(mode, source, target), - timeout: mode === "worker" ? 600_000 : 60_000, - maxBuffer: 1024 * 1024, - stdio, - }); + const launch = (stdio) => + spawn(command.command, command.args, { + cwd: probe?.cwd ?? root, + encoding: "utf8", + env: childEnvironment(mode, source, target), + timeout: mode === "worker" ? 600_000 : 60_000, + maxBuffer: 1024 * 1024, + stdio, + }); if (dependencies.spawn) result = launch(undefined); else { - const paths = mode === "unsafe-probe" ? [syscallProbe] : mode === "hardened-probe" ? [helper, syscallProbe] : [helper, childArgs[0]]; + const paths = + mode === "unsafe-probe" + ? [syscallProbe] + : mode === "hardened-probe" + ? [helper, syscallProbe] + : [helper, childArgs[0]]; const fds = []; try { - for (const path of paths) fds.push(openSync(path, constants.O_RDONLY | constants.O_CLOEXEC)); + for (const path of paths) + fds.push(openSync(path, constants.O_RDONLY | constants.O_CLOEXEC)); result = launch(["pipe", "pipe", "pipe", ...fds]); } finally { for (const fd of fds) closeSync(fd); @@ -463,7 +470,18 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur try { console.log( JSON.stringify( - await superviseCredentialChild({ mode, source, target, helper, syscallProbe, childArgs: childArgs.map((path) => /^\/proc\/self\/fd\/\d+$/u.test(path) ? `/proc/${process.pid}/fd/${path.slice("/proc/self/fd/".length)}` : path) }), + await superviseCredentialChild({ + mode, + source, + target, + helper, + syscallProbe, + childArgs: childArgs.map((path) => + /^\/proc\/self\/fd\/\d+$/u.test(path) + ? `/proc/${process.pid}/fd/${path.slice("/proc/self/fd/".length)}` + : path, + ), + }), ), ); } catch (error) { diff --git a/scripts/cp06-probe-fixture.mjs b/scripts/cp06-probe-fixture.mjs index 41b1575..a690fdb 100644 --- a/scripts/cp06-probe-fixture.mjs +++ b/scripts/cp06-probe-fixture.mjs @@ -239,9 +239,12 @@ export function readAnchoredFile(parentAnchor, name, { expected } = {}) { } const bytes = readFileSync(fd); const current = fstatSync(fd, { bigint: true }); - if (!sameInode(pinned, lstatSync(path, { bigint: true })) || - current.size !== pinned.size || current.mtimeNs !== pinned.mtimeNs || - current.ctimeNs !== pinned.ctimeNs) { + if ( + !sameInode(pinned, lstatSync(path, { bigint: true })) || + current.size !== pinned.size || + current.mtimeNs !== pinned.mtimeNs || + current.ctimeNs !== pinned.ctimeNs + ) { throw new Error("DUMMY proof artifact identity changed"); } return bytes; @@ -256,7 +259,9 @@ export function withArtifactParent(path, action) { const components = match[2].split("/"); const name = components.pop(); assertArtifactName(name); - const parent = components.length ? openAnchoredDirectory(match[1], components.join("/"), { create: false }) : null; + const parent = components.length + ? openAnchoredDirectory(match[1], components.join("/"), { create: false }) + : null; try { return action(parent?.anchor ?? match[1], name); } finally { @@ -269,7 +274,9 @@ export function writeArtifactFile(path, data, options) { } export function writeArtifactEntry(path, data, options) { - return withArtifactParent(path, (anchor, name) => writeAnchoredEntry(anchor, name, data, options)); + return withArtifactParent(path, (anchor, name) => + writeAnchoredEntry(anchor, name, data, options), + ); } export function readArtifactFile(path, encoding, options) { diff --git a/scripts/cp06-worker-evidence.mjs b/scripts/cp06-worker-evidence.mjs index c0a2f71..e0393d2 100644 --- a/scripts/cp06-worker-evidence.mjs +++ b/scripts/cp06-worker-evidence.mjs @@ -9,7 +9,13 @@ export function workerEvidenceContext(inputPath, target, source, { provenance } const input = readWorkerInput(inputPath); const paths = canonicalWorkerPaths(input, target, source); const snapshot = readPinnedRegularFile(paths.originalPath); - return { input, paths, original: snapshot.bytes, originalIdentity: snapshot.identity, provenance }; + return { + input, + paths, + original: snapshot.bytes, + originalIdentity: snapshot.identity, + provenance, + }; } // The supervisor resolves the selected installation itself without executing @@ -28,40 +34,123 @@ export function readWorkerInput(path) { const fd = openSync(path, constants.O_RDONLY | constants.O_CLOEXEC); try { const identity = fstatSync(fd); - if (!identity.isFile() || identity.nlink !== 1 || identity.size < 1 || identity.size > 50 * 1024) throw new Error("CP06_CHILD_OUTPUT_INVALID"); + if ( + !identity.isFile() || + identity.nlink !== 1 || + identity.size < 1 || + identity.size > 50 * 1024 + ) + throw new Error("CP06_CHILD_OUTPUT_INVALID"); return JSON.parse(readFileSync(fd, "utf8")); - } finally { closeSync(fd); } + } finally { + closeSync(fd); + } } return JSON.parse(readPinnedRegularFile(path).bytes.toString("utf8")); } -export function validateWorkerEvidence(child, { input, paths, original, originalIdentity, provenance }) { +export function validateWorkerEvidence( + child, + { input, paths, original, originalIdentity, provenance }, +) { if (originalIdentity) { const current = readPinnedRegularFile(paths.originalPath); - for (const key of ["dev", "ino", "size", "nlink", "mode", "mtimeNs", "ctimeNs"]) requireValue(current.identity[key] === originalIdentity[key]); + for (const key of ["dev", "ino", "size", "nlink", "mode", "mtimeNs", "ctimeNs"]) + requireValue(current.identity[key] === originalIdentity[key]); requireValue(current.bytes.equals(original)); } - exact(child, ["credential_store", "event_stream", "model", "oauth_preflight", "pi_install", "pi_version", "provider", "raw_transcript_retained", "response", "response_sha256", "result", "schema_version", "session"]); - requireValue(child.schema_version === 1 && child.result === "pass" && child.pi_version === "0.85.1" && child.raw_transcript_retained === false); + exact(child, [ + "credential_store", + "event_stream", + "model", + "oauth_preflight", + "pi_install", + "pi_version", + "provider", + "raw_transcript_retained", + "response", + "response_sha256", + "result", + "schema_version", + "session", + ]); + requireValue( + child.schema_version === 1 && + child.result === "pass" && + child.pi_version === "0.85.1" && + child.raw_transcript_retained === false, + ); requireValue(child.provider === input.provider && child.model === input.model); - exact(child.pi_install, ["install_root", "package_root", "package_artifact", "dependency", "executable", "executable_entry", "sdk_entry", "package_sha256", "executable_sha256", "sdk_entry_sha256"]); - for (const key of ["install_root", "package_root", "executable", "executable_entry", "sdk_entry"]) requireValue(typeof child.pi_install[key] === "string" && child.pi_install[key].startsWith("/")); - for (const key of ["package_sha256", "executable_sha256", "sdk_entry_sha256"]) requireValue(digest(child.pi_install[key])); + exact(child.pi_install, [ + "install_root", + "package_root", + "package_artifact", + "dependency", + "executable", + "executable_entry", + "sdk_entry", + "package_sha256", + "executable_sha256", + "sdk_entry_sha256", + ]); + for (const key of ["install_root", "package_root", "executable", "executable_entry", "sdk_entry"]) + requireValue( + typeof child.pi_install[key] === "string" && child.pi_install[key].startsWith("/"), + ); + for (const key of ["package_sha256", "executable_sha256", "sdk_entry_sha256"]) + requireValue(digest(child.pi_install[key])); exact(child.pi_install.package_artifact, ["manifest_sha256", "tarball_integrity"]); - requireValue(digest(child.pi_install.package_artifact.manifest_sha256) && typeof child.pi_install.package_artifact.tarball_integrity === "string" && child.pi_install.package_artifact.tarball_integrity.startsWith("sha512-")); - exact(child.pi_install.dependency, ["entry", "entry_sha256", "manifest_sha256", "name", "package_sha256", "resolution", "root", "tarball_integrity", "version"]); + requireValue( + digest(child.pi_install.package_artifact.manifest_sha256) && + typeof child.pi_install.package_artifact.tarball_integrity === "string" && + child.pi_install.package_artifact.tarball_integrity.startsWith("sha512-"), + ); + exact(child.pi_install.dependency, [ + "entry", + "entry_sha256", + "manifest_sha256", + "name", + "package_sha256", + "resolution", + "root", + "tarball_integrity", + "version", + ]); const dependency = child.pi_install.dependency; - requireValue(dependency.version === "0.85.1" && dependency.name === "@earendil-works/pi-ai" && dependency.resolution === "esm-import-condition"); - for (const key of ["entry_sha256", "manifest_sha256", "package_sha256"]) requireValue(digest(dependency[key])); - for (const key of ["root", "entry"]) requireValue(typeof dependency[key] === "string" && dependency[key].startsWith("/")); - requireValue(typeof dependency.tarball_integrity === "string" && dependency.tarball_integrity.startsWith("sha512-")); + requireValue( + dependency.version === "0.85.1" && + dependency.name === "@earendil-works/pi-ai" && + dependency.resolution === "esm-import-condition", + ); + for (const key of ["entry_sha256", "manifest_sha256", "package_sha256"]) + requireValue(digest(dependency[key])); + for (const key of ["root", "entry"]) + requireValue(typeof dependency[key] === "string" && dependency[key].startsWith("/")); + requireValue( + typeof dependency.tarball_integrity === "string" && + dependency.tarball_integrity.startsWith("sha512-"), + ); requireValue(provenance !== undefined && sameJson(child.pi_install, provenance)); exact(child.oauth_preflight, ["minimum_validity_ms", "remaining_validity_ms", "refreshed"]); - requireValue(child.oauth_preflight.refreshed === false && child.oauth_preflight.minimum_validity_ms === input.timeout_ms + 300_000 && Number.isSafeInteger(child.oauth_preflight.remaining_validity_ms) && child.oauth_preflight.remaining_validity_ms >= child.oauth_preflight.minimum_validity_ms); + requireValue( + child.oauth_preflight.refreshed === false && + child.oauth_preflight.minimum_validity_ms === input.timeout_ms + 300_000 && + Number.isSafeInteger(child.oauth_preflight.remaining_validity_ms) && + child.oauth_preflight.remaining_validity_ms >= child.oauth_preflight.minimum_validity_ms, + ); exact(child.credential_store, ["reads", "lists", "modify_denials", "delete_denials"]); - requireValue(Number.isSafeInteger(child.credential_store.reads) && child.credential_store.reads >= 1 && Number.isSafeInteger(child.credential_store.lists) && child.credential_store.lists >= 0 && child.credential_store.modify_denials === 0 && child.credential_store.delete_denials === 0); + requireValue( + Number.isSafeInteger(child.credential_store.reads) && + child.credential_store.reads >= 1 && + Number.isSafeInteger(child.credential_store.lists) && + child.credential_store.lists >= 0 && + child.credential_store.modify_denials === 0 && + child.credential_store.delete_denials === 0, + ); exact(child.session, ["in_memory", "active_tools"]); - requireValue(child.session.in_memory === true && JSON.stringify(child.session.active_tools) === '["read"]'); + requireValue( + child.session.in_memory === true && JSON.stringify(child.session.active_tools) === '["read"]', + ); exact(child.event_stream, ["sha256", "event_count", "read_audit"]); requireValue(digest(child.event_stream.sha256) && digest(child.response_sha256)); validateReadAudit(child.event_stream.read_audit, { @@ -70,23 +159,41 @@ export function validateWorkerEvidence(child, { input, paths, original, original original, eventCount: child.event_stream.event_count, }); - requireValue(typeof child.event_stream.read_audit.tool_call_id === "string" && child.event_stream.read_audit.tool_call_id.length > 0); - requireValue(auditWorkerOutcome(child.response, { - root: paths.root, - referenceRoot: paths.fixtureRoot, - expectedOriginalPath: paths.originalPath, - }).ok); + requireValue( + typeof child.event_stream.read_audit.tool_call_id === "string" && + child.event_stream.read_audit.tool_call_id.length > 0, + ); + requireValue( + auditWorkerOutcome(child.response, { + root: paths.root, + referenceRoot: paths.fixtureRoot, + expectedOriginalPath: paths.originalPath, + }).ok, + ); return child; } function sameJson(actual, expected) { if (expected === null || typeof expected !== "object") return actual === expected; - if (actual === null || typeof actual !== "object" || Array.isArray(actual) !== Array.isArray(expected)) return false; + if ( + actual === null || + typeof actual !== "object" || + Array.isArray(actual) !== Array.isArray(expected) + ) + return false; const keys = Object.keys(expected).sort(); - return Object.keys(actual).sort().join(",") === keys.join(",") && keys.every((key) => sameJson(actual[key], expected[key])); + return ( + Object.keys(actual).sort().join(",") === keys.join(",") && + keys.every((key) => sameJson(actual[key], expected[key])) + ); } function exact(value, keys) { - requireValue(value !== null && typeof value === "object" && !Array.isArray(value) && Object.keys(value).sort().join(",") === keys.sort().join(",")); + requireValue( + value !== null && + typeof value === "object" && + !Array.isArray(value) && + Object.keys(value).sort().join(",") === keys.sort().join(","), + ); } function requireValue(value) { if (!value) throw new Error("CP06_CHILD_OUTPUT_INVALID"); diff --git a/scripts/replay-cp06-p07.mjs b/scripts/replay-cp06-p07.mjs index 1b6ab9e..f5aac14 100644 --- a/scripts/replay-cp06-p07.mjs +++ b/scripts/replay-cp06-p07.mjs @@ -5,7 +5,12 @@ import { readFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -import { openAnchoredDirectory, openProbeOutput, writeArtifactFile, readArtifactFile } from "./cp06-probe-fixture.mjs"; +import { + openAnchoredDirectory, + openProbeOutput, + writeArtifactFile, + readArtifactFile, +} from "./cp06-probe-fixture.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const outputRoot = resolve(root, process.env.CP06_OUTPUT ?? ".factory/state/cp06-correction"); @@ -136,6 +141,8 @@ function sha256(text) { async function fileHash(path) { return createHash("sha256") - .update(path.startsWith(`/proc/${process.pid}/fd/`) ? readArtifactFile(path) : await readFile(path)) + .update( + path.startsWith(`/proc/${process.pid}/fd/`) ? readArtifactFile(path) : await readFile(path), + ) .digest("hex"); } diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index 3d4c926..aa0b35a 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -5,7 +5,14 @@ import { stat } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -import { openAnchoredDirectory, openProbeOutput, writeArtifactEntry, readArtifactFile, withArtifactParent, removeAnchoredEntry } from "./cp06-probe-fixture.mjs"; +import { + openAnchoredDirectory, + openProbeOutput, + writeArtifactEntry, + readArtifactFile, + withArtifactParent, + removeAnchoredEntry, +} from "./cp06-probe-fixture.mjs"; import { readPinnedRegularFile } from "./cp06-guarded-read.mjs"; const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); @@ -708,7 +715,10 @@ function requireRead(path) { } function requireWrite(path, value, options) { - const written = writeArtifactEntry(path, value, { ...options, expected: preparedArtifacts.get(path) }); + const written = writeArtifactEntry(path, value, { + ...options, + expected: preparedArtifacts.get(path), + }); preparedArtifacts.set(path, written.identity); return written.sha256; } @@ -719,7 +729,9 @@ async function readFile(path, encoding) { } async function writeFile(path, value) { - return requireWrite(path, value, { replace: path === join(fixtureRoot, ".factory/tasks/AUTH-001.yaml") }); + return requireWrite(path, value, { + replace: path === join(fixtureRoot, ".factory/tasks/AUTH-001.yaml"), + }); } async function rm(path) { @@ -731,10 +743,16 @@ function copyFixtureTree(source, destination, logical = destination) { for (const entry of readdirSync(source, { withFileTypes: true })) { if (entry.isDirectory()) { const child = openAnchoredDirectory(destination, entry.name); - try { copyFixtureTree(join(source, entry.name), child.anchor, join(logical, entry.name)); } - finally { child.close(); } + try { + copyFixtureTree(join(source, entry.name), child.anchor, join(logical, entry.name)); + } finally { + child.close(); + } } else if (entry.isFile()) { - const written = writeArtifactEntry(join(destination, entry.name), readPinnedRegularFile(join(source, entry.name)).bytes); + const written = writeArtifactEntry( + join(destination, entry.name), + readPinnedRegularFile(join(source, entry.name)).bytes, + ); preparedArtifacts.set(join(logical, entry.name), written.identity); } else throw new Error("DUMMY fixture contains a nonregular entry"); } diff --git a/test/cp06-pi-dependency.test.mjs b/test/cp06-pi-dependency.test.mjs index 7df343c..f9745ff 100644 --- a/test/cp06-pi-dependency.test.mjs +++ b/test/cp06-pi-dependency.test.mjs @@ -145,8 +145,14 @@ test("supervisor trusted provenance verifies the selected installation without e }).provenance; assert.deepEqual(trusted, reported); assert.equal(existsSync(join(projectRoot, "DUMMY-home")), false); - await writeFile(join(packages, "pi-ai/dist/index.js"), "DUMMY same-version dependency replacement"); - assert.throws(() => trustedWorkerProvenance({ root: projectRoot, environment }), /pinned artifact/); + await writeFile( + join(packages, "pi-ai/dist/index.js"), + "DUMMY same-version dependency replacement", + ); + assert.throws( + () => trustedWorkerProvenance({ root: projectRoot, environment }), + /pinned artifact/, + ); } finally { await rm(projectRoot, { recursive: true, force: true }); } diff --git a/test/cp06-proof-boundary.test.mjs b/test/cp06-proof-boundary.test.mjs index 6d27f03..4b526ba 100644 --- a/test/cp06-proof-boundary.test.mjs +++ b/test/cp06-proof-boundary.test.mjs @@ -159,11 +159,15 @@ test("DUMMY SDK proof outer cleanup removes child leftovers only through the ret mkdirSync(join(proofRoot, "DUMMY-worker", "nested"), { recursive: true }); writeFileSync(join(proofRoot, "DUMMY-worker", "nested", "DUMMY-partial"), "DUMMY"); symlinkSync(external, join(proofRoot, "DUMMY-worker", "DUMMY-escape")); - return { status: null, error: Object.assign(new Error("DUMMY"), { code: "ETIMEDOUT" }) }; + return { + status: null, + error: Object.assign(new Error("DUMMY"), { code: "ETIMEDOUT" }), + }; }, }), (error) => - error.code === "CP06_ISOLATION_UNSUPPORTED" && /CP06_DUMMY_SDK_TIMEOUT/u.test(error.message), + error.code === "CP06_ISOLATION_UNSUPPORTED" && + /CP06_DUMMY_SDK_TIMEOUT/u.test(error.message), ); await assert.rejects(readdir(leftovers[0]), { code: "ENOENT" }); assert.equal(await readFile(sentinel, "utf8"), "DUMMY SENTINEL UNCHANGED"); @@ -245,9 +249,46 @@ test("DUMMY SDK proof consumer rejects incomplete and extra payloads", () => { for (const payload of [ { ...validBase, auth_cases: [] }, { ...validBase, sdk_worker: { ...validBase.sdk_worker, event_count: 2 } }, - { ...validBase, sdk_worker: { ...validBase.sdk_worker, read_audit: { ...validBase.sdk_worker.read_audit, expected_sha256: "a".repeat(64), returned_sha256: "a".repeat(64), returned_bytes: 1 } } }, - { ...validBase, actual_sdk_outcomes: { ...validBase.actual_sdk_outcomes, cases: validBase.actual_sdk_outcomes.cases.map((entry) => entry.scenario === "valid" ? { ...entry, event_count: 2 } : entry) } }, - { ...validBase, actual_sdk_outcomes: { ...validBase.actual_sdk_outcomes, cases: validBase.actual_sdk_outcomes.cases.map((entry) => entry.scenario === "valid" ? { ...entry, read_audit: { ...entry.read_audit, expected_sha256: "a".repeat(64), returned_sha256: "a".repeat(64), returned_bytes: 1 } } : entry) } }, + { + ...validBase, + sdk_worker: { + ...validBase.sdk_worker, + read_audit: { + ...validBase.sdk_worker.read_audit, + expected_sha256: "a".repeat(64), + returned_sha256: "a".repeat(64), + returned_bytes: 1, + }, + }, + }, + { + ...validBase, + actual_sdk_outcomes: { + ...validBase.actual_sdk_outcomes, + cases: validBase.actual_sdk_outcomes.cases.map((entry) => + entry.scenario === "valid" ? { ...entry, event_count: 2 } : entry, + ), + }, + }, + { + ...validBase, + actual_sdk_outcomes: { + ...validBase.actual_sdk_outcomes, + cases: validBase.actual_sdk_outcomes.cases.map((entry) => + entry.scenario === "valid" + ? { + ...entry, + read_audit: { + ...entry.read_audit, + expected_sha256: "a".repeat(64), + returned_sha256: "a".repeat(64), + returned_bytes: 1, + }, + } + : entry, + ), + }, + }, { ...validBase, auth_cases: [...validBase.auth_cases, authCase("expired", "blocked", 0)] }, { ...validBase, extra: "DUMMY-SECRET" }, { ...validBase, sdk_dependency: { ...validBase.sdk_dependency, version: "0.99.2" } }, @@ -477,9 +518,15 @@ function readAudit(toolCallId, originalPath) { other_tool_completions: [], completed_successfully: true, exact_original: true, - expected_sha256: createHash("sha256").update(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL).digest("hex"), - returned_sha256: createHash("sha256").update(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL).digest("hex"), - returned_bytes: Buffer.byteLength(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL), + expected_sha256: createHash("sha256") + .update(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL) + .digest("hex"), + returned_sha256: createHash("sha256") + .update(toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL) + .digest("hex"), + returned_bytes: Buffer.byteLength( + toolCallId === "DUMMY-read" ? DUMMY_SDK_ORIGINAL : DUMMY_OUTCOME_ORIGINAL, + ), tool_call_id: toolCallId, start_event_index: 12, end_event_index: 13, diff --git a/test/cp06-review-boundaries.test.mjs b/test/cp06-review-boundaries.test.mjs index 52ff037..48391e1 100644 --- a/test/cp06-review-boundaries.test.mjs +++ b/test/cp06-review-boundaries.test.mjs @@ -1,12 +1,35 @@ import { strict as assert } from "node:assert"; import { spawnSync } from "node:child_process"; import { chmodSync, closeSync, constants, openSync, readdirSync, writeFileSync } from "node:fs"; -import { mkdir, mkdtemp, readdir, readFile, rename, rm, symlink, writeFile } from "node:fs/promises"; +import { + mkdir, + mkdtemp, + readdir, + readFile, + rename, + rm, + symlink, + writeFile, +} from "node:fs/promises"; import { join, resolve } from "node:path"; import test from "node:test"; -import { assertHardenedProof, assertUnsafeControl, runIsolationProbe } from "../scripts/cp06-auth-security.mjs"; -import { createProbeFixture, openProbeOutput, openAnchoredDirectory, readArtifactFile, writeArtifactEntry, writeArtifactFile } from "../scripts/cp06-probe-fixture.mjs"; -import { superviseCredentialChild, supervisorFailureRecord } from "../scripts/cp06-namespace-supervisor.mjs"; +import { + assertHardenedProof, + assertUnsafeControl, + runIsolationProbe, +} from "../scripts/cp06-auth-security.mjs"; +import { + createProbeFixture, + openProbeOutput, + openAnchoredDirectory, + readArtifactFile, + writeArtifactEntry, + writeArtifactFile, +} from "../scripts/cp06-probe-fixture.mjs"; +import { + superviseCredentialChild, + supervisorFailureRecord, +} from "../scripts/cp06-namespace-supervisor.mjs"; import { dummyWorkerContext, dummyWorkerEvidence } from "../scripts/cp06-dummy-worker-evidence.mjs"; import { withCompiledHelper } from "./fixtures/cp06-compiled-helper.mjs"; import { openPinnedDirectory } from "../scripts/cp06-guarded-read.mjs"; @@ -22,17 +45,48 @@ test("real hardened probe executes retained read-only descriptors and inherited const proof = await runIsolationProbe({ ...options, mode: "hardened-probe" }); assertHardenedProof(proof); const missing = structuredClone(proof); - missing.child.direct_namespace_syscalls.stdout = missing.child.direct_namespace_syscalls.stdout.split("\n").filter((line) => !line.startsWith("inherited_ptrace_attach=")).join("\n"); + missing.child.direct_namespace_syscalls.stdout = + missing.child.direct_namespace_syscalls.stdout + .split("\n") + .filter((line) => !line.startsWith("inherited_ptrace_attach=")) + .join("\n"); assert.throws(() => assertHardenedProof(missing)); const probeFd = openSync(binaries.syscallProbe, constants.O_RDONLY | constants.O_CLOEXEC); try { - const closed = spawnSync("unshare", ["--user", "--map-root-user", "--mount", "--propagation", "private", "--", "setpriv", "--no-new-privs", "--bounding-set=-all", "--inh-caps=-all", "--ambient-caps=-all", "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", "--", "/proc/self/fd/3", "/proc/self/fd/4", "/DUMMY-no-mount"], { - stdio: ["ignore", "pipe", "pipe", helperFd, probeFd], encoding: "utf8", timeout: 20_000, - }); + const closed = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + "setpriv", + "--no-new-privs", + "--bounding-set=-all", + "--inh-caps=-all", + "--ambient-caps=-all", + "--securebits=+noroot,+noroot_locked,+no_setuid_fixup,+no_setuid_fixup_locked", + "--", + "/proc/self/fd/3", + "/proc/self/fd/4", + "/DUMMY-no-mount", + ], + { + stdio: ["ignore", "pipe", "pipe", helperFd, probeFd], + encoding: "utf8", + timeout: 20_000, + }, + ); assert.equal(closed.status, 70); assert.equal(closed.stdout, ""); - } finally { closeSync(probeFd); } - } finally { await rm(join(state, "DUMMY-review-kernel"), { recursive: true, force: true }); } + } finally { + closeSync(probeFd); + } + } finally { + await rm(join(state, "DUMMY-review-kernel"), { recursive: true, force: true }); + } }); }); @@ -44,35 +98,98 @@ test("full semantic contracts reject forged nested evidence inside independently { ...good, response: { ...good.response, evidence_gaps: [] } }, { ...good, event_stream: { ...good.event_stream, extra: "DUMMY-SECRET" } }, { ...good, event_stream: { ...good.event_stream, event_count: 2 } }, - { ...good, event_stream: { ...good.event_stream, read_audit: { ...good.event_stream.read_audit, expected_sha256: "a".repeat(64), returned_sha256: "a".repeat(64), returned_bytes: 1 } } }, + { + ...good, + event_stream: { + ...good.event_stream, + read_audit: { + ...good.event_stream.read_audit, + expected_sha256: "a".repeat(64), + returned_sha256: "a".repeat(64), + returned_bytes: 1, + }, + }, + }, { ...good, pi_install: { ...good.pi_install, extra: "DUMMY-SECRET" } }, - { ...good, pi_install: { ...good.pi_install, package_root: "/usr/lib/node_modules/@earendil-works/pi-coding-agent" } }, + { + ...good, + pi_install: { + ...good.pi_install, + package_root: "/usr/lib/node_modules/@earendil-works/pi-coding-agent", + }, + }, { ...good, pi_install: { ...good.pi_install, sdk_entry_sha256: "a".repeat(64) } }, - { ...good, pi_install: { ...good.pi_install, package_artifact: { ...good.pi_install.package_artifact, tarball_integrity: "sha512-DUMMY-SECRET" } } }, - { ...good, pi_install: { ...good.pi_install, dependency: { ...good.pi_install.dependency, root: "/usr/lib/node_modules/@earendil-works/pi-ai" } } }, - { ...good, pi_install: { ...good.pi_install, dependency: { ...good.pi_install.dependency, tarball_integrity: "sha512-DUMMY-SECRET" } } }, + { + ...good, + pi_install: { + ...good.pi_install, + package_artifact: { + ...good.pi_install.package_artifact, + tarball_integrity: "sha512-DUMMY-SECRET", + }, + }, + }, + { + ...good, + pi_install: { + ...good.pi_install, + dependency: { + ...good.pi_install.dependency, + root: "/usr/lib/node_modules/@earendil-works/pi-ai", + }, + }, + }, + { + ...good, + pi_install: { + ...good.pi_install, + dependency: { ...good.pi_install.dependency, tarball_integrity: "sha512-DUMMY-SECRET" }, + }, + }, ]; for (const child of [good, ...variants]) { for (const cleanupFails of [false, true]) { let cleaned = 0; let caught; try { - await superviseCredentialChild({ mode: "worker", source: "/DUMMY-source", target: "/DUMMY-target", helper: "/DUMMY-helper", syscallProbe: "/DUMMY-probe" }, { - workerContext: context, - mount: async () => ({ mountIds: { source: "1", target: "2" }, parentMounts: ["DUMMY"], verifyIntegrity: async () => true, cleanup: async () => { - await new Promise((done) => setTimeout(done, 1)); cleaned++; - if (cleanupFails) throw new Error("DUMMY-SECRET"); - } }), - spawn: () => ({ status: 0, stdout: JSON.stringify(child) }), - }); - } catch (error) { caught = error; } + await superviseCredentialChild( + { + mode: "worker", + source: "/DUMMY-source", + target: "/DUMMY-target", + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + }, + { + workerContext: context, + mount: async () => ({ + mountIds: { source: "1", target: "2" }, + parentMounts: ["DUMMY"], + verifyIntegrity: async () => true, + cleanup: async () => { + await new Promise((done) => setTimeout(done, 1)); + cleaned++; + if (cleanupFails) throw new Error("DUMMY-SECRET"); + }, + }), + spawn: () => ({ status: 0, stdout: JSON.stringify(child) }), + }, + ); + } catch (error) { + caught = error; + } assert.equal(cleaned, 1); if (child === good && !cleanupFails) assert.equal(caught, undefined); else { const record = supervisorFailureRecord(caught); assert.equal(JSON.stringify(record).includes("DUMMY-SECRET"), false); - const codes = record.causes.length ? record.causes.map((cause) => cause.code) : [record.code]; - assert.deepEqual(codes, [...(child === good ? [] : ["CP06_CHILD_OUTPUT_INVALID"]), ...(cleanupFails ? ["CP06_CREDENTIAL_CLEANUP_FAILED"] : [])]); + const codes = record.causes.length + ? record.causes.map((cause) => cause.code) + : [record.code]; + assert.deepEqual(codes, [ + ...(child === good ? [] : ["CP06_CHILD_OUTPUT_INVALID"]), + ...(cleanupFails ? ["CP06_CREDENTIAL_CLEANUP_FAILED"] : []), + ]); } } } @@ -83,11 +200,30 @@ test("worker provenance without an independently trusted expectation is rejected assert.ok(provenance); let cleaned = 0; await assert.rejects( - superviseCredentialChild({ mode: "worker", source: "/DUMMY-source", target: "/DUMMY-target", helper: "/DUMMY-helper", syscallProbe: "/DUMMY-probe" }, { - workerContext: untrusted, - mount: async () => ({ mountIds: { source: "1", target: "2" }, parentMounts: ["DUMMY"], verifyIntegrity: async () => true, cleanup: async () => { cleaned++; } }), - spawn: () => ({ status: 0, stdout: JSON.stringify(dummyWorkerEvidence(dummyWorkerContext())) }), - }), + superviseCredentialChild( + { + mode: "worker", + source: "/DUMMY-source", + target: "/DUMMY-target", + helper: "/DUMMY-helper", + syscallProbe: "/DUMMY-probe", + }, + { + workerContext: untrusted, + mount: async () => ({ + mountIds: { source: "1", target: "2" }, + parentMounts: ["DUMMY"], + verifyIntegrity: async () => true, + cleanup: async () => { + cleaned++; + }, + }), + spawn: () => ({ + status: 0, + stdout: JSON.stringify(dummyWorkerEvidence(dummyWorkerContext())), + }), + }, + ), { code: "CP06_CHILD_OUTPUT_INVALID" }, ); assert.equal(cleaned, 1); @@ -101,15 +237,36 @@ test("artifact replacement and reads are bound to the prepared inode", async () const path = join(child.anchor, "DUMMY-task.yaml"); const prepared = writeArtifactEntry(path, "DUMMY prepared"); assert.equal(readArtifactFile(path, "utf8", { expected: prepared.identity }), "DUMMY prepared"); - assert.equal(writeArtifactEntry(path, "DUMMY replaced", { replace: true, expected: prepared.identity }).identity.ino, prepared.identity.ino); - assert.throws(() => writeArtifactFile(path, "DUMMY unbound", { replace: true }), /identity changed/); + assert.equal( + writeArtifactEntry(path, "DUMMY replaced", { replace: true, expected: prepared.identity }) + .identity.ino, + prepared.identity.ino, + ); + assert.throws( + () => writeArtifactFile(path, "DUMMY unbound", { replace: true }), + /identity changed/, + ); await writeFile(join(outputRoot, "raw", "DUMMY-substitute"), "DUMMY SUBSTITUTE UNCHANGED"); - await rename(join(outputRoot, "raw", "DUMMY-substitute"), join(outputRoot, "raw", "DUMMY-task.yaml")); - assert.throws(() => writeArtifactFile(path, "DUMMY write", { replace: true, expected: prepared.identity }), /identity changed/); - assert.throws(() => readArtifactFile(path, "utf8", { expected: prepared.identity }), /identity changed/); - assert.equal(await readFile(join(outputRoot, "raw", "DUMMY-task.yaml"), "utf8"), "DUMMY SUBSTITUTE UNCHANGED"); + await rename( + join(outputRoot, "raw", "DUMMY-substitute"), + join(outputRoot, "raw", "DUMMY-task.yaml"), + ); + assert.throws( + () => writeArtifactFile(path, "DUMMY write", { replace: true, expected: prepared.identity }), + /identity changed/, + ); + assert.throws( + () => readArtifactFile(path, "utf8", { expected: prepared.identity }), + /identity changed/, + ); + assert.equal( + await readFile(join(outputRoot, "raw", "DUMMY-task.yaml"), "utf8"), + "DUMMY SUBSTITUTE UNCHANGED", + ); } finally { - child.close(); output.close(); await rm(outputRoot, { recursive: true, force: true }); + child.close(); + output.close(); + await rm(outputRoot, { recursive: true, force: true }); } }); @@ -118,13 +275,22 @@ test("ten-pack refuses a prepared contract replaced by another regular inode", a const directoryRoot = await mkdtemp(join(state, "DUMMY-runner-")); const contract = join(directoryRoot, "ten-pack/repo/.factory/tasks/AUTH-001.yaml"); const hook = join(directoryRoot, "DUMMY-hook.mjs"); - await writeFile(hook, `import fs from 'node:fs'; import {syncBuiltinESMExports} from 'node:module'; const real = fs.openSync; let done = false; fs.openSync = (path, ...rest) => { if (!done && String(path).endsWith('/AUTH-before.yaml')) { done = true; fs.writeFileSync(${JSON.stringify(`${contract}.DUMMY`)}, 'DUMMY SUBSTITUTE UNCHANGED'); fs.renameSync(${JSON.stringify(`${contract}.DUMMY`)}, ${JSON.stringify(contract)}); } return real(path, ...rest); }; syncBuiltinESMExports();`); + await writeFile( + hook, + `import fs from 'node:fs'; import {syncBuiltinESMExports} from 'node:module'; const real = fs.openSync; let done = false; fs.openSync = (path, ...rest) => { if (!done && String(path).endsWith('/AUTH-before.yaml')) { done = true; fs.writeFileSync(${JSON.stringify(`${contract}.DUMMY`)}, 'DUMMY SUBSTITUTE UNCHANGED'); fs.renameSync(${JSON.stringify(`${contract}.DUMMY`)}, ${JSON.stringify(contract)}); } return real(path, ...rest); }; syncBuiltinESMExports();`, + ); try { - const result = spawnSync(process.execPath, ["--import", hook, resolve("scripts/replay-cp06-ten-pack.mjs")], { env: { ...process.env, CP06_OUTPUT: directoryRoot }, encoding: "utf8", timeout: 30_000 }); + const result = spawnSync( + process.execPath, + ["--import", hook, resolve("scripts/replay-cp06-ten-pack.mjs")], + { env: { ...process.env, CP06_OUTPUT: directoryRoot }, encoding: "utf8", timeout: 30_000 }, + ); assert.notEqual(result.status, 0); assert.match(result.stderr, /DUMMY proof artifact identity changed/); assert.equal(await readFile(contract, "utf8"), "DUMMY SUBSTITUTE UNCHANGED"); - } finally { await rm(directoryRoot, { recursive: true, force: true }); } + } finally { + await rm(directoryRoot, { recursive: true, force: true }); + } }); test("DUMMY fixture cleanup closes every descriptor and removes through its anchor despite failures", async () => { @@ -140,7 +306,12 @@ test("DUMMY fixture cleanup closes every descriptor and removes through its anch if (scenario !== "alias-umount-fails") writeFileSync(join(path, "source"), "DUMMY"); } else if (command === "umount") { umounts.push(path.split("/").at(-1)); - if (scenario === "alias-umount-fails" ? path.endsWith("/DUMMY-alias") : scenario === "setup-and-umount-fail") return { status: 32 }; + if ( + scenario === "alias-umount-fails" + ? path.endsWith("/DUMMY-alias") + : scenario === "setup-and-umount-fail" + ) + return { status: 32 }; } return { status: 0 }; }; @@ -148,17 +319,30 @@ test("DUMMY fixture cleanup closes every descriptor and removes through its anch try { const fixture = await createProbeFixture({ root: process.cwd(), outputRoot, spawn }); await fixture.cleanup(); - } catch (error) { caught = error; } + } catch (error) { + caught = error; + } assert.equal(openDescriptors(), before, scenario); - assert.deepEqual(umounts.map((name) => name.replace(/^DUMMY-probe-.*/u, "DUMMY-probe")), scenario === "alias-umount-fails" ? ["DUMMY-alias", "DUMMY-probe"] : ["DUMMY-probe"]); - const remaining = (await readdir(outputRoot)).filter((name) => name.startsWith("DUMMY-probe-")); + assert.deepEqual( + umounts.map((name) => name.replace(/^DUMMY-probe-.*/u, "DUMMY-probe")), + scenario === "alias-umount-fails" ? ["DUMMY-alias", "DUMMY-probe"] : ["DUMMY-probe"], + ); + const remaining = (await readdir(outputRoot)).filter((name) => + name.startsWith("DUMMY-probe-"), + ); if (scenario === "alias-umount-fails") { assert.ok(caught instanceof AggregateError); - assert.deepEqual(caught.errors.map((error) => error.message), ["private DUMMY fixture mount unavailable"]); + assert.deepEqual( + caught.errors.map((error) => error.message), + ["private DUMMY fixture mount unavailable"], + ); assert.deepEqual(remaining, []); } else if (scenario === "setup-and-umount-fail") { assert.ok(caught instanceof AggregateError); - assert.deepEqual(caught.errors.map((error) => error.code ?? error.message), ["EEXIST", "private DUMMY fixture mount unavailable"]); + assert.deepEqual( + caught.errors.map((error) => error.code ?? error.message), + ["EEXIST", "private DUMMY fixture mount unavailable"], + ); assert.equal(remaining.length, 1); await rm(join(outputRoot, remaining[0]), { recursive: true }); } else { @@ -166,43 +350,97 @@ test("DUMMY fixture cleanup closes every descriptor and removes through its anch assert.deepEqual(remaining, []); } } - } finally { await rm(resolve(state, `DUMMY-fixture-${process.pid}`), { recursive: true, force: true }); } + } finally { + await rm(resolve(state, `DUMMY-fixture-${process.pid}`), { recursive: true, force: true }); + } }); test("supervisor CLI serializes fixture setup and independent fixture cleanup causes", async () => { - const locate = (command) => spawnSync("sh", ["-c", `command -v ${command}`], { encoding: "utf8" }).stdout.trim(); + const locate = (command) => + spawnSync("sh", ["-c", `command -v ${command}`], { encoding: "utf8" }).stdout.trim(); const [realMount, realUmount] = [locate("mount"), locate("umount")]; const secret = "DUMMY-SECRET-FIXTURE-MARKER"; const base = resolve(state, `DUMMY-fixture-cli-${process.pid}`); const shims = join(base, "shims"); const outputRoot = join(base, "auth-security"); - const setup = { stage: "setup", code: "CP06_ISOLATION_SETUP_FAILED", exit_code: 73, description: "credential namespace setup failed" }; - const cleanup = { stage: "cleanup", code: "CP06_CREDENTIAL_CLEANUP_FAILED", exit_code: 74, description: "credential mount cleanup failed" }; + const setup = { + stage: "setup", + code: "CP06_ISOLATION_SETUP_FAILED", + exit_code: 73, + description: "credential namespace setup failed", + }; + const cleanup = { + stage: "cleanup", + code: "CP06_CREDENTIAL_CLEANUP_FAILED", + exit_code: 74, + description: "credential mount cleanup failed", + }; try { for (const umountFails of [true, false]) { await rm(base, { recursive: true, force: true }); await mkdir(shims, { recursive: true }); for (const [name, body] of [ - ["mount", `"${realMount}" "$@" || exit $?\ncase "$last" in */DUMMY-probe-*) : > "$last/source" ;; esac`], - ["umount", `case "$last" in */DUMMY-probe-*) ${umountFails ? `printf '%s' '${secret}' >&2; exit 32` : ":"} ;; esac\nexec "${realUmount}" "$@"`], + [ + "mount", + `"${realMount}" "$@" || exit $?\ncase "$last" in */DUMMY-probe-*) : > "$last/source" ;; esac`, + ], + [ + "umount", + `case "$last" in */DUMMY-probe-*) ${umountFails ? `printf '%s' '${secret}' >&2; exit 32` : ":"} ;; esac\nexec "${realUmount}" "$@"`, + ], ]) { - writeFileSync(join(shims, name), `#!/bin/sh\nfor argument do last="$argument"; done\n${body}\n`); + writeFileSync( + join(shims, name), + `#!/bin/sh\nfor argument do last="$argument"; done\n${body}\n`, + ); chmodSync(join(shims, name), 0o700); } - const result = spawnSync("unshare", ["--user", "--map-root-user", "--mount", "--propagation", "private", "--", process.execPath, "--experimental-import-meta-resolve", resolve("scripts/cp06-namespace-supervisor.mjs"), "hardened-probe", outputRoot, "-", "/DUMMY-helper", "/DUMMY-probe"], { - encoding: "utf8", timeout: 30_000, env: { ...process.env, PATH: `${shims}:${process.env.PATH}` }, - }); + const result = spawnSync( + "unshare", + [ + "--user", + "--map-root-user", + "--mount", + "--propagation", + "private", + "--", + process.execPath, + "--experimental-import-meta-resolve", + resolve("scripts/cp06-namespace-supervisor.mjs"), + "hardened-probe", + outputRoot, + "-", + "/DUMMY-helper", + "/DUMMY-probe", + ], + { + encoding: "utf8", + timeout: 30_000, + env: { ...process.env, PATH: `${shims}:${process.env.PATH}` }, + }, + ); const record = umountFails - ? { schema_version: 1, stage: "cleanup", code: "CP06_CLEANUP_FAILED", exit_code: 74, description: "credential namespace cleanup failed", causes: [setup, cleanup] } + ? { + schema_version: 1, + stage: "cleanup", + code: "CP06_CLEANUP_FAILED", + exit_code: 74, + description: "credential namespace cleanup failed", + causes: [setup, cleanup], + } : { schema_version: 1, ...setup, causes: [] }; assert.equal(result.status, record.exit_code, result.stderr); assert.equal(result.stdout, ""); assert.equal(result.stderr, `${JSON.stringify(record)}\n`); assert.equal(`${result.stdout}${result.stderr}`.includes(secret), false); - const remaining = (await readdir(outputRoot)).filter((name) => name.startsWith("DUMMY-probe-")); + const remaining = (await readdir(outputRoot)).filter((name) => + name.startsWith("DUMMY-probe-"), + ); assert.equal(remaining.length, umountFails ? 1 : 0); } - } finally { await rm(base, { recursive: true, force: true }); } + } finally { + await rm(base, { recursive: true, force: true }); + } }); test("artifact writes, replacements, copies and reads reject redirected leaves and intermediate directories", async () => { @@ -221,30 +459,48 @@ test("artifact writes, replacements, copies and reads reject redirected leaves a assert.equal(await readFile(sentinel, "utf8"), "DUMMY SENTINEL UNCHANGED"); } await symlink(outputRoot, join(outputRoot, "raw", "redirected")); - assert.throws(() => writeArtifactFile(join(child.anchor, "redirected", "DUMMY-sentinel"), "DUMMY write", { replace: true })); + assert.throws(() => + writeArtifactFile(join(child.anchor, "redirected", "DUMMY-sentinel"), "DUMMY write", { + replace: true, + }), + ); assert.throws(() => openPinnedDirectory(join(outputRoot, "raw", "redirected", "raw"))); const source = join(child.anchor, "DUMMY-original.txt"); writeArtifactFile(source, "DUMMY bytes"); writeArtifactFile(join(child.anchor, "DUMMY-copy.txt"), readArtifactFile(source)); assert.equal(readArtifactFile(join(child.anchor, "DUMMY-copy.txt"), "utf8"), "DUMMY bytes"); } finally { - child.close(); output.close(); await rm(outputRoot, { recursive: true, force: true }); + child.close(); + output.close(); + await rm(outputRoot, { recursive: true, force: true }); } }); -for (const [runner, directory, rawName] of [["replay-cp06-ten-pack.mjs", "ten-pack", "git-init.txt"], ["replay-cp06-p07.mjs", "p07", "build.txt"]]) { +for (const [runner, directory, rawName] of [ + ["replay-cp06-ten-pack.mjs", "ten-pack", "git-init.txt"], + ["replay-cp06-p07.mjs", "p07", "build.txt"], +]) { test(`${runner} refuses a substituted raw artifact at its executable boundary`, async () => { await mkdir(state, { recursive: true }); const directoryRoot = await mkdtemp(join(state, "DUMMY-runner-")); const sentinel = join(directoryRoot, "DUMMY-sentinel"); const hook = join(directoryRoot, "DUMMY-hook.mjs"); await writeFile(sentinel, "DUMMY SENTINEL UNCHANGED"); - await writeFile(hook, `import child from 'node:child_process'; import {symlinkSync} from 'node:fs'; import {syncBuiltinESMExports} from 'node:module'; const real = child.spawnSync; child.spawnSync = (command,args,options) => { if (${JSON.stringify(directory)} === 'ten-pack' ? command === 'git' && args[0] === 'init' : command === 'npm') { symlinkSync(${JSON.stringify(sentinel)}, ${JSON.stringify(join(directoryRoot, directory, "raw", rawName))}); return {status:0, stdout:'DUMMY', stderr:''}; } return real(command,args,options); }; syncBuiltinESMExports();`); + await writeFile( + hook, + `import child from 'node:child_process'; import {symlinkSync} from 'node:fs'; import {syncBuiltinESMExports} from 'node:module'; const real = child.spawnSync; child.spawnSync = (command,args,options) => { if (${JSON.stringify(directory)} === 'ten-pack' ? command === 'git' && args[0] === 'init' : command === 'npm') { symlinkSync(${JSON.stringify(sentinel)}, ${JSON.stringify(join(directoryRoot, directory, "raw", rawName))}); return {status:0, stdout:'DUMMY', stderr:''}; } return real(command,args,options); }; syncBuiltinESMExports();`, + ); try { - const result = spawnSync(process.execPath, ["--import", hook, resolve("scripts", runner)], { env: { ...process.env, CP06_OUTPUT: directoryRoot }, encoding: "utf8", timeout: 30_000 }); + const result = spawnSync(process.execPath, ["--import", hook, resolve("scripts", runner)], { + env: { ...process.env, CP06_OUTPUT: directoryRoot }, + encoding: "utf8", + timeout: 30_000, + }); assert.notEqual(result.status, 0); assert.match(result.stderr, /EEXIST/); assert.equal(await readFile(sentinel, "utf8"), "DUMMY SENTINEL UNCHANGED"); - } finally { await rm(directoryRoot, { recursive: true, force: true }); } + } finally { + await rm(directoryRoot, { recursive: true, force: true }); + } }); } diff --git a/test/cp06-worker-lifecycle.test.mjs b/test/cp06-worker-lifecycle.test.mjs index 140c8df..8c420a8 100644 --- a/test/cp06-worker-lifecycle.test.mjs +++ b/test/cp06-worker-lifecycle.test.mjs @@ -509,12 +509,20 @@ test("supervisor executable serializes compound worker failures as typed causes" await writeFile(join(directory, "docs/AUTH.md"), DUMMY_OUTCOME_ORIGINAL); await writeFile(join(directory, "DUMMY-contract.yaml"), "DUMMY contract"); await writeFile(join(directory, "DUMMY-pack.md"), "DUMMY pack"); - await writeFile(join(directory, "DUMMY-input.json"), JSON.stringify({ - schema_version: 1, root: directory, fixture_root: directory, - original_path: join(directory, "docs/AUTH.md"), - contract_path: join(directory, "DUMMY-contract.yaml"), pack_path: join(directory, "DUMMY-pack.md"), - provider: "openai-codex", model: "DUMMY-model", timeout_ms: 1_000, - })); + await writeFile( + join(directory, "DUMMY-input.json"), + JSON.stringify({ + schema_version: 1, + root: directory, + fixture_root: directory, + original_path: join(directory, "docs/AUTH.md"), + contract_path: join(directory, "DUMMY-contract.yaml"), + pack_path: join(directory, "DUMMY-pack.md"), + provider: "openai-codex", + model: "DUMMY-model", + timeout_ms: 1_000, + }), + ); try { const result = spawnSync( "unshare", @@ -595,7 +603,12 @@ test("supervisor executable resolves the selected installation without DUMMY tru causes: [], }; const cases = [ - { name: "installation-missing", scenario: "dummy-provenance-untrusted", install: {}, record: setup }, + { + name: "installation-missing", + scenario: "dummy-provenance-untrusted", + install: {}, + record: setup, + }, ...(installed ? Object.entries(DUMMY_CLI_INSTALLED_SCENARIOS).map(([scenario, { record }]) => ({ name: scenario, @@ -609,7 +622,9 @@ test("supervisor executable resolves the selected installation without DUMMY tru const probeFd = openSync(binaries.syscallProbe, constants.O_RDONLY | constants.O_CLOEXEC); try { for (const { name, scenario, install: selected, record } of cases) { - const directory = realpathSync(await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-installed-"))); + const directory = realpathSync( + await mkdtemp(join(tmpdir(), "factory-cp06-DUMMY-installed-")), + ); const shims = join(directory, "shims"); const source = join(directory, "credential", "DUMMY-source.json"); const sourceBytes = '{"DUMMY":"credential fixture only"}'; @@ -620,12 +635,20 @@ test("supervisor executable resolves the selected installation without DUMMY tru await writeFile(join(directory, "docs/AUTH.md"), DUMMY_OUTCOME_ORIGINAL); await writeFile(join(directory, "DUMMY-contract.yaml"), "DUMMY contract"); await writeFile(join(directory, "DUMMY-pack.md"), "DUMMY pack"); - await writeFile(join(directory, "DUMMY-input.json"), JSON.stringify({ - schema_version: 1, root: directory, fixture_root: directory, - original_path: join(directory, "docs/AUTH.md"), - contract_path: join(directory, "DUMMY-contract.yaml"), pack_path: join(directory, "DUMMY-pack.md"), - provider: "openai-codex", model: "DUMMY-model", timeout_ms: 1_000, - })); + await writeFile( + join(directory, "DUMMY-input.json"), + JSON.stringify({ + schema_version: 1, + root: directory, + fixture_root: directory, + original_path: join(directory, "docs/AUTH.md"), + contract_path: join(directory, "DUMMY-contract.yaml"), + pack_path: join(directory, "DUMMY-pack.md"), + provider: "openai-codex", + model: "DUMMY-model", + timeout_ms: 1_000, + }), + ); const environment = { ...process.env, ...selected, HOME: directory }; delete environment.CP06_DUMMY_TRUSTED_PROVENANCE; if (selected.CP06_PI_PACKAGE_ROOT === undefined) { diff --git a/test/fixtures/cp06-credential-mount.mjs b/test/fixtures/cp06-credential-mount.mjs index 9316e2e..723fecd 100644 --- a/test/fixtures/cp06-credential-mount.mjs +++ b/test/fixtures/cp06-credential-mount.mjs @@ -110,7 +110,8 @@ try { ? "no-follow-refused" : error.message; } -if (scenario.endsWith("-ancestor-symlink")) result.external_entries = (await readdir(external)).sort(); +if (scenario.endsWith("-ancestor-symlink")) + result.external_entries = (await readdir(external)).sort(); markerReads.push(await readFile(externalLeaf, "utf8")); result.marker_unchanged = markerReads.every((value) => value === marker); const originalSource = From b9bfceb2164487dc5e2bc7a778bc1df86c4bfd18 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 6 Oct 2026 06:03:56 +0000 Subject: [PATCH 43/45] no-mistakes: apply CI fixes --- .github/workflows/ci.yml | 7 +++++++ docs/probes/CP-06.md | 2 +- scripts/replay-cp06-ten-pack.mjs | 5 +++-- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 239a63e..49c6933 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -18,4 +18,11 @@ jobs: - run: npm run format:check - run: npm run lint - run: npm run typecheck + # Ubuntu 24.04 runners restrict unprivileged user namespaces through AppArmor. + # CP-06 isolation tests need real disposable DUMMY namespaces and have no fallback. + - name: Allow unprivileged user namespaces + run: | + if [ -e /proc/sys/kernel/apparmor_restrict_unprivileged_userns ]; then + sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 + fi - run: npm test diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 652d3c3..0b80196 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -23,7 +23,7 @@ The original PR10 CLI worker is superseded by a bounded Pi 0.85.1 public-SDK pat The historical 2026-10-05 WSL2 mount identity failure belonged to the util-linux path handoff. Preserved head `fdc8321` replaces credential bind/remount commands with native `open_tree`, `mount_setattr` and `move_mount`, retaining the source inode and an identity-checked `O_PATH | O_NOFOLLOW` destination. Leaf and parent substitution regressions retain outside DUMMY sentinels unchanged. The post-mount identity check is unchanged. At that head mount setup passed, but preflight still failed with "direct syscall probe failed to execute": the zero-capability adversary could not reopen the supervisor's descriptor and the helper closed inherited descriptors. The current correction supplies bounded read-only descriptors through each actual launch instead. Final-head full SDK/security, worker-cleanup and genuine semantic acceptance must still be run by the outer executor; this source correction alone does not close `CP06-KERNEL-PROOF-INCOMPLETE` as an acceptance blocker. -Source and target remain distinct `ro,nosuid,nodev,noexec` mount instances of one pinned inode, with containing directories separately read-only inside the private namespace and writable DUMMY scratch. Host namespace and permissions are unchanged. Support is Linux x86-64, including WSL2 Linux guests with the required kernel primitives; unsupported namespaces, mount APIs, libseccomp, compilation, SDK versions or cleanup fail closed, without fallback. This is not Windows host filesystem/credential coverage; Windows and macOS are not claimed. +Source and target remain distinct `ro,nosuid,nodev,noexec` mount instances of one pinned inode, with containing directories separately read-only inside the private namespace and writable DUMMY scratch. Host namespace and permissions are unchanged. Support is Linux x86-64, including WSL2 Linux guests with the required kernel primitives; unsupported namespaces, mount APIs, libseccomp, compilation, SDK versions or cleanup fail closed, without fallback. This is not Windows host filesystem/credential coverage; Windows and macOS are not claimed. Hosts that restrict unprivileged user namespaces (for example the Ubuntu 24.04 AppArmor `kernel.apparmor_restrict_unprivileged_userns` default) fail these tests closed; CI lifts that runner-only restriction before `npm test` rather than skipping them. The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup retains source and target parent descriptors, checks the prepared target identity, and consumes inode descriptors in the native mount helper; executable transition regressions reject replaced ancestors, leaves and parents without touching external DUMMY sentinels. Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal, `openAnchoredDirectory(..., { reset: true })`, and the replay runner's removal of its evaluation home (created by `mkdtemp` relative to a retained no-follow `TMPDIR` descriptor) use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink or for a different real directory. Sibling ten-pack/P-07 manifests, raw outputs, destination copies and hashes now also consume descriptor-relative no-follow artifact operations. Replaced task contracts are opened without truncation, checked for a single regular inode equal to the inode the runner itself created when preparing the fixture, then truncated through that retained descriptor; reads of runner-created artifacts require the same recorded inode, and the ten-pack manifest, pack and receipt digests are the digests returned by their writes rather than a rehash of the path. The DUMMY SDK proof root is created relative to a retained no-follow `TMPDIR` descriptor and, after success, failure or timeout, removed with `removeAnchoredEntry` only while its name still identifies the retained root descriptor; otherwise the cleanup failure is reported beside the initiating cause. DUMMY SDK/outcome fixture writes and exit counters use the same bounded artifact operations. A security preflight failure still blocks worker-cleanup proof before any configured credential use. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. diff --git a/scripts/replay-cp06-ten-pack.mjs b/scripts/replay-cp06-ten-pack.mjs index aa0b35a..b525d23 100644 --- a/scripts/replay-cp06-ten-pack.mjs +++ b/scripts/replay-cp06-ten-pack.mjs @@ -26,8 +26,6 @@ const fixtureDirectory = openAnchoredDirectory(proofRoot, "repo"); const fixtureRoot = fixtureDirectory.anchor; const rawRoot = rawDirectory.anchor; const cli = join(root, "dist/src/cli.js"); -const ctx = resolveTool(process.env.CTX_BIN ?? "ctx"); -configureOfflineCtxModelDir(); const reviewer = process.env.CP06_REVIEWER ?? "Pi CP-06 correction worker"; const commands = []; const preparedArtifacts = new Map(); @@ -51,6 +49,9 @@ run("git-add", "git", ["add", "."], { cwd: fixtureRoot }); run("git-commit", "git", ["commit", "-qm", "seed CP-06 ten-pack fixture"], { cwd: fixtureRoot, }); +// Resolve CTX only when first needed so artifact-boundary refusals do not depend on it. +const ctx = resolveTool(process.env.CTX_BIN ?? "ctx"); +configureOfflineCtxModelDir(); run("ctx-init", ctx, ["init", fixtureRoot, "--json"], { cwd: fixtureRoot }); const globalSources = ["PROJECT.md", "ARCHITECTURE.md"]; From a8c4c3d5588a2e5bc19dd93a3bc1068f5fc850f8 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 6 Oct 2026 06:57:31 +0000 Subject: [PATCH 44/45] Reject any offset or limit on the CP06 original read The accepted R2 protocol requires one complete read of the supplied original without offset or limit. The guard and event audit accepted an explicit offset:1/limit range when it returned the complete bytes. Reject the presence of either argument before the SDK read executes and again at the event-audit boundary, restore the negative tests (including the SDK first-line range with complete bytes), keep the unbounded positive, add ranged-call rejections through the guard around the pinned SDK's actual read tool in the DUMMY SDK proof, and correct the probe documentation. Co-Authored-By: Claude Opus 5.5 --- docs/probes/CP-06.md | 16 +++++++++------- scripts/cp06-dummy-sdk-proof.mjs | 6 ++++++ scripts/cp06-guarded-read.mjs | 14 ++++---------- scripts/cp06-worker-audit.mjs | 12 +++++++----- test/cp06-guarded-read.test.mjs | 12 ++++++++---- test/cp06-worker-audit.test.mjs | 32 ++++++++++++++++++++------------ 6 files changed, 54 insertions(+), 38 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 0b80196..70a3ddc 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -256,14 +256,16 @@ file; Pi's SDK read implementation receives only the immutable snapshot, so a replacement race cannot redirect the provider-visible bytes. The audit correlates `tool_execution_start` and `tool_execution_end` by tool-call ID and event order, requires `isError: false`, and forbids all other start or -completion events. An omitted range is accepted; the SDK's explicit first-line -form is accepted only as `offset: 1` with a positive integer `limit`, and only -when the returned UTF-8 bytes and SHA-256 still equal the complete tracked exact -original. Offset-only, limit-only, wrong-start, and partial results fail. The -final response must follow the successful completion. Deterministic protocol -tests reject omitted, failed, partial, mismatched, out-of-order, +completion events. The original must be read exactly once without `offset` or +`limit`: the guard rejects the presence of either argument before the SDK read +executes, and the audit independently rejects a start event carrying either, +even an `offset: 1` range whose result would equal the complete original. The +returned UTF-8 bytes and SHA-256 must equal the complete tracked exact original. +The final response must follow the successful completion. Deterministic protocol +tests reject omitted, failed, partial, any-range, mismatched, out-of-order, malformed-orphan, extra-project, credential-path, symlink-alias, and -post-construction replacement-race cases. +post-construction replacement-race cases; the DUMMY SDK proof also rejects +ranged calls through the guard wrapping the pinned SDK's actual read tool. The genuine Pi SDK process runs in a private Linux user/mount namespace only after the complete DUMMY auth and OS preflight passes. A supervisor walks the diff --git a/scripts/cp06-dummy-sdk-proof.mjs b/scripts/cp06-dummy-sdk-proof.mjs index 0e8fa3a..d0e1d45 100644 --- a/scripts/cp06-dummy-sdk-proof.mjs +++ b/scripts/cp06-dummy-sdk-proof.mjs @@ -149,6 +149,12 @@ async function fauxWorkerProof() { root: workerRoot, expectedOriginalPath: originalPath, }); + for (const range of [{ offset: 1, limit: 2_000 }, { offset: 1 }, { limit: 2_000 }]) { + await assert.rejects( + guardedRead.tool.execute("DUMMY-ranged-read", { path: originalPath, ...range }), + /without offset or limit/, + ); + } const faux = ai.fauxProvider({ provider: "DUMMY-readonly", models: [{ id: "DUMMY-model" }], diff --git a/scripts/cp06-guarded-read.mjs b/scripts/cp06-guarded-read.mjs index c4c9542..abed174 100644 --- a/scripts/cp06-guarded-read.mjs +++ b/scripts/cp06-guarded-read.mjs @@ -58,7 +58,7 @@ export function createGuardedReadTool({ sdk, root, expectedOriginalPath }) { tool: { ...read, async execute(toolCallId, args, ...rest) { - assertAllowedRead(args, canonicalOriginal, pinned.lineCount); + assertAllowedRead(args, canonicalOriginal); let current; try { current = lstatSync(canonicalOriginal, { bigint: true }); @@ -74,7 +74,7 @@ export function createGuardedReadTool({ sdk, root, expectedOriginalPath }) { }; } -export function assertAllowedRead(args, expectedOriginalPath, completeLineLimit = 1) { +export function assertAllowedRead(args, expectedOriginalPath) { if ( args === null || typeof args !== "object" || @@ -84,14 +84,8 @@ export function assertAllowedRead(args, expectedOriginalPath, completeLineLimit ) { throw new Error("read denied: only the canonical supplied original is allowed"); } - const hasOffset = Object.hasOwn(args, "offset"); - const hasLimit = Object.hasOwn(args, "limit"); - if (hasOffset !== hasLimit) throw new Error("read denied: incomplete explicit range"); - if ( - hasOffset && - (args.offset !== 1 || !Number.isSafeInteger(args.limit) || args.limit < completeLineLimit) - ) { - throw new Error("read denied: explicit range must cover the complete original"); + if (Object.hasOwn(args, "offset") || Object.hasOwn(args, "limit")) { + throw new Error("read denied: the original must be read without offset or limit"); } } diff --git a/scripts/cp06-worker-audit.mjs b/scripts/cp06-worker-audit.mjs index 22f8166..a492dfa 100644 --- a/scripts/cp06-worker-audit.mjs +++ b/scripts/cp06-worker-audit.mjs @@ -9,7 +9,7 @@ export const AUDIT_REJECTION_REASONS = Object.freeze({ unexpectedTools: "unexpected tool events", readCount: "expected exactly one read of the supplied original", missingCallId: "original read start omitted toolCallId", - range: "original read used a range other than offset 1 with an explicit limit", + range: "original read used offset or limit", matchingCompletion: "expected one matching read completion", duplicateCompletion: "read event stream contains an unmatched or duplicate completion", completionOrder: "read completion did not occur after its matching start", @@ -84,10 +84,12 @@ export function auditReadEvents(events, options) { if (typeof callId !== "string" || callId.length === 0) { return reject(AUDIT_REJECTION_REASONS.missingCallId); } - const offset = start.event.args?.offset; - const limit = start.event.args?.limit; - const usesExplicitRange = offset !== undefined || limit !== undefined; - if (usesExplicitRange && (offset !== 1 || !Number.isSafeInteger(limit) || limit < 1)) { + const args = start.event.args; + if ( + args !== null && + typeof args === "object" && + (Object.hasOwn(args, "offset") || Object.hasOwn(args, "limit")) + ) { return reject(AUDIT_REJECTION_REASONS.range); } diff --git a/test/cp06-guarded-read.test.mjs b/test/cp06-guarded-read.test.mjs index ef3ccd8..9a05faa 100644 --- a/test/cp06-guarded-read.test.mjs +++ b/test/cp06-guarded-read.test.mjs @@ -47,21 +47,25 @@ test("guarded read denies noncanonical and unauthorized paths before tool access { path: original, offset: 2, limit: 20 }, { path: original, offset: 1, limit: 0 }, { path: original, offset: 1, limit: 1 }, + { path: original, offset: 1, limit: 20 }, + { path: original, offset: 1, limit: 2_000 }, + { path: original, offset: 0 }, + { path: original, limit: Number.MAX_SAFE_INTEGER }, + { path: original, offset: undefined }, { path: original, extra: true }, ]) { await assert.rejects(tool.execute("DUMMY-read", args)); } assert.equal(accesses, 0); const complete = await tool.execute("DUMMY-read", { path: original }); - await tool.execute("DUMMY-read", { path: original, offset: 1, limit: 20 }); assert.equal(complete.content[0].text, "DUMMY exact\noriginal"); - assert.deepEqual(exported, ["DUMMY exact\noriginal", "DUMMY exact\noriginal"]); - assert.equal(accesses, 2); + assert.deepEqual(exported, ["DUMMY exact\noriginal"]); + assert.equal(accesses, 1); await unlink(original); await symlink(credential, original); await assert.rejects(tool.execute("DUMMY-raced-read", { path: original })); - assert.equal(accesses, 2); + assert.equal(accesses, 1); assert.equal(exported.includes("DUMMY secret canary"), false); } finally { await rm(root, { recursive: true, force: true }); diff --git a/test/cp06-worker-audit.test.mjs b/test/cp06-worker-audit.test.mjs index ccc45eb..eebfd0f 100644 --- a/test/cp06-worker-audit.test.mjs +++ b/test/cp06-worker-audit.test.mjs @@ -3,6 +3,7 @@ import { readFile } from "node:fs/promises"; import { resolve } from "node:path"; import test from "node:test"; import { + AUDIT_REJECTION_REASONS, assistantResponseText, auditReadEvents, isExpectedOriginalReference, @@ -182,19 +183,26 @@ test("worker missing-source identity accepts only the canonical original path", ); }); -for (const [name, acceptedStart] of [ - ["without an explicit range", start], - [ - "with the SDK first-line range when it returns the complete original", - { ...start, args: { path: originalPath, offset: 1, limit: 2_000 } }, - ], +test("worker read audit accepts one completed exact-original read without offset or limit", () => { + const audit = auditReadEvents([start, successfulEnd(), finalMessage], options); + assert.equal(audit.ok, true); + assert.equal(audit.summary.completed_successfully, true); + assert.equal(audit.summary.exact_original, true); + assert.equal(audit.summary.expected_sha256, audit.summary.returned_sha256); +}); + +for (const [name, args] of [ + ["the SDK first-line range", { offset: 1, limit: 2_000 }], + ["an explicit offset", { offset: 1 }], + ["an explicit limit", { limit: Number.MAX_SAFE_INTEGER }], ]) { - test(`worker read audit accepts one completed exact-original read ${name}`, () => { - const audit = auditReadEvents([acceptedStart, successfulEnd(), finalMessage], options); - assert.equal(audit.ok, true); - assert.equal(audit.summary.completed_successfully, true); - assert.equal(audit.summary.exact_original, true); - assert.equal(audit.summary.expected_sha256, audit.summary.returned_sha256); + test(`worker read audit rejects ${name} even when it returns the complete original`, () => { + const audit = auditReadEvents( + [{ ...start, args: { path: originalPath, ...args } }, successfulEnd(), finalMessage], + options, + ); + assert.equal(audit.ok, false); + assert.equal(audit.reason, AUDIT_REJECTION_REASONS.range); }); } From 604744e3d09fd11a1eac6a4c80ffab18d6cc64d0 Mon Sep 17 00:00:00 2001 From: Factory Maintainer Date: Tue, 6 Oct 2026 07:10:38 +0000 Subject: [PATCH 45/45] no-mistakes(document): Align CP-06 probe docs with no-range original read --- docs/probes/CP-06.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/probes/CP-06.md b/docs/probes/CP-06.md index 70a3ddc..3e09baf 100644 --- a/docs/probes/CP-06.md +++ b/docs/probes/CP-06.md @@ -27,7 +27,7 @@ Source and target remain distinct `ro,nosuid,nodev,noexec` mount instances of on The unsafe and hardened probes accept only an auth-security output directory under task-local `.factory/state/`, not a credential filename. The proof runner validates the ignored output location and opens its existing components without following symlinks before helper compilation or artifact writes; compiler output is written through pre-created `O_EXCL | O_NOFOLLOW` writable descriptors. After compilation, each artifact is reopened through its descriptor as `O_RDONLY`, with inode and anchored-leaf identity checks; the writable descriptor is closed before any execution. Execution and hashing consume the retained read-only inode descriptors, not the leaf paths, so later leaf replacement does not select replacement bytes. The namespace supervisor opens that directory without following symlink components, creates a private tmpfs fixture and literal `DUMMY ORIGINAL` source itself, and retains directory and source descriptors for a complete-byte DUMMY audit. Mount setup retains source and target parent descriptors, checks the prepared target identity, and consumes inode descriptors in the native mount helper; executable transition regressions reject replaced ancestors, leaves and parents without touching external DUMMY sentinels. Runner artifacts (`input.json`, `evidence.json`, `blocked.json`, auth-security `evidence.json`, and the worker-cleanup proof's ten-pack inputs and `evidence.json`) are created through `writeAnchoredFile`, an `O_CREAT | O_EXCL | O_NOFOLLOW` write relative to a pinned directory descriptor whose returned SHA-256 replaces a later path-based rehash, so a substituted leaf symlink fails with `EEXIST` before any external write. The worker-cleanup proof also creates its DUMMY scratch, evaluation-home, `tmp`, and credential directories with `openAnchoredDirectory`, writes the DUMMY source with `writeAnchoredFile`, and passes HOME, TMPDIR, and the credential path to the runner under the retained home descriptor. Its scratch removal, `openAnchoredDirectory(..., { reset: true })`, and the replay runner's removal of its evaluation home (created by `mkdtemp` relative to a retained no-follow `TMPDIR` descriptor) use `removeAnchoredEntry`, which walks the tree through no-follow directory descriptors, checks each opened directory against its `lstat` identity, unlinks symlinks without following them, and fails rather than descending when a directory is swapped for a symlink or for a different real directory. Sibling ten-pack/P-07 manifests, raw outputs, destination copies and hashes now also consume descriptor-relative no-follow artifact operations. Replaced task contracts are opened without truncation, checked for a single regular inode equal to the inode the runner itself created when preparing the fixture, then truncated through that retained descriptor; reads of runner-created artifacts require the same recorded inode, and the ten-pack manifest, pack and receipt digests are the digests returned by their writes rather than a rehash of the path. The DUMMY SDK proof root is created relative to a retained no-follow `TMPDIR` descriptor and, after success, failure or timeout, removed with `removeAnchoredEntry` only while its name still identifies the retained root descriptor; otherwise the cleanup failure is reported beside the initiating cause. DUMMY SDK/outcome fixture writes and exit counters use the same bounded artifact operations. A security preflight failure still blocks worker-cleanup proof before any configured credential use. Both documented proof entrypoints default to `.factory/state/cp06-correction/auth-security/`. Arbitrary source paths cannot initiate a probe; a failed private fixture setup blocks before byte reads or adversarial operations. This DUMMY fixture is not a live credential or model acceptance result. -The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path and range allowlist. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. +The evaluated worker injects `Cp06ReadOnlyCredentialStore` into `ModelRuntime.create({ refreshOnCreate:false, allowModelNetwork:false, modelsPath:null })` and passes that runtime to `createAgentSession` with an in-memory session, an empty resource loader, and one SDK `read` tool wrapped by a pre-execution canonical-path allowlist that rejects any `offset` or `limit`. The wrapper rejects every path except the already-canonical exact original, including credential paths, arbitrary project paths, and symlink aliases, before Pi's SDK read implementation can consume the immutable pinned snapshot. `modify` and `delete` reject before any callback or persistence. Expired and near-expiry DUMMY OAuth cases record zero refresh callbacks, zero `toAuth`, and zero persistence; an unexpired offline case resolves through `toAuth` with zero refresh/network/mutation. The live worker preflights enough validity for its timeout plus Pi's five-minute window and records only compact event/transcript and non-credential source hashes, counts, and parsed output, never credential contents or raw events. Real credential source integrity uses pinned device, inode, link, mode, size, and timestamp metadata plus mount identity; credential bytes are never hashed. Byte hashes in the isolation adversary are literal DUMMY-only controls. The current global Pi may be newer than the proven interface. Validation therefore installs exact Pi `0.85.1` under ignored task-local `.factory/state/cp06-sdk/` and supplies that installation's canonical package root and bin path explicitly. Both paths must equal the expected task-local locations, the bin symlink must resolve into that package, and its package metadata, declared entry, executable version, and complete published file manifests must agree with the independently pinned exact-version npm artifacts in `.factory/cp06-sdk-integrity.json`; inherited `PATH`, a global installation, an unrelated same-version package, and same-version file substitutions are rejected before SDK import without fallback. The pin records npm tarball SHA-512 SRI and sorted `[relative path, file SHA-256]` manifest SHA-256 and file count for both packages; it excludes nested installed `node_modules` from each package manifest, while separately verifying the resolved `pi-ai` package. It does not attest the registry itself, transitive packages other than `pi-ai`, or immutable files after this check. The global installation/configuration remains unchanged. The DUMMY provider resolver uses the selected package's ESM dependency context and the dependency's supported `import` export; it rejects a non-`0.85.1` `pi-ai`, a dependency outside the selected installation, or absent import resolution. Evidence records the selected Pi and resolved dependency package/entry paths, trusted artifact pins, manifest digests, and SHA-256 entry digests. To reproduce a pin independently, download the named exact-version tarballs into ignored `.factory/state/`, compare each tarball's SHA-512 against the tracked SRI, extract into a disposable directory, sort all regular published files by relative path and compute SHA-256 of JSON-serialized `[path, SHA-256(file)]` pairs (with no extra files or symlinks); compare count and digest to the tracked pin. Never derive a new trusted pin from the selected installation under evaluation. This resolver requires Pi's own Node floor (22.19 or newer) and Node's `--experimental-import-meta-resolve` support; inability to use it blocks rather than selecting a global fallback. @@ -100,7 +100,7 @@ Tracked, secret-free inputs and runners: records the unscoped bad path, one-condition AUTH correction, exact ten fixed packs and receipts, and a missing-original disconfirming case; - `scripts/cp06-worker-audit.mjs`, `test/cp06-worker-audit.test.mjs`, and - `scripts/replay-cp06-worker.mjs` — reject incomplete, failed, wrong-range, + `scripts/replay-cp06-worker.mjs` — reject incomplete, failed, ranged, mismatched, out-of-order, and extra-project reads, then give a new Pi process only the bounded AUTH handoff and permit one exact-original read; - `test/fixtures/workflow/history/` and