fix(native): restore a sourceless graph after its project files were touched by roodboi · Pull Request #630 · hack-dance/hack

roodboi · 2026-11-06T20:56:37Z

Summary

On a native project, editing the Compose file and putting the original bytes back left the stopped run unrestorable: every later up failed with graph_source_compatibility. It was first seen on a project with a build: service after a refused up with an edited Compose file.

Cause

Neither the refused attempt nor the pool restart persisted anything. Host reruns with an isolated home showed the following:

up, down, up. The run restores.
up, down, runtime down, up. The run restores.
up, down, edited up (refused), original bytes back, up. Refused. The graph receipt, the run mapping and every file under the native home and .hack/.internal were byte-identical before and after the refused attempt.
Image-only project: up, down, touch the Compose file, up. Refused as well.

A native plan ID also binds the project source inventory, which records the size and modification time of every selected project file. Rewriting the Compose file with its original bytes, or touching any project file, changes the plan ID. The reviewed Compose input is still identical, but the review no longer equals the retained plan. A graph that mounts no project source has no shared-source contract that such a change could be proven against. So source compatibility, fresh restore selection and source replay all refused it.

With a logging wrapper around hack-native, the restore after the edit sent a normalized Compose document byte-identical to a working restore, under the same --expect-original, and still got a different --expect-plan. hack-native project plan on a copy of the project changed its plan ID after each touch.

Fix

The retained review is now accepted in one more case. The review must carry the retained graph's exact original and normalized Compose input. Neither the graph nor the reviewed plan may read project source: no source binding, bind mount, dependency cache or cache provenance. Three checks accept this case:

source_compatibility;
the stopped-restore selection check in restore_inputs, where the review replays under the retained plan ID;
prepare_replay, for ordinary sourceless replay.

The predicate is normalized::source_independent_review. Restore still keeps the retained ownership plan. It also still requires identical resource bindings (names, images, networks, caches, routing), readiness and absent compute, so no compute, data or ownership changes. Graphs that read source keep their shared-source proofs. A changed Compose input is still refused on this path; HACK-1914 (#614) covers that case.

hack doctor --native-run-mapping inspect used to refuse every mapping that did not need a device rebind, with "unsafe, changed, or owned by another run". A current mapping now gets a read-only report with these fields:

state: "current" and repairRequired: false;
the mapping SHA-256 and the run;
the native graph phase, or null while the pool is stopped;
whether the graph's run, owner, namespace and plan match the mapping.

A renumbered mapping still goes through the legacy rebind inspection unchanged, and repair is unchanged.

Fit with HACK-1914 (#614)

This branch merges cleanly with claude/hack-1914-native-reconcile. With #614, a sourceless graph whose plan ID changed only through the source inventory gets plan_reconcile::applies == true (plan IDs differ), so it is reconciled rather than strictly replayed. Both paths are correct and recreate compute from the same review. To keep strict replay for unchanged Compose input after both land, applies can treat normalized::source_independent_review(receipt, plan) as unchanged; that is a follow-up for the merge train. The Rust tests of both changes pass on the merged tree (below).

Validation

Rust: a new source_metadata_change_keeps_a_sourceless_review_restorable test uses a synthetic project. It covers edit-and-revert and touching an unrelated file, both accepted with a different plan ID. It also covers the cases that must still refuse: a changed Compose input, a receipt with a source binding, a foreign namespace, legacy receipts without normalized input, and a bind-mounted service. A mutation that accepts any normalized receipt fails this test.
TS: tests/native-project-run-filesystem.test.ts covers inspecting a current mapping. The report is read-only (only graph inspect runs, bytes unchanged), a foreign plan is reported as ownershipMatches: false, an unreadable graph gives graph: null, repair still refuses, and an unsafe mode still refuses. A renumbered mapping still returns the legacy selection.
Focused bun tests (bounded runner): 55 pass, 0 fail across mapping, mapping command, builds, restore and retained-image suites.
bun run --cwd packages/cli typecheck: pass. bun run --cwd packages/cli check: pass (warnings only). Biome on changed files: clean.
RUSTUP_TOOLCHAIN=1.97.1 bun run check:local: pass. RUSTUP_TOOLCHAIN=1.97.1 bun run test:local from a short-path checkout: pass, 1346 passed, 0 failed, 110 ignored across 21 test binaries (lib: 1190 passed).
Merged with claude/hack-1914-native-reconcile (clean text merge, scratch tree): provider::graph::{normalized,restore,plan_reconcile} tests pass, 25 passed, 0 failed, including the 6 reconcile tests and the new test.
docs:cli-reference regenerated; docs:prepare and docs:check: pass. gitleaks protect/detect: no leaks.
The full bun suite was not run locally; this relies on hosted CI.

Host proof (Apple Silicon, isolated HOME/TMPDIR, one VM at a time)

The candidate bundle was built from this branch with pinned provider archives.

Build fixture, exact repro. The steps were: up (cold build), down, runtime down (pool stopped), up with an added env var, original Compose back, then up. The edited up is still refused (graph_source_compatibility; fix(native): reconcile stopped graphs after Compose edits #614 covers it). The original up now restores the same run with the same ownership plan ID and the same built image ID, healthy. The built file and context output appear in the logs. doctor --native-run-mapping inspect reports state: "current", the graph ready-observed (then stopped-data-retained after down) and ownershipMatches: true.
Image-only control with a named volume. The steps were: up, down, runtime down, up, down, touch the Compose file, then up. Every start restored the same run, and the volume marker written on first start was the same across all three starts. On the base bundle, the touch step was refused with graph_source_compatibility.
Already-stuck runs. Homes left unrestorable by the base bundle (one build project after the edit-and-revert, one image-only project after touch) restore their original runs with this bundle, without repair.

Remaining gaps

An edited Compose input on a stopped graph is still refused on this base; HACK-1914 (fix(native): reconcile stopped graphs after Compose edits #614) reconciles it.
HACK_NATIVE_REBUILD=1 with an unchanged Compose file still restores the retained image rather than rebuilding (existing behavior, unchanged here).
Graphs with bind-mounted source or dependency caches keep their shared-source proofs, which may still refuse after unrelated file touches when no shared contract was recorded. This change does not widen those proofs.

Refs HACK-1926

🤖 Generated with Claude Code

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

…touched A native plan ID binds the project source inventory, which records the size and modification time of every selected project file. Editing the Compose file and putting the original bytes back, or touching any other project file, therefore changed the plan ID of an otherwise identical review. A stopped graph that mounts no project source has no shared-source contract to prove such a change against, so `up` refused every later restore with graph_source_compatibility. Build services made this common: the refused attempt after a Compose edit was not the cause, the edit itself was. When the review carries the retained graph's exact original and normalized Compose input, and neither the graph nor the reviewed plan reads project source (no source binding, bind mount, dependency cache or cache provenance), source compatibility, fresh restore selection and source replay now treat it as the retained review. Restore keeps the retained ownership plan and still requires identical resource bindings, images and readiness. Graphs that read source keep their shared-source proofs, and a changed Compose input is still refused here. `doctor --native-run-mapping inspect` refused every mapping that did not need a device rebind with "unsafe, changed, or owned by another run". A current mapping is now reported read-only with its run, the native graph phase and whether the graph's ownership matches; repair is unchanged. Refs HACK-1926 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

roodboi · 2026-11-06T20:56:42Z

@codex review

chatgpt-codex-connector · 2026-11-06T20:56:55Z

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-07T06:39:37.134604Z	`a93db84`	Manual request
🔒 Security Review	✅ Completed 2026-11-07T06:42:57.388892Z	`a93db84`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

linear-code · 2026-11-06T20:57:33Z

HACK-1926

chatgpt-codex-connector · 2026-11-06T20:58:13Z

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 009989c73c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-11-06T21:02:03Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 009989c73c

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…uild-restore

roodboi · 2026-11-07T06:38:14Z

@codex review

chatgpt-codex-connector · 2026-11-07T06:39:35Z

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: a93db84213

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-11-07T06:42:56Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: a93db84213

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@codesmith-bot

, #612, #628, #629, #630) (#639) ## What this is Merge train batch 6 into `next`: the native image builds stack and its fixes, as one batch. Builds stay off by default: they run only with `HACK_NATIVE_BUILDS=1`, which native `up`, restart and doctor all check. Each component's reviewed head is merged with `git merge --no-ff`, in stack order. Integration conflicts are resolved in the batch's merge commits, described below. Hosted CI on this branch is the full-suite gate for the combined result. | PR | Title | Linear | Head included | Review | | --- | --- | --- | --- | --- | | #583 | feat(native): pin a BuildKit client and scan Dockerfile image references | HACK-1270 | `f21d5a04` | Codex code + security: no findings; 0 open threads | | #606 | feat(native): build Compose images in the guest engine (runtime build-image) | HACK-1270 | `460ddff2` | Codex code + security: no findings; 0 open threads | | #607 | feat(native): admit build storage, budget build cache and reclaim superseded build images | HACK-1270 | `9f33a1d0` | Codex code + security: no findings; 0 open threads | | #612 | feat(native): build Compose build services in native up behind HACK_NATIVE_BUILDS | HACK-1270 | `7f42e328` | Codex clean at `8ec7460e`; `7f42e328` adds #628's squash (re-review requested) | | #628 | fix(native): admit native build fields and name services in build refusals | HACK-1925 | `f07ea583` | Codex code + security clean; already squashed into #612's branch | | #629 | fix(native): keep the VM responsive during image builds and stop a silent guest | HACK-1924 | `a0e3e797` | Codex code + security: no findings; 0 open threads | | #630 | fix(native): restore a sourceless graph after its project files were touched | HACK-1914 | `a93db842` | Codex clean at `009989c7`; `a93db842` only merges #629 into it (re-review requested) | ## Security properties in the merged code - **Build arguments:** they reach the BuildKit client only through a private 0600 bake definition, never its environment. The client environment is cleared to an allowlist. Names must be plain identifiers, and BuildKit-interpreted names such as `BUILDKIT_*`, `BUILDX_*` and `SOURCE_DATE_EPOCH` are refused. A build argument equal to a managed secret value is refused without echoing the value. Build arguments are not secrets, matching Docker. - **Source policy:** a source policy from `build_policy::document`, set through `EXPERIMENTAL_BUILDKIT_SOURCE_POLICY` in a private file, applies to every build. It allows only the scanner's digest-pinned base images and denies other images, `oci-layout`, `http`, `https`, `git` and blob sources. Pinned bases whose config has `OnBuild` triggers are refused. - **Dockerfile scanner:** - remote `ADD` sources are refused, including the JSON form, multiple sources and variables; - JSON-form arguments are read as BuildKit reads them, and a lenient-decode differential fails closed; - heredocs are recognized as BuildKit's lexer does, including whitespace after `<<`; - a stage named `scratch` is refused; - `ARG` names must be plain identifiers; - platform arguments must resolve to `linux/arm64`; - expanded values are size-bounded. - **Build output:** forwarded output is sanitized and then redacted, including Go-quoted and JSON-escaped forms and values split across a line cut. It is bounded by size. ## Integration changes - **#606 with #605 / #609:** `resolve_build_base` runs on #605's guarded registry client, so the resolver and address policy, origin-bound authentication and the realm rules apply. It uses #609's native platform selection, so builds stay Docker Hub and linux/arm64 only. The older Docker Hub-only fetch helpers are dropped. - **#606 / #607 with #605 / #609 / #615:** the `build-image` and `reclaim-build-images` commands sit beside the image-policy and emulation options. The build error helpers take `impl Into<ErrorMessage>`. - **#607:** the native-build-retention TLA+ contract is registered beside the restart-policy, pool-emulation and multi-root contracts. Its model-coverage row now says an orphaned build client keeps the fence and per-key lock until it exits (`9f33a1d0`). - **#612 with batch 5:** - The builds-enabled classification flows through `classifyAdapted`, the Compose-backend refusal and doctor. - With the gate off, native `up` refuses a build with a message that names `HACK_NATIVE_BUILDS`, so doctor's basic-build advice names it too; one test expectation changed to match. - Startup and restart keep the dependency-file selection, host aliases, emulation policy and the inactive-profile skip alongside the build services. - **New (#609 x #612):** a build service whose adaptation selects amd64 emulation is refused before any effect, because native builds are linux/arm64 only. `prefer-native` passes, and a test covers it. - **#629:** the socket cleanup after a clean stop is kept inside #629's new guest stop. #629's two flaky relay tests are fixed on its branch (`a0e3e797`): each test gets its own fixture root, and EINVAL from `set_read_timeout` after the peer closes is accepted. - **`next` merge:** the last merge brings in Kit's CI shard and SIGKILL harness changes (#624, #627). The commit hook formatted `tests/native-compose-generation.test.ts` with Biome, a whitespace-only change to that Kit file. ## Validation on this branch (`6337e0ed`) - Clean CLI typecheck: pass. Biome on the 20 changed files: no errors. - `RUSTUP_TOOLCHAIN=1.97.1 bun run check:local`: pass. All-features clippy `-D warnings` (Zig 0.15.2): pass. - Cargo `--lib`: 1346 pass. `--features native-config-plan` `provider::graph`: 673 pass. Contracts: `project_contract` 66, `compose_fields_contract` 10, `interpolation_budget_contract` 1, `local_contract` 14. - `bun run test:models` (TLA+ 1.7.4): exit 0, 172 controls verified. - Focused bun tests, bounded runner, the changed test files: 242 pass, 0 fail. - `docs:prepare` + `docs:check`, `privacy:check`, gitleaks over `origin/next..HEAD`: pass. Host evidence: 11 of 11 builds passed on an M5 host with the relay (#629). Still open: the source-policy host check (a pinned build succeeds, and a base trigger that pulls an unpinned image or fetches https fails with "denied by policy") and the bake `--allow fs.read` / metadata shape on the pinned buildx 0.34.1. Both are listed in #606. After this merges, each component PR is closed with a link to the merge commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code)  --- <a href="https://backend.blacksmith.sh/track/run-proof?expires=1796625833&installation_model_id=17053&pr_number=639&ref=codesmith_pr_footer&repository=hack-dance%2Fhack&return_to=https%3A%2F%2Fgithub.com%2Fhack-dance%2Fhack%2Fpull%2F639&variant=see_it_in_action&signature=c3ba624f6f34813ba26773c5b4c3caf941bd59515e8c3016c92df8f0e2a6048f"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/proof-see-it-in-action-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/proof-see-it-in-action-light.svg"><img alt="See it in action" src="https://pr-comments-assets.blacksmith.sh/codesmith/proof-see-it-in-action-dark-3x.png" width="132" height="28"></picture></a> <sup><a href="https://app.blacksmith.sh/hack-dance/codesmith/hack/pr/639?autoLogin=true&ref=codesmith_pr_footer">Open in [code]smith</a> · Need help on this PR? Tag <code>@codesmith-bot</code> with what you need.</sup>   --------- Co-authored-by: Dimitri Kennedy <344842+roodboi@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

roodboi · 2026-11-07T08:57:48Z

Merged into next through the train batch PR #639 (squash commit e59fcc2). This PR's reviewed head a93db842134582ea79754b4ab4e34f06f3297e9d was included. #639 describes the integration resolutions and the batch security fixes: Go-style image config key reading, non-ASCII key refusal, the build base size bound (unpacked size tracked in HACK-1945), build-base descriptor validation and case-insensitive stage names. Closing as merged by batch; the branch is kept.

🤖 Addressed by Claude Code

Merge branch 'claude/hack-1924-build-vm-hang' into claude/hack-1926-b…

a93db84

…uild-restore

roodboi mentioned this pull request Nov 7, 2026

feat(native): merge train batch 6, native image builds (#583, #606, #607, #612, #628, #629, #630) #639

Merged

roodboi closed this Nov 7, 2026

Navigation Menu

Repository navigation

fix(native): restore a sourceless graph after its project files were touched - #630

fix(native): restore a sourceless graph after its project files were touched#630
roodboi wants to merge 2 commits into
claude/hack-1924-build-vm-hangfrom
claude/hack-1926-build-restore

roodboi commented Nov 6, 2026 •

edited by blacksmith-sh Bot

Loading

Uh oh!

roodboi commented Nov 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026 •

edited

Loading

Uh oh!

linear-code Bot commented Nov 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026

Uh oh!

roodboi commented Nov 7, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 7, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 7, 2026

Uh oh!

roodboi commented Nov 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

roodboi commented Nov 6, 2026 • edited by blacksmith-sh Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Cause

Fix

Fit with HACK-1914 (#614)

Validation

Host proof (Apple Silicon, isolated HOME/TMPDIR, one VM at a time)

Remaining gaps

Uh oh!

roodboi commented Nov 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear-code Bot commented Nov 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

roodboi commented Nov 7, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 7, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 7, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

roodboi commented Nov 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

roodboi commented Nov 6, 2026 •

edited by blacksmith-sh Bot

Loading

chatgpt-codex-connector Bot commented Nov 6, 2026 •

edited

Loading