Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/guides/native-candidate.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,12 @@ With it:
- Restoring an unchanged retained run uses the retained images and never builds; if
one is missing, `up` refuses and asks you to start fresh. `hack restart` also never
builds: it reuses the running or retained image, or refuses before stopping anything.
- "Unchanged" means the same Compose bytes. A run that mounts no project source and
uses no dependency cache still restores after its Compose file was edited and put
back, or after any project file was touched; build-context edits do not rebuild it.
Its review then differs only in the project source inventory (file sizes and
modification times), which such a run never reads, and restore keeps the retained
ownership plan, images and data.
- As with Compose `up`, only build services in the selected profiles are built and
pinned. A build service in a profile you did not select is not built, and restore
and restart need no image for it; selecting its profile in a later fresh `up` builds
Expand Down Expand Up @@ -444,6 +450,11 @@ the mapping, inspect the exact instance with the current bundle:

Omitting `--branch` uses the same linked-worktree default as project commands;
detached linked worktrees require an explicit instance. Inspection writes nothing.
A mapping that still matches the checkout reports `state: "current"` with its run,
the native graph phase (or `null` while the pool is stopped), whether the graph's
run, owner, namespace and plan match the mapping, and `repairRequired: false`.
Only a mapping whose scope no longer matches goes through the legacy device-rebind
inspection below, which returns the `selectionSha256` that repair requires.
Repair requires the exact selection and explicit acceptance of unproven original
filesystem volume continuity. It changes only the three scope directory device
numbers, preserving canonical paths, inodes, branch, run, owner, plan, environment
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1429,7 +1429,7 @@ hack doctor [options]
| `--browser-result unknown|works|fails|permission-denied` | Your manual browser observation for --browser-url (default: unknown) |
| `--branch <name>` | Run against a branch-specific instance (compose name + hostnames) |
| `--native-cleanup inspect` | Inspect one mapped native graph's pending cleanup without recovery or stop replay |
| `--native-run-mapping inspect|repair` | Inspect or explicitly repair a native run mapping after filesystem device renumbering |
| `--native-run-mapping inspect|repair` | Inspect a native run mapping, or explicitly repair it after filesystem device renumbering |
| `--expect-selection <64-hex>` | Require the exact run-mapping recovery inspection selection |
| `--accept-legacy-device-rebind` | Explicitly accept legacy migration without proof of original filesystem volume continuity |
| `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) |
Expand Down
4 changes: 3 additions & 1 deletion packages/runtime-core/src/provider/graph/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1529,7 +1529,9 @@ pub fn source_compatibility(
if needs_source && revision.is_none() {
return Err(refused());
}
if !unchanged_normalized_review(&receipt, plan_id, normalized) {
if !unchanged_normalized_review(&receipt, plan_id, normalized)
&& !normalized::source_independent_review(&receipt, plan)
{
let revision = revision.ok_or_else(refused)?;
if hostname_change::prepare(&engine, plan, source_receipt, normalized)?.is_none() {
source::prepare_shared_changed(
Expand Down
144 changes: 144 additions & 0 deletions packages/runtime-core/src/provider/graph/normalized.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,42 @@ impl NormalizedInputIdentity {
}
}

/// Whether a stopped or active graph reads nothing from the project tree: no
/// source binding, dependency cache or cache provenance was recorded for it.
pub(super) fn source_free_receipt(receipt: &Receipt) -> bool {
receipt.source.is_none()
&& receipt.initializer_cache_release.is_empty()
&& receipt
.resources
.values()
.all(|resource| resource.cache.is_none() && resource.cache_provenance.is_none())
}

/// Whether `plan` reviews exactly the retained graph's Compose input for a graph
/// that reads nothing from the project tree.
///
/// A plan ID also binds the project source inventory, which records the size and
/// modification time of every selected project file. Rewriting the Compose file
/// with its original bytes, or touching any project file, therefore changes the
/// plan ID although the reviewed Compose input is identical. Only graphs that
/// mount project source or use a dependency cache consume that inventory; they
/// keep their shared-source proofs. For every other graph the identical original
/// and normalized Compose bytes are the whole reviewed input. Restore then keeps
/// the retained ownership plan and still requires identical resource bindings,
/// images and readiness, so no compute, data or ownership changes.
pub(super) fn source_independent_review(receipt: &Receipt, plan: &project::PlanData) -> bool {
receipt
.normalized_input
.as_ref()
.is_some_and(|input| input.valid(&receipt.namespace) && input.matches_plan(plan))
&& source_free_receipt(receipt)
&& plan.services.values().all(|service| {
!service.active
|| (service.dependency_cache.is_none()
&& service.mounts.iter().all(|mount| mount.kind == "volume"))
})
}

pub(super) fn replay_refused() -> CandidateError {
error(
"graph_normalized_replay",
Expand Down Expand Up @@ -261,6 +297,114 @@ mod tests {
);
}

/// Rewriting the Compose file with its original bytes, or touching any project
/// file, changes only the source inventory. A graph without project source or
/// caches restores from that review; anything that reads source, or any change
/// of the reviewed Compose input itself, keeps the strict proofs.
#[test]
fn source_metadata_change_keeps_a_sourceless_review_restorable() {
let fixture = super::super::tests::Fixture::new();
let project = fixture.0.join("project");
fs::create_dir_all(project.join("app")).unwrap();
fs::write(project.join("app/hello.txt"), b"hello\n").unwrap();
let candidate = Candidate::discover(&fixture.0).unwrap();
let namespace = candidate.plan(&project).unwrap().namespace;
let review = |bytes: &[u8]| {
let original = format!("{:x}", Sha256::digest(bytes));
let report = project::plan_normalized(
&candidate,
project::NormalizedComposeOptions {
branch: None,
project: &project,
compose_file: Path::new("compose.yaml"),
profiles: &[],
expected_namespace: &namespace,
expected_compose_sha256: &original,
compose_bytes: bytes,
},
)
.unwrap();
let identity = NormalizedInputIdentity {
namespace: namespace.clone(),
original_compose_sha256: original.clone(),
normalized_compose_sha256: original,
};
(report, identity)
};
let rewrite = |bytes: &[u8]| {
// Some filesystems keep coarse timestamps; make the rewrite observable.
std::thread::sleep(std::time::Duration::from_millis(20));
fs::write(project.join("compose.yaml"), bytes).unwrap();
};
let receipt_for = |plan_id: &str, identity: &NormalizedInputIdentity| -> Receipt {
serde_json::from_value(json!({"version":1,"run":"a".repeat(32),"owner":"b".repeat(32),"namespace":namespace,"plan_id":plan_id,"phase":"stopped-data-retained","readiness":{},"resources":{},"normalized_input":identity})).unwrap()
};
let image = b"services: {app: {image: 'sha256:1111111111111111111111111111111111111111111111111111111111111111', volumes: ['data:/data']}}\nvolumes: {data: {}}\n";
rewrite(image);
let (before, identity) = review(image);
let receipt = receipt_for(&before.plan_id, &identity);
assert!(super::super::unchanged_normalized_review(
&receipt,
&before.plan_id,
&identity
));
// Edit, refuse, put back: same bytes, new modification time.
rewrite(b"services: {app: {image: alpine}}\n");
rewrite(image);
let (restored, restored_identity) = review(image);
assert_eq!(restored_identity, identity);
assert_ne!(restored.plan_id, before.plan_id);
assert!(!super::super::unchanged_normalized_review(
&receipt,
&restored.plan_id,
&identity
));
assert!(source_independent_review(&receipt, &restored.plan));
// Touching an unrelated project file is the same kind of change.
std::thread::sleep(std::time::Duration::from_millis(20));
fs::write(project.join("app/hello.txt"), b"hello\n").unwrap();
let (touched, _) = review(image);
assert_ne!(touched.plan_id, restored.plan_id);
assert!(source_independent_review(&receipt, &touched.plan));

// A changed Compose input is not this path (Compose edits are reconciled elsewhere).
let edited = b"services: {app: {image: 'sha256:1111111111111111111111111111111111111111111111111111111111111111', environment: {REVISION: '2'}, volumes: ['data:/data']}}\nvolumes: {data: {}}\n";
rewrite(edited);
let (changed, _) = review(edited);
assert!(!source_independent_review(&receipt, &changed.plan));
rewrite(image);

// A graph that recorded source, a cache or cache provenance keeps its proofs.
let mut sourced = receipt.clone();
sourced.source = Some(
serde_json::from_value(json!({"revision":"e".repeat(64),"archive_sha256":"f".repeat(64),"selection_sha256":"0".repeat(64)}))
.unwrap(),
);
assert!(!source_free_receipt(&sourced));
let (current, _) = review(image);
assert!(!source_independent_review(&sourced, &current.plan));
// A foreign namespace or missing provenance never qualifies.
let mut foreign = receipt.clone();
foreign.namespace = "f".repeat(64);
assert!(!source_independent_review(&foreign, &current.plan));
let mut legacy = receipt.clone();
legacy.normalized_input = None;
assert!(!source_independent_review(&legacy, &current.plan));

// A bind mount reads project source: the inventory is part of what it runs.
let bind = b"services: {app: {image: 'sha256:1111111111111111111111111111111111111111111111111111111111111111', volumes: ['./app:/app:ro']}}\n";
rewrite(bind);
let (mounted, mounted_identity) = review(bind);
let mounted_receipt = receipt_for(&mounted.plan_id, &mounted_identity);
rewrite(bind);
let (remounted, _) = review(bind);
assert_ne!(remounted.plan_id, mounted.plan_id);
assert!(!source_independent_review(
&mounted_receipt,
&remounted.plan
));
}

#[test]
fn normalized_receipt_cannot_be_replayed_as_original() {
let mut receipt: Receipt = serde_json::from_value(json!({"version":1,"run":"a".repeat(32),"owner":"b".repeat(32),"namespace":"c".repeat(64),"plan_id":"d".repeat(64),"phase":"ready","readiness":{},"resources":{}})).unwrap();
Expand Down
9 changes: 8 additions & 1 deletion packages/runtime-core/src/provider/graph/restore.rs
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,14 @@ fn restore_inputs(
Some(change),
)?;
} else {
verify_fresh(&receipt, &inputs.review.plan_id, fresh, &generation)?;
// A source-independent review of the exact retained Compose input
// replays under the retained plan; resource bindings are rechecked below.
let plan = if normalized::source_independent_review(&receipt, &inputs.review.plan) {
&receipt.plan_id
} else {
&inputs.review.plan_id
};
verify_fresh(&receipt, plan, fresh, &generation)?;
}
} else {
normalized::require_file_replay(&receipt)?;
Expand Down
6 changes: 5 additions & 1 deletion packages/runtime-core/src/provider/graph/source.rs
Original file line number Diff line number Diff line change
Expand Up @@ -390,7 +390,11 @@ pub(super) fn prepare_replay(
.as_ref()
.and_then(|source| source.live.as_ref());
if !live || retained.is_none() {
if live || retained.is_some() || inputs.review.plan_id != receipt.plan_id {
if live
|| retained.is_some()
|| (inputs.review.plan_id != receipt.plan_id
&& !super::normalized::source_independent_review(receipt, &inputs.review.plan))
{
return Err(error(
"graph_source_changed",
"Replay source mode or immutable review changed.",
Expand Down
10 changes: 8 additions & 2 deletions src/backends/native-project-mapping-command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import {
sanitizeBranchSlug,
} from "../lib/project.ts";
import {
inspectNativeProjectRunFilesystemRecovery,
inspectNativeProjectRunMapping,
recoverNativeProjectRunFilesystem,
} from "./native-project-run.ts";
import type { NativeRuntimeSelection } from "./native-runtime-client.ts";
Expand Down Expand Up @@ -108,7 +108,7 @@ export async function runNativeProjectMappingCommand(opts: {
};
const data =
opts.selection.action === "inspect"
? await inspectNativeProjectRunFilesystemRecovery({
? await inspectNativeProjectRunMapping({
scope,
runtime: opts.runtime,
})
Expand All @@ -120,6 +120,12 @@ export async function runNativeProjectMappingCommand(opts: {
});
if (opts.json) {
emitCliResult({ result: okResult({ data }) });
} else if ("state" in data) {
process.stdout.write(
`Native run mapping is current: branch=${selected.branch ?? "base"}, run=${data.run.run}\n` +
`graph=${data.graph ? `${data.graph.phase}, ownership ${data.graph.ownershipMatches ? "matches" : "differs"}` : "unavailable"}\n` +
"No repair is needed. Graph recovery, retained-data readback and application readiness require separate verification.\n"
);
} else {
process.stdout.write(
`Native run mapping ${opts.selection.action === "repair" ? "repaired" : "inspected"}: branch=${selected.branch ?? "base"}, run=${data.run.run}\n` +
Expand Down
93 changes: 93 additions & 0 deletions src/backends/native-project-run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -858,6 +858,99 @@ export async function inspectNativeProjectRunFilesystemRecovery(opts: {
return (await recoverySelection(opts)).inspection;
}

export type NativeProjectRunCurrentInspection = {
readonly schema: typeof CURRENT_SCHEMA;
readonly state: "current";
readonly mappingSha256: string;
readonly run: NativeProjectRun;
/** Absent when the native graph cannot be read, for example while the pool is stopped. */
readonly graph: {
readonly phase: string;
readonly ownershipMatches: boolean;
} | null;
readonly repairRequired: false;
};

const CURRENT_SCHEMA = "hack.native-project-run-mapping-inspection/v1";
const GRAPH_PHASE = /^[a-z][a-z-]{0,63}$/;

async function currentGraph(opts: {
readonly projectRoot: string;
readonly run: NativeProjectRun;
readonly runtime: NativeRuntimeSelection;
readonly invoke?: typeof invokeNativeRuntime;
}): Promise<NativeProjectRunCurrentInspection["graph"]> {
let observed: unknown;
try {
observed = await inspectNativeProjectGraph({
runtime: opts.runtime,
projectRoot: opts.projectRoot,
run: opts.run.run,
invoke: opts.invoke ?? invokeNativeRuntime,
});
} catch {
return null;
}
const receipt = isRecord(observed) ? observed.receipt : undefined;
if (
!(
isRecord(receipt) &&
typeof receipt.phase === "string" &&
GRAPH_PHASE.test(receipt.phase)
)
) {
return null;
}
return {
phase: receipt.phase,
ownershipMatches:
isRecord(observed) &&
observed.journal_incomplete === false &&
receipt.run === opts.run.run &&
receipt.owner === opts.run.owner &&
receipt.namespace === opts.run.namespace &&
receipt.plan_id === opts.run.planId,
};
}

/**
* Read-only Doctor inspection of the saved run mapping. A mapping that still
* matches this checkout is reported with its native graph phase and whether the
* graph's ownership matches it; nothing needs repair. Only a mapping whose scope
* no longer matches falls through to the explicit legacy device-rebind
* inspection, which refuses every other difference.
*/
export async function inspectNativeProjectRunMapping(opts: {
readonly scope: NativeProjectRunScope;
readonly runtime: NativeRuntimeSelection;
readonly invoke?: typeof invokeNativeRuntime;
}): Promise<
NativeProjectRunCurrentInspection | NativeProjectRunFilesystemInspection
> {
const p = await paths(opts.scope, false);
await recoveryDirectories(p);
const { bytes, value } = await readRecoveryFile(p.file);
let run: NativeProjectRun;
try {
run = record(value, p.identity);
} catch {
return await inspectNativeProjectRunFilesystemRecovery(opts);
}
return {
schema: CURRENT_SCHEMA,
state: "current",
mappingSha256: createHash("sha256").update(bytes).digest("hex"),
run,
graph: await currentGraph({
projectRoot: p.identity.projectRoot,
run,
runtime: opts.runtime,
invoke: opts.invoke,
}),
repairRequired: false,
};
}

/** Explicitly rebind only three mapping device numbers; graph history is untouched. */
export async function recoverNativeProjectRunFilesystem(opts: {
readonly scope: NativeProjectRunScope;
Expand Down
2 changes: 1 addition & 1 deletion src/commands/doctor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,7 @@ const doctorOptions = [
long: "--native-run-mapping",
valueHint: "inspect|repair",
description:
"Inspect or explicitly repair a native run mapping after filesystem device renumbering",
"Inspect a native run mapping, or explicitly repair it after filesystem device renumbering",
} as const),
defineOption({
name: "expectSelection",
Expand Down
Loading
Loading