Skip to content

feat: filesystem.sync() + snapshot copy on archives (ST-127) - #107

Open
sfroment wants to merge 15 commits into
files-multipartfrom
files-sync
Open

sfroment wants to merge 15 commits into
files-multipartfrom
files-sync

Conversation

@sfroment

Copy link
Copy Markdown
Member

Summary

sync(local, remote, *, direction="push", delete=False, exclude=(),
     checksum=False, dry_run=False)                         both fs twins
  list both sides   walk_local(local, exclude) vs NDJSON files_list(remote)
  plan              filter the remote listing through path_excluded
                    (own path or any directory prefix — the walker prunes
                    a matching dir whole, the plan must agree)
  diff              path/type/size/mtime, whole seconds; dirs skip size
                    (a filesystem artifact that never converges);
                    checksum=True adds sha256 stats for pairs that pass
  plan_batches      changed set -> ~50 MiB tar batches;
                    files over one batch -> upload_file/download_file
                    (multipart sessions on executors v0.0.24+)
  push              build_archive -> PUT /files/archive (creates the root)
  pull              GET /files/archive (server-side excludes + oversized
                    paths, so their bytes travel once) -> extract only the
                    changed set, replacing in-the-way files/links/dirs
                    without following symlinks out of the destination
  collision guard   oversized patterns that would also drop planned
                    members -> SandboxFilesystemError before any transfer
  delete=True       prune the destination via the legacy routes, sparing
                    excluded trees whole and tolerating paths a type flip
                    in this sync already replaced

DeclarativeSnapshot.copy() rides the same routes: a directory builds as one archive push (binary-safe, modes/mtimes/symlinks kept), a single file through upload_file; pre-v0.0.24 executors keep the per-file legacy walk, pinned unchanged.

A five-pass review loop (parallel per-file lanes + tools lens, then four focused verification passes) landed 24 red-first findings on top of the build: the exclude contract (push+delete no longer deletes what exclude promised to skip; pull no longer plans or counts prunable members), directory size convergence, Go-parity for escaped class members, extraction make-room, loud unreadable-subtree walks, async failure-arm pinning with multipart-capable fakes, the no-retry/build-seam/no-op tokens watched, and docs regenerated with the whole sync surface (koyeb.sandbox.sync added to the doc module list).

Evidence

  • Before: files-multipart tip — 448 sandbox tests green; SandboxFilesystem has no sync surface and snapshot copy walks per-file as text.
    After: 607 sandbox tests green (uv run --no-sync pytest koyeb/sandbox/ -q → 607 passed in 9.77s): +112 build tests and +47 review tests, every review test red-first or an explicit coverage pin. Mutation runs scoped to the delta report zero survivors across every pass (crapper: all delta-touched functions under CRAP 10; the ancestor-blind path_excluded mutant is killed by all six of its tests).
  • The exclude contract is pinned end to end: push with delete=True, exclude=("*.log",) leaves skip.log on the remote with deleted == 0; pull with exclude=("backup",) never plans x/backup/small.txt (0 downloads, no archive GET, local copy untouched); an oversized backup next to a changed x/backup/small.txt raises SandboxFilesystemError naming the member before any transfer.

Merge Danger

Door: two-way — additive surface on both fs twins; reverting the branch reverts the feature. No wire changes below the base.

Blast Radius: the sandbox filesystem sync/copy surface only. The archive/list routes and multipart rides require executor v0.0.24+ (has_files_routes() gates them); older executors keep the legacy walk for snapshot copy. delete=True prunes the destination — new capability, explicit opt-in. Stacked on #106 (base files-multipart); retarget to main when #106 merges.

GET /files/list streams NDJSON entries (recursive walk, error entries
pass through); GET /files/archive streams a gzip'd tar with repeated
exclude params; PUT /files/archive streams a tar into a directory and
returns {files, bytes}. All three idle-bounded like the other /files
routes; the archive PUT shares files_put's no-mid-stream-retry contract.
Async twins cover bytes, sync iterables, async iterables and file
objects at the MockTransport wire seam.
Pure engine behind filesystem.sync() in koyeb/sandbox/sync.py: local walk
in the executor's FileEntry shape (relative /-separated, %04o perms,
RFC3339Nano UTC mtimes, symlinks listed not followed, special files
skipped, excluded dirs pruned whole); path_match is a faithful port of
Go's path.Match (* and ? never cross /) so client-side excludes agree
with the executor's archive route byte for byte — cross-checked against
real Go on 24 arms; diff on path/type/size/mtime with whole-second mtime
(PAX tar cannot round-trip ns; rsync's default), optional sha256 arms via
injected hash callables, unreadable remote entries skipped not guessed;
plan_batches fills ~part_size batches without ever splitting a file,
oversized files separated for upload_file/download_file; prune_descendants
keeps topmost deletes; relative_remote resolves remote=None.
build_archive spools a changed set as a gzip'd PAX tar — relative
/-separated members, modes and whole-second mtimes kept, symlinks as
links, owners dropped — the exact shape the executor's PUT /files/archive
extracts. extract_archive mirrors the executor's extractor: parents
created, members in the way replaced, dir modes/mtimes restored last
(writing into a dir bumps its mtime), a keep filter for the pull side's
changed set, and the same traversal refusal (absolute paths, ..). Round-
trips pinned: content, mode, mtime, symlinks, empty dirs, replacements,
keep-filter isolation, traversal + absolute refusal.
sync(local, remote=None, direction, delete, exclude, checksum, dry_run)
on SandboxFilesystem and AsyncSandboxFilesystem: list both sides (local
walk vs files_list NDJSON), diff on path/type/size/whole-second mtime,
send the changed set as ~50 MiB tar batches spooled per batch and
streamed into files_archive_put; files over one part ride
upload_file/download_file (multipart once ST-126 slice 2 lands — the
call sites are the public seams); push treats a 404 listing as an empty
tree (the archive PUT creates the root), pull refuses a missing source;
delete=True prunes to topmost deletes — remote via the legacy routes,
local via rmtree/unlink; checksum=True compares sha256 for files that
pass size+mtime, with the executor-side digests prefetched through
hash_candidates (the pure diff cannot await) and a 404 hash read as
unprovable, not changed; a no-op sync sends nothing. dry_run computes
counts only. Adds files_stat(hash=) for the stat?hash=sha256 arm.
Pinned against a miniature executor: the fake backs /files routes with a
real tree using the engine's own walk/build/extract, so push and pull
round-trip real tar bytes.
DeclarativeSnapshot._copy: a directory reaches the builder sandbox as one
archive push through filesystem.sync on executors v0.0.24+ (binary-safe,
modes/mtimes/symlinks kept — the legacy per-file walk read files as text
and lost all of that), a single file through the streamed upload_file
(binary-safe; no probe needed — it carries its own legacy fallback). The
legacy walk stays byte-for-byte for old executors, pinned by its shape.
Also fixes the pre-existing unbound SandboxFilesystem.write_file call in
write_files (deferred since the files-api round 2 review) — the loop now
calls self.write_file, so the async override chain holds.
Crapper pass on the diff-touched functions, every one now under the
CRAP-10 bar with coverage: the Go-parity matcher restructured around a
_match_class parser, a _match_unit single-character matcher, and a
_validate_chunk pass (the two-phase shape drops Go's fused failed-flag
protocol — the post-failure state is unobservable, verified against the
24 Go-cross-checked arms); path_match's star scan extracted into
_star_skip; diff's checksum verdict into _hash_verdict and the delete
tail into _remaining_deletes; extract_archive's member dispatch into
_extract_member/_clear_member/_extract_file. Also pins the async twins
of the files_list and files_archive_get 400 arms (the round-5 lesson:
async failure arms drift unpinned), and the unarchivable-pair rule —
both sides having an 'other' entry at one path is never a delete.
The two-phase _match_chunk conflated chunk exhaustion with match
failure: a chunk whose last unit mismatched consumed the chunk, so the
'return the leftover chunk verdict' check read failure as success —
path_match('a*b', 'b') matched (Go's path.Match says false). The verdict
now travels with ok explicitly. Found chasing the uncovered tail-
validation lines the CRAP pass flagged; pinned with the regression rows
cross-checked against real Go, plus the malformed-later-chunk error
(Go validates the whole pattern even past an earlier mismatch), the
async checksum and async pull+delete twins (the round-5 lesson again —
the async arms were the uncovered ones), and diff's pair-verdict
extraction — every engine function now at CRAP ≤ 10 with coverage.
A directory's st_size is a filesystem artifact (APFS ~64, ext4 4096)
that never converges cross-filesystem; every sync re-planned every
directory. Only real content carries size.

[\]] and [\-] are Go path.Match class literals; rejecting the
backslash made the escape arm dead code and aborted walks on
patterns the executor accepts.
The remote listing rode the diff unfiltered: push with delete removed
the very files exclude promised to skip, and pull planned excluded
entries — counted as downloads that never arrived, oversized ones
fully downloaded. Both listings now filter through path_excluded (own
path or any directory prefix, mirroring the walker's whole-dir
pruning), a pull's archive GET leaves oversized files out so their
bytes travel once, and a collision guard refuses loudly when an
oversized pattern would also drop planned members — suffix, class or
dir-prune — before any transfer.
A type flip at one path aborted the pull permanently — open() on a
directory, makedirs over a file, or a symlinked dir silently written
through to arbitrary paths outside the destination. Extraction now
clears the in-the-way object without following links, deletes
tolerate paths an earlier flip already replaced, the walk is loud on
unreadable subtrees, and pull treats a missing destination root as
an empty tree while push refuses a missing source like rsync.
The async fake had no multipart routes, so the async oversized arm
was untestable and every failure arm — bad direction, legacy
executor, listing failure, missing remote — went unpinned while the
sync twin pinned all four. The fakes delegate to the sync tree, the
legacy executor gets an async shape, and the checksum stat-failure
guard is pinned on both twins.
The no-retry promise hung on one unwatched token — dropping
idempotent=False silently re-enables blind 5xx retries and replays
a consumed stream as a truncated body. Mid-stream resets and 502s
are now single-call, on both twins.
build's CRAP was 120 — the recipe loops and the sandbox lifecycle
in one body. _apply_operations carries the loops; the seam is
pinned through build() itself, including a failed command.
The sync surface shipped undocumented — no sync(), no list/archive
transports, three stale signatures — while sibling surfaces publish.
The engine module joins the doc list.
@sfroment
sfroment changed the base branch from files-multipart to main October 28, 2026 15:04
@sfroment
sfroment changed the base branch from main to files-multipart October 28, 2026 15:09
@sfroment
sfroment added this pull request to stack #108 October 28, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant