Repository navigation
fix(utils): SparseCopy from a FUSE or NFS source leaves zero blocks as holes - #331
Merged
Merged
Conversation
A FUSE daemon that does not answer FUSE_LSEEK makes the kernel report the whole file as data, so SparseCopy from a JuiceFS checkpoint dir wrote every zero byte of a 10 GiB COW. From a FUSE source the copy now reads 1 MiB blocks and skips all-zero ones. Every other source keeps io.CopyN and its copy_file_range: a userspace block copy measured 4-6% slower on ext4.
Contributor
Author
Hardware verification: passSetup. Testbed host, kernel 6.12, sandboxd 707ef2cf, CH 750aa2b. The two arms differ only in the cocoon binary: A = 2afbc27, F = 5d2e580. Each run used a fresh sandboxd, so the golden, the archive and the wake clone all run on the arm's cocoon. Workload. A small claim with 512 MiB of urandom data is archived to a dir checkpoint store, then woken by an exec. The run dir is on xfs.
|
…sAllZero An NFS client without SEEK support falls back to generic_file_llseek like a FUSE daemon without lseek, and docs/cli.md offers --to-dir for NFS golden images that clone --from-dir reads. The block check reuses tar.go's isAllZero.
Contributor
Author
|
Follow-up 90aa33b, from an independent whole-file review:
The zero-skip path was verified on JuiceFS hardware. NFS adds only the magic constant; there was no NFS hardware run. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ReflinkCopyfalls back toSparseCopywhenever FICLONE fails, for example across filesystems.SparseCopywrites every range thatSEEK_DATAreports.FUSE_LSEEK, the kernel falls back togeneric_file_llseek, which reports the whole file as one data range. JuiceFS behaves this way.vm clone --from-dirwhose source dir is on JuiceFS writes every zero byte of the COW. That happens on every archived wake, checkpoint claim or template refill from a JuiceFScheckpoint_dirin sandboxd.Evidence (hardware, cocoon 2afbc27)
Archived wake through sandboxd, with the run dir on xfs in both cases:
Change
fstatfsreturnsFUSE_SUPER_MAGIC), each data segment is copied in 1 MiBReadAt/WriteAtblocks, and all-zero blocks are left unwritten. The destination is truncated to the source size first, so those blocks stay holes.io.CopyN, which uses copy_file_range. A userspace block copy measured 4–6% slower than copy_file_range on ext4, and this is the clone path.TestCopySegmentSkipsZeroBlocks: it fails when the zero skip is removed (16384 of 16384 blocks allocated).Hot path
fstatfsper copied file.Gates
make lint:0 issues.linux + darwin.make fmt-check: rc 0.asl -forwarder=false ./...: clean on both GOOS.go test -race ./utils/ ./hypervisor/...: ok, also in a Linux container.Hardware run of this fix
Pending. JuiceFS archived wake with 2afbc27 vs this commit, plus one ext4 run per arm. Results will be posted here.