Repository navigation
List and report disk-passthrough saves as zarr backend files - #34
Merged
Merged
Conversation
ZarrStorageBackend saves .json/.csv/.txt/.roi.zip/.zip outputs through the disk backend, and exists() already answers for them, but list_files() and is_dir() only looked inside the zarr store. A directory holding only such saves (for example a result directory of CSV tables and ROI archives) was therefore reported as missing and empty by the backend that wrote it. list_files() now includes passthrough files beside the arrays (never the store's own metadata files), and is_dir() is true for a directory holding passthrough saves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
delete() kept its own extension list (.json/.csv/.txt) instead of the declared DISK_PASSTHROUGH_EXTENSIONS, so .roi.zip and .zip files this backend saved on disk were opened as zarr groups and failed to delete. Re-running a pipeline over existing zarr-mode outputs therefore failed when it replaced a saved ROI archive. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 10, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cause
ZarrStorageBackendstores.json/.csv/.txt/.roi.zip/.zipoutputs through the disk backend.save(),load(),exists()anddelete()already follow those files.list_files()andis_dir()only looked inside the zarr store, so the backend that wrote a directory of CSV tables and ROI archives then said that directory was missing and empty.In OpenHCS this made zarr-mode result-only directories (
results/, published on purpose since OpenHCS #134) fail publication. The step-time check (exists) admitted the directory and wrote an empty projection entry. The completion check (is_dir+list_files) then skipped writing its header fields, soopenhcs_metadata.jsonended up with aresultsentry that had noimage_files,available_backendsormicroscope_handler_name.Fix
list_files()also returns the passthrough files saved in the directory, filtered the same way as array outputs and never including the store's own metadata files (zarr.json,.zgroup, …). A directory that holds only passthrough saves is listed without opening a zarr store.is_dir()is true for a directory that holds passthrough saves.Tests
Two regression tests in
tests/test_zarr_backend.py. Full suite:591 passed, 2 skipped.Consumed by the OpenHCS B2 PR (submodule bump).
🤖 Generated with Claude Code