feat(zarr-metadata)!: a chunk grid is judged against the shape it chunks

[d-v-b]

🤖 AI text below 🤖

The second of four pieces of layer 4, depending on #4440 (4a, fill values): a v3 array's chunk_grid is judged against its shape, by the grid's definition. The first four commits are #4440's; this PR's are the last three.

document["shape"] = [4, 4]
document["chunk_grid"] = {"name": "regular", "configuration": {"chunk_shape": [4]}}
validate_array_metadata_v3(document)
# (ValidationProblem(loc=('chunk_grid', 'configuration', 'chunk_shape'), message='expected one chunk length per dimension of shape, got 1', kind='invalid_value'),)

What a grid knows. ChunkGridDefinition gains shape_rules(configuration, nested, shape): what the spec disallows in a grid of that configuration over an array of a given shape, located in the configuration. It takes the same arguments as 4a's fill_value_rules: the configuration, the fields it holds as the scope read them, and the thing judged. A grid that says nothing of the shape fits every one, so a third-party grid is unaffected. The two package grids:

• regular: one chunk length per dimension (regular grid), and 0 only for a dimension of length 0 (core: "non-zero when the corresponding dimensions of the arrays have non-zero length"). A chunk longer than its dimension is fine.
• rectilinear: one chunk_shapes entry per dimension, with chunk lengths that sum to at least the dimension's length (extension). A bare integer repeats until it covers, so it always does. A run-length entry counts as length times count and is never expanded. [] covers a dimension of length 0.

Where it is judged. validate_array_metadata_v3 judges the grid once both the grid and the shape are read. A grid the scope did not read, being out of scope or invalid, is left unjudged. So is a grid beside a shape that fails its own check, so no problem is reported twice. The check reaches from_json, from_key_value, to_key_value, the pydantic types and each array in an inline consolidated_metadata. Reading the shape now has one owner, which returns the lengths or None with its problems. The v3 grid and dimension_names checks use it, and so does the v2 rank check between shape and chunks.

chunk_grid_problems stays private, in v3._definition. 4c's door returns each axis's chunk lengths along with the problems, and replaces it.

Breaking. A document whose grid does not fit its shape now has a problem in chunk_grid.configuration, where the package accepted it before. create_default(chunk_grid=...) without shape keeps the scalar default shape=(), which a grid of another rank does not fit, so that model now fails at to_key_value. The docstring says to pass the two together.

A fix on the way. create_default(shape=...) set chunk_shape equal to shape, so an empty dimension got a chunk length of 0. zarr refuses to open that grid: "integer chunk edge length must be >= 1". The default grid now has a chunk length of 1 there, which still fits the shape. This is its own commit, with a bugfix fragment.

Choices worth a look

• A regular chunk length of 0 on a dimension of length 0 is accepted when read. The core spec's condition allows it, and zarr-python 3.0 and 3.1 wrote it, although the regular grid page says chunk sizes must be greater than zero. The package's own writer no longer produces it (above).
• zarr-python refuses a rectilinear [] for a dimension of length 0 ("has no chunk edges"). This PR accepts it, since there is nothing to cover.
• The v2 create_default has the same derivation: chunks equals shape, with 0 for an empty dimension. zarr opens that document, so this PR leaves it alone.
• shape_rules returns only problems. In 4c the lengths come from a separate function, which is only called on a grid its shape_rules accepted. The design review considered a single chunks -> (lengths, problems) hook and kept the split, for the same reason rules and canonical are split: the rules report problems, and derivation works from what they accepted.

Reviews. Two, with separate lenses. The correctness review found no bug in the rules. A Hypothesis differential against a reference written from the spec text agreed on 8,000 examples, and its reach was measured: rank mismatches, 0 on an empty dimension, uncovered dimensions, run-length entries, exact cover and [] for length 0 all came up. Canonical and written rectilinear spellings agreed on 3,000 examples. zarr-python agrees everywhere except [], noted above. The review's one finding was the chunk length of 0 from create_default. The design review's changes are in too:

• chunk_grid_problems stays private.
• One no_rules(*_) default serves every hook, instead of one per arity.
• Reading the shape has one owner.
• Messages and citations are fixed.
• The tests are split one per case.

The stack. Layers 0 (#4420, #4421, #4422), 1 (#4432), 2 (#4434) and 3 (#4436) have merged. 4a is #4440; this is 4b; 4c (#4442) and 4d (#4443) follow.

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 zarr-metadata | 🛠️ Build #34798158 | 📁 Comparing e0f0db1 against latest (0401a7f)

  🔍 Preview build  

5 files changed · ± 5 modified

± Modified

• index.html
• api/model/index.html
• api/v3/chunk_grid/index.html
• api/v3/data_type/index.html
• api/v3/definition/index.html

[d-v-b]

🤖 AI text below 🤖

This landed with #4443, which carried its commits and merged all of layer 4. Nothing is left to merge here. #4444 renames this PR's changelog fragment to #4443.