Repository navigation
Migration to zarr 3 - #112
Open
konstntokas wants to merge 11 commits into
Open
konstntokas wants to merge 11 commits into
konstntokas wants to merge 11 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #112 +/- ##
===========================================
+ Coverage 99.92% 100.00% +0.07%
===========================================
Files 30 30
Lines 1418 1467 +49
===========================================
+ Hits 1417 1467 +50
+ Misses 1 0 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
konstntokas
marked this pull request as ready for review
October 29, 2026 16:59
forman
requested changes
Nov 2, 2026
Member
There was a problem hiding this comment.
Implementation looks good!
However, I'd like to go for the following changes which would need also some documentation changes:
- Make Zarr 3 the new default format, because it is the better format.
zarr_versionshould be deprecated,zarr_formatbe used instead to align with the API.Requirezarr_formatsetting in new configurations. If it is missing assume an old config that refers to the old defaultzarr_version=2and sozarr_format=2. Emit a warning in this case.shardssetting should be ignored with a warning if it is not applicable.
EDIT
- Require
zarr_format=2|3setting in new configurations and acceptzarr_version=2in old configurations. If either is missing, raise.
| desired and the variable's data array will be persisted as one block. | ||
|
|
||
| For Zarr v3, `shards` may additionally be configured for a variable. A shard | ||
| contains an integer number of regular chunks, reducing object counts while |
Member
There was a problem hiding this comment.
Suggested change
| contains an integer number of regular chunks, reducing object counts while | |
| contains an integer number of regular chunks, reducing file or object counts while |
| contains an integer number of regular chunks, reducing object counts while | ||
| retaining smaller read/write chunks. Each shard size must be at least its | ||
| corresponding chunk size and an exact multiple of it. Shards are not available | ||
| for Zarr v2. |
Member
There was a problem hiding this comment.
Suggested change
| for Zarr v2. | |
| for Zarr v2, hence the `shards` setting will be ignored. |
| @@ -603,7 +611,7 @@ | |||
| "zarr_version": { | |||
Member
There was a problem hiding this comment.
Deprecate zarr_version and introduce zarr_format (to align with Zarr API).
| "description": "The Zarr version to be used.", | ||
| "const": DEFAULT_ZARR_VERSION, | ||
| "enum": [2, 3], | ||
| "default": DEFAULT_ZARR_VERSION, |
Member
There was a problem hiding this comment.
Suggested change
| "default": DEFAULT_ZARR_VERSION, | |
| "default": DEFAULT_ZARR_FORMAT, |
Member
There was a problem hiding this comment.
Suggested change
| DEFAULT_ZARR_FORMAT, |
| target_store: collections.abc.MutableMapping, | ||
| target_ds: xr.Dataset, | ||
| target_attrs: dict[str, Any], | ||
| zarr_version: int, |
Member
There was a problem hiding this comment.
Suggested change
| zarr_version: int, | |
| zarr_format: int, |
| @@ -1,5 +1,7 @@ | |||
| ## Version 0.8.1 (in development) | |||
|
|
|||
| * Migrated to the Zarr v3 Python API. Supports writing Zarr v2 and v3 stores, | |||
Member
There was a problem hiding this comment.
Suggested change
| * Migrated to the Zarr v3 Python API. Supports writing Zarr v2 and v3 stores, | |
| * Migrated to use `zarr >=3`. Supports writing Zarr v2 and v3 stores, |
| @@ -1,5 +1,7 @@ | |||
| ## Version 0.8.1 (in development) | |||
Member
There was a problem hiding this comment.
Suggested change
| ## Version 0.8.1 (in development) | |
| ## Version 0.9.0 (in development) |
Member
There was a problem hiding this comment.
This PR is a major change of the implementation and a capability enhancement. This justifies a minor number increase.
| ## Version 0.8.1 (in development) | ||
|
|
||
| * Migrated to the Zarr v3 Python API. Supports writing Zarr v2 and v3 stores, | ||
| with Zarr v2 as the default. (#111) |
Member
There was a problem hiding this comment.
Suggested change
| with Zarr v2 as the default. (#111) | |
| with Zarr 3 as the new default format. (#111) |
Comment on lines
+31
to
+32
| [Zarr format](https://zarr.readthedocs.io/en/stable/). Targets use Zarr v2 by | ||
| default and may opt into Zarr v3. If an error |
Member
There was a problem hiding this comment.
Suggested change
| [Zarr format](https://zarr.readthedocs.io/en/stable/). Targets use Zarr v2 by | |
| default and may opt into Zarr v3. If an error | |
| [Zarr format](https://zarr.readthedocs.io/en/stable/). Targets use Zarr 3 format by | |
| default and may opt into Zarr 2 format, if needed. If an error |
Collaborator
Author
Linking related issue: #107 |
konstntokas
force-pushed
the
konstntokas/migrate_zarr3
branch
from
November 2, 2026 15:43
34577db to
242b875
Compare
Collaborator
Author
|
Closes #107 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrated to the Zarr v3 Python API. Supports writing Zarr v2 and v3 stores, with Zarr v2 as the default.
closes #111
Checklist (strike out non-applicable):
CHANGES.mdCHANGES.mddocs/*