Update stale rows in the Kerchunk comparison FAQ - #1045
Merged
TomNicholas merged 3 commits intoJul 15, 2026
Conversation
The "Splitting uncompressed data into chunks" row linked PR zarr-developers#199, which was closed without merging. Sub-chunk slicing landed instead via zarr-developers#996: slicing an uncompressed ManifestArray along its largest-stride axis rewrites the reference's byte offset/length. A finer chunk grid along that axis can be built by combining isel with xarray.concat, so no dedicated rechunk method is needed. Also note that chunk selection now covers sub-chunk slicing of uncompressed arrays, not just chunk-aligned selections. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two Parallelization rows predated open_virtual_mfdataset's `parallel` kwarg: - Generation: point at open_virtual_mfdataset(parallel="dask"|"lithops"| Executor), which parallelizes the per-file open map step, rather than only describing a manual dask.delayed wrap. - Combining (tree-reduce): drop the stale "wrap ManifestArray in dask.array (Untested)" description. VirtualiZarr deliberately combines references serially on the client since they are small, avoiding a distributed tree-reduce (per the Scaling docs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TomNicholas
changed the title
- to_kerchunk format strings: 'JSON' -> 'json' (the format arg only accepts the lowercase literals "dict"/"json"/"parquet"), and fix the missing opening quote in format='parquet'. - DMRPPParser: pass an instance (DMRPPParser()) like every other parser row, not the bare class. - Mark the Kerchunk cell for reading a Zarr v3 store as ❌ (ZarrToZarr is v2-only) instead of leaving it blank. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Several rows in the Kerchunk comparison table in
docs/explanation/faq.mdwere stale or contained incorrect snippets.Stale feature rows
Sub-dividing an uncompressed chunk — was marked "❌ Not yet implemented" linking #199 (closed without merging). Sub-chunk slicing actually landed via #996: slicing an uncompressed
ManifestArrayalong its largest-stride axis rewrites the reference's byte offset/length. A finer chunk grid along that axis can be built by combiningiselwithxarray.concat, so no dedicated rechunk method is needed.Selecting specific chunks — now also covers sub-chunk slicing of uncompressed arrays, not just chunk-aligned selections.
Parallelized generation of references — point at
open_virtual_mfdataset(..., parallel="dask" | "lithops" | Executor), which parallelizes the per-file open map step, instead of only describing a manualdask.delayedwrap.Parallelized combining (tree-reduce) — drop the stale "wrap⚠️ Untested)" description. VirtualiZarr deliberately combines references serially on the client since they are small, avoiding a distributed tree-reduce (per the Scaling docs).
ManifestArrayindask.array(Incorrect snippets
to_kerchunkformat strings:'JSON'→'json'(theformatarg only accepts the lowercase literals"dict"/"json"/"parquet"), and fix the missing opening quote informat='parquet'.DMRPPParser→DMRPPParser(), matching every other parser row (it is a class needing instantiation).ZarrToZarris v2-only) instead of leaving it blank.