A Zarr Version of DAP

[DennisHeimbigner]

All-
While it is on my mind, I have been thinking about a DAP equivalent that uses Zarr as the data model. Hope you are familiar, at least in passing, with Zarr.

Anyway, Zarr would provide data model much simpler than DAP4. No sequences, for example. This would make server and client implementations much easier to build, and might lower the adoptions costs for users. I think we could build a standard quite quicky by just reusing much of the DAP4 documentation but using JSON (the Zarr metadata format) instead of XML. We could draw heavily from the Zarr Version 3 specification.

Do any of you have an opinion on the usability/market for this? Would there be some possible source for money to support the development of this: NOAA for example. Of course the money issue is irrelevant to me since I am retired. But it matters to OPeNDAP and Unidata.

Anyway, an idea to kick around.

[ndp-opendap]

I think you might have a look at our NASA funded work on dmr++ and how that is getting some attention from the Zarr project. @Mikejmnez has been in conversation with them and might have more information.

[ndp-opendap]

I found VirtualiZarr thanks to @Mikejmnez documentation.

[Mikejmnez]

Yes - VirtualiZarr is a python package that can generate / take metadata information and generate virtual zarr references / Zarr Arrays. This makes it possible to get Zarr-like performance from HDF5/Netcdf4 datasets . There is now a parser for Hyrax's dmr++ in the VirtualiZarr package, dmr++ have the right kind of metadata (chunk references) associated with HDF5/Netcdf4 files, and therefore one can user to generate Xarray object that performs similar to Xarray+Zarr.

It does not provide a point of access the same way an OPeNDAP server would, but it enables performance when data is already on S3.