fix: accept universal-pathlib UPath as a StoreLike value - #280
Open
d-v-b wants to merge 15 commits into
Open
Conversation
…#176) Bumps the actions group with 8 updates in the / directory: | Package | From | To | | --- | --- | --- | | [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) | `0.9.5` | `0.9.6` | | [codecov/codecov-action](https://github.com/codecov/codecov-action) | `6.0.0` | `6.0.1` | | [github/issue-metrics](https://github.com/github/issue-metrics) | `4.2.2` | `4.2.7` | | [j178/prek-action](https://github.com/j178/prek-action) | `2.0.3` | `2.0.4` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` | | [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` | | [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.0` | | [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) | `0.5.3` | `0.5.6` | Updates `prefix-dev/setup-pixi` from 0.9.5 to 0.9.6 - [Release notes](https://github.com/prefix-dev/setup-pixi/releases) - [Commits](prefix-dev/setup-pixi@1b2de7f...5185adf) Updates `codecov/codecov-action` from 6.0.0 to 6.0.1 - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@57e3a13...e79a696) Updates `github/issue-metrics` from 4.2.2 to 4.2.7 - [Release notes](https://github.com/github/issue-metrics/releases) - [Commits](github-community-projects/issue-metrics@c9e9838...1e38d5e) Updates `j178/prek-action` from 2.0.3 to 2.0.4 - [Release notes](https://github.com/j178/prek-action/releases) - [Commits](j178/prek-action@6ad8027...bdca6f1) Updates `actions/upload-artifact` from 7.0.0 to 7.0.1 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v7...043fb46) Updates `actions/download-artifact` from 7.0.0 to 8.0.1 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v7...3e5f45b) Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0 - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](pypa/gh-action-pypi-publish@v1.13.0...cef2210) Updates `zizmorcore/zizmor-action` from 0.5.3 to 0.5.6 - [Release notes](https://github.com/zizmorcore/zizmor-action/releases) - [Commits](zizmorcore/zizmor-action@b1d7e1f...5f14fd0) --- updated-dependencies: - dependency-name: prefix-dev/setup-pixi dependency-version: 0.9.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: codecov/codecov-action dependency-version: 6.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: github/issue-metrics dependency-version: 4.2.7 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: j178/prek-action dependency-version: 2.0.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/download-artifact dependency-version: 8.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: pypa/gh-action-pypi-publish dependency-version: 1.14.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions - dependency-name: zizmorcore/zizmor-action dependency-version: 0.5.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
A remote UPath now creates an FsspecStore from the filesystem and storage
options the UPath already carries, and a local UPath creates a LocalStore so
that UPath("/data") and Path("/data") agree.
This previously worked only by accident. In universal-pathlib < 0.3 every
UPath subclassed pathlib.Path and implemented __fspath__, so a remote path was
either converted to a URI string by the caller (xarray does this) or wrapped
in a LocalStore that happened to dispatch through fsspec. Since
universal-pathlib 0.3 remote paths do neither, and passing one raised
TypeError: Unsupported type for store_like.
The UPath branch is checked before the Path branch so that routing is
identical on both universal-pathlib 0.2 and 0.3.
FsspecStore.from_upath now converts the UPath's filesystem to async mode
instead of raising TypeError for synchronous filesystems and warning for
sync-mode instances of async ones.
The memory:// routing test wraps a synchronous MemoryFileSystem, which needs
fsspec's AsyncFileSystemWrapper, so it is skipped below fsspec 2024.12.0 as
test_wrap_sync_filesystem already does.
Closes zarr-developers#4244
Assisted-by: ClaudeCode:claude-fable-5
d-v-b
force-pushed
the
claude/zarr-python-issue-4244-3863cc
branch
from
August 10, 2026 20:04
63e3930 to
e023851
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Closes zarr-developers#4244.
What was going on
Passing a remote
UPathto the top-level API raisedTypeError: Unsupported type for store_like: 'S3Path'. Nothing changed on zarr's side — universal-pathlib 0.3.0 (2025-09-29) rebasedUPathontopathlib_abc, and remote paths likeS3Pathstopped beingpathlib.Pathsubclasses and stopped implementing__fspath__.That broke two things that had been working by accident:
os.PathLikestore to a string before handing it to zarr, so zarr used to receive"s3://..."and route it toFsspecStore.from_url. Zarr never actually saw aUPath. With no__fspath__, the raw object now comes through.zarr.open(S3Path). I confirmed against zarr 3.1.3 + upath 0.2.6 that this used to hit theisinstance(store_like, Path)branch and build aLocalStorerooted at anS3Path— which worked only because upath 0.2 overrode every pathlib method to dispatch through fsspec.So the report is filed as a feature request, and that framing is accurate: remote
UPathsupport was never deliberate. ButFsspecStore.from_upathalready exists and universal-pathlib is already an optional dependency, so wiring it up is small.The change
make_storegains aUPathbranch. Remote UPaths go toFsspecStore.from_upath; local ones (protocol""or"file") go to aLocalStore, soUPath('/data')andPath('/data')agree, mirroring how the equivalent strings are already routed.The branch is checked before the
Pathbranch. On universal-pathlib 0.2 everyUPath— includingS3Path— subclassespathlib.Path, so otherwise remote paths would keep landing in aLocalStorethere. I verified routing is now identical on 0.2.0 and 0.3.10:UPath('s3://bucket/foo.zarr')FsspecStoreFsspecStoreUPath('memory://bucket/foo.zarr')FsspecStoreFsspecStoreUPath('file:///tmp/x.zarr')LocalStoreLocalStoreUPath('/tmp/x.zarr')LocalStoreLocalStoreDrive-by fix:
FsspecStore.from_upathpassedupath.fsstraight to the constructor without going through_make_async(unlikefrom_mapper). AUPathhands back a filesystem in whatever mode it was built with, which is synchronous unless the caller passedasynchronous=True. Sofrom_upathraisedTypeError: Filesystem needs to support async operations.for genuinely sync filesystems, and emitted the "not created withasynchronous=True" warning for sync-mode instances of async ones. Both reproduce on upath 0.3.10.storage_optionspassed alongside aUPathstill raises, since theUPathcarries its own — the existing check already covers this, and there's now a test pinning it.Notes for review
UPathwithstorage_optionsis an error rather than a merge. Ambiguous either way; erroring seemed better than silently picking one. Happy to change. Note xarray always forwardsstorage_optionsto zarr, but asNonewhen unset, so the reported case is unaffected.LocalStorerather than through fsspec'sLocalFileSystem. This is a judgement call for consistency with the string handling (and it's faster); the alternative is to send everything toFsspecStoreand keep the branch simpler.UPathre-export in_utils.pyneeded an explicitas UPathalias to satisfyno_implicit_reexport.Testing
storage_options.open_group(UPath(...))round trip reproducing the issue, and one assertingfrom_upathon a sync-modeUPathyields an async filesystem with storage options preserved and no warning.filterwarningsfor aDeprecationWarningthat universal-pathlib 0.2.0 raises from its own subclass registry when any localUPathis constructed. It fires on a bareUPath('/tmp/x')with zarr uninvolved, and would otherwise break themin_depsCI env, which pinsuniversal_pathlib==0.2.0.ruffandmypyclean. Store tests also verified against universal-pathlib 0.2.0.🤖 Generated with Claude Code