Add safetensors to the pytorch extra - #75
Open
manjunathshiva wants to merge 1 commit into
Open
Conversation
pip install tabfm[pytorch] cannot load the released checkpoint:
tabfm_v1_0_0.load() fails with
NameError: name 'safetensors' is not defined
inside huggingface_hub's _load_as_safetensor. huggingface_hub declares
safetensors optional and imports it lazily, but the PyTorch loader always
goes through PyTorchModelHubMixin -> model.safetensors (since 1.0.1
dropped pytorch_model.bin), so safetensors is a hard runtime dependency
of the pytorch extra. Torch does not pull it in transitively.
Adds it to the pytorch extra and to the pinned requirements.txt lock.
The JAX path is unaffected (weights restore via orbax).
manjunathshiva
requested review from
abhidas,
erzel,
rajatsen91,
siriuz42,
tamannarayan and
weihaokong
as code owners
July 20, 2026 04:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A fresh
pip install tabfm[pytorch]cannot load the released checkpoint. On tabfm 1.0.1 (cb6ba46), Python 3.13, torch 2.13.0:Cause:
huggingface_hubdeclaressafetensorsas an optional dependency and imports it lazily, but the PyTorch loader always goes throughPyTorchModelHubMixin, and since 1.0.1 the Hugging Face checkpoint ships onlymodel.safetensors(#33 / the 1.0.1 weight-loading fix dropped thepytorch_model.binpath). Sosafetensorsis a hard runtime dependency of thepytorchextra — andtorchdoes not pull it in transitively. Anyone installing per the README's PyTorch instructions hits theNameErroron their firstload().It likely went unnoticed because CI's
pip install -e .[dev,jax,pytorch]never loads the released checkpoint (tests build small models directly), and any environment that has ever installedtransformers/acceleratealready hassafetensorspresent.Change: add
safetensorsto thepytorchextra inpyproject.toml, and a matching pinned entry (safetensors==0.8.0, current release) to therequirements.txtlock in pip-compile format/ordering. The JAX path is unaffected — its weights restore via orbax.Verified in a clean venv:
pip install -e .[pytorch]on this branch pullssafetensorsandtabfm_v1_0_0.load(model_type="classification")succeeds; onmainthe same steps reproduce theNameErrorabove.Note on overlap with #49: the MLX backend PR bundles this same dependency fix (both the
safetensorsline in thepytorchextra and thesafetensors==0.8.0pin inrequirements.txt). This PR extracts it as a standalone fix so it isn't gated on the larger feature review; if this lands first, the corresponding hunks in #49 resolve trivially on its rebase, and if #49 lands first this PR can simply be closed.