Fix open bugs: MPS float64 crash (#68), OOF calibration with max_num_rows (#55), sklearn transform_output leak (#58), safetensors dep (#56) - #78
Open
p4vlos wants to merge 1 commit into
Conversation
p4vlos
requested review from
abhidas,
erzel,
rajatsen91,
siriuz42,
tamannarayan and
weihaokong
as code owners
July 21, 2026 14:25
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
…ows, sklearn transform_output leak, missing safetensors dep - Cast float64 targets to float32 before the device move in _predict_step_pytorch and the context-cache prefill: MPS rejects float64 tensors at transfer time, so the existing guard was unreachable on Apple Silicon (google-research#68). - Classifier calibration with max_num_rows: average out-of-fold probabilities per row over the members that actually predicted it, instead of slicing all members by member 0's validation indices, which mixed all-zero rows into the calibration fit (google-research#55). - Run public fit/predict entry points under sklearn.config_context(transform_output="default") so a global sklearn.set_config(transform_output="pandas") cannot break the numpy-based internal pipeline (google-research#58). - Declare safetensors in the pytorch extra: the published HF checkpoint is safetensors-only, and loading it without the package fails with NameError inside huggingface_hub (google-research#56).
p4vlos
force-pushed
the
fix-open-issue-bugs
branch
from
July 21, 2026 19:13
073f8d9 to
f649eae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes four open issues (rebased on current main; an earlier revision also addressed #43, which 1.0.1 has since fixed upstream). Each fix is small and independent; happy to split into separate PRs if preferred.
#68 — PyTorch backend crash on Apple MPS with float64 targets
_predict_step_pytorch(and the context-cache prefill added in #62) movedyto the device before the float64→float32 guard, but MPS rejects float64 tensors at transfer time, so the guard was unreachable on Apple Silicon. The cast now happens host-side before the move in both places (numerically identical on CPU/CUDA). Added a CPU test asserting the model receives float32 targets for float64 (numpy-default) input.#55 — calibration fit on partially-zeroed OOF probabilities with
max_num_rowsWith
max_num_rows, each ensemble member draws its own row subsample, so on the single-validation-split path the members' OOF rows do not coincide.fit()sliced all members by member 0's validation indices, so_fit_calibrationreceived rows that were all-zero for every other member (rows summing to 1/E, 2/E, …).predict_oof_probanow records which (member, row) pairs were actually predicted (oof_pred_mask_), andfit()averages each row over the members that predicted it, dropping rows no member predicted. For all previously-correct configurations (nomax_num_rows, or full CV without subsampling) the computation is unchanged — the mask is full there, so the count-normalized sum equals the old mean, and the NNLS path (which already forbidsmax_num_rows) sees identical inputs. Added a test that spies on_fit_calibrationand asserts every calibration row is a valid probability vector undermax_num_rows+ single-split.#58 —
'DataFrame' object has no attribute 'flatten'under global pandas outputA user-level
sklearn.set_config(transform_output="pandas")made the internal transformers (y_scaler_, preprocessors,ColumnTransformer) return DataFrames, crashing the numpy-based pipeline. Publicfit/predict/predict_proba/predict_oof*entry points now run undersklearn.config_context(transform_output="default")via a small decorator. (Instance-levelset_outputpinning was not an option:ColumnTransformer.set_outputrequires every sub-transformer to supportset_output, which the custom encoders here do not.) Added a test running both estimators under the global pandas config.#56 —
NameError: name 'safetensors' is not definedThe published HF checkpoint is safetensors-only (see #53/#77), but the
pytorchextra does not declaresafetensors, sohuggingface_hub's mixin fails withNameErrorwhen the package is missing. Addedsafetensorsto thepytorchextra and the pinnedrequirements.txt.Testing
pytest(torch env, no JAX): 26 passed, 30 skipped (JAX-only) — includes 3 new tests.max_num_rows+ calibration: single-split path fits calibration on partially-zeroed OOF probabilities #55 and Regression example not working AttributeError: 'DataFrame' object has no attribute 'flatten' #58 tests fail (the MPS-cast test is contract-only, since the crash needs MPS hardware).🤖 Generated with Claude Code