Enable Pandas 3#803
Draft
Scienfitz wants to merge 8 commits into
Draft
Conversation
pandas 3 uses Arrow-backed string dtypes by default. Calling .values on such a Series returns an ArrowExtensionArray which does not support NumPy-style multi-dimensional indexing ([:, None]). Replace .values with np.asarray() to guarantee a NumPy array in all pandas versions.
DataFrame.applymap was deprecated in pandas 2.1 and removed in pandas 3.0. Use getattr fallback to DataFrame.map (added in 2.1) while retaining compatibility with pandas < 2.1.
In pandas 3, assigning an object value into a typed column raises a TypeError instead of emitting a FutureWarning as in pandas 2. Cast affected columns to object dtype before the Dummy assignment in DiscreteDependenciesConstraint to make the operation valid in all pandas versions. Update tests accordingly and remove the now-unnecessary FutureWarning suppression from pytest.ini.
pandas 3 with Copy-on-Write may return read-only NumPy arrays from .values. torch.from_numpy() raises a UserWarning for non-writable arrays, which pytest treats as an error. Use DataFrame.to_numpy(copy=True) to guarantee a writable array in all pandas versions.
- hypothesis_strategies/parameters.py: pandas 3 Arrow-backed string indices return ArrowStringArray from .index.values, which hypothesis sampled_from() cannot draw from; convert to list instead. - utils/dataframe.py: pandas 3 Copy-on-Write may return read-only NumPy arrays from to_numpy(copy=False); add a writability check and copy before passing to torch.from_numpy().
comp_rep_bounds.values returns a read-only NumPy array under pandas 3 Copy-on-Write semantics. Use to_numpy(copy=True) at the two call sites in sample_uniform so that torch.from_numpy() receives a writable array.
astype(copy=False) returns a read-only view when the array already has the target dtype. Add the same writability check as the pd.Series/ DataFrame branch to ensure torch.from_numpy() always receives a writable array.
The method accepts a bounds array that callers may source directly from comp_rep_bounds.values, which under pandas 3 CoW can be read-only. Guard with a writability check inside the method so all call paths are covered, including tests that call the method directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TBD: Refinement and Fixes
Pandas 3 is only supported for Python 3.11+ so the 3.10 test serves as a verification that everything (still) works in Pandas 2