Migrate Python worker containers to Python 3.11#59
Open
alptaciroglu wants to merge 3 commits into
Open
Conversation
…ry support All 14 Python worker containers now use FROM python:3.11 (8 migrated from Python 2, 6 pinned from python:3). Includes full compatibility fixes for pandas 2.0+, numpy 2.0, and htslib 1.21. Tested end-to-end with CD.gwas example data on a local FUMA installation (SNP2GENE pipeline, all steps pass). Changes: - Dockerfiles: pin all Python containers to python:3.11 - Python 2→3 syntax: print statements, configparser, .to_numpy() - Pandas 2.0+: delim_whitespace→sep, DataFrame.append→pd.concat, dtype fix - NumPy 2.0: np.in1d→np.isin (10 files) - subprocess.check_output bytes decode (getLD.py) - tabix -p vcf → -s 1 -b 2 -e 2 (allSNPs.py, htslib 1.21 compat) - R Dockerfiles: add CMD ["R", "--vanilla"] (5 files) - MAGMA Dockerfiles: curl -L for SURFsara URL redirect (2 files) - requirements.txt: remove Python 2 packages, unpin versions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
np.array() on lists of different lengths raises ValueError in modern NumPy. Adding dtype=object preserves the ragged structure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
numpy int64 values in gene_order/label_order arrays are not JSON-serializable in Python 3. Use .tolist() which recursively converts to native Python types. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FROM python:3.11(8 migrated from Python 2, 6 pinned frompython:3)delim_whitespace→sep=r'\s+',DataFrame.append→pd.concat)np.in1d→np.isinacross 10 files, 14 occurrences)tabix -p vcf→tabix -s 1 -b 2 -e 2in allSNPs.py)subprocess.check_output().decode()in getLD.py)astype(int)→astype(int).astype(str)in allSNPs.py)Files changed (40 files, all in
scripts/)Dockerfiles (14): All set to
FROM python:3.11getld,geteqtl,getci,snpannot,genemap,qqsnps_filt,locusPlot,celltype_plot_datapython:3:allsnps,gwas_file,getgwascatalog,annotPlot,magma,create_circos_plotPython scripts: pandas, numpy, subprocess, and tabix compatibility fixes
Test plan
🤖 Generated with Claude Code