From aaa8bd1ded67d52f6e6944021c4efcfacd168e5c Mon Sep 17 00:00:00 2001 From: Robert Samples Date: Mon, 29 Jun 2026 01:38:37 -0400 Subject: [PATCH 1/4] Add PyInstaller-based portable Windows build (no Anaconda required) Adds requirements.txt, build/mpact.spec, and build/build_windows.bat to produce a standalone dist/MPACT/ folder end users can run without installing Anaconda or any packages themselves. Built and smoke-tested: the frozen exe imports PyQt5/pandas/numpy/scipy/matplotlib/seaborn/ scikit-learn/epam.indigo cleanly and reaches the styled main window under an offscreen Qt platform. Two non-obvious choices, documented in build/README.md and CLAUDE.md: --onedir (not --onefile), since main.py reads/writes npatlas.tsv and compoundimages/*.png via cwd-relative paths that need to persist next to the exe; and contents_directory='.', since PyInstaller 6.x's default onedir layout nests those files under _internal/ instead, breaking the same assumption. epam.indigo's native DLL needed an explicit collect_all('indigo') -- its hand-rolled load path is invisible to PyInstaller's static import analysis. Co-Authored-By: Claude Sonnet 4.6 --- .gitignore | 8 +++ CLAUDE.md | 52 ++++++++++++++++++++ build/README.md | 97 +++++++++++++++++++++++++++++++++++++ build/build_windows.bat | 57 ++++++++++++++++++++++ build/mpact.spec | 105 ++++++++++++++++++++++++++++++++++++++++ requirements.txt | 31 ++++++++++++ 6 files changed, 350 insertions(+) create mode 100644 build/README.md create mode 100644 build/build_windows.bat create mode 100644 build/mpact.spec create mode 100644 requirements.txt diff --git a/.gitignore b/.gitignore index 0ec0604..6b57e20 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,13 @@ site/ code/__pycache__/ + +# PyInstaller portable-build artifacts (see build/mpact.spec, CLAUDE.md). +# dist/ and build/mpact/ are generated by `pyinstaller build/mpact.spec`; +# build/.buildvenv/ is the throwaway venv build_windows.bat creates. +dist/ +build/mpact/ +build/.buildvenv/ + code/__pycache__/stats.cpython-39.pyc code/__pycache__/plotting.cpython-39.pyc code/__pycache__/MSFaST.cpython-39.pyc diff --git a/CLAUDE.md b/CLAUDE.md index 3fd1a72..a51cf75 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,6 +19,58 @@ in `code/`. only resolves when run as a script. So the GUI stack cannot be imported headlessly. +## Portable build (no Anaconda required) + +`build/mpact.spec` + `build/build_windows.bat` produce a standalone Windows +folder build via PyInstaller that end users can unzip and run without +installing Anaconda, Python, or any packages. See `build/README.md` for the +build steps and the reasoning behind the two non-obvious spec choices +(`--onedir` not `--onefile`; `contents_directory='.'`) and the dependency +gotchas (`epam.indigo`'s native DLL needs `collect_all`, not just +`hiddenimports`; `requirements.txt` intentionally allows NumPy 2.x, unlike the +Anaconda-env cap below). Built and smoke-tested 2026-06-29 (offscreen launch, +all imports including indigo's native lib resolved cleanly); not yet run +against real data end-to-end by a human. + +### Portable builds for Linux / Mac (not yet implemented) + +Not attempted yet — logged here for whoever picks this up next, since the +Windows build surfaced platform-specific issues that will very likely recur: + +- **PyInstaller itself is cross-platform but builds aren't** — you cannot + cross-compile a Linux or Mac build from this Windows dev machine; each + needs to be built (and ideally smoke-tested per `build/README.md`) on that + OS. The existing `build/mpact.spec` should mostly carry over unchanged + (`pathex`, `datas`, `hiddenimports` are all OS-agnostic Python), but the + `epam.indigo` fix is the one item proven to be OS-specific: its DLL lookup + path is `indigo/lib//...` (`windows-x86_64` here) — confirm + `epam.indigo` actually ships Linux (`linux-x86_64`) and Mac + (`mac-x86_64`/`mac-arm64`) native libraries in its PyPI wheel for the + target platform/architecture before assuming `collect_all('indigo')` just + works there too; if not, that dependency may need a different + installation path (e.g. a system package) on that OS. +- **macOS app bundling**: a plain PyInstaller onedir folder runs on Mac, but + for something distributable as a normal double-clickable app, the spec + would need a `BUNDLE(...)` step (PyInstaller supports this directly in the + same spec file) to produce a `MPACT.app`. Unsigned/unnotarized `.app` + bundles get Gatekeeper warnings on other people's Macs — fine for the + team's own use, a blocker for wider distribution without an Apple + Developer signing certificate. +- **Linux**: PyInstaller onedir output runs fine as a plain executable + + folder (no installer step strictly required), but Qt's platform plugin + selection is more failure-prone than on Windows/Mac — expect to need + `QT_QPA_PLATFORM`/`xcb` library checks on whatever distro it's tested on + first, the same class of issue the offscreen-platform smoke test exists to + catch early (`build/README.md`). +- **`run.bat`'s cwd-relative-path assumption** (`code/main.py` reading + `npatlas.tsv`/`compoundimages/` relative to cwd, not `__file__`) is + platform-agnostic in principle — a Linux/Mac launcher script just needs + the equivalent `cd` into the build's own folder before exec'ing the binary, + same as `--contents-directory .` solves it for the Windows PyInstaller + build. +- Given all of the above is unverified, treat any of it as a starting + hypothesis to confirm on real Linux/Mac hardware, not a finished plan. + ## ⚠️ NumPy 2.x / Anaconda dependency hazard (read before touching deps) This runs in a conda env whose pandas/matplotlib/scipy are compiled against diff --git a/build/README.md b/build/README.md new file mode 100644 index 0000000..778b632 --- /dev/null +++ b/build/README.md @@ -0,0 +1,97 @@ +# Portable Windows build (no Anaconda required) + +Produces a folder you can zip and hand to someone with no Python, no +Anaconda, and no `pip install` of their own -- they just unzip and double-click +`MPACT.exe`. + +## Build it + +``` +build\build_windows.bat +``` + +This creates a throwaway venv at `build\.buildvenv\` (never touches your dev +Anaconda env), installs `requirements.txt` + PyInstaller into it, and runs +`pyinstaller build\mpact.spec`. Output: `dist\MPACT\` -- a folder, **not** a +single exe. Ship the whole folder; `dist\MPACT\MPACT.exe` is the launcher. + +Re-running the script reuses the existing venv. Delete `build\.buildvenv\` to +force a clean dependency reinstall; delete `dist\` and `build\mpact\` to force +a clean PyInstaller rebuild (the batch script does the build step +unconditionally, but PyInstaller itself caches via its own `build\mpact\*.toc` +files). + +## Why onedir, not onefile, and why `--contents-directory .` + +- **`--onedir` (the PyInstaller default this spec uses), not `--onefile`.** + `code/main.py` reads/writes `npatlas.tsv` and `compoundimages/*.png` via + bare relative paths (e.g. `'compoundimages/' + cmpd + '.png'`), resolved + against the process's current working directory -- not against the + installed package location. A onefile build extracts everything into a + fresh temp directory on *every launch* and deletes it on exit, so anything + main.py wrote into `compoundimages/` (newly-rendered structure PNGs) would + vanish the moment the app closed, and onefile startup is also slower + (extract-then-run vs. run-in-place). Onedir keeps those files persistently + next to the exe, matching how `run.bat` already works (it `cd`s into + `code\` before launching, for the same cwd-relative-paths reason -- see + CLAUDE.md). +- **`contents_directory='.'` on the `EXE(...)` call in `mpact.spec`.** + PyInstaller 6.0 changed the onedir default to nest everything except the + exe itself under `dist/MPACT/_internal/`, to keep the top-level folder + clean. That breaks the same cwd-relative-path assumption above (cwd is + `dist/MPACT/`, but `npatlas.tsv` would land in `dist/MPACT/_internal/`). + Setting `contents_directory='.'` restores the pre-6.0 flat layout instead + of patching main.py's path handling to be build-system-aware. + +## Dependency notes specific to this build + +- `requirements.txt` deliberately does **not** carry the `numpy<2` cap that + `importdependencies.py` / CLAUDE.md document for the Anaconda dev env. That + cap exists only to protect conda-compiled pandas/matplotlib/scipy binaries + built against NumPy 1.x's ABI -- it doesn't apply to a clean pip venv, where + every wheel is built against NumPy 2.x from the start. Capping it here + would actually make pip's resolve fail outright: the PyPI `fastcluster` + wheel for Python 3.12 requires `numpy>=2`. (The codebase has no NumPy + 1.x-only API usage -- `np.float`/`np.int`/etc. were checked and aren't + used -- so NumPy 2.x is safe here.) +- `epam.indigo` ships a compiled cheminformatics engine + (`indigo/lib//*.dll`) that it loads via its own hand-rolled path + lookup in `indigo/_common/lib.py`, completely invisible to PyInstaller's + static import analysis -- a plain `hiddenimports=['indigo']` is not + enough; the DLL itself never gets copied and the app fails at + `Indigo()` construction with `RuntimeError: Could not find native + libraries`. Fixed in `mpact.spec` via + `PyInstaller.utils.hooks.collect_all('indigo')`, which pulls in the + package's data files (the DLLs included) and binaries, not just its + Python modules. +- scikit-learn and scipy ship compiled extension submodules that their + PyInstaller hooks don't always discover via static analysis (import-time + C-level plugin lookups). A few are listed explicitly in + `hiddenimports` in `mpact.spec`; if a future scikit-learn/scipy upgrade + introduces an `ImportError`/`ModuleNotFoundError` only in the *frozen* exe + (not when running `python main.py` normally), that's almost always this + category of problem -- check `build\mpact\warn-mpact.txt` after a build + for "not found" hidden-import warnings as a starting point. + +## Smoke-testing a build + +The frozen exe can be sanity-checked headlessly (no display needed) the same +way the test suite's `conftest.py` does it, via Qt's offscreen platform: + +``` +set QT_QPA_PLATFORM=offscreen +dist\MPACT\MPACT.exe +``` + +It won't get far without real peak-table data, but a clean launch with no +`Traceback`/`ImportError` in the console confirms every dependency (PyQt5, +pandas, indigo's native DLL, etc.) is actually bundled and importable in the +frozen environment -- which is the failure mode this kind of build most +commonly hits, not application logic bugs. + +## Linux / Mac builds + +Not yet built or tested on those platforms. See the "Portable builds for +Linux / Mac" section in the repo-root `CLAUDE.md` for what's expected to +carry over from this Windows build vs. what would need platform-specific +work, for whoever picks this up next. diff --git a/build/build_windows.bat b/build/build_windows.bat new file mode 100644 index 0000000..6912da1 --- /dev/null +++ b/build/build_windows.bat @@ -0,0 +1,57 @@ +@echo off +setlocal + +rem Builds a portable, Anaconda-free Windows distribution of MPACT using +rem PyInstaller, in a throwaway venv (so it never touches the dev Anaconda +rem env). Resolves paths via %~dp0 so it works regardless of where the repo +rem is cloned. Run from anywhere; output lands in dist\MPACT\ (a folder -- +rem ship the whole folder, launch via dist\MPACT\MPACT.exe). +rem +rem Requires: a plain (non-Anaconda) Python 3.9-3.12 on PATH for creating the +rem build venv. Anaconda's own Python works too if that's all you have. + +set "REPO_DIR=%~dp0.." +set "VENV_DIR=%REPO_DIR%\build\.buildvenv" + +if not exist "%VENV_DIR%\Scripts\python.exe" ( + echo Creating build venv at "%VENV_DIR%"... + python -m venv "%VENV_DIR%" + if errorlevel 1 ( + echo Failed to create venv. Is Python installed and on PATH? + pause + exit /b 1 + ) +) + +call "%VENV_DIR%\Scripts\activate.bat" + +echo Installing runtime dependencies... +pip install --upgrade pip +pip install -r "%REPO_DIR%\requirements.txt" +if errorlevel 1 ( + echo Dependency install failed. + pause + exit /b 1 +) + +echo Installing PyInstaller... +pip install pyinstaller +if errorlevel 1 ( + echo PyInstaller install failed. + pause + exit /b 1 +) + +echo Building MPACT.exe... +cd /d "%REPO_DIR%" +pyinstaller build\mpact.spec --noconfirm +if errorlevel 1 ( + echo Build failed. + pause + exit /b 1 +) + +echo. +echo Build complete: %REPO_DIR%\dist\MPACT\MPACT.exe +echo Ship the entire dist\MPACT\ folder -- it is not a single-file exe. +pause diff --git a/build/mpact.spec b/build/mpact.spec new file mode 100644 index 0000000..3ab9daf --- /dev/null +++ b/build/mpact.spec @@ -0,0 +1,105 @@ +# -*- mode: python ; coding: utf-8 -*- +# +# PyInstaller spec for a portable MPACT build (no Anaconda / conda env needed +# on the end user's machine). See build/README.md for how to run this and +# CLAUDE.md for the background on why --onedir (not --onefile) was chosen. +# +# Build from the repo root with: +# pyinstaller build/mpact.spec --noconfirm +# +# Output lands in dist/MPACT/ (a folder, not a single exe) -- ship the whole +# folder. dist/MPACT/MPACT.exe is the launcher. + +import os + +from PyInstaller.utils.hooks import collect_all + +block_cipher = None + +REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(SPEC), os.pardir)) +CODE_DIR = os.path.join(REPO_ROOT, "code") + +# Data files main.py reads/writes via cwd-relative paths (run.bat / this spec +# both assume the process cwd is the folder main.py itself lives in -- here, +# the dist/MPACT/ folder next to the exe). npatlas.tsv is read-only reference +# data; compoundimages/ is a read+write cache of rendered structure PNGs that +# main.py creates new entries in at runtime (code/main.py ~line 664), so it +# must land in a writable location next to the exe, not inside the PyInstaller +# onefile temp-extraction dir -- this is the main reason this build uses +# --onedir instead of --onefile. +datas = [ + (os.path.join(CODE_DIR, "npatlas.tsv"), "."), + (os.path.join(CODE_DIR, "compoundimages"), "compoundimages"), +] +binaries = [] + +# Packages whose PyInstaller hooks don't always pick up every native/data +# file automatically. epam.indigo ships its own compiled cheminformatics +# engine (indigo/lib//*.dll, loaded by a hand-rolled path lookup in +# indigo's own _common/lib.py -- not detected by static import analysis at +# all, so it must be pulled in explicitly via collect_all); scikit-learn/scipy +# ship compiled extension modules with import-time plugin discovery that +# static analysis can miss. +hiddenimports = [ + "sklearn.utils._cython_blas", + "sklearn.neighbors._quad_tree", + "sklearn.tree._utils", + "scipy.special.cython_special", +] +for pkg in ("indigo",): + pkg_datas, pkg_binaries, pkg_hiddenimports = collect_all(pkg) + datas += pkg_datas + binaries += pkg_binaries + hiddenimports += pkg_hiddenimports + +a = Analysis( + [os.path.join(CODE_DIR, "main.py")], + pathex=[CODE_DIR], + binaries=binaries, + datas=datas, + hiddenimports=hiddenimports, + hookspath=[], + hooksconfig={}, + runtime_hooks=[], + excludes=[], + noarchive=False, + cipher=block_cipher, +) + +pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher) + +exe = EXE( + pyz, + a.scripts, + [], + exclude_binaries=True, + name="MPACT", + debug=False, + bootloader_ignore_signals=False, + strip=False, + upx=False, + # Keep a console window, matching run.bat's existing behavior: MPACT + # prints non-fatal errors/progress to stdout and there's no in-app log + # viewer yet (see the main.py TODO about a status-bar terminal view), so + # console=False would silently swallow diagnostics a user might need to + # report a bug. + console=True, + icon=os.path.join(REPO_ROOT, "cog.ico") if os.path.isfile(os.path.join(REPO_ROOT, "cog.ico")) else None, + # PyInstaller >=6.0 defaults onedir builds to nesting everything except + # the exe under an _internal/ subfolder. main.py's data paths + # ('npatlas.tsv', 'compoundimages/...') are bare and resolved against the + # process cwd, which is dist/MPACT/ (the exe's own folder) when launched + # normally -- so keep the flat pre-6.0 layout, putting datas/binaries + # directly alongside MPACT.exe instead of in _internal/. + contents_directory=".", +) + +coll = COLLECT( + exe, + a.binaries, + a.zipfiles, + a.datas, + strip=False, + upx=False, + name="MPACT", +) diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..53f94f9 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,31 @@ +# Runtime dependencies for MPACT, pinned to versions verified against the +# current code/ tree. Used by the PyInstaller portable build +# (build/build_windows.bat), which installs these into a throwaway venv. +# +# NOTE this is NOT the same numpy constraint as importdependencies.py / the +# Anaconda dev env hazard documented in CLAUDE.md. That cap (numpy<2) exists +# only to protect conda-compiled pandas/matplotlib/scipy binaries built +# against the NumPy 1.x ABI -- it does not apply here, because this file +# provisions a clean pip-only venv where pandas/scipy/etc. are pip wheels +# built against NumPy 2.x's ABI from the start. Capping numpy<2 here instead +# makes the resolve impossible (fastcluster's PyPI wheel for Python 3.12 +# requires numpy>=2), and the codebase has no numpy 1.x-only API usage +# (`np.float`/`np.int`/etc. were checked and are not used), so numpy 2.x is +# safe for this build. +pandas==2.2.2 +scipy==1.13.1 +matplotlib==3.8.4 +seaborn==0.13.2 +PyQt5==5.15.10 +scikit-learn==1.4.2 +tqdm==4.66.4 +epam.indigo==1.23.0 +UpSetPlot==0.9.0 +squarify==0.4.4 + +# Optional: faster hierarchical clustering for the heatmap/bootstrap dendrogram. +# importdependencies.py installs this automatically at runtime if missing and +# never fails startup if it can't be installed -- listed here only so the +# portable build ships with it preinstalled instead of trying to pip install +# at first launch (which the frozen exe's end users won't be able to do). +fastcluster==1.3.0 From 2d42b9c04949297ff01a8134d146325b2733b267 Mon Sep 17 00:00:00 2001 From: Robert Samples Date: Mon, 29 Jun 2026 09:23:43 -0400 Subject: [PATCH 2/4] Fix multiprocessing fork bomb in the frozen exe (freeze_support) pvclust.py's bootstrap dendrogram uses multiprocessing.Pool(), which on Windows needs multiprocessing.freeze_support() called before any other code when running from a frozen (PyInstaller) executable. Without it, each worker process re-executes the entire bundled program from scratch -- including creating a brand new QApplication and MainWindow -- instead of just running the pool target. Confirmed this exact failure mode with a minimal standalone repro exe: without freeze_support(), a 4-worker Pool() in a frozen build recurses exponentially (observed tens of thousands of relaunched processes before the host OS ran out of resources). With the guard added here, multiprocessing.freeze_support() is a documented no-op under any non-frozen interpreter (plain `python main.py`, Spyder), so this is safe for every other way MPACT is launched. Co-Authored-By: Claude Sonnet 4.6 --- code/main.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/code/main.py b/code/main.py index 4a25052..bd432d9 100644 --- a/code/main.py +++ b/code/main.py @@ -6,6 +6,7 @@ import re import sys +import multiprocessing import numpy as np import pandas as pd pd.options.mode.chained_assignment = None # default='warn' @@ -1350,6 +1351,13 @@ def mousePressEvent(self, event): if __name__ == "__main__": + # Required for pvclust.py's multiprocessing.Pool() (bootstrap dendrogram) + # to work in a PyInstaller-frozen exe: without it, each worker process + # re-executes the whole frozen program from scratch instead of running + # just the pool target, spawning a new QApplication/MainWindow per + # worker. A no-op everywhere else (plain `python main.py`, Spyder), + # so safe to call unconditionally. + multiprocessing.freeze_support() app = QtWidgets.QApplication(sys.argv) if sys.platform != 'win32': app.setStyle('Fusion') From 87ae4d6cfb6ac4003b3aa4bc9319b9ecc6866d7c Mon Sep 17 00:00:00 2001 From: Robert Samples Date: Mon, 29 Jun 2026 09:33:09 -0400 Subject: [PATCH 3/4] Bump version to v1.00.01 Co-Authored-By: Claude Sonnet 4.6 --- code/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/code/main.py b/code/main.py index bd432d9..c9d3528 100644 --- a/code/main.py +++ b/code/main.py @@ -237,7 +237,7 @@ def __init__(self): #self.show() # makes window reappear, acts like normal window now (on top now but can be underneath if you raise another window) self.ui.setupUi(self) - self.ui.label_credits.setText('v1.00 r24.11.17') + self.ui.label_credits.setText('v1.00.01 r26.06.29') #initialize other dialog windows self.dialog = dialog() From a3c2890a3a5ca72a8567bccf508fffb4f8033202 Mon Sep 17 00:00:00 2001 From: Robert Samples Date: Mon, 29 Jun 2026 09:36:22 -0400 Subject: [PATCH 4/4] format devnotes --- CLAUDE.md => devnotes.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) rename CLAUDE.md => devnotes.md (98%) diff --git a/CLAUDE.md b/devnotes.md similarity index 98% rename from CLAUDE.md rename to devnotes.md index a51cf75..b99c9dd 100644 --- a/CLAUDE.md +++ b/devnotes.md @@ -1,8 +1,7 @@ -# CLAUDE.md — MPACT +# devnotes.md — MPACT -Guidance for Claude Code (and humans) working in this repo. MPACT is a PyQt5 -desktop tool for mass-spectrometry / natural-products data analysis. Code lives -in `code/`. +MPACT is a PyQt5 desktop tool for mass-spectrometry / natural-products data analysis. +Code runs in ./code. ## Running the app @@ -71,7 +70,7 @@ Windows build surfaced platform-specific issues that will very likely recur: - Given all of the above is unverified, treat any of it as a starting hypothesis to confirm on real Linux/Mac hardware, not a finished plan. -## ⚠️ NumPy 2.x / Anaconda dependency hazard (read before touching deps) +## NumPy 2.x / Anaconda dependency hazard (read before touching deps) This runs in a conda env whose pandas/matplotlib/scipy are compiled against **NumPy 1.x**. A bare `pip install ` can upgrade NumPy to 2.x, which breaks