Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .strip-manifest
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ sha256 32cb5b2719fd0f50a76a63e92ed0d81f2770d4a260bcc1c096f7d8cec3dc0dc7 pre_bu
sha256 fddaf7592554e14867cef501599350affa19d95b408a7c1f4a75b043625f3253 pre_built/el8.x86_64.glibc2p28/bin/stty.bz2
sha256 ea8663a62529908ee65ac75c41adc1874f13c98bd6a2887d9192ddc62810794a pre_built/el8.x86_64.glibc2p28/bin/stylua.bz2
sha256 6d62d65b8dcba1d9330f7e9375af1eddec1933d381917d43853a420df469f55c pre_built/el8.x86_64.glibc2p28/bin/sum.bz2
sha256 ef1ab201d70c0142e06817393ab229c0923003623aad7d6293afee1816aff32e pre_built/el8.x86_64.glibc2p28/bin/surfer.bin.bz2
sha256 66570d88c48d7b347f39c21dda2d137908b1b1d2573ae7bd6451c1d2cbe9a241 pre_built/el8.x86_64.glibc2p28/bin/surfer.bz2
sha256 0685493a201147e1bf8e62d9102d39c8a624d9b4837e2593daec4c73e1760f0a pre_built/el8.x86_64.glibc2p28/bin/sync.bz2
sha256 5eacac3388fae845870fa103a742a0bbf83397476ea844085d825f78ba3d21dc pre_built/el8.x86_64.glibc2p28/bin/tac.bz2
sha256 6c68418ef5fa20f1a20693b29f9a631c6be0f4d3f9bcdb74a978be1d98bb2ae4 pre_built/el8.x86_64.glibc2p28/bin/tail.bz2
Expand Down
9 changes: 7 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ pre_built/
build-octave.sh - Build GNU Octave from source (without Qt/Java/X11; gnuplot backend)
build-gvim.sh - Build GTK3 GUI vim (gvim.bin + gvim wrapper script) from source
build-nedit-ng.sh - Build nedit-ng Qt5 NEdit rewrite from source (CMake, single binary)
build-surfer.sh - Build Surfer waveform viewer (Rust egui/glow OpenGL GUI) from stable tag; submodule-aware, fresh CARGO_HOME, strip+patchelf+wrapper
reproduce-llvm-build.sh - LLVM build reproduction script
.strip-manifest - sha256/tar-meta cache for strip_all_elf_binaries

Expand Down Expand Up @@ -220,7 +221,7 @@ When installer runs, `resolve_tool_selection(args, registry)` performs:
5. Walk soft `recommends` -- silently drop skipped/unknown.
6. Filter by current platform (`linux`/`macos`/`windows`).

Group expansion (`expand_groups`) recursive, cycle-detected. Synthetic groups expand at runtime (not stored in `packages.json`): `@shared` -> every non-group package whose `kind != env` (everything for a shared/read-only tree); `@envs` -> its complement (every `kind == env` config bundle). `all` is also accepted as a name and expands to every non-group package. Tools and env config bundles are **fully decoupled** -- no cross-`recommends` in either direction (a tool never recommends its env bundle, an env bundle never recommends its tool) -- so `./loadout install @shared --dest-dir ...` (shared tree) and `./loadout install @envs` (per-user config) are clean complements needing no `--skip`/`--no-deps`. Both synthetic groups surface in `list --groups` and `describe <@group>` (driven by `_SYNTHETIC_GROUPS`) despite not being registry entries.
Group expansion (`expand_groups`) recursive, cycle-detected. Synthetic groups expand at runtime (not stored in `packages.json`): `@shared` -> every non-group, **non-`optional`** package whose `kind != env` (everything for a shared/read-only tree); `@envs` -> its complement (every `kind == env` config bundle). `all` is also accepted as a name and expands to every non-group, non-`optional` package. **`optional: true` is the only gating field** (there is no `default` field): packages so flagged are skipped by the `all` / `@shared` / `@engineering-loadout` sweeps and install only when named explicitly or pulled by a group that lists them in `members` -- e.g. `surfer` (name-only) and the `@rust` trio (`rust`, `rust-crate-store`, `env-cargo`, reachable via `@rust`). Tools and env config bundles are **fully decoupled** -- no cross-`recommends` in either direction (a tool never recommends its env bundle, an env bundle never recommends its tool) -- so `./loadout install @shared --dest-dir ...` (shared tree) and `./loadout install @envs` (per-user config) are clean complements needing no `--skip`/`--no-deps`. Both synthetic groups surface in `list --groups` and `describe <@group>` (driven by `_SYNTHETIC_GROUPS`) despite not being registry entries.

### Subcommands

Expand All @@ -240,7 +241,7 @@ Installer uses Click/rich-click subcommands (dnf/apt verbs). Bare `loadout` and

### Selection flags

- Positional `PKG...` -- install exactly these packages/`@groups` (deps still walked unless `--no-deps`). `all` = every non-group package.
- Positional `PKG...` -- install exactly these packages/`@groups` (deps still walked unless `--no-deps`). `all` = every non-group, non-`optional` package.
- `--skip NAMES` -- remove packages/`@groups` from install set (`install`, `reinstall`, `upgrade`, `resolve`).
- `--no-deps` -- install named set verbatim; no `depends`/`recommends` walk (`install`, `reinstall`, `upgrade`, `resolve`).
- `--force` -- continue past resolver errors (e.g. hard dep in skip set), printing `WARNING:` row instead of erroring (`install`, `reinstall`, `upgrade`, `resolve`).
Expand Down Expand Up @@ -293,6 +294,8 @@ Each phase installer (`install_prebuilt_binaries`, `install_fonts`, `install_tld

**VCD toggle profiler runtime behavior**: `vcd-toggle-profiler` is a C++17 runtime package built from `github.com/smprather/vcd-toggle-profiler` (C++ implementation only). The archive `pre_built/<platform>/runtime/vcd-toggle-profiler.tar.bz2` extracts to `~/.local/` and installs `bin/vcd-toggle-profiler` (POSIX-sh wrapper), `lib/vcd-toggle-profiler/vcd-toggle-profiler.bin` (real ELF), `share/vcd-toggle-profiler/uplot/{uPlot.iife.js,uPlot.min.css}`, and license files. The wrapper derives its prefix from its installed path and supplies `--uplot-js/--uplot-css` defaults so report generation works from any current directory while still allowing explicit user overrides. Build with `pre_built/build_scripts/build-vcd-toggle-profiler.sh --rev <git-ref>` or `--source <checkout>`; it deliberately uses EL8 system `/usr/bin/g++` 8.5 with `-lstdc++fs`, not upstream CMake Release (`-march=native`), keeping the deployed binary portable across farm CPUs and capped at `GLIBCXX_3.4.21`. `pigz` is recommended so `.vcd.gz` input uses the bundled fast gzip path when available. Re-run `./loadout completion bash > envs/bash/global/completions/loadout.bash` after changing the package registry.

**Surfer behavior**: `surfer` is an `optional` (name-only) `bin` package -- a Rust egui/**glow (OpenGL)** waveform viewer (VCD/FST/GHW). Built from the latest stable tag on EL8 by `pre_built/build_scripts/build-surfer.sh --tag vX.Y.Z` (native glibc-2.28; release profile has no `-march=native`). Packaged gvim-style: `bin/surfer.bz2` (POSIX-sh wrapper) + `bin/surfer.bin.bz2` (stripped ELF, RPATH `$ORIGIN/../lib64`). The wrapper mirrors wezterm's GL block (prepends `<prefix>/lib64` to `LD_LIBRARY_PATH`, sets `LIBGL_DRIVERS_PATH`/`__EGL_VENDOR_LIBRARY_DIRS`) before exec'ing `surfer.bin`. `depends: [gui_libs, mesa3d_libs]`; the ELF NEEDs only glibc + libgcc_s, winit/glow **dlopen** libGL/X11/wayland/xkbcommon at runtime (GLVND dispatcher stays host-provided, never bundled). Install: `./loadout install surfer` (auto-pulls gui_libs + mesa3d_libs). `surfer server` is a built-in headless mode. Build quirks (in `ADDING_BINARIES.md`): the v0.7.0 tag vendors f128 + instruction-decoder as **git submodules** (clone `--recurse-submodules`); the loadout's offline `~/.cargo/config.toml` registry-store must be bypassed with a fresh `CARGO_HOME`.

**GObject typelib behavior**: Installer copies `*.typelib` files from `pre_built/<platform>/typelibs/` to `~/.local/lib/girepository-1.0/`. `envs/bash/global/bashrc` exports `GI_TYPELIB_PATH=$HOME/.local/lib/girepository-1.0` when that dir exists, allowing Python tools using `import gi` (PyGObject) to find bundled typelibs. Required typelibs documented in `"typelibs"` key of `packages.json` entries (reference only -- installer copies all typelibs unconditionally). Typelib files from EL8 RPMs: `gobject-introspection` (GLib/GObject/Gio/GIRepository), `gtk3` (Gtk/Gdk/GdkPixbuf), `gtksourceview3` (GtkSource-3.0). Plain files, no strip/patchelf needed.

**Meld runtime behavior (shanghai bundle)**: Meld 3.20.4 bundled as "shanghai" -- extracted from system-installed EL8 package, repacked into `pre_built/<platform>/runtime/meld.tar.bz2`. Uses **system Python 3.6** (`/usr/bin/python3.6`) with bundled PyGObject 3.28.3 (gi) and pycairo 1.16.3. Bundle extracts to `~/.local/` and installs: `lib/python3.6/site-packages/{gi,cairo,meld,meld3}/`, `share/meld/`, `bin/meld` (Python 3.6 launcher). Launcher derives its local prefix from installed `bin/meld`, exports `LOADOUT_LOCAL_PREFIX`, and sets `sys.path`, `GI_TYPELIB_PATH`, `LD_LIBRARY_PATH` before `import gi` -- `LD_LIBRARY_PATH` must be set before `import gi` so `dlopen()` for `_gi.cpython-36m.so` finds installed `lib64/libgirepository-1.0.so.1`. Bundled `meld/conf.py` has DATADIR patched to `LOADOUT_LOCAL_PREFIX/share/meld` (was `/usr/share/meld`; fallback remains `~/.local/share/meld` only for direct import outside the launcher). Exclusively-owned libs: `libgirepository-1.0.so.1`, `libgtksourceview-3.0.so.1` (both patchelf'd RPATH `$ORIGIN`). Installer function: `install_meld_runtime()`. Typically installed with gui_libs: `./loadout install gui_libs,meld`. Installer warns if meld selected without gui_libs (GTK3/Qt5/X11 libs all from gui_libs). To update meld: re-`yum install meld` on EL8 build machine, re-run shanghai extract steps (copy gi/cairo/meld packages + data files into bundle dir, patch conf.py, create tar.bz2, bzip2 new libs if version changed, update packages.json version), run `./strip_all_elf_binaries`, commit. **Why shanghai instead of PyPI wheels**: PyGObject has no binary wheels on PyPI (source-only, requires gobject-introspection headers). EL8 ships GLib 2.56.4; PyGObject >= 3.36 requires GLib >= 2.62; PyGObject <= 3.30 (works with GLib 2.56) incompatible with Python 3.14 (`Py_TYPE()` assignment removed). System Python 3.6 sidesteps all of this.
Expand All @@ -309,6 +312,8 @@ Each phase installer (`install_prebuilt_binaries`, `install_fonts`, `install_tld

**Rolling-git wheel behavior**: First-party `python-tool` packages can be tagged `"rolling_git": "<clone-url>"` in `packages.json` (currently `liberty-tools`, `text-serdes`, `time-plot`). `./update <name>` (and bare `./update`, which now includes all rolling pkgs) rebuilds these from source instead of using a hand-bundled wheel: cheap `git ls-remote` skip-check -> fresh `git clone --filter=blob:none` (keeps tags so `describe` works; falls back to full clone) -> `uv build --wheel` (handles maturin **and** hatchling backends) -> prune previous `<dist>-*.whl` -> copy new wheel into `pre_built/<platform>/wheels/` -> surgically stamp the package's `version` to `git describe --tags --always --dirty`. Rebuild happens **only when the source commit changed**; `./update <name> --rebuild` forces. `./update --list[-outdated]` shows rolling rows with latest = remote HEAD sha. **maturin gotcha:** under PEP517 maturin tags a bare `linux_x86_64` wheel; `update` sets `MATURIN_PEP517_ARGS="--compatibility manylinux_2_28"` for maturin backends so the wheel is portable + auditwheel-checked. Builds need network + `~/.cache/uv` write (run on the EL8 build machine). Add a 4th rolling project by adding the `rolling_git` field -- `./update` discovers it automatically. To add a new third-party (pinned) Python tool instead, see *Python tool behavior* below.

**cicwave Python tool behavior**: `cicwave` optional `uv_tool` (`optional: true` -- opt in with `./loadout install cicwave`; not in any bulk group). PyQtGraph waveform viewer (ngspice/Xyce/VCD/CSV). It is a **loadout PyQt6 fork** of the upstream PySide6 app: PySide6 has **no wheel that is both EL8 (glibc 2.28) and Python 3.14** -- 6.9.x is `manylinux_2_28` but `python<3.14`; 6.10+ added 3.14 but jumped to `manylinux_2_34` (glibc 2.34/RHEL9, a one-way ratchet). PyQt6 satisfies both (`PyQt6 6.9.1` abi3 cp39, `PyQt6-Qt6 6.9.2` `manylinux_2_28` with `libQt6Core` floor `GLIBC_2.28`, `PyQt6-sip` cp314). The fork lives as `pre_built/build_scripts/cicwave/0001-port-pyside6-to-pyqt6.patch` (PySide6->PyQt6 imports, `pyqtSignal as Signal`, **25 strict-enum scopings** like `Qt.AlignCenter -> Qt.AlignmentFlag.AlignCenter`, two dynamic `getattr(QPalette.ColorRole, ...)` / `ShortcutContext` fixes, and the pyproject `PySide6 -> PyQt6` dep so the wheel resolves offline). Build/bundle with `pre_built/build_scripts/build-cicwave.sh --tag 0.5.2` (clones stable tag, applies patch, `uv build`, downloads the PyQt6 + matplotlib closure as EL8/cp314 wheels, splits the ~79 MB `pyqt6_qt6` wheel into `.whl.part-NNN` -- the installer's `_prepare_wheels_dir` rejoins it like the polars/pyarrow wheels). numpy/pandas/click/pyyaml/packaging/python_dateutil/six are reused from the existing bundle. Headless-verify with `QT_QPA_PLATFORM=offscreen` + `CICSIM_USE_OPENGL=0` (offscreen Qt has no GL); interactive paths need a real X/WSLg smoke after a version bump. See `ADDING_BINARIES.md` -> "cicwave".

**JupyterLab Python tool behavior**: `jupyterlab` optional `uv_tool` (`optional: true` -- opt in with `./loadout install jupyterlab`). Installed via `uv tool install jupyterlab` using bundled wheels. After install, `jupyter` and `jupyter-lab` launchers at `~/.local/bin/`. Users run `jupyter lab`; JupyterLab opens in system browser. Requires accessible browser (WSL2: Windows browser via WSL interop; headless farm nodes: point `BROWSER` to VNC-accessible browser or use `--no-browser --port=8888` with port forwarding). Wheels must be downloaded with `PIP_REQUIRE_VIRTUALENV=0 pip3.14 download jupyterlab --platform manylinux_2_28_x86_64 --python-version 3.14 --only-binary :all: -d pre_built/<platform>/wheels/` -- JupyterLab has ~80 dependency packages.

**Backup behavior**: Numbered backups in `loadout_backups/backup.N/` (always starts at `.1`; never bare `backup`). Skips files already pointing to repo. Never overwrites existing backups. At end of successful install, backup dir compressed to `loadout_backups/backup.N.tar.bz2`, uncompressed dir removed; numbering checks both `backup.N/` and `backup.N.tar.bz2` when picking next N. Post-install hooks (receive `LOADOUT_BACKUP_DIR`) run before compression. `./loadout snapshot restore <path>` accepts uncompressed dirs or `.tar.bz2` archives (extracts to `/tmp`, restores). Main backups exclude font files (`*.ttf`, `*.otf`, `*.pcf`, `*.bdf`, `*.woff`, `*.woff2`, etc.) -- large and reproducible. The fonts phase has its own `fonts.bak*` move for normal installs and suppresses it under `--no-backup`.
Expand Down
Loading
Loading