From 76216c96e55b8b46a595de56df1e8d87f1a56393 Mon Sep 17 00:00:00 2001 From: Alexandre Boucaud Date: Fri, 8 May 2026 00:01:14 +0200 Subject: [PATCH 1/8] Improve installation instructions --- docs/user/install.md | 77 +++++++++++++++++++++++++------------------- 1 file changed, 43 insertions(+), 34 deletions(-) diff --git a/docs/user/install.md b/docs/user/install.md index c3ba817a..f734de31 100644 --- a/docs/user/install.md +++ b/docs/user/install.md @@ -1,50 +1,60 @@ # Install -You need three things on your machine: Python 3.11+, the `lc` CLI, and -Claude Code. A container runtime is optional but recommended. +To get started on a lightcone project, you need three things on your machine: Python 3.11+, the lightcone command line tool `lc`, and +an agent-based CLI (currently supporting Claude Code). +A container runtime is optional but recommended. -## 1. Python 3.11+ +## 1. Python -If you don't already have a recent Python: +If you don't already have a recent Python -- macOS: `brew install python@3.12` -- Linux: your package manager (`apt install python3.12`, etc.) or - [pyenv](https://github.com/pyenv/pyenv). -- Windows: [python.org](https://www.python.org/downloads/) or WSL. +=== "macOS" + ```bash + brew install python@3.12 + ``` -Confirm: +=== "Linux" + Your package manager (`apt install python3.12`, etc.) or + [pyenv](https://github.com/pyenv/pyenv) -```bash -python3 --version # → Python 3.11.x or newer -``` +=== "Windows" + [python.org](https://www.python.org/downloads/) or WSL + + +!!! tip "Recommendation" + We highly recommend the use of [uv](https://docs.astral.sh/uv/) to manage Python installation and virtual environments. + + `uv` can be installed in a single commandline + ```bash + curl -LsSf https://astral.sh/uv/install.sh | sh + ``` + and a subsequent version of Python + ``` + uv python install 3.12 + ``` ## 2. lightcone-cli The published name on PyPI is `lightcone-cli`; the command it provides is `lc`. -```bash -pip install lightcone-cli -``` - -If you use [uv](https://docs.astral.sh/uv/) (recommended): +=== "uv" + ```bash + uv tool install lightcone-cli + ``` -```bash -uv pip install lightcone-cli -# or, project-local: -uv tool install lightcone-cli -``` +=== "pip" + ```bash + python -m pip install lightcone-cli + ``` -Confirm: +Get a confirmation of the proper installation by running ```bash lc --version # → lightcone-cli, version ... ``` -> **Heads-up about the `lc` name.** `lc` is not a standard Unix tool, -> but a few people have a personal shell alias `lc='ls --color'`. If -> that's you, installing lightcone-cli will shadow the alias — -> rebind it (e.g. `alias l='ls --color'`). +> **Note** Some people may have already set a personal shell alias `lc='ls --color'`. If that's you, installing lightcone-cli will shadow the alias — make sure to rebind it (e.g. `alias l='ls --color'`). ## 3. Global configuration @@ -58,25 +68,24 @@ container: ``` `auto` detects whichever of `podman`, `docker`, or `podman-hpc` is on -your PATH (and skips docker if its daemon isn't running). You can pin -the runtime later by editing this file directly. +your PATH (and skips docker if its daemon isn't running). Feel free to pin the runtime later by editing this file directly. -## 4. Claude Code +## 4. Agentic CLI -Most of your interactions with lightcone-cli happen *through* Claude -Code, the CLI that drives Claude. +Most of your interactions with a lightcone project happen *through* an agent-based CLI, for now we are supporting Claude Code. +Install Claude Code ```bash curl -fsSL https://claude.ai/install.sh | bash ``` -Open a project (in the next page we make one) with: +Open a project in your terminal or editor (see [Getting Started](getting-started.md)) and run ```bash claude ``` -Inside Claude Code you'll type slash commands like `/lc-new` and +Inside Claude Code you will have dedicated lightcone CLI slash commands available like `/lc-new` and `/lc-build` — see [The Claude Code Workflow](claude-workflow.md). ## 5. (Optional) Docker or Podman From 29bec15153cfc547e7da902e7c827d375734ade7 Mon Sep 17 00:00:00 2001 From: dkn16 Date: Thu, 7 May 2026 13:58:35 -0700 Subject: [PATCH 2/8] docs(user): add NERSC (Perlmutter) guide under user/ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reorganized away from the now-removed docs/hpc/ tree and into the new docs/user/ user-guide structure. The page is a focused, site- specific walkthrough: install Claude Code → pick a Python env (with the 40 GB home-quota → \$SCRATCH symlink note) → install lightcone-cli from PyPI or source → init a project → run on compute nodes via salloc + claude. Wired into the User Guide nav as "NERSC (Perlmutter)" right after "Running on a Cluster" (which it complements with site-specific overlays), and cross-linked from cluster.md so NERSC users land here naturally. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/user/cluster.md | 5 + docs/user/nersc.md | 300 +++++++++++++++++++++++++++++++++++++++++++ zensical.toml | 1 + 3 files changed, 306 insertions(+) create mode 100644 docs/user/nersc.md diff --git a/docs/user/cluster.md b/docs/user/cluster.md index dec7f8e5..29aa557c 100644 --- a/docs/user/cluster.md +++ b/docs/user/cluster.md @@ -5,6 +5,11 @@ a SLURM HPC system. There's no separate configuration to learn — the same `lc run` command works inside an allocation, just with more hardware to spread across. +> On NERSC Perlmutter, the filesystem layout (DVS-mounted home, Lustre +> scratch) and the `module load conda` workflow add a few site-specific +> considerations. See [NERSC (Perlmutter)](nersc.md) for a focused +> walkthrough. + ## The big picture `lc run` always dispatches through a Dask cluster. Three branches: diff --git a/docs/user/nersc.md b/docs/user/nersc.md new file mode 100644 index 00000000..eeb41505 --- /dev/null +++ b/docs/user/nersc.md @@ -0,0 +1,300 @@ +# lightcone-cli on NERSC (Perlmutter) + +A practical guide for running [`lightcone-cli`](https://github.com/LightconeResearch/lightcone-cli) on **Perlmutter**. The CLI itself behaves the same as on a laptop — the wrinkles are in the filesystem layout (DVS-mounted home, Lustre scratch), the container runtime (`podman-hpc`), and SLURM submission. This page covers all three. + +!!! tip "Already familiar with the basics?" + The generic [Install](install.md) and [Running on a Cluster](cluster.md) pages cover the cross-platform story. This page is the NERSC-specific overlay — read it first if Perlmutter is your home base. + +--- + +## 0. Agentic CLI + +`lightcone-cli` is the execution layer of the `lightcone` project — it harnesses an agent-based CLI (currently [Claude Code](https://docs.claude.com/en/docs/claude-code/setup)) to follow the `astra` standard while building and running an analysis. So the very first step, even before touching `lightcone-cli` itself, is to install the agent: + +```bash +curl -fsSL https://claude.ai/install.sh | bash # installs to ~/.local/bin/claude +``` + +Make sure `~/.local/bin` is on your `PATH`, then verify and authenticate: + +```bash +claude --version +claude # first run prompts for login (claude.ai or API key) +``` + +Other install routes (npm, native package managers) are documented in the [Claude Code installation docs](https://docs.claude.com/en/docs/claude-code/setup). + +--- + +## 1. Python + +NERSC's `python` module gives you a ready-to-use Python distribution with `conda`, `pip`, and many common scientific packages already installed — no env creation needed for the basics: + +```bash +module load python # NERSC Python (3.11+); brings conda and pip onto PATH +``` + +That's enough for installing `lightcone-cli` on top. Skip ahead to [§2](#2-install-lightcone-cli). + +!!! note "When you'd want your own conda env" + The NERSC python module is shared and read-only. You *can* layer user-level packages on top, but you can't pin a different Python version or guarantee dependency isolation. If you need either, build a conda env on top of the module: + + ```bash + module load python + conda create -n your-env-name python=3.11 -y + conda activate your-env-name + ``` + + This is also NERSC's [recommended path for `pip install`](https://docs.nersc.gov/development/languages/python/nersc-python/) when you need custom packages: pip-into-conda-env rather than pip-into-base. + +!!! warning "Storage note: 40 GB home quota" + Conda envs land under `~/.conda/envs/` by default. The Perlmutter home quota is **40 GB**, which gets eaten quickly. NERSC recommends `/global/common/software//` for larger envs. If you really want them on `$SCRATCH` (note: 12-week purge!), move and symlink: + + ```bash + conda deactivate + mv ~/.conda/envs/your-env-name $SCRATCH/conda-envs/ + ln -s $SCRATCH/conda-envs/your-env-name ~/.conda/envs/your-env-name + ``` + + See [NERSC's Python guide](https://docs.nersc.gov/development/languages/python/nersc-python/) for the full storage strategy. + +--- + +## 2. Install lightcone-cli + +With Python in place, install the package itself. Pick the path that matches your environment: + +### Path A — On top of NERSC's `python` module (no conda env) + +The module is read-only, so install with `--user` to land into your home directory's site-packages: + +```bash +python -m pip install --user lightcone-cli +``` + +This drops the `lc` console script into `~/.local/bin/`. Make sure that's on your `PATH` — Perlmutter usually has it by default; check with: + +```bash +echo $PATH | tr : '\n' | grep .local/bin +``` + +!!! tip "Already use `uv`?" + [`uv`](https://docs.astral.sh/uv/) isn't shipped by NERSC, but if you've installed it yourself (`curl -LsSf https://astral.sh/uv/install.sh | sh`), `uv tool install` is a cleaner alternative — it isolates `lc` in its own venv and exposes the same `~/.local/bin/lc` wrapper: + + ```bash + uv tool install lightcone-cli + ``` + +### Path B — Inside a conda env + +```bash +conda activate your-env-name +python -m pip install lightcone-cli # or: uv pip install lightcone-cli +``` + +`astra-tools` is a transitive dependency, so a single `lightcone-cli` install pulls it in automatically. + +### Path C — From source (contributors only) + +If you want to track the latest commits or contribute back, clone the repo and install editably. **Most users should stick with PyPI** and skip this section. + +```bash +cd ~/.lightcone # or wherever you keep clones +git clone https://github.com/LightconeResearch/lightcone-cli.git +pip install -e ./lightcone-cli # editable: tracks local edits +``` + +If you also want to hack on `astra-tools` (note: PyPI name `astra-tools`, GitHub repo name `ASTRA`): + +```bash +git clone https://github.com/LightconeResearch/ASTRA.git +pip install -e ./ASTRA +``` + +For development tooling (pytest, ruff, mypy), add the `dev` extras: + +```bash +pip install -e "./lightcone-cli[dev]" +``` + +### One-time setup + +After install, run setup once: + +```bash +lc setup +``` + +This creates `~/.lightcone/config.yaml` with `runtime: auto`. You'll pin it to `podman-hpc` for compute nodes in [§5](#5-running-on-compute-nodes). + +### Verify + +```bash +which lc # should resolve inside your active env's bin/ +lc --version +lc --help +``` + +--- + +## 3. Initialize a new project + +Scaffold a project directory and drop into it with the agent: + +```bash +lc init your-analysis # scaffolds a fresh project tree +cd your-analysis +claude # launch Claude Code inside the project +``` + +--- + +## 4. Start your research + +Once Claude Code is open, drive everything from there. The `lc-*` skills are how you tell the agent what to build: + +=== "Start fresh" + ```text + /lc-new Please sample a standard Gaussian distribution using numpy. + ``` + +=== "Migrate existing code" + ```text + /lc-migrate I have code that samples a standard Gaussian distribution using numpy at @../gaussian_sampling. Please create an analysis based on it. + ``` + +After that, just keep talking to the agent in plain English about what you want to build next. + +!!! warning "You're still on a login node" + Everything from `lc init` through your first `/lc-new` runs on a Perlmutter **login node**. That's fine for scaffolding and small recipes, but anything heavyweight needs a compute node — see [§5](#5-running-on-compute-nodes). + +--- + +## 5. Running on compute nodes + +Login nodes are shared and rate-limited — fine for `lc init`, `lc status`, and small `lc build` calls, but anything heavyweight belongs on a compute node. + +### Pre-flight: pin the container runtime and build images + +Perlmutter compute nodes ship `podman-hpc`. Pin it once globally: + +```yaml +# ~/.lightcone/config.yaml +container: + runtime: podman-hpc +``` + +Then, on a login node, build and migrate your project's images: + +```bash +cd /path/to/your-analysis +lc build +``` + +`lc build` runs `podman-hpc build` followed by `podman-hpc migrate`, which copies the image into each compute node's local container cache. See [Running on a Cluster → Pre-flight](cluster.md#pre-flight-pick-the-right-container-runtime) for the underlying mechanics. + +### Interactive runs (agent-driven) + +The agent (Claude Code) calls `lc run` for you whenever a recipe needs to materialize — you never call it directly. What you *do* control is **where Claude Code is running**: it inherits the shell environment you launched it from. To put the agent's recipes onto a compute node, simply launch `claude` from inside a SLURM allocation: + +```bash +salloc -A -q interactive -C gpu --nodes=1 -t 00:30:00 +# salloc drops you onto a compute node; from there: +cd /path/to/your-analysis +claude +``` + +Now everything the agent triggers (`lc run`, scripts, etc.) executes on the allocated node. + +!!! note "Picking a QoS" + The `interactive` QoS on the GPU partition is right for development. For longer or larger sessions, see [NERSC's queue policy reference](https://docs.nersc.gov/jobs/policy/). + +### Unattended batch runs (no agent in the loop) + +For production sweeps where the recipes are already nailed down, you can submit `lc run` directly as a batch job. See [Running on a Cluster → A typical SLURM workflow](cluster.md#a-typical-slurm-workflow) for the generic template; on Perlmutter, the only addition is the `-A` / `-q` directives: + +```bash +#!/bin/bash +#SBATCH -A +#SBATCH -q regular +#SBATCH -C gpu +#SBATCH -N 4 +#SBATCH -t 04:00:00 + +cd $SCRATCH/your-analysis +source ~/.conda/envs/your-env-name/bin/activate # or your venv +lc run -j 16 +``` + +!!! note "When to use this path" + The agent-driven flow above is the right tool during development. Reach for batch submission when you've finished iterating and want a hands-off sweep. + +### Storage gotcha: Snakemake state must live on `$SCRATCH` + +!!! danger "DVS silently ignores `flock()`" + `$HOME` and `/global/cfs/` are mounted on compute nodes via DVS, which silently ignores `flock()`. Snakemake (and any sane locking system) relies on `flock`, so its `.snakemake/` directory and Dask spill files **must** live on Lustre (`$SCRATCH`), which honors `flock`. Otherwise you get intermittent silent rule-rerun loops or hangs. + +`lc` redirects state automatically when it detects Perlmutter, so this usually just works. To pin explicitly at project creation: + +```bash +lc init your-analysis --scratch '$SCRATCH' # kept verbatim, expanded at run time +``` + +Or, after the fact, edit `/.lightcone/lightcone.yaml`: + +```yaml +scratch_root: $SCRATCH +``` + +!!! warning "12-week purge on `$SCRATCH`" + Perlmutter purges `$SCRATCH` on a rolling 12-week window. For outputs you need to keep, copy or symlink to `/global/cfs/cdirs//`. + +### Further reading + +- [NERSC interactive jobs](https://docs.nersc.gov/jobs/interactive/) — `salloc` patterns and reservation queues +- [Perlmutter system overview](https://docs.nersc.gov/systems/perlmutter/) — node types and partitions +- [NERSC Python guide](https://docs.nersc.gov/development/languages/python/nersc-python/) — module, conda, and pip layering + +--- + +## 6. Common troubleshooting + +| Symptom | Likely cause | Fix | +|---|---|---| +| `lc: command not found` | Wrong env active, or `~/.local/bin` not on `PATH` | `which lc`; reinstall in the active env, or fix `PATH` | +| `lc` runs but uses unexpected code | Two installs across two envs shadowing each other on `PATH` | `which lc` and uninstall the stale one | +| `ModuleNotFoundError: lightcone.cli.__main__` | Tried `python -m lightcone.cli` (the package isn't directly executable) | Use the `lc` console script instead | +| Snakemake locking errors / silent rule rerun loops | `.snakemake/` ended up on DVS-mounted storage | Set `scratch_root: $SCRATCH` in the project's `.lightcone/lightcone.yaml` | +| `ImportError: cannot import name 'resolve_analysis_tree' from 'astra.helpers'` | Stale `astra-tools` (pre-0.2.5) | `pip install -U astra-tools` | +| `PermissionError` reading another user's symlinked `results/` | Cross-user scratch path without group ACLs | Request access from the data owner, or copy the manifests into your own scratch | +| `pip install` hangs or times out on a compute node | Compute nodes have no public internet | Always install from a login node | + +--- + +## 7. Updating + +=== "PyPI install" + ```bash + pip install -U lightcone-cli astra-tools + ``` + +=== "Source install" + ```bash + cd ~/.lightcone/lightcone-cli + git pull + pip install -e . # only needed if pyproject.toml changed + ``` + + Editable installs auto-follow source edits — switching branches or pulling new commits is reflected immediately in `lc`. Re-run `pip install -e .` only when `pyproject.toml` adds a new dependency or changes the `[project.scripts]` table. + +--- + +## 8. Uninstalling + +```bash +pip uninstall lightcone-cli # remove from the active env +rm -rf ~/.lightcone/lightcone-cli # only for source installs +``` + +!!! note "Keep your config?" + `~/.lightcone/config.yaml` survives the uninstall. Delete it too if you want to start fresh. diff --git a/zensical.toml b/zensical.toml index f38bc093..04164e13 100644 --- a/zensical.toml +++ b/zensical.toml @@ -16,6 +16,7 @@ nav = [ {"Tutorial: Your First Analysis" = "user/tutorial.md"}, {"Multiverse Analyses" = "user/multiverse.md"}, {"Running on a Cluster" = "user/cluster.md"}, + {"NERSC (Perlmutter)" = "user/nersc.md"}, {"Troubleshooting" = "user/troubleshooting.md"}, {"Glossary" = "user/glossary.md"}, ]}, From 415e243d5467a8df562d05362c06ee4c565ee18d Mon Sep 17 00:00:00 2001 From: dkn16 Date: Fri, 8 May 2026 11:58:55 -0700 Subject: [PATCH 3/8] =?UTF-8?q?docs(nersc):=20polish=20=C2=A71,=20=C2=A72,?= =?UTF-8?q?=20=C2=A77,=20=C2=A78=20=E2=80=94=20uv-first,=20admonitions,=20?= =?UTF-8?q?tabs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bring the NERSC guide in line with the conventions of the improved install.md (uv recommended, pip as fallback) and tighten the prose: - §1 Python: lead with the uv install one-liner + \`uv python install 3.12\`. NERSC's \`module load python\` is now the alternative inside a !!! note, with the conda-env subsection nested under it (only relevant if you went the module route). - §2 Install: \`uv tool install lightcone-cli\` is the headline command. The pip path is preserved as a !!! note alternative with === "NERSC python module" / === "Conda env" tabs so users only see the variant relevant to their Python choice. Source install commands switched to \`uv pip install -e\`. "One-time setup" + "Verify" merged into a single short block. - §7 Updating: added a \`uv tool upgrade\` tab as the default; pip and source as alternatives. - §8 Uninstalling: symmetric uv-tool / pip tabs. - Drops the stale reference to \`~/.lightcone/targets/\` (folder no longer exists; only config.yaml lives there). The blockquote-style callouts converted to !!! note / !!! warning / !!! danger admonitions throughout, matching the vocabulary already used in user/index.md and user/install.md. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/user/nersc.md | 118 ++++++++++++++++++++++----------------------- 1 file changed, 59 insertions(+), 59 deletions(-) diff --git a/docs/user/nersc.md b/docs/user/nersc.md index eeb41505..a178d34a 100644 --- a/docs/user/nersc.md +++ b/docs/user/nersc.md @@ -28,24 +28,30 @@ Other install routes (npm, native package managers) are documented in the [Claud ## 1. Python -NERSC's `python` module gives you a ready-to-use Python distribution with `conda`, `pip`, and many common scientific packages already installed — no env creation needed for the basics: +Like the generic [Install](install.md#1-python) page, we recommend [`uv`](https://docs.astral.sh/uv/) for managing Python on Perlmutter — it's faster than pip and gives you a Python independent of NERSC's `module` system. NERSC doesn't ship it, but it installs into your home dir with a single curl: ```bash -module load python # NERSC Python (3.11+); brings conda and pip onto PATH +curl -LsSf https://astral.sh/uv/install.sh | sh +uv python install 3.12 ``` -That's enough for installing `lightcone-cli` on top. Skip ahead to [§2](#2-install-lightcone-cli). +This drops both `uv` and an isolated Python 3.12 under `~/.local/`. Make sure `~/.local/bin` is on your `PATH`. -!!! note "When you'd want your own conda env" - The NERSC python module is shared and read-only. You *can* layer user-level packages on top, but you can't pin a different Python version or guarantee dependency isolation. If you need either, build a conda env on top of the module: +!!! note "Alternative: NERSC's `python` module" + If you'd rather use NERSC's pre-built environment, `module load python` gives you a ready-to-use distribution with `conda`, `pip`, and many scientific packages already installed: + + ```bash + module load python # NERSC Python (3.11+); brings conda and pip onto PATH + ``` + + Convenient, but the module is shared and read-only — you can't pin a different Python version or guarantee dependency isolation. For that, build a conda env on top: ```bash - module load python conda create -n your-env-name python=3.11 -y conda activate your-env-name ``` - This is also NERSC's [recommended path for `pip install`](https://docs.nersc.gov/development/languages/python/nersc-python/) when you need custom packages: pip-into-conda-env rather than pip-into-base. + This is also NERSC's [recommended path for `pip install`](https://docs.nersc.gov/development/languages/python/nersc-python/) when you need custom packages. !!! warning "Storage note: 40 GB home quota" Conda envs land under `~/.conda/envs/` by default. The Perlmutter home quota is **40 GB**, which gets eaten quickly. NERSC recommends `/global/common/software//` for larger envs. If you really want them on `$SCRATCH` (note: 12-week purge!), move and symlink: @@ -62,79 +68,62 @@ That's enough for installing `lightcone-cli` on top. Skip ahead to [§2](#2-inst ## 2. Install lightcone-cli -With Python in place, install the package itself. Pick the path that matches your environment: - -### Path A — On top of NERSC's `python` module (no conda env) - -The module is read-only, so install with `--user` to land into your home directory's site-packages: - -```bash -python -m pip install --user lightcone-cli -``` - -This drops the `lc` console script into `~/.local/bin/`. Make sure that's on your `PATH` — Perlmutter usually has it by default; check with: +The package on PyPI is `lightcone-cli`; the command it provides is `lc`. The recommended install uses `uv tool`, which isolates `lc` in its own venv under `~/.local/share/uv/tools/` and exposes a wrapper at `~/.local/bin/lc`: ```bash -echo $PATH | tr : '\n' | grep .local/bin +uv tool install lightcone-cli ``` -!!! tip "Already use `uv`?" - [`uv`](https://docs.astral.sh/uv/) isn't shipped by NERSC, but if you've installed it yourself (`curl -LsSf https://astral.sh/uv/install.sh | sh`), `uv tool install` is a cleaner alternative — it isolates `lc` in its own venv and exposes the same `~/.local/bin/lc` wrapper: - - ```bash - uv tool install lightcone-cli - ``` +`astra-tools` is a transitive dependency — pulled in automatically. -### Path B — Inside a conda env +!!! note "Alternative: pip" + If you'd rather not use uv, install with pip. The exact command depends on which Python you're using: -```bash -conda activate your-env-name -python -m pip install lightcone-cli # or: uv pip install lightcone-cli -``` + === "NERSC python module" + ```bash + module load python + python -m pip install --user lightcone-cli # --user lands in ~/.local/bin/ + ``` -`astra-tools` is a transitive dependency, so a single `lightcone-cli` install pulls it in automatically. + === "Conda env" + ```bash + conda activate your-env-name + python -m pip install lightcone-cli + ``` -### Path C — From source (contributors only) +### From source (contributors only) -If you want to track the latest commits or contribute back, clone the repo and install editably. **Most users should stick with PyPI** and skip this section. +If you want to track the latest commits or contribute back, clone the repo and install editably. **Most users should stick with PyPI.** ```bash cd ~/.lightcone # or wherever you keep clones git clone https://github.com/LightconeResearch/lightcone-cli.git -pip install -e ./lightcone-cli # editable: tracks local edits +uv pip install -e ./lightcone-cli # or: pip install -e ./lightcone-cli ``` -If you also want to hack on `astra-tools` (note: PyPI name `astra-tools`, GitHub repo name `ASTRA`): +To hack on `astra-tools` itself (PyPI name `astra-tools`, GitHub repo `ASTRA`): ```bash git clone https://github.com/LightconeResearch/ASTRA.git -pip install -e ./ASTRA -``` - -For development tooling (pytest, ruff, mypy), add the `dev` extras: - -```bash -pip install -e "./lightcone-cli[dev]" +uv pip install -e ./ASTRA ``` -### One-time setup - -After install, run setup once: +For development tooling (pytest, ruff, mypy): ```bash -lc setup +uv pip install -e "./lightcone-cli[dev]" ``` -This creates `~/.lightcone/config.yaml` with `runtime: auto`. You'll pin it to `podman-hpc` for compute nodes in [§5](#5-running-on-compute-nodes). - -### Verify +### One-time setup and sanity check ```bash -which lc # should resolve inside your active env's bin/ +lc setup # creates ~/.lightcone/config.yaml with runtime: auto +which lc # should resolve under ~/.local/bin/ or your active env lc --version -lc --help ``` +You'll pin `runtime: podman-hpc` for compute nodes in [§5](#5-running-on-compute-nodes). + --- ## 3. Initialize a new project @@ -273,28 +262,39 @@ scratch_root: $SCRATCH ## 7. Updating -=== "PyPI install" +=== "uv tool" + ```bash + uv tool upgrade lightcone-cli + ``` + +=== "pip" ```bash pip install -U lightcone-cli astra-tools ``` -=== "Source install" +=== "Source" ```bash cd ~/.lightcone/lightcone-cli git pull - pip install -e . # only needed if pyproject.toml changed + uv pip install -e . # only needed if pyproject.toml changed ``` - Editable installs auto-follow source edits — switching branches or pulling new commits is reflected immediately in `lc`. Re-run `pip install -e .` only when `pyproject.toml` adds a new dependency or changes the `[project.scripts]` table. + Editable installs auto-follow source edits — switching branches or pulling new commits is reflected immediately in `lc`. Re-install only when `pyproject.toml` adds a new dependency or changes the `[project.scripts]` table. --- ## 8. Uninstalling -```bash -pip uninstall lightcone-cli # remove from the active env -rm -rf ~/.lightcone/lightcone-cli # only for source installs -``` +=== "uv tool" + ```bash + uv tool uninstall lightcone-cli + ``` + +=== "pip" + ```bash + pip uninstall lightcone-cli + rm -rf ~/.lightcone/lightcone-cli # only for source installs + ``` !!! note "Keep your config?" `~/.lightcone/config.yaml` survives the uninstall. Delete it too if you want to start fresh. From a991e100e904f29c16db181071740f8aeaf42246 Mon Sep 17 00:00:00 2001 From: dkn16 Date: Mon, 11 May 2026 09:29:38 -0700 Subject: [PATCH 4/8] docs(nersc): drop lc setup (removed), align batch example with uv MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verified the doc against the actual CLI surface in src/lightcone/cli/commands.py. The canonical command list is init / run / status / verify / build / export / eval — there is no \`lc setup\`. Global config is auto-created by _ensure_global_config() on any \`lc\` invocation (commands.py:75). - §2 "One-time setup and sanity check" renamed to "Sanity check". Dropped the stale \`lc setup\` command. The pin-runtime hint moved into a !!! note that explains the auto-creation, matching how install.md §3 frames it. - §5 sbatch example: now leads with \`export PATH=\$HOME/.local/bin\` (the uv tool install default), with conda activation commented as the alternative. Previously assumed a conda env, which is inconsistent with the uv-first install path in §1/§2. Other commands and flags in the doc were verified to exist: - lc init --scratch (commands.py:137) - lc run -j / --jobs (commands.py:382) - lc build → podman-hpc build + migrate (container.py:558,635) - /.lightcone/lightcone.yaml + scratch_root (scratch.py:8,67) - /lc-new and /lc-migrate skills (claude/lightcone/skills/) The troubleshooting table entry about \`resolve_analysis_tree\` ImportError was reproduced live against the venv's astra-tools, so it stays. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/user/nersc.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/docs/user/nersc.md b/docs/user/nersc.md index a178d34a..25f127f6 100644 --- a/docs/user/nersc.md +++ b/docs/user/nersc.md @@ -114,15 +114,16 @@ For development tooling (pytest, ruff, mypy): uv pip install -e "./lightcone-cli[dev]" ``` -### One-time setup and sanity check +### Sanity check ```bash -lc setup # creates ~/.lightcone/config.yaml with runtime: auto which lc # should resolve under ~/.local/bin/ or your active env lc --version +lc --help ``` -You'll pin `runtime: podman-hpc` for compute nodes in [§5](#5-running-on-compute-nodes). +!!! note "Global config is auto-created" + The first `lc` invocation writes `~/.lightcone/config.yaml` with `runtime: auto` — no manual setup step needed. You'll pin it to `podman-hpc` for compute nodes in [§5](#5-running-on-compute-nodes). --- @@ -211,7 +212,11 @@ For production sweeps where the recipes are already nailed down, you can submit #SBATCH -t 04:00:00 cd $SCRATCH/your-analysis -source ~/.conda/envs/your-env-name/bin/activate # or your venv + +# Make `lc` available — pick the line that matches your install: +export PATH=$HOME/.local/bin:$PATH # uv tool install (default) +# source ~/.conda/envs/your-env-name/bin/activate # conda env + lc run -j 16 ``` From f453278baa67ee3faccb32cc555fa3077a0b9a73 Mon Sep 17 00:00:00 2001 From: dkn16 Date: Mon, 11 May 2026 09:39:28 -0700 Subject: [PATCH 5/8] docs(nersc): frame Claude Code as the running example, not the only agent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit lightcone-cli is designed to work with any agent-based CLI; Claude Code is currently the best-supported one, but it's not the only intended target. The earlier doc spoke as if Claude Code = the agent. - §0: rewrote the opening paragraph to make the "any agent CLI; we use Claude Code as the running example" framing explicit. The Claude install commands move under a clearly scoped "Installing Claude Code:" subsection. - §3: \`claude\` launch line now reads "launch your agent CLI (Claude Code shown here)". - §4: "Once Claude Code is open" → "Once your agent CLI is open (Claude Code in this guide's examples)". - §5 interactive runs: prose now refers to "the agent" generically; the parenthetical "(Claude Code)" is gone since the framing is set in §0. The shell command keeps \`claude\` but adds a comment "# or whichever agent CLI you use". - §5 batch runs: adjusted to "no agent CLI involved" instead of "no Claude Code in the loop" — same meaning, agent-neutral. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/user/nersc.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/user/nersc.md b/docs/user/nersc.md index 25f127f6..e619535d 100644 --- a/docs/user/nersc.md +++ b/docs/user/nersc.md @@ -9,7 +9,9 @@ A practical guide for running [`lightcone-cli`](https://github.com/LightconeRese ## 0. Agentic CLI -`lightcone-cli` is the execution layer of the `lightcone` project — it harnesses an agent-based CLI (currently [Claude Code](https://docs.claude.com/en/docs/claude-code/setup)) to follow the `astra` standard while building and running an analysis. So the very first step, even before touching `lightcone-cli` itself, is to install the agent: +`lightcone-cli` is the execution layer of the `lightcone` project — it harnesses an **agent-based CLI** to follow the `astra` standard while building and running an analysis. The choice of agent is open: anything that can drive a project shell works. Right now the tooling (skills, hooks, slash commands) is best supported on [Claude Code](https://docs.claude.com/en/docs/claude-code/setup), so this guide uses Claude Code as the running example — substitute your preferred agent CLI throughout if you use a different one. + +Installing Claude Code: ```bash curl -fsSL https://claude.ai/install.sh | bash # installs to ~/.local/bin/claude @@ -134,14 +136,14 @@ Scaffold a project directory and drop into it with the agent: ```bash lc init your-analysis # scaffolds a fresh project tree cd your-analysis -claude # launch Claude Code inside the project +claude # launch your agent CLI (Claude Code shown here) ``` --- ## 4. Start your research -Once Claude Code is open, drive everything from there. The `lc-*` skills are how you tell the agent what to build: +Once your agent CLI is open (Claude Code in this guide's examples), drive everything from there. The `lc-*` skills are how you tell the agent what to build: === "Start fresh" ```text @@ -185,13 +187,13 @@ lc build ### Interactive runs (agent-driven) -The agent (Claude Code) calls `lc run` for you whenever a recipe needs to materialize — you never call it directly. What you *do* control is **where Claude Code is running**: it inherits the shell environment you launched it from. To put the agent's recipes onto a compute node, simply launch `claude` from inside a SLURM allocation: +The agent calls `lc run` for you whenever a recipe needs to materialize — you never call it directly. What you *do* control is **where the agent is running**: it inherits the shell environment you launched it from. To put the agent's recipes onto a compute node, simply launch it from inside a SLURM allocation: ```bash salloc -A -q interactive -C gpu --nodes=1 -t 00:30:00 # salloc drops you onto a compute node; from there: cd /path/to/your-analysis -claude +claude # or whichever agent CLI you use ``` Now everything the agent triggers (`lc run`, scripts, etc.) executes on the allocated node. @@ -201,7 +203,7 @@ Now everything the agent triggers (`lc run`, scripts, etc.) executes on the allo ### Unattended batch runs (no agent in the loop) -For production sweeps where the recipes are already nailed down, you can submit `lc run` directly as a batch job. See [Running on a Cluster → A typical SLURM workflow](cluster.md#a-typical-slurm-workflow) for the generic template; on Perlmutter, the only addition is the `-A` / `-q` directives: +For production sweeps where the recipes are already nailed down, you can submit `lc run` directly as a batch job — no agent CLI involved. See [Running on a Cluster → A typical SLURM workflow](cluster.md#a-typical-slurm-workflow) for the generic template; on Perlmutter, the only addition is the `-A` / `-q` directives: ```bash #!/bin/bash From 796f5b0a6111db409f323fe71047018a47e296b5 Mon Sep 17 00:00:00 2001 From: dkn16 Date: Mon, 11 May 2026 09:42:02 -0700 Subject: [PATCH 6/8] =?UTF-8?q?docs(nersc):=20list=20other=20agent=20CLIs?= =?UTF-8?q?=20(Codex,=20opencode)=20at=20end=20of=20=C2=A70?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The §0 intro already says the choice of agent is open and Claude Code is just the running example. Concrete pointers help readers act on that — added a !!! note at the end of §0 with two examples: - OpenAI Codex (github.com/openai/codex) - opencode (opencode.ai) Each entry links to its install docs. Kept the list short and the prose neutral so the section doesn't read like an endorsement. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/user/nersc.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/user/nersc.md b/docs/user/nersc.md index e619535d..d35a2c05 100644 --- a/docs/user/nersc.md +++ b/docs/user/nersc.md @@ -9,7 +9,7 @@ A practical guide for running [`lightcone-cli`](https://github.com/LightconeRese ## 0. Agentic CLI -`lightcone-cli` is the execution layer of the `lightcone` project — it harnesses an **agent-based CLI** to follow the `astra` standard while building and running an analysis. The choice of agent is open: anything that can drive a project shell works. Right now the tooling (skills, hooks, slash commands) is best supported on [Claude Code](https://docs.claude.com/en/docs/claude-code/setup), so this guide uses Claude Code as the running example — substitute your preferred agent CLI throughout if you use a different one. +`lightcone-cli` is the execution layer of the `lightcone` project — it harnesses an **agent-based CLI** to follow the `astra` standard while building and running an analysis. The choice of agent is open: anything that can drive a project shell works. This guide uses Claude Code as the running example — substitute your preferred agent CLI throughout if you use a different one. Installing Claude Code: @@ -26,6 +26,14 @@ claude # first run prompts for login ( Other install routes (npm, native package managers) are documented in the [Claude Code installation docs](https://docs.claude.com/en/docs/claude-code/setup). +!!! note "Other agent CLIs" + Other agentic CLIs work too — for example: + + - [OpenAI Codex](https://github.com/openai/codex) — see the repo README for install options. + - [opencode](https://opencode.ai/docs/#install) — install via `curl -fsSL https://opencode.ai/install | bash`. + + Pick whichever you prefer; the rest of this guide writes `claude` in concrete commands, but the workflow is the same with any agent CLI. + --- ## 1. Python From 2b7bbb05cdee553ab235a5dd5007010e456d1425 Mon Sep 17 00:00:00 2001 From: Alexandre Boucaud Date: Mon, 11 May 2026 19:45:40 +0200 Subject: [PATCH 7/8] Update first pages --- docs/skills/index.md | 2 +- docs/user/agent-workflow.md | 2 +- docs/user/getting-started.md | 38 +++++++++++++++++------------------- docs/user/index.md | 8 ++++---- docs/user/install.md | 33 ++++++++++++------------------- docs/user/tutorial.md | 3 +-- 6 files changed, 37 insertions(+), 49 deletions(-) diff --git a/docs/skills/index.md b/docs/skills/index.md index 5100013b..44a1d6d8 100644 --- a/docs/skills/index.md +++ b/docs/skills/index.md @@ -5,7 +5,7 @@ plugin. They give the agent a structured, phase-by-phase workflow for the most common research operations. If you're a researcher trying to *use* these, the -[Claude Code Workflow](../user/claude-workflow.md) page in the user +[Claude Code Workflow](../user/agent-workflow.md) page in the user guide is the friendly version. This page is for maintainers. ## Available skills diff --git a/docs/user/agent-workflow.md b/docs/user/agent-workflow.md index 502194cc..2cec5e0e 100644 --- a/docs/user/agent-workflow.md +++ b/docs/user/agent-workflow.md @@ -1,4 +1,4 @@ -# The Agent Workflow +# The Agentic Workflow The agentic surface is five slash commands. Each one is a structured prompt — the agent follows a specific phased flow, not free-form chat. diff --git a/docs/user/getting-started.md b/docs/user/getting-started.md index 3f022e0b..fc9ccaf4 100644 --- a/docs/user/getting-started.md +++ b/docs/user/getting-started.md @@ -8,27 +8,27 @@ Make sure you've finished the [install](install.md) first. ## 1. Create a project -```bash -lc init my-analysis -cd my-analysis -``` + lc init my-analysis + cd my-analysis `lc init` is a one-shot setup. It creates a small, opinionated directory layout and stops; it doesn't ask any questions. ## 2. What you got -``` -my-analysis/ -├── astra.yaml # the spec — this is where everything lives -├── CLAUDE.md # short note for the agent (resumes context across sessions) -├── .gitignore -├── .venv/ # Python virtual env (skip with --no-venv) -├── .lightcone/ # internal scratchpad — don't edit by hand -├── .claude/ # Claude Code plugin — skills, agents, hooks -├── universes/ # placeholder for now -└── results/ # placeholder for now -``` + my-analysis/ + ├── astra.yaml # the spec — this is where everything lives + ├── CLAUDE.md # short note for the agent (resumes context across sessions) + ├── .gitignore + ├── .git # initialized git repository (skip with --no-git) + ├── .venv/ # Python virtual env (skip with --no-venv) + ├── .claude/ # Claude Code plugin — skills, agents, hooks + ├── .lightcone/ # internal scratchpad — don't edit by hand + ├── Containerfile # build instructions for a local testing container — don't edit by hand + ├── requirements.txt # software dependencies — don't edit by hand + ├── universes/ # + ├── src/ # placeholder directories for now + └── results/ # The two files you'll actually look at: @@ -50,9 +50,7 @@ can edit it by hand whenever you want. ## 3. Open Claude Code -```bash -claude -``` + claude That opens an interactive session inside `my-analysis/`. Claude Code reads `astra.yaml` and `CLAUDE.md` so it has context. @@ -69,7 +67,7 @@ Inside Claude Code: | `/lc-migrate` | You have an existing codebase you want wrapped in ASTRA. | | `/lc-feedback` | Something broke and you want to file a GitHub issue without leaving the session. | -The next page, [The Claude Code Workflow](claude-workflow.md), +The next page, [The Agentic Workflow](agent-workflow.md), explains each of these in more detail. ## 5. The four CLI commands you'll actually type @@ -90,7 +88,7 @@ exact flags. ## 6. Read on -- [The Claude Code Workflow](claude-workflow.md) — how each slash +- [The Agentic Workflow](agent-workflow.md) — how each slash command actually flows. - [Tutorial: Your First Analysis](tutorial.md) — end-to-end, with the agent doing most of the typing. diff --git a/docs/user/index.md b/docs/user/index.md index c96a3316..14978f68 100644 --- a/docs/user/index.md +++ b/docs/user/index.md @@ -2,12 +2,12 @@ `lightcone-cli` is a small toolchain that turns a research question into a reproducible analysis. You describe what you're trying to learn, -Claude Code helps you turn that into a precise specification, and the +a cli agent helps you turn that into a precise specification, and the `lc` command line keeps the resulting code, decisions, and outputs in sync — forever. You don't need to write Python or YAML by hand. The agent handles the -implementation; you stay in charge of the scientific choices. +implementation; **you stay in charge of the scientific choices**. ## What this guide covers @@ -15,7 +15,7 @@ implementation; you stay in charge of the scientific choices. machine. - [Getting Started](getting-started.md) — your first `lc init` and what every directory means. -- [The Claude Code Workflow](claude-workflow.md) — `/lc-new`, +- [The Agentic Workflow](agent-workflow.md) — `/lc-new`, `/lc-build`, `/lc-verify`, `/lc-migrate`, `/lc-feedback` — what each one does and when to reach for it. - [Tutorial: Your First Analysis](tutorial.md) — an end-to-end worked @@ -57,7 +57,7 @@ unhurried version. Python commands, not a DSL. There's no learning curve beyond what's in the [tutorial](tutorial.md). - **An IDE.** `lc` is a command-line tool; the agent surface lives - inside Claude Code. + inside the agent harness (Claude Code for now). If you'd rather skim the design and architecture, the [maintainer docs](../index.md) are the other half of this site. diff --git a/docs/user/install.md b/docs/user/install.md index f734de31..66e69228 100644 --- a/docs/user/install.md +++ b/docs/user/install.md @@ -20,18 +20,16 @@ If you don't already have a recent Python === "Windows" [python.org](https://www.python.org/downloads/) or WSL - !!! tip "Recommendation" We highly recommend the use of [uv](https://docs.astral.sh/uv/) to manage Python installation and virtual environments. `uv` can be installed in a single commandline - ```bash - curl -LsSf https://astral.sh/uv/install.sh | sh - ``` + + curl -LsSf https://astral.sh/uv/install.sh | sh + and a subsequent version of Python - ``` - uv python install 3.12 - ``` + + uv python install 3.12 ## 2. lightcone-cli @@ -50,9 +48,7 @@ is `lc`. Get a confirmation of the proper installation by running -```bash -lc --version # → lightcone-cli, version ... -``` + lc --version # → lightcone-cli, version ... > **Note** Some people may have already set a personal shell alias `lc='ls --color'`. If that's you, installing lightcone-cli will shadow the alias — make sure to rebind it (e.g. `alias l='ls --color'`). @@ -75,18 +71,15 @@ your PATH (and skips docker if its daemon isn't running). Feel free to pin the r Most of your interactions with a lightcone project happen *through* an agent-based CLI, for now we are supporting Claude Code. Install Claude Code -```bash -curl -fsSL https://claude.ai/install.sh | bash -``` + + curl -fsSL https://claude.ai/install.sh | bash Open a project in your terminal or editor (see [Getting Started](getting-started.md)) and run -```bash -claude -``` + claude Inside Claude Code you will have dedicated lightcone CLI slash commands available like `/lc-new` and -`/lc-build` — see [The Claude Code Workflow](claude-workflow.md). +`/lc-build` — see [The Agentic Workflow](agent-workflow.md). ## 5. (Optional) Docker or Podman @@ -105,10 +98,8 @@ isolation. ## Sanity check -```bash -lc --help -lc init --help -``` + lc --help + lc init --help Both should print help text. If `lc` is shadowed by an `ls` alias, unset it (`unalias lc`) or use the full path diff --git a/docs/user/tutorial.md b/docs/user/tutorial.md index 2076f6b8..c4d7786b 100644 --- a/docs/user/tutorial.md +++ b/docs/user/tutorial.md @@ -42,8 +42,7 @@ Type: The agent banner switches to **RESEARCH QUESTION** and asks something like "What are you trying to learn?" Reply in plain prose: -> I want to know how much R² changes on the diabetes dataset depending -> on whether I standardize features before fitting a linear regression. + I want to know how much R² changes on the diabetes dataset depending on whether I standardize features before fitting a linear regression. A few follow-ups will sharpen this. After Phase 1 your `astra.yaml` already has a `name`, `description`, and `version` — open it in another From 2e548cb70d63d875105e656238bbbc0300de8b04 Mon Sep 17 00:00:00 2001 From: Alexandre Boucaud Date: Wed, 13 May 2026 00:22:19 +0200 Subject: [PATCH 8/8] =?UTF-8?q?docs:=20address=20PR=20checklist=20?= =?UTF-8?q?=E2=80=94=20narrative=20step,=20code=20tags,=20nav=20and=20accu?= =?UTF-8?q?racy=20fixes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add narrative block population step to lc-new docs, agent-workflow, and tutorial - Add Quick Start snippet to docs/index.md researcher section - Fix commands.py annotation (setup→export) in index.md repo tree - Remove stale lc setup entry from zensical.toml nav - Replace stale Dagster reference in lc-feedback.md maintainer note - Add text/bash language tags to ~30 bare code fences across CLI reference, skills, tutorial, contributing, and user docs Co-Authored-By: Claude Sonnet 4.6 --- docs/cli/build.md | 4 ++-- docs/cli/export.md | 6 +++--- docs/cli/index.md | 2 +- docs/cli/init.md | 4 ++-- docs/cli/run.md | 2 +- docs/cli/status.md | 2 +- docs/cli/verify.md | 2 +- docs/contributing/setup.md | 2 +- docs/contributing/testing.md | 2 +- docs/index.md | 22 +++++++++++++++++++--- docs/skills/authoring.md | 2 +- docs/skills/index.md | 2 +- docs/skills/lc-build.md | 2 +- docs/skills/lc-feedback.md | 12 ++++++------ docs/skills/lc-migrate.md | 2 +- docs/skills/lc-new.md | 6 ++++-- docs/skills/lc-verify.md | 4 ++-- docs/user/agent-workflow.md | 7 +++++-- docs/user/multiverse.md | 4 ++-- docs/user/troubleshooting.md | 2 +- docs/user/tutorial.md | 22 ++++++++++++---------- zensical.toml | 1 - 22 files changed, 68 insertions(+), 46 deletions(-) diff --git a/docs/cli/build.md b/docs/cli/build.md index c3a01f5d..eab38f0a 100644 --- a/docs/cli/build.md +++ b/docs/cli/build.md @@ -5,7 +5,7 @@ images so `lc run` can use `--pull=never`). ## Synopsis -``` +```text lc build [OPTIONS] ``` @@ -35,7 +35,7 @@ nothing to build. ## Tag computation -``` +```text lc-- ``` diff --git a/docs/cli/export.md b/docs/cli/export.md index 3e3b52f4..49ecc00e 100644 --- a/docs/cli/export.md +++ b/docs/cli/export.md @@ -6,7 +6,7 @@ future formats without breaking the CLI surface. ## Synopsis -``` +```text lc export wrroc [OPTIONS] ``` @@ -53,7 +53,7 @@ lc export wrroc --license https://opensource.org/licenses/MIT ### Output -``` +```text ✓ Wrote WRROC directory: ./wrroc Captured 7 runs across universes: baseline, alt_method ``` @@ -61,7 +61,7 @@ lc export wrroc --license https://opensource.org/licenses/MIT If no materialized outputs are found, the bundle still writes — but only contains the workflow definition, and a warning is printed: -``` +```text ✓ Wrote WRROC directory: ./wrroc Warning: no materialized outputs were found — the bundle contains only the workflow definition. diff --git a/docs/cli/index.md b/docs/cli/index.md index fdd641bd..33ffba0b 100644 --- a/docs/cli/index.md +++ b/docs/cli/index.md @@ -24,7 +24,7 @@ Code skills, with the CLI as the durable, scriptable backstop. ## Global options -``` +```text lc [OPTIONS] COMMAND [ARGS]... Options: diff --git a/docs/cli/init.md b/docs/cli/init.md index 98f5abf9..4db4979f 100644 --- a/docs/cli/init.md +++ b/docs/cli/init.md @@ -4,7 +4,7 @@ Scaffold a new ASTRA project with Claude Code integration. ## Synopsis -``` +```text lc init [OPTIONS] [DIRECTORY] ``` @@ -14,7 +14,7 @@ lc init [OPTIONS] [DIRECTORY] Inside `DIRECTORY` (creating it if needed): -``` +```text astra.yaml # tiny boilerplate spec with one example output CLAUDE.md # short note pointing future agents at the project .gitignore # Python + lightcone state diff --git a/docs/cli/run.md b/docs/cli/run.md index 657ee36c..3e311be9 100644 --- a/docs/cli/run.md +++ b/docs/cli/run.md @@ -5,7 +5,7 @@ and dispatches through Snakemake on a Dask cluster. ## Synopsis -``` +```text lc run [OPTIONS] [OUTPUTS]... ``` diff --git a/docs/cli/status.md b/docs/cli/status.md index 33a8b138..7985c3a5 100644 --- a/docs/cli/status.md +++ b/docs/cli/status.md @@ -5,7 +5,7 @@ Manifest-driven status report for every output declared in ## Synopsis -``` +```text lc status [OPTIONS] ``` diff --git a/docs/cli/verify.md b/docs/cli/verify.md index e3b1270e..6e1ae216 100644 --- a/docs/cli/verify.md +++ b/docs/cli/verify.md @@ -5,7 +5,7 @@ input chain. Catches tampering, drift, and forged manifests. ## Synopsis -``` +```text lc verify [OPTIONS] ``` diff --git a/docs/contributing/setup.md b/docs/contributing/setup.md index 581bc580..b4d4698b 100644 --- a/docs/contributing/setup.md +++ b/docs/contributing/setup.md @@ -77,7 +77,7 @@ relative to the repo root. ## Repo layout -``` +```text src/lightcone/ # main namespace (PEP 420; no __init__.py at the package root) src/snakemake_executor_plugin_dask/ # Snakemake → Dask executor plugin claude/lightcone/ # Claude Code plugin (force-included into the wheel) diff --git a/docs/contributing/testing.md b/docs/contributing/testing.md index f64eea7e..f1d3e3da 100644 --- a/docs/contributing/testing.md +++ b/docs/contributing/testing.md @@ -2,7 +2,7 @@ ## Test layout -``` +```text tests/ ├── conftest.py # shared fixtures ├── test_cli.py # Click CliRunner integration tests diff --git a/docs/index.md b/docs/index.md index 56144227..0d3521d9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,6 +13,22 @@ This site has two halves. Start at the [User Guide](user/index.md). Friendly, step-by-step, with worked examples. You will not need to read any Python. +The shortest possible path: + +=== "uv" + ```bash + uv tool install lightcone-cli + lc init my-analysis && cd my-analysis + claude # then, inside Claude Code: /lc-new + ``` + +=== "pip" + ```bash + pip install lightcone-cli + lc init my-analysis && cd my-analysis + claude # then, inside Claude Code: /lc-new + ``` + ## I work on lightcone-cli Welcome — keep reading. The rest of this page is a fast tour for @@ -34,11 +50,11 @@ spec itself (validation, paper management, evidence verification); the ## Repository at a glance -``` +```text src/lightcone/ # PEP 420 namespace package — NO __init__.py ├── cli/ # Click surface │ ├── __init__.py # exposes main() -│ ├── commands.py # init, run, status, verify, build, setup +│ ├── commands.py # init, run, status, verify, build, export │ └── plugin.py # plugin source-dir discovery ├── engine/ # execution substrate │ ├── manifest.py # write_manifest, sha256_dir, code_version @@ -74,7 +90,7 @@ regular package and break coexistence with future sibling distributions ## Execution flow -``` +```text astra.yaml ── snakefile.generate() ──► .lightcone/Snakefile + .lightcone/snakefile-config.json │ snakemake -s … -d … --executor dask diff --git a/docs/skills/authoring.md b/docs/skills/authoring.md index 00c6e6a2..1f4ecade 100644 --- a/docs/skills/authoring.md +++ b/docs/skills/authoring.md @@ -7,7 +7,7 @@ Skills are markdown files with YAML frontmatter. Each one lives in ## File layout -``` +```text claude/lightcone/skills/ └── my-skill/ ├── SKILL.md diff --git a/docs/skills/index.md b/docs/skills/index.md index 44a1d6d8..5c933101 100644 --- a/docs/skills/index.md +++ b/docs/skills/index.md @@ -41,7 +41,7 @@ files, anti-patterns. The skill bundles its own helper scripts under ## Plugin layout -``` +```text claude/lightcone/ ├── skills/ │ ├── lc-new/SKILL.md diff --git a/docs/skills/lc-build.md b/docs/skills/lc-build.md index f1d8524a..53c36526 100644 --- a/docs/skills/lc-build.md +++ b/docs/skills/lc-build.md @@ -11,7 +11,7 @@ Defaults: universe `baseline`, max-iterations `25`. ## Allowed tools -``` +```text Read, Write, Edit, Glob, Grep, Bash(astra:*), Bash(lc:*), Bash(python:*), Bash(git:*), Bash(pip:*), Bash(mkdir:*), Bash(setup-lc-build:*), diff --git a/docs/skills/lc-feedback.md b/docs/skills/lc-feedback.md index 3db3e3de..f1dab913 100644 --- a/docs/skills/lc-feedback.md +++ b/docs/skills/lc-feedback.md @@ -10,7 +10,7 @@ Argument hint: ``. ## Allowed tools -``` +```text Bash(gh:*), Bash(python:*), Bash(uname:*), AskUserQuestion ``` @@ -48,7 +48,7 @@ create`. ## Issue body template -``` +```text ## What happened [1–3 sentences combining user description + session context] @@ -77,7 +77,7 @@ Sections that don't apply are dropped. ## Notes for the maintainer who's looking -The triage hint in the prompt currently says "lightcone-cli — `lc` CLI, -**Dagster execution**, recipes, container builds, scaffolding, skills." -That's stale — the Dagster mention should be replaced with -"Snakemake/Dask execution." See the `SKILL.md` source. +The triage hint in the skill prompt distinguishes ASTRA (schema, validation, +`astra` CLI) from lightcone-cli (Snakemake/Dask execution, `lc` CLI, +container builds, scaffolding, skills). When in doubt, the skill defaults +to lightcone-cli. See the `SKILL.md` source for the exact wording. diff --git a/docs/skills/lc-migrate.md b/docs/skills/lc-migrate.md index 3ba5f9f2..230fcc07 100644 --- a/docs/skills/lc-migrate.md +++ b/docs/skills/lc-migrate.md @@ -9,7 +9,7 @@ Source: [`claude/lightcone/skills/lc-migrate/SKILL.md`](https://github.com/Light ## Allowed tools -``` +```text Read, Write, Edit, Glob, Grep, Bash(astra:*), Bash(lc:*), Bash(python:*), Bash(pip:*), Bash(git:*), Bash(mkdir:*), Bash(ls:*), Agent, AskUserQuestion diff --git a/docs/skills/lc-new.md b/docs/skills/lc-new.md index a86bcf4a..89fa234d 100644 --- a/docs/skills/lc-new.md +++ b/docs/skills/lc-new.md @@ -8,7 +8,7 @@ Source: [`claude/lightcone/skills/lc-new/SKILL.md`](https://github.com/Lightcone ## Allowed tools -``` +```text Read, Write(astra.yaml), Write(universes/*), Write(CLAUDE.md), Edit(astra.yaml), Edit(universes/*), Edit(CLAUDE.md), Glob, Grep, Bash(astra:*), Bash(lc:*), Bash(mkdir:*), Bash(echo:*), @@ -34,7 +34,9 @@ arbitrary files. The lc-extractor subagent is invoked via `Task`. and write them to `astra.yaml`. 4. **Finalize** — `astra validate astra.yaml`, `astra validate --verify-evidence` if quotes exist, `astra universe generate -n - baseline`, populate the `## Working Notes` section of `CLAUDE.md` + baseline`, populate the `narrative:` block in `astra.yaml` (`summary`, + `methods`, `inputs`, `outputs` — `findings` stays TODO until results + exist), then populate the `## Working Notes` section of `CLAUDE.md` with conversational context not captured in the spec. The skill writes to `astra.yaml` after each phase rather than in bulk diff --git a/docs/skills/lc-verify.md b/docs/skills/lc-verify.md index 47ea30a1..08105967 100644 --- a/docs/skills/lc-verify.md +++ b/docs/skills/lc-verify.md @@ -7,7 +7,7 @@ Source: [`claude/lightcone/skills/lc-verify/SKILL.md`](https://github.com/Lightc ## Allowed tools -``` +```text Read, Glob, Grep, Bash(astra:*), Bash(lc:*), Bash(python:*), Bash(ls:*), AskUserQuestion @@ -32,7 +32,7 @@ No `Write`, no `Edit`. The skill cannot modify the project. ## Report format -``` +```text | Check | Status | |--------------------------|--------| | Spec validation | ✓/✗ | diff --git a/docs/user/agent-workflow.md b/docs/user/agent-workflow.md index 2cec5e0e..de317c45 100644 --- a/docs/user/agent-workflow.md +++ b/docs/user/agent-workflow.md @@ -34,8 +34,11 @@ The skill walks you through four phases: (methodological choices that could shift results). `→ astra.yaml`. 4. **Finalize.** `astra validate astra.yaml` to make sure the spec is valid; `astra universe generate -n baseline` to seed a baseline - universe; the `## Working Notes` section of `CLAUDE.md` gets the - conversational context that wouldn't otherwise survive a `/clear`. + universe; the `narrative:` block in `astra.yaml` gets filled in + (`summary`, `methods`, `inputs`, `outputs` — `findings` stays TODO + until results exist); the `## Working Notes` section of `CLAUDE.md` + gets the conversational context that wouldn't otherwise survive a + `/clear`. You don't write any code or YAML during `/lc-new`. By the time it finishes, you have a precise specification. The agent enforces this: diff --git a/docs/user/multiverse.md b/docs/user/multiverse.md index e36e1ce4..41ab116a 100644 --- a/docs/user/multiverse.md +++ b/docs/user/multiverse.md @@ -79,7 +79,7 @@ lc run accuracy --universe baseline # one of each Each universe gets its own results directory: -``` +```text results/ ├── baseline/ │ ├── r2/... @@ -92,7 +92,7 @@ results/ `lc status` shows a per-universe column, so you can see at a glance which universes are caught up: -``` +```text Universe baseline ✓ ok r2 ✓ ok fit_plot diff --git a/docs/user/troubleshooting.md b/docs/user/troubleshooting.md index 8f33b921..194d8dd3 100644 --- a/docs/user/troubleshooting.md +++ b/docs/user/troubleshooting.md @@ -179,7 +179,7 @@ claude Inside Claude Code: -``` +```text /lc-feedback the lc-extractor agent crashed on PDF X ``` diff --git a/docs/user/tutorial.md b/docs/user/tutorial.md index c4d7786b..ca427da7 100644 --- a/docs/user/tutorial.md +++ b/docs/user/tutorial.md @@ -35,7 +35,7 @@ recipes yet." Type: -``` +```text /lc-new ``` @@ -95,16 +95,18 @@ outputs: container: Containerfile ``` -Phase 4 (**FINALIZE**) runs `astra validate astra.yaml` and writes -`universes/baseline.yaml`. You're handed back a short summary table — -two outputs, one decision, zero prior insights. +Phase 4 (**FINALIZE**) runs `astra validate astra.yaml`, writes +`universes/baseline.yaml`, and fills in the `narrative:` block in +`astra.yaml` (`summary`, `methods`, `inputs`, `outputs`). You're handed +back a short summary table — two outputs, one decision, zero prior +insights. The agent suggests `/clear` to free up context, then `/lc-build`. Take its advice. ## 3. Build it with `/lc-build` -``` +```text /clear /lc-build ``` @@ -114,7 +116,7 @@ empty `scripts/` dir, the references in `.claude/guides/`) and writes a build plan to `.lightcone/plans/build-plan-baseline.md`. It might look like this: -``` +```text 1. Add Python deps (scikit-learn, matplotlib) to requirements.txt 2. Write Containerfile if missing 3. scripts/fit.py — accepts --standardize {standardized,raw}, writes r2.json @@ -129,7 +131,7 @@ It asks you to approve. Pick "Approve and start building." **Phase 2: loop.** The agent works through the plan one item at a time. You'll see lines like: -``` +```text ▶ scripts/fit.py — writing ▶ lc build — building image lc-r2-decision-demo-9a1f3... ▶ lc run accuracy --universe baseline @@ -144,13 +146,13 @@ clean record of the build. ## 4. Verify it with `/lc-verify` -``` +```text /lc-verify ``` Read-only audit: -``` +```text | Check | Status | |--------------------------|--------| | Spec validation | ✓ | @@ -181,7 +183,7 @@ lc status You should see: -``` +```text Universe baseline ✓ ok r2 ✓ ok fit_plot diff --git a/zensical.toml b/zensical.toml index 04164e13..019583f6 100644 --- a/zensical.toml +++ b/zensical.toml @@ -30,7 +30,6 @@ nav = [ {"lc status" = "cli/status.md"}, {"lc verify" = "cli/verify.md"}, {"lc export" = "cli/export.md"}, - {"lc setup" = "cli/setup.md"}, ]}, {"Python API" = [ {"Overview" = "api/index.md"},