docs: science projects use dataset/ not data/ (workspace convention)#70
Merged
Conversation
On CPU the correct accelerator depends on the source model, and the docs
previously implied JAX was GPU-only ("## JAX on GPU") while telling users to set
number_of_cores to their core count unconditionally. That combination leads
straight to the slow path: a real SLaM run spent >12h stuck in source_lp.
The rule, from autolens_workspace's cpu_fast_modeling.py:
- Parametric source (source_lp): JAX (use_jax=True) vectorises and parallelises
well on CPU; JAX disables multiprocessing so number_of_cores is left unset.
- Pixelized source (source_pix/light/mass): sparse operator formalism
(dataset.apply_sparse_operator_cpu()) + use_jax=False + number_of_cores=N.
It is numba-based and does not support JAX, so the two never combine.
A SLaM pipeline spans both regimes -> two SettingsSearch objects, two datasets.
- al_configure_search: new "Branch — CPU acceleration" section; corrected the
misleading number_of_cores bullet.
- init-slam: pointer at Step 3, since the workspace SLaM scripts don't do this.
- wiki/core/operations/hpc: replaced "CPU parallelism"/"JAX on GPU" framing with
the two-regime table; added cpu_fast_modeling.py as a source.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…pedantic) start-new-project scaffolded datasets under data/, diverging from the PyAuto workspace convention (dataset/) used by this clone, every autolens_workspace example, PyAutoReduce's scripts/docs, and the --sample/--dataset CLI idiom (dataset/<sample>/<dataset_name>/). A project on data/ silently breaks scripts, hpc/sync DATA_DIRS and anything copied from the workspace. Switch the scaffold, .gitignore, manifest example and Publish gate to dataset/, and add a pedantic note telling the agent never to use data/ or accept it silently. project.yaml's metadata key is unchanged (different thing). Also un-ignore dataset/**/info.json (small, needed to reproduce). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HD3ZkpFRydshFLeu2V6xDi
This was referenced Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
start-new-projectscaffolded datasets underdata/, diverging from the workspace conventiondataset/— used by this clone, everyautolens_workspaceexample, PyAutoReduce's scripts/docs, and the--sample/--datasetCLI idiom.A project on
data/silently breaks scripts,hpc/sync'sDATA_DIRS, and anything copied from the workspace (surfaced in a real project: nothing synced to HPC untilDATA_DIRSwas hand-patched).Switches the scaffold tree,
.gitignore, manifest example and Publish gate todataset/, and adds a pedantic note telling the agent never to usedata/nor accept it silently.project.yaml'sdata:key is untouched (metadata, not the folder). Also un-ignoresdataset/**/info.json.PyAutoReduce needs no change — already
dataset/throughout.🤖 Generated with Claude Code