Shrink pre_build to what its run produces (audit step 3)

autobuild/generate.py

@@ -1,5 +1,6 @@
 import os
 import shutil
+import subprocess
 import sys
 import time
 from argparse import ArgumentParser
@@ -104,7 +105,10 @@ def copy_to_notebooks(source):
         try:
             notebook = build_util.py_to_notebook(start_here_file)
             build_util.inject_colab_setup(notebook, project)
-            os.system(f"git add -f {notebook}")
+            # Checked call, no -f: start_here.ipynb is tracked and unignored in
+            # every workspace that has one (measured, #156); a failure here must
+            # surface, not vanish into an ignored os.system return.
+            subprocess.run(["git", "add", "--", str(notebook)], check=True)
             if report is not None:
                 from result_collector import ScriptResult, Status
                 report.results.append(ScriptResult(

pre_build.sh

@@ -46,7 +46,12 @@ run_workspace() {
     cd "$dir"
 
     echo "  Running black..."
-    black .
+    # Format only the dirs this script stages — black'ing "." reformatted files
+    # no staging rule covered (13 files across HowTo*/assistant), leaving them
+    # perpetually dirty-local on every release (#156).
+    for d in scripts slam_pipeline; do
+        if [ -d "$d" ]; then black "$d/"; fi
+    done
 
     if [ "$generate" = "true" ]; then
         echo "  Running generate.py ($project)..."
@@ -59,20 +64,15 @@ run_workspace() {
         exit 1
     }
 
-    echo "  Staging safe directories..."
-    # Honor .gitignore for dataset/ — each workspace ignores dataset/** except a
-    # small allowlist of REAL observational data (un-ignored via ! lines). A prior
-    # `git add -f` here force-committed simulated datasets that are meant to be
-    # generated at runtime via al.util.dataset.should_simulate(); only allowlisted
-    # real data may be staged. See PyAutoBuild#126.
-    # A fully-ignored dataset/ is the normal case, not an error: git add exits
-    # non-zero when every path it matches is ignored, and under `set -e` that
-    # aborts the whole release. Staging nothing is the correct outcome there.
-    if [ -d dataset ]; then
-        git add dataset/ 2>/dev/null || true
-    fi
-    for d in config notebooks scripts; do
-        [ -d "$d" ] && git add "$d/"
+    echo "  Staging what this run produced..."
+    # Stage only what the run itself modifies: black'd scripts and the
+    # generated notebooks. The former dataset/ and config/ adds staged nothing
+    # the run produced — they swept pre-existing human-uncommitted work into
+    # release commits, which is the mechanism that leaked simulated datasets
+    # (#126). Releases require clean mains (Heart gates on it); human work is
+    # committed by humans. See docs/pre_build_failure_audit.md §3/§6 (#156).
+    for d in notebooks scripts; do
+        if [ -d "$d" ]; then git add "$d/"; fi
     done
     if [ "$slam" = "true" ] && [ -d "slam_pipeline" ]; then
         git add slam_pipeline/

skills/pre_build/pre_build.md

@@ -55,7 +55,7 @@ bash $HOME/Code/PyAutoLabs/PyAutoBuild/bin/autobuild pre_build <minor_version>
 The script handles every mechanical step of the pre-build flow:
 
 1. Ensures the canonical `pending-release` label exists on each release-window repo.
-2. For every workspace, runs `black .`, runs `generate.py` for projects with a notebook target, and stages only the safe directories (`config/`, `notebooks/`, `scripts/`, `dataset/`, plus `slam_pipeline/` for `autolens_workspace`). Root-level artifacts and README Colab URLs are committed by `release.yml` on the runner, not here.
+2. For every workspace, runs black on the staged dirs (`scripts/`, `slam_pipeline/`), runs `generate.py` for projects with a notebook target, and stages only what the run itself produced (`notebooks/`, `scripts/`, plus `slam_pipeline/` for `autolens_workspace`). It does not stage `dataset/` or `config/` — nothing in the run modifies them, and sweeping pre-existing human work into release commits was the #126 leak mechanism. Root-level artifacts and README Colab URLs are committed by `release.yml` on the runner, not here.
 3. Commits and pushes each workspace (skipping if no changes are staged).
 4. Commits and pushes PyAutoBuild itself.
 5. Dispatches `gh workflow run release.yml --repo PyAutoLabs/PyAutoBuild --field minor_version=<N>`.