diff --git a/.agents/skills/bootstrap-diagnostics/SKILL.md b/.agents/skills/bootstrap-diagnostics/SKILL.md index 3e6d47371..ba935aa7b 100644 --- a/.agents/skills/bootstrap-diagnostics/SKILL.md +++ b/.agents/skills/bootstrap-diagnostics/SKILL.md @@ -2,7 +2,7 @@ name: bootstrap-diagnostics description: >- Agent-only handling playbook for session-start bootstrap diagnostics. - Use whenever the session-start digest's bootstrap section prints any diagnostic or capability line - MISSING, MISSING_MANUAL, BACKEND_INVALID, NEEDS_GH_AUTH, TANGLE, CREW_HARNESS_OVERRIDE, CREW_DISPATCH, FLEET_SYNC, SECONDMATE_SYNC, SECONDMATE_LIVENESS, TASKS_AXI, NUDGE_SECONDMATES, or FMX - or when a standalone bin/fm-bootstrap.sh run prints one. + Use whenever the session-start digest's bootstrap section prints any diagnostic or capability line - MISSING, MISSING_MANUAL, BACKEND_INVALID, NEEDS_GH_AUTH, TANGLE, CREW_HARNESS_OVERRIDE, CREW_DISPATCH, FLEET_SYNC, SECONDMATE_SYNC, SECONDMATE_LIVENESS, TASKS_AXI, NUDGE_SECONDMATES, SERVICE_FAILED, FMX, or UPSTREAM_DRIFT - or when a standalone bin/fm-bootstrap.sh run prints one. A silent bootstrap section means all good and needs no skill load. user-invocable: false metadata: @@ -47,3 +47,12 @@ The inline rules in `AGENTS.md` section 3 still bind: detect, then consent, then A secondmate that was skipped, already current, or whose advance changed no instructions is not listed and must not be disturbed. - `FMX: X mode on ...` / `FMX: X mode off ...` - bootstrap confirmed or removed the local X-mode poll artifacts (`docs/configuration.md` "X mode (.env)"). Only when a running watcher needs the cadence transition applied immediately, restart the home-scoped watcher through the emitted harness supervision protocol; bootstrap deliberately never restarts the watcher itself. +- `SERVICE_FAILED: - failed since ` - a systemd unit named in the optional local `config/critical-services` file is in the failed state; a pure read-only detection (`systemctl is-failed`, no root), so a read-only session still surfaces it. + Report it to the captain in plain language; never restart the unit yourself - a unit that failed may be unsafe to restart without knowing why. + An absent or empty config file, a host without `systemctl`, or no failed unit all print nothing. +- `UPSTREAM_DRIFT: local main is ahead / behind upstream/main, last reconciled d ago ()` - the always-on FYI variant: firstmate's OWN repo tracking how far it has drifted from its read-only upstream template (`kunchenguid/firstmate`, the `upstream` remote after the remote swap). + It prints every session; record it silently like any capability fact and take no action while the wording stays FYI. + It only reports - the network fetch that refreshes the ref runs in the locked fleet-sync sweep, and reconciling upstream is never a bootstrap side effect. +- `UPSTREAM_DRIFT: this repo's upstream sync needs attention - ...` - the escalated variant (local main more than ~30 commits behind `upstream/main`, or the merge-base older than 10 days). + Surface it to the captain in plain outcome language and, on their go-ahead, dispatch a deliberately-reviewed firstmate-repo reconciliation ship task - fetch `upstream`, merge it into local `main`, resolve conflicts, and land local-only (`AGENTS.md` section 1) - the same shape that reconciled the divergence before. + Never automate the merge; keeping the gap small also keeps the pipeline's opportunistic upstream PR's diff clean (`AGENTS.md` section 1). diff --git a/.agents/skills/harness-adapters/SKILL.md b/.agents/skills/harness-adapters/SKILL.md index 63f378bb7..ec6bb5bc0 100644 --- a/.agents/skills/harness-adapters/SKILL.md +++ b/.agents/skills/harness-adapters/SKILL.md @@ -132,6 +132,13 @@ Its broader dark-TRUECOLOR placeholder handling and dark-theme tradeoff are docu That styled capture is internal to the boolean detector only. `fm-peek` and every other human or LLM-facing capture path stays plain `tmux capture-pane` with no escape codes. +**Auto-compaction fact (verified 2026-07-09, Claude Code 2.1.206; shipped constant raised to 300000 2026-07-11).** +`bin/fm-spawn.sh` launches every claude crewmate and secondmate with `CLAUDE_CODE_AUTO_COMPACT_WINDOW=300000`, scoped the same way as `CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false` above, so these separately-launched processes auto-compact around 300k tokens the way the primary already does via its own `.claude/settings.local.json` (which they never inherit). +The var is read once at process startup and is capped at the model's real context window, so it can only lower the effective trigger, never raise it past the real ceiling: `min(300000, window)` is `300000` once `window` exceeds it, but a no-op (stays at `window`) below that. +This is not a universal guarantee for every possible future model choice - a differently-configured account/model capped at the plain ~200k window would see no change - but it is true for the window tier crew actually runs on today: `--model sonnet` on this account, the standard crewmate spawn, resolves to the ~1M-context beta, well above 300000. +Verified live in a tmux pane at the original 200000 value (the mechanism is unchanged, only the shipped constant moved to 300000): `claude --model sonnet` on this account resolves to a ~967k-token auto-compact window with no override (`/context` reported `Auto-compact window: 967k tokens`, i.e. the 1M-context beta, not the 200k a plain Sonnet window would imply); relaunched with `CLAUDE_CODE_AUTO_COMPACT_WINDOW=200000` set, `/context` reported `39.5k/200k tokens (20%)` and `Auto-compact window: 200k tokens` - the window pinned to the override as expected. +This is why the primary's own two-var form (`CLAUDE_CODE_AUTO_COMPACT_WINDOW=1000000` + `CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=20`) is not copied here: that pair assumes a 1M window, and would silently cap a smaller-window model/account combination to a far more aggressive trigger. + **Primary-session guard fact (verified 2026-07-04, Claude Code 2.1.201; preserved 2026-07-08, Claude Code 2.1.204).** This is separate from the per-task crewmate turn-end hook above (that one just `touch`es a marker file in a task's own `.claude/settings.local.json`). The firstmate PRIMARY's own `.claude/settings.json` registers `bin/fm-turnend-guard.sh` as a Stop hook, and exiting with status 2 plus stderr reliably forces the model to continue. diff --git a/.agents/skills/updatefirstmate/SKILL.md b/.agents/skills/updatefirstmate/SKILL.md index 015bc1350..f50d50c47 100644 --- a/.agents/skills/updatefirstmate/SKILL.md +++ b/.agents/skills/updatefirstmate/SKILL.md @@ -18,6 +18,9 @@ It never forces, never creates a merge commit, never stashes, and advances a tar A tracked-files fast-forward leaves the gitignored operational dirs (data/, state/, config/, projects/, .no-mistakes/) untouched, so a secondmate's in-flight work is never disrupted. This touches only the firstmate repo and its own worktrees, never anything under `projects/`. +Post-swap, this pull follows `origin`, which is the captain's fork, so `/updatefirstmate` converges the whole fleet onto that fork and does nothing on the primary checkout, which is itself the source every fork tracks. +It deliberately never reaches for the read-only `upstream` template: folding template improvements into local `main` is a separate, reviewed reconciliation ship task, the one the `UPSTREAM_DRIFT:` bootstrap diagnostic surfaces, and it never rides in on a self-update. + ## What it does 1. **Run the updater:** diff --git a/.gitignore b/.gitignore index 5ed2da0c3..cc1d02e14 100644 --- a/.gitignore +++ b/.gitignore @@ -16,3 +16,4 @@ config/backend config/x-mode.env config/cmux-socket-password config/wedge-alarm +config/critical-services diff --git a/AGENTS.md b/AGENTS.md index 1df624bc7..ffe034814 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -41,6 +41,9 @@ Shared tracked material is `AGENTS.md`, `README.md`, `CONTRIBUTING.md`, `.tasks. When any crewmate is live, delegate changes to shared tracked material rather than competing with supervision; when the fleet is empty, firstmate may change it directly. This repo is a shared template, while `.env`, `data/`, `state/`, `config/`, `projects/`, and `.no-mistakes/` are captain-private and gitignored. Ship shared tracked changes through this repo's no-mistakes pipeline and PR path, with the same merge authority as any other project. +This repo's own upstream (`kunchenguid/firstmate`, the `upstream` remote after the remote swap) is read-only to this fleet with no merge rights, ever, so firstmate-repo ship tasks land local-only: branch, commit, firstmate reviews the diff, the captain approves, and firstmate fast-forwards local `main`. +`bin/fm-merge-local.sh` then also best-effort pushes that fast-forwarded `main` to `origin` (the captain's fork) when a remote is present, so the fork never silently drifts behind local `main`; a push failure is reported but never fails the merge, and a local-only project with no remote at all is skipped silently. +The pipeline still auto-opens a PR against that upstream, but it is a non-blocking, purely opportunistic contribution back - never a merge gate here and never waited on - and its diff stays clean only while the `UPSTREAM_DRIFT:` gap (section 3) is small, since a large gap makes the PR carry a lot of unrelated local-only history. Never add an agent name as a commit co-author. ## 2. Layout and state @@ -85,6 +88,10 @@ Routing precedence is an explicit per-task captain override, then the best-fit c The generic effort fallback and its precedence are owned by `harness-adapters`: explicit captain and standing configured effort win; otherwise use low for well-understood explicit work, xhigh for ambiguous investigation or design, intermediate levels proportionally, and never max without explicit captain preference. Do not add model-specific versions of that policy. +A profile's `use.ultracode` marks a task that needs a genuinely independent second pass on its finished diff before PR-ready, launched as its own separately dispatched task, never a sub-task the flagged crewmate spawns itself; run `bin/fm-ultracode-guard.sh flag ` right after spawn and `check ` before PR-ready (section 7's Validate). +Run `bin/fm-risk-tripwire.sh ` after resolving the profile and again once the task has a brief or diff; a hit floors the model/effort to `opus`/`xhigh` with ultracode `independent-review` regardless of the matched rule, and also run `bin/fm-ultracode-guard.sh flag independent-review` so the requirement is tracked mechanically even when the matched rule did not set ultracode; a hit discovered only at Validate time still floors the task in place and is never silently downgraded afterward. +The unverified-adapter discipline also covers models: verify a non-default `model` end-to-end on a trivial supervised task, including any sub-agent path its harness exposes, before a dispatch rule may name it, and never copy an unverified model into a live `config/crew-dispatch.json`. + `secondmate-provisioning` owns secondmate harness pins and config inheritance, while `harness-adapters` owns the harness consequences. Dispatch only on a backend that `fm-spawn` validates as spawn-capable. A missing dependency, authentication failure, unsupported backend, or version refusal is a blocker; never silently retry on another backend. @@ -206,6 +213,15 @@ Running, fixing, or CI states remain working; parked approval or fix-review stat A worker hand-editing, committing, aborting, or restarting during an active validation run duplicates pipeline ownership; steer it back to the gate response flow. The worker reports the PR when CI first becomes green rather than waiting for merge monitoring to finish. +**Escalation triggers (mechanical, never rest on self-report alone).** +For a task assigned the trivial tier (Haiku/low), run `bin/fm-tier-guard.sh ` during Validate, or whenever a heartbeat review touches it, to check whether its diff or elapsed time has outgrown that tier's envelope; any tier also escalates once its diff crosses the script's general heavy-scale ceiling. +A report escalates the task to at least Sonnet/high in place, without losing its branch or context, mirroring `bin/fm-promote.sh`'s in-place promotion. +A crewmate's own report - it cannot find root cause, a "confirmed" fix touches a shared path, or it raised a tradeoff buried in seemingly mechanical work - is a second, non-mechanical trigger with the same effect. +Either way, escalate the model/effort in place and never silently de-escalate for the rest of that task's life. + +**Ultracode confirmation.** +Before advancing an ultracode-flagged task to PR-ready, run `bin/fm-ultracode-guard.sh check `; it refuses until a genuinely separate task - dispatched independently, never a sub-task the flagged crewmate spawned itself - has reviewed the finished diff and its findings were addressed, recorded with `bin/fm-ultracode-guard.sh reviewed `. + ### PR ready, landing, and teardown For a ready PR, use `bin/fm-pr-check.sh` to record the PR and authoritative head and to arm merge monitoring. @@ -215,6 +231,7 @@ A captain instruction to merge is explicit authority; `yolo` is the only standin Tear down a ship task only after landing is confirmed. A teardown refusal for uncommitted or unlanded work is a stop-and-investigate result, never an obstacle to bypass. Never force teardown without explicit discard authority. +Any non-scout, non-secondmate teardown - PR-based or local-only - also best-effort syncs and restarts that project's live NAS deployment, if one is recorded in `data/nas-deployments.md`, through `bin/fm-nas-deploy-sync.sh` (section 2); a project with no recorded deployment is a silent no-op. After successful teardown, record completion, retain only the configured recent Done history, and re-evaluate queued work whose blockers and time gates have cleared. A secondmate is persistent and an empty queue is healthy. @@ -348,12 +365,14 @@ Firstmate's shared instruction surface reaches running homes only after it lands Only `AGENTS.md`, `bin/`, and `.agents/skills/` are loaded by a running firstmate; public `skills/` is an installer-facing surface. When the captain invokes `/updatefirstmate` or asks to update firstmate, load the `/updatefirstmate` skill. It performs guarded fast-forward updates of firstmate and registered secondmate homes, refreshes instructions, and never touches anything under `projects/`. +Post-swap, `origin` is the captain's fork, so `/updatefirstmate` only ever converges the fleet onto that fork and is a no-op on the primary checkout, which is itself the fleet's source of truth. +It never pulls the read-only `upstream` template; folding upstream-template improvements into local `main` is the separate reviewed reconciliation task the `UPSTREAM_DRIFT:` bootstrap line surfaces (section 3), never a self-update. ## 13. Agent-only reference skills These skills are not captain-invocable; load them only at their precise triggers. -- `bootstrap-diagnostics` - load whenever the session-start digest's bootstrap section prints any diagnostic or capability line (`MISSING:`, `MISSING_MANUAL:`, `BACKEND_INVALID:`, `NEEDS_GH_AUTH`, `TANGLE:`, `CREW_HARNESS_OVERRIDE:`, `CREW_DISPATCH:`, `FLEET_SYNC:`, `SECONDMATE_SYNC:`, `SECONDMATE_LIVENESS:`, `TASKS_AXI:`, `NUDGE_SECONDMATES:`, or `FMX:`); silence needs no load. +- `bootstrap-diagnostics` - load whenever the session-start digest's bootstrap section prints any diagnostic or capability line (`MISSING:`, `MISSING_MANUAL:`, `BACKEND_INVALID:`, `NEEDS_GH_AUTH`, `TANGLE:`, `CREW_HARNESS_OVERRIDE:`, `CREW_DISPATCH:`, `FLEET_SYNC:`, `SECONDMATE_SYNC:`, `SECONDMATE_LIVENESS:`, `TASKS_AXI:`, `NUDGE_SECONDMATES:`, `FMX:`, `SERVICE_FAILED:`, or `UPSTREAM_DRIFT:`); silence needs no load. - `diagnostic-reasoning` - load before scoping a reported bug and before acting on a diagnostic report. - `harness-adapters` - load before spawning or recovering a crewmate or secondmate, handling a trust dialog, sending a harness-specific skill invocation, interrupting or exiting an agent, resuming an exited agent, or verifying a new harness adapter. - `firstmate-orca` - load before switching to Orca, spawning or supervising Orca-backed work, smoke-testing Orca backend behavior, debugging Orca task state, or reconciling Orca-backed task metadata. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4e7bd8154..52e790d91 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,6 +30,8 @@ Dependency bots are exempt so their automation keeps working, but regular contri See the [no-mistakes quick start](https://kunchenguid.github.io/no-mistakes/start-here/quick-start/) for the full first-run walkthrough. +Note: a firstmate maintainer's own working copy uses the inverted remote layout - there `origin` is the maintainer's fork and `upstream` is this parent repo - so if you are ever reading such a copy's git config rather than your own fresh clone, expect `origin` and `upstream` swapped from the step 1 layout above. + ## Repo conventions - This repo is a template for running a firstmate orchestrator agent. diff --git a/bin/backends/cmux.sh b/bin/backends/cmux.sh index 69dc0b53b..478e9ee50 100644 --- a/bin/backends/cmux.sh +++ b/bin/backends/cmux.sh @@ -398,8 +398,15 @@ fm_backend_cmux_parse_target() { # # (fm_backend_zellij_pane_exists) rather than the design sketch's original # read-screen-based suggestion. fm_backend_cmux_surface_exists() { # - local wsid=$1 sfid=$2 - fm_backend_cmux_cli list-panes --workspace "$wsid" --json --id-format uuids 2>/dev/null \ + local wsid=$1 sfid=$2 out + # Capture the list-panes call and gate on its OWN exit/output before piping to + # jq: an absent target makes list-panes fail with no output, and `jq -e` on + # empty input exits 0 on jq 1.6 (only 1.7+ reports the no-result exit 4), so + # relying on the pipe's trailing jq status alone would false-positive the + # surface as present on older jq. A failed or empty list-panes means no surface. + out=$(fm_backend_cmux_cli list-panes --workspace "$wsid" --json --id-format uuids 2>/dev/null) || return 1 + [ -n "$out" ] || return 1 + printf '%s' "$out" \ | jq -e --arg s "$sfid" '[.panes[]? | select(.surface_ids // [] | index($s))] | length > 0' >/dev/null 2>&1 } diff --git a/bin/fm-afk-launch.sh b/bin/fm-afk-launch.sh index a6ddcbcc6..0e17371de 100755 --- a/bin/fm-afk-launch.sh +++ b/bin/fm-afk-launch.sh @@ -603,10 +603,18 @@ fm_afk_launch_stop() { fm_afk_launch_main() { local result - fm_afk_launch_lock_acquire || return 1 + # Traps go in BEFORE the lock is taken: a TERM/INT landing inside + # fm_afk_launch_lock_acquire (after mkdir, during the pid-identity write) + # would otherwise kill the launcher with the just-created lock left behind. + # fm_afk_launch_lock_release is pid-guarded, so running it without the lock + # (or while another process owns it) is a safe no-op. trap fm_afk_launch_lock_release EXIT trap 'exit 130' INT trap 'exit 143' TERM + if ! fm_afk_launch_lock_acquire; then + trap - EXIT INT TERM + return 1 + fi case "${1:-start}" in start) fm_afk_launch_start ;; start-native) fm_afk_launch_start_native ;; diff --git a/bin/fm-bootstrap.sh b/bin/fm-bootstrap.sh index 1f5cc5d80..ec9f2ad5f 100755 --- a/bin/fm-bootstrap.sh +++ b/bin/fm-bootstrap.sh @@ -14,7 +14,10 @@ # "SECONDMATE_SYNC: secondmate : skipped: ", # "NUDGE_SECONDMATES: fm-...", # "SECONDMATE_LIVENESS: secondmate : already-live|respawned|skipped: |respawn failed: ", -# "FMX: X mode on ..." or "FMX: X mode off ...". +# "FMX: X mode on ..." or "FMX: X mode off ...", +# "SERVICE_FAILED: - failed since ", +# "UPSTREAM_DRIFT: local main ahead / behind +# upstream/main ... (needs-attention wording when far behind or stale)". # A NUDGE_SECONDMATES line lists the RUNNING secondmate task selectors # (fm-) whose worktree was fast-forwarded to firstmate's own # current default-branch commit (a purely LOCAL fast-forward, never an @@ -56,8 +59,30 @@ # X mode is OPTIONAL and inert unless FM_HOME/.env has a non-empty # FMX_PAIRING_TOKEN. When opted in, bootstrap requires curl+jq, writes # the relay poll shim and 30s cadence config, and prints an FMX line. +# A SERVICE_FAILED line means a systemd unit named in the optional local +# config/critical-services file is in the failed state; it is a pure +# read-only detection (systemctl is-failed, no root) so a read-only +# session still surfaces it. Absent config file, no systemctl, or no +# failed unit all print nothing. The timestamp clause is omitted when +# StateChangeTimestamp is unavailable. +# An UPSTREAM_DRIFT line reports, every session, how far firstmate's OWN +# local main and its read-only upstream template (upstream/main after the +# remote swap; kunchenguid/firstmate) have diverged in both directions, +# plus days since their last shared merge-base. It reads whatever +# refs/remotes/upstream/main currently holds and NEVER fetches, so it +# prints in the read-only/no-lock path too; the network fetch that +# refreshes that ref runs only in the locked mutating fleet-sync sweep +# below (bounded by FM_UPSTREAM_FETCH_TIMEOUT, default 20s). The wording +# escalates from a plain FYI to "this repo's upstream sync needs +# attention" once local main is more than 30 commits behind upstream/main +# or the merge-base is older than 10 days. It only REPORTS; reconciling +# upstream is always a deliberately dispatched, reviewed ship task, never +# automated. No-op when there is no upstream remote or no upstream/main +# ref yet. # Fleet sync fetches, fast-forwards safe default-branch states, reports -# recovered and STUCK clone drift, and prunes gone local branches; it is +# recovered and STUCK clone drift, and prunes gone local branches, and it +# also refreshes firstmate's own upstream template for the UPSTREAM_DRIFT +# report above; it is # bounded by FM_FLEET_SYNC_BOOTSTRAP_TIMEOUT when it is a non-empty # numeric override, while non-numeric values fall back to 20s. # When the override is unset or blank, the timeout is @@ -148,7 +173,76 @@ fleet_sync_relay_all_output() { done < "$tmp" } +# Upstream-drift diagnostic for firstmate's OWN repo (both halves defined here). +# firstmate's upstream template (upstream/main after the remote swap; +# kunchenguid/firstmate) is read-only to this fleet - it can never be pushed or +# merged from here - so local main and upstream/main drift apart silently over +# time, and firstmate-repo ship tasks land local-only instead (AGENTS.md prime +# directives). The report reads whatever refs/remotes/upstream/main currently +# holds WITHOUT fetching, so it is safe in the read-only/no-lock detect path; the +# fetch runs only in the locked fleet-sync sweep. It only REPORTS - reconciling +# upstream stays a deliberately dispatched, reviewed ship task, never automated. +UPSTREAM_DRIFT_BEHIND_MAX=30 +UPSTREAM_DRIFT_DAYS_MAX=10 + +# Count-and-report half: unconditional (detect section), no network. Prints one +# FYI line every session with how far local main is ahead/behind upstream/main and +# days since their merge-base, escalating the wording to "needs attention" once +# local main is more than UPSTREAM_DRIFT_BEHIND_MAX commits behind OR the +# merge-base is older than UPSTREAM_DRIFT_DAYS_MAX days. No-op without an upstream +# remote, an upstream/main ref, or a local main. +upstream_drift_report() { + local behind ahead base="" base_ct="" base_date="unknown" now="" days="?" attention=0 + git -C "$FM_ROOT" remote get-url upstream >/dev/null 2>&1 || return 0 + git -C "$FM_ROOT" rev-parse --verify --quiet refs/remotes/upstream/main >/dev/null 2>&1 || return 0 + git -C "$FM_ROOT" rev-parse --verify --quiet refs/heads/main >/dev/null 2>&1 || return 0 + behind=$(git -C "$FM_ROOT" rev-list --count main..upstream/main 2>/dev/null) || return 0 + ahead=$(git -C "$FM_ROOT" rev-list --count upstream/main..main 2>/dev/null) || return 0 + base=$(git -C "$FM_ROOT" merge-base main upstream/main 2>/dev/null || true) + if [ -n "$base" ]; then + base_date=$(git -C "$FM_ROOT" log -1 --format=%cd --date=short "$base" 2>/dev/null || echo unknown) + base_ct=$(git -C "$FM_ROOT" log -1 --format=%ct "$base" 2>/dev/null || true) + now=$(date +%s 2>/dev/null || true) + if [ -n "$base_ct" ] && [ -n "$now" ]; then + days=$(( (now - base_ct) / 86400 )) + fi + fi + case "$behind" in ''|*[!0-9]*) ;; *) [ "$behind" -gt "$UPSTREAM_DRIFT_BEHIND_MAX" ] && attention=1 ;; esac + case "$days" in ''|*[!0-9]*) ;; *) [ "$days" -gt "$UPSTREAM_DRIFT_DAYS_MAX" ] && attention=1 ;; esac + if [ "$attention" -eq 1 ]; then + echo "UPSTREAM_DRIFT: this repo's upstream sync needs attention - local main is $behind behind / $ahead ahead of upstream/main, last reconciled ${days}d ago ($base_date); dispatch a reviewed reconciliation ship task" + else + echo "UPSTREAM_DRIFT: local main is $ahead ahead / $behind behind upstream/main, last reconciled ${days}d ago ($base_date)" + fi +} + +# Fetch half: MUTATING/network, called only from fleet_sync (the locked sweep), so +# the detect-only/read-only path never fetches. Best-effort and bounded - offline, +# a transient failure, or a stalled connection just leaves the last-fetched ref in +# place. A timeout/gtimeout binary is required for the refresh; without one, +# bootstrap leaves the last-fetched ref in place rather than running an +# unbounded network fetch. +# No-op without an upstream remote. +upstream_drift_fetch() { + local timeout_bin="" + git -C "$FM_ROOT" remote get-url upstream >/dev/null 2>&1 || return 0 + if command -v timeout >/dev/null 2>&1; then + timeout_bin=timeout + elif command -v gtimeout >/dev/null 2>&1; then + timeout_bin=gtimeout + fi + [ -n "$timeout_bin" ] || return 0 + "$timeout_bin" "${FM_UPSTREAM_FETCH_TIMEOUT:-20}" git -C "$FM_ROOT" fetch --quiet upstream 2>/dev/null || true +} + fleet_sync() { + # Refresh firstmate's own read-only upstream template first, independent of the + # project-clone sync below (so it runs even with no clones present). This is the + # mutating/network half of the UPSTREAM_DRIFT diagnostic; upstream_drift_report + # in the detect section never fetches, keeping that report safe in the + # read-only detect-only path. + upstream_drift_fetch + [ -x "$FM_ROOT/bin/fm-fleet-sync.sh" ] || return 0 [ -d "$PROJECTS" ] || return 0 @@ -501,6 +595,7 @@ crew_dispatch_validate() { fi err=$(jq -r ' def verified($h): ["claude","codex","opencode","pi","grok"] | index($h); + def safe_token: test("^[A-Za-z0-9._-]+$"); def effort_ok($h; $e): if $e == null then true elif ($e | type) != "string" then false @@ -516,9 +611,11 @@ crew_dispatch_validate() { elif ($u | type) == "object" then [$u] else [] end; + def all_profiles: + ([(.rules // [])[]? | use_profiles(.use?)[]?] + + (if (.default? | type) == "object" then [.default] else [] end)); def bad_efforts: - ([(.rules // [])[]? | use_profiles(.use?)[]? | {h: .harness, e: .effort}] - + (if (.default? | type) == "object" then [{h: .default.harness, e: .default.effort}] else [] end)) + (all_profiles | map({h: .harness, e: .effort})) | map(select(.e != null)) | map(select((.h | type) == "string" and verified(.h))) | map(select(. as $p | effort_ok($p.h; $p.e) | not)) @@ -537,6 +634,9 @@ crew_dispatch_validate() { "unknown select: " + ([ (.rules // [])[]? | .select? // empty | select(. != "quota-balanced") ] | unique | join(", ")) elif has("default") and (.default | type) != "object" then "default must be an object" elif has("default") and ((.default.harness? | type) != "string" or (.default.harness | length) == 0) then "default needs harness when present" + elif (all_profiles | map(select(has("ultracode") and (.ultracode | type) != "boolean")) | length) > 0 then "ultracode must be boolean" + elif (all_profiles | map(select(has("ultracode_role") and (.ultracode_role | type) != "string")) | length) > 0 then "ultracode_role must be a string" + elif (all_profiles | map(select(has("ultracode_role") and (.ultracode_role | type) == "string" and ((.ultracode_role | safe_token) | not))) | length) > 0 then "ultracode_role must be a safe token" else ([(.rules // [])[]? | use_profiles(.use?)[]?.harness] + [.default?.harness?] | map(select(. != null)) @@ -572,6 +672,32 @@ crew_dispatch_validate() { ' "$file" } +# Detect critical systemd units in the failed state. Reads one unit name per +# line from config/critical-services (blank lines and full-line # comments +# ignored) and prints one SERVICE_FAILED line per failed unit. Pure read-only +# detection via systemctl is-failed (no root); absent config file, no systemctl, +# or no failed unit all print nothing. +critical_services_check() { + local file unit ts + file="$CONFIG/critical-services" + [ -f "$file" ] || return 0 + command -v systemctl >/dev/null 2>&1 || return 0 + while IFS= read -r unit || [ -n "$unit" ]; do + unit="${unit#"${unit%%[![:space:]]*}"}" + unit="${unit%"${unit##*[![:space:]]}"}" + [ -n "$unit" ] || continue + case "$unit" in '#'*) continue ;; esac + if systemctl is-failed --quiet "$unit" >/dev/null 2>&1; then + ts=$(systemctl show "$unit" --property=StateChangeTimestamp --value 2>/dev/null || true) + if [ -n "$ts" ]; then + echo "SERVICE_FAILED: $unit - failed since $ts" + else + echo "SERVICE_FAILED: $unit - in failed state" + fi + fi + done < "$file" +} + if [ "${1:-}" = "install" ]; then shift [ $# -gt 0 ] || { echo "usage: fm-bootstrap.sh install ..." >&2; exit 1; } @@ -631,6 +757,8 @@ crew_dispatch_validate if ! fm_backlog_backend_manual "$CONFIG" && fm_tasks_axi_compatible; then echo "TASKS_AXI: available" fi +critical_services_check +upstream_drift_report if [ "${FM_BOOTSTRAP_DETECT_ONLY:-0}" != 1 ]; then secondmate_sync secondmate_liveness_sweep diff --git a/bin/fm-brief.sh b/bin/fm-brief.sh index a1b9cb540..1e2672030 100755 --- a/bin/fm-brief.sh +++ b/bin/fm-brief.sh @@ -34,6 +34,9 @@ # local-only implement on branch, stop and report "ready in branch" (no push/PR); # captain approves, firstmate merges to local main # Ship briefs begin with a worktree-isolation assertion before the branch step. +# Both ship and scout briefs include a docker-compose isolation note telling the +# crewmate to read .treehouse-compose-project and pass it on every docker +# compose invocation (docs/configuration.md "Docker Compose project isolation"). # Scout tasks ignore mode - their deliverable is a report, not a merge. # Every scaffold's status protocol distinguishes the configured # declared-external-wait verb (FM_CLASSIFY_PAUSED_VERB, default "paused") from @@ -236,6 +239,12 @@ This is a SCOUT task: the deliverable is a written report, not a PR. The worktree is your laboratory - install, run, edit, and make scratch commits freely; all of it is discarded at teardown. The report is the only thing that survives, so anything worth keeping must be in it. +This worktree also carries a docker-compose isolation marker at \`.treehouse-compose-project\` in the worktree root. +If this task starts docker or docker-compose, read that file and pass its contents explicitly on every \`docker compose\` invocation, e.g. \`docker compose -p "\$(cat .treehouse-compose-project)" up -d\`. +Exported env vars do not reliably persist between separate tool-call invocations, so if you use \`export COMPOSE_PROJECT_NAME=\$(cat .treehouse-compose-project)\` instead, restate it at the start of every docker-related shell command rather than exporting once and assuming it sticks. +Without this, Compose's default project-naming (derived from the worktree's directory basename) collides with every other worktree of this same project, and both fight over the same container/volume/network names. +A second, related collision vector is fixed host ports: prefer not publishing a hard-coded host port and query the assigned one with \`docker compose port \`, or otherwise pick a deterministic per-worktree-derived port, since two isolated stacks can still fight over a shared host port even with different project names. + # Rules 1. Never push to any remote and never open a PR. 2. Stay inside this worktree; the only files you may write outside it are the report and the status file below. @@ -341,6 +350,12 @@ You are in a disposable git worktree of $REPO, at a detached HEAD on a clean def The path check is authoritative: \`git rev-parse --git-dir\` and \`git rev-parse --git-common-dir\` can help inspect the repo, but they do not prove you are outside the primary checkout. If the top-level path is the primary checkout or not the worktree you were launched in, STOP - do not branch or commit here - append \`blocked: launched in primary checkout, not an isolated worktree\` to the status file and stop. +This worktree also carries a docker-compose isolation marker at \`.treehouse-compose-project\` in the worktree root. +If this task starts docker or docker-compose, read that file and pass its contents explicitly on every \`docker compose\` invocation, e.g. \`docker compose -p "\$(cat .treehouse-compose-project)" up -d\`. +Exported env vars do not reliably persist between separate tool-call invocations, so if you use \`export COMPOSE_PROJECT_NAME=\$(cat .treehouse-compose-project)\` instead, restate it at the start of every docker-related shell command rather than exporting once and assuming it sticks. +Without this, Compose's default project-naming (derived from the worktree's directory basename) collides with every other worktree of this same project, and both fight over the same container/volume/network names. +A second, related collision vector is fixed host ports: prefer not publishing a hard-coded host port and query the assigned one with \`docker compose port \`, or otherwise pick a deterministic per-worktree-derived port, since two isolated stacks can still fight over a shared host port even with different project names. + 1. First action: create your branch: \`git checkout -b fm/$ID\`$SETUP2 # Rules diff --git a/bin/fm-dispatch-select.sh b/bin/fm-dispatch-select.sh index c626e9a95..dfe100379 100755 --- a/bin/fm-dispatch-select.sh +++ b/bin/fm-dispatch-select.sh @@ -115,7 +115,9 @@ first_profile() { def clean($p): {harness: $p.harness} + (if ($p.model? | type) == "string" then {model: $p.model} else {} end) - + (if ($p.effort? | type) == "string" then {effort: $p.effort} else {} end); + + (if ($p.effort? | type) == "string" then {effort: $p.effort} else {} end) + + (if ($p.ultracode? | type) == "boolean" then {ultracode: $p.ultracode} else {} end) + + (if ($p.ultracode_role? | type) == "string" then {ultracode_role: $p.ultracode_role} else {} end); clean(.[0]) ' } @@ -169,7 +171,9 @@ selection=$(printf '%s\n' "$quota_json" | jq -ec \ def clean($p): {harness: $p.harness} + (if ($p.model? | type) == "string" then {model: $p.model} else {} end) - + (if ($p.effort? | type) == "string" then {effort: $p.effort} else {} end); + + (if ($p.effort? | type) == "string" then {effort: $p.effort} else {} end) + + (if ($p.ultracode? | type) == "boolean" then {ultracode: $p.ultracode} else {} end) + + (if ($p.ultracode_role? | type) == "string" then {ultracode_role: $p.ultracode_role} else {} end); def provider_for($h): [.providers[]? | select(.provider == $h)][0]; def general_ids($h): if $h == "claude" then ["five_hour", "seven_day"] diff --git a/bin/fm-merge-local.sh b/bin/fm-merge-local.sh index fdc801148..07e5ca50a 100755 --- a/bin/fm-merge-local.sh +++ b/bin/fm-merge-local.sh @@ -66,3 +66,20 @@ before=$(git -C "$PROJ" rev-parse --short "$DEFAULT") git -C "$PROJ" merge --ff-only "$BRANCH" >/dev/null after=$(git -C "$PROJ" rev-parse --short "$DEFAULT") echo "merged $BRANCH into local $DEFAULT ($before -> $after) in $PROJ" + +# Keep a push-backed default branch synced with the merge that just landed, so +# the remote never silently drifts behind local main. This is what keeps +# firstmate's own fork alive after the remote swap that made origin the captain's +# fork (AGENTS.md prime directives note on the firstmate repo's local-only +# landing pattern). Best-effort by design: the local fast-forward above is the +# operation that matters, so a push failure (offline, transient network, or a +# non-fast-forward rejection) is reported but never fails the merge. A pure +# local-only project with no origin remote has nowhere to push and is skipped +# silently - not every local-only project is remote-backed. +if git -C "$PROJ" remote get-url origin >/dev/null 2>&1; then + if git -C "$PROJ" push origin "$DEFAULT" >/dev/null 2>&1; then + echo "pushed $DEFAULT to origin in $PROJ" + else + echo "warning: local $DEFAULT merged, but syncing it to origin failed in $PROJ (best-effort; merge succeeded)" >&2 + fi +fi diff --git a/bin/fm-nas-deploy-sync.sh b/bin/fm-nas-deploy-sync.sh new file mode 100755 index 000000000..b04238a67 --- /dev/null +++ b/bin/fm-nas-deploy-sync.sh @@ -0,0 +1,338 @@ +#!/usr/bin/env bash +# Best-effort, non-fatal deploy sync for a single project's live NAS checkout: a +# merged PR lands in the project's GitHub default branch, but the live app on this +# host runs from its own separate checkout under /mnt/nas/experiments//, +# managed by pm2 - that checkout does not update itself (see the 2026-07-04 entry +# in data/learnings.md). This script closes that gap after a landed ship task. +# +# Looks the project up in data/nas-deployments.md, a pipe-table with one row per +# deployed project: "| | | |" (comma- +# separated when more than one process serves the app). A project absent from +# that table - or a fresh checkout with no data/nas-deployments.md at all - has no +# known live deployment, which is not an error: prints one skip line and exits 0. +# +# When present: fetches the NAS checkout's origin and fast-forward-only merges to +# origin/, mirroring bin/fm-fleet-sync.sh's dirty/diverged safety exactly +# - never force, never discard local changes, never touch a checkout that is off +# its default branch, dirty, or diverged (reported as STUCK, left untouched). +# Restarts the recorded pm2 process(es) only when the merge actually advanced +# HEAD, then verifies each is back online via `pm2 jlist`. Always prints exactly +# one clear result line and exits 0 - this script is best-effort and must never +# fail or block its caller (see the post-teardown call in bin/fm-teardown.sh). +# +# Usage: fm-nas-deploy-sync.sh +# +# Overrides (test injection): FM_DATA_OVERRIDE points at an alternate data/ dir +# (same knob bin/fm-project-mode.sh uses); FM_NAS_DEPLOYMENTS_OVERRIDE points at +# an alternate mapping file directly. pm2 is invoked via PATH, so a fakebin shim +# ahead of it on PATH intercepts restart/list during tests - real tests must never +# reach the real /mnt/nas/experiments or a real pm2 process. +# +# Every filesystem/git touch of $NAS_PATH requires a timeout/gtimeout binary and +# is wrapped in a bounded timeout (FM_NAS_SYNC_TIMEOUT seconds, default 15): the +# mount is a NAS, and an unreachable one is a hang risk (a stuck stat/read), not +# a fast failure, which would otherwise stall fm-teardown.sh's caller and break +# this script's non-blocking contract. +# +# $NAS_PATH is one single shared live-deployment checkout - unlike a project's +# projects/ dev clone, which is scoped to one firstmate home, two ship +# tasks for the same project landing moments apart can both reach the SAME +# NAS checkout from concurrent teardowns. That races the fetch on an orphaned +# .git/packed-refs.lock exactly like bin/fm-fleet-sync.sh's fetch had to guard +# against; fetch_with_packed_refs_lock_guard below mirrors that recovery, +# sharing the staleness proof from bin/fm-lock-lib.sh, bounded by +# FM_NAS_SYNC_PACKED_REFS_LOCK_RETRIES / _RETRY_WAIT_SECS / _AGE_SECS. +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" +DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" +MAP="${FM_NAS_DEPLOYMENTS_OVERRIDE:-$DATA/nas-deployments.md}" +NAS_SYNC_TIMEOUT="${FM_NAS_SYNC_TIMEOUT:-15}" +# shellcheck source=bin/fm-lock-lib.sh +. "$SCRIPT_DIR/fm-lock-lib.sh" +FM_LOCK_LOG_PREFIX=nas-deploy-sync + +NAS_SYNC_PACKED_REFS_LOCK_RETRIES=${FM_NAS_SYNC_PACKED_REFS_LOCK_RETRIES:-3} +NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS=${FM_NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS:-1} +NAS_SYNC_PACKED_REFS_LOCK_AGE_SECS=${FM_NAS_SYNC_PACKED_REFS_LOCK_AGE_SECS:-30} +case "$NAS_SYNC_PACKED_REFS_LOCK_RETRIES" in ''|*[!0-9]*) NAS_SYNC_PACKED_REFS_LOCK_RETRIES=3 ;; esac +case "$NAS_SYNC_PACKED_REFS_LOCK_AGE_SECS" in ''|*[!0-9]*) NAS_SYNC_PACKED_REFS_LOCK_AGE_SECS=30 ;; esac +if ! [[ "$NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS" =~ ^([0-9]+([.][0-9]*)?|[.][0-9]+)$ ]]; then + echo "nas-deploy-sync: invalid packed-refs lock retry wait '$NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS'; using 1s" >&2 + NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS=1 +fi + +HAVE_TIMEOUT=none +if command -v timeout >/dev/null 2>&1; then + HAVE_TIMEOUT=timeout +elif command -v gtimeout >/dev/null 2>&1; then + HAVE_TIMEOUT=gtimeout +fi + +# bounded : run , killed after $NAS_SYNC_TIMEOUT seconds. +bounded() { + case "$HAVE_TIMEOUT" in + timeout) timeout "$NAS_SYNC_TIMEOUT" "$@" ;; + gtimeout) gtimeout "$NAS_SYNC_TIMEOUT" "$@" ;; + *) return 124 ;; + esac +} + +# git_nas : git -C "$NAS_PATH" , bounded. +git_nas() { + bounded git -C "$NAS_PATH" "$@" +} + +pm2_bounded() { + bounded pm2 "$@" +} + +usage() { + echo "usage: fm-nas-deploy-sync.sh " >&2 +} + +if [ "${1:-}" = "--help" ] || [ "${1:-}" = "-h" ]; then + usage + exit 0 +fi +[ $# -eq 1 ] || { usage; exit 1; } +NAME=$1 + +first_line() { + printf '%s\n' "$1" | sed -n '1s/[[:space:]]\{1,\}/ /g;1p' +} + +# True when git stderr shows the packed-refs.lock "File exists" race, mirroring +# bin/fm-fleet-sync.sh's is_packed_refs_lock_error. +is_packed_refs_lock_error() { + printf '%s\n' "$1" | grep -Eq "Unable to create ['\"].*packed-refs\\.lock['\"]: File exists" +} + +# Absolute path to $NAS_PATH's packed-refs.lock, or empty when it cannot be +# resolved. +packed_refs_lock_path() { + local lock abs + lock=$(git_nas rev-parse --git-path packed-refs.lock 2>/dev/null) || return 1 + [ -n "$lock" ] || return 1 + case "$lock" in + /*) printf '%s\n' "$lock" ;; + *) + abs=$(cd "$NAS_PATH" && pwd -P) || return 1 + printf '%s/%s\n' "$abs" "$lock" + ;; + esac +} + +# Run `git_nas fetch origin --quiet`, tolerating an orphaned packed-refs.lock +# left by a second landed-teardown racing this exact NAS checkout. Sets +# FETCH_OUTPUT and returns the fetch's exit status; mirrors bin/fm-fleet-sync.sh's +# fetch_with_packed_refs_lock_guard retry-then-provably-stale-clear contract, +# sharing the staleness proof from bin/fm-lock-lib.sh. +fetch_with_packed_refs_lock_guard() { + local rc attempt=0 lock lock_desc + FETCH_OUTPUT=$(git_nas fetch origin --quiet 2>&1); rc=$? + [ "$rc" -eq 0 ] && return 0 + is_packed_refs_lock_error "$FETCH_OUTPUT" || return "$rc" + + lock=$(packed_refs_lock_path) || lock="" + lock_desc=${lock:-packed-refs.lock} + while [ "$attempt" -lt "$NAS_SYNC_PACKED_REFS_LOCK_RETRIES" ]; do + attempt=$(( attempt + 1 )) + echo "$NAME: fetch blocked by packed-refs lock ($lock_desc) at $NAS_PATH; waiting ${NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS}s and retrying ($attempt/${NAS_SYNC_PACKED_REFS_LOCK_RETRIES}) (owning process may be exiting)" >&2 + sleep "$NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS" + FETCH_OUTPUT=$(git_nas fetch origin --quiet 2>&1); rc=$? + if [ "$rc" -eq 0 ]; then + echo "$NAME: fetch succeeded on retry; packed-refs lock cleared on its own" >&2 + return 0 + fi + is_packed_refs_lock_error "$FETCH_OUTPUT" || return "$rc" + done + + # Retries exhausted and still the lock signature. Clear ONLY if provably stale. + lock=$(packed_refs_lock_path) || lock="" + if [ -n "$lock" ] && [ -e "$lock" ]; then + if fm_lock_is_provably_stale "$lock" "$NAS_PATH" "$NAS_SYNC_PACKED_REFS_LOCK_AGE_SECS"; then + if ! rm -f "$lock"; then + echo "$NAME: failed to remove provably-stale packed-refs lock $lock; leaving it in place" >&2 + return "$rc" + fi + echo "$NAME: removed provably-stale packed-refs lock $lock (age >= ${NAS_SYNC_PACKED_REFS_LOCK_AGE_SECS}s, no live holder) and retrying fetch" >&2 + FETCH_OUTPUT=$(git_nas fetch origin --quiet 2>&1); rc=$? + if [ "$rc" -eq 0 ]; then + echo "$NAME: fetch succeeded after stale packed-refs lock cleanup" >&2 + return 0 + fi + return "$rc" + fi + echo "$NAME: fetch blocked by packed-refs lock $lock that persisted across ${NAS_SYNC_PACKED_REFS_LOCK_RETRIES} retries and is not provably stale (may belong to a live process); leaving it in place" >&2 + return "$rc" + fi + echo "$NAME: fetch packed-refs lock signature persisted across ${NAS_SYNC_PACKED_REFS_LOCK_RETRIES} retries even after the lock file disappeared" >&2 + return "$rc" +} + +# lookup_row: print "|" for $NAME's row in $MAP, or +# nothing when the file is absent or has no matching row. +lookup_row() { + [ -f "$MAP" ] || return 0 + # $3 ~ /^\// excludes the header ("nas_repo_path") and separator ("---") rows - + # a real NAS path is always absolute - so a project literally named "project" + # cannot collide with the header's own column label. + awk -F'|' -v n="$NAME" ' + { for (i = 1; i <= NF; i++) { gsub(/^[ \t]+|[ \t]+$/, "", $i) } } + NF >= 4 && $2 == n && $3 ~ /^\// { print $3 "|" $4; exit } + ' "$MAP" +} + +row=$(lookup_row) +if [ -z "$row" ]; then + echo "$NAME: skipped: no recorded NAS deployment" + exit 0 +fi +NAS_PATH=${row%%|*} +PROCS=${row#*|} + +if [ "$HAVE_TIMEOUT" = none ]; then + echo "$NAME: skipped: no timeout/gtimeout binary available for bounded NAS access" + exit 0 +fi +if ! command -v pm2 >/dev/null 2>&1; then + echo "$NAME: skipped: pm2 is unavailable" + exit 0 +fi +if ! command -v jq >/dev/null 2>&1; then + echo "$NAME: skipped: jq is unavailable" + exit 0 +fi + +if ! bounded test -d "$NAS_PATH"; then + echo "$NAME: skipped: NAS path $NAS_PATH not a directory" + exit 0 +fi +if ! git_nas rev-parse --is-inside-work-tree >/dev/null 2>&1; then + echo "$NAME: skipped: $NAS_PATH not a git repo" + exit 0 +fi +if ! git_nas remote get-url origin >/dev/null 2>&1; then + echo "$NAME: skipped: no origin remote at $NAS_PATH" + exit 0 +fi + +if ! fetch_with_packed_refs_lock_guard; then + reason="fetch failed" + [ -z "$FETCH_OUTPUT" ] || reason="$reason: $(first_line "$FETCH_OUTPUT")" + echo "$NAME: skipped: $reason" + exit 0 +fi + +default_branch() { + local ref branch + ref=$(git_nas symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || true) + if [ -n "$ref" ]; then + echo "${ref#origin/}" + return 0 + fi + for branch in main master; do + if git_nas show-ref --verify --quiet "refs/heads/$branch"; then + echo "$branch" + return 0 + fi + done + return 1 +} + +DEFAULT=$(default_branch) || { + echo "$NAME: skipped: cannot determine default branch at $NAS_PATH" + exit 0 +} +BASE="origin/$DEFAULT" +if ! git_nas rev-parse --verify --quiet "$BASE^{commit}" >/dev/null; then + echo "$NAME: skipped: $BASE does not exist at $NAS_PATH" + exit 0 +fi + +cur=$(git_nas symbolic-ref --quiet --short HEAD 2>/dev/null || echo "") +dirty=no +status_rc=0 +status_out=$(git_nas status --porcelain 2>/dev/null) || status_rc=$? + +if [ "$cur" != "$DEFAULT" ]; then + echo "$NAME: STUCK: NAS checkout at $NAS_PATH is not on $DEFAULT - needs attention" + exit 0 +fi +if [ "$status_rc" -ne 0 ]; then + echo "$NAME: STUCK: cannot inspect worktree status at $NAS_PATH - needs attention" + exit 0 +fi +[ -z "$(printf '%s\n' "$status_out" | head -1)" ] || dirty=yes +if [ "$dirty" = yes ]; then + echo "$NAME: STUCK: NAS checkout at $NAS_PATH has uncommitted changes - needs attention" + exit 0 +fi + +local_rev="" +if ! local_rev=$(git_nas rev-parse --verify --quiet "$DEFAULT"); then + echo "$NAME: skipped: cannot read local $DEFAULT at $NAS_PATH" + exit 0 +fi +remote_rev="" +if ! remote_rev=$(git_nas rev-parse --verify --quiet "$BASE"); then + echo "$NAME: skipped: cannot read $BASE at $NAS_PATH" + exit 0 +fi +if [ "$local_rev" = "$remote_rev" ]; then + echo "$NAME: already current" + exit 0 +fi +if ! git_nas merge-base --is-ancestor "$DEFAULT" "$BASE"; then + behind=$(git_nas rev-list --count "HEAD..$BASE" 2>/dev/null) || behind="?" + echo "$NAME: STUCK: NAS checkout at $NAS_PATH diverged from $BASE, $behind commits behind - needs attention" + exit 0 +fi + +before=$(git_nas rev-parse --short "$DEFAULT") +merge_output="" +if ! merge_output=$(git_nas merge --ff-only "$BASE" 2>&1); then + reason="fast-forward failed" + [ -z "$merge_output" ] || reason="$reason: $(first_line "$merge_output")" + echo "$NAME: skipped: $reason" + exit 0 +fi +after=$(git_nas rev-parse --short "$DEFAULT") + +# process_online : true when pm2 currently reports EVERY instance of +# as online (a clustered app can register several jlist entries under +# the same name; one lagging instance must not be masked by the others). +process_online() { + local proc=$1 status + status=$(pm2_bounded jlist 2>/dev/null | jq -r --arg n "$proc" \ + '[.[] | select(.name == $n) | .pm2_env.status] as $s + | if ($s | length) == 0 then "missing" + elif all($s[]; . == "online") then "online" + else "not-online" + end' 2>/dev/null) || status="missing" + [ "$status" = "online" ] +} + +restart_problems="" +IFS=',' read -r -a procs <<< "$PROCS" +for proc in "${procs[@]}"; do + proc="${proc#"${proc%%[![:space:]]*}"}" + proc="${proc%"${proc##*[![:space:]]}"}" + [ -n "$proc" ] || continue + if ! pm2_bounded restart "$proc" >/dev/null 2>&1; then + restart_problems="$restart_problems $proc(restart failed)" + continue + fi + if ! process_online "$proc"; then + restart_problems="$restart_problems $proc(not online after restart)" + fi +done + +if [ -n "$restart_problems" ]; then + echo "$NAME: synced $before..$after but restart verification failed:$restart_problems - needs attention" + exit 0 +fi +echo "$NAME: synced $before..$after, restarted $PROCS and verified online" diff --git a/bin/fm-risk-tripwire.sh b/bin/fm-risk-tripwire.sh new file mode 100755 index 000000000..fceb0cd37 --- /dev/null +++ b/bin/fm-risk-tripwire.sh @@ -0,0 +1,304 @@ +#!/usr/bin/env bash +# Mechanical risk trip-wire (guardrail #2): greps a task's brief text and, once +# code exists, its changed file paths for migration/auth/schema/security +# signals - a second, structurally different check from the natural-language +# "when" match a crew-dispatch rule already made, so a misclassified risky task +# cannot slip to a cheap model/effort tier on a single judgment call +# (data/research-resource-tiering-synthesis.md). +# +# Usage: fm-risk-tripwire.sh +# +# Checks whatever is available for : +# - data//brief.md, if it exists (works before spawn, right after +# bin/fm-brief.sh scaffolds it - the first checkpoint) +# - the changed file paths between its project's default branch and either +# the worktree's local HEAD, or the open PR's head when state/.meta +# records pr=/pr_head= (mirroring bin/fm-review-diff.sh, so a no-mistakes fix +# round already pushed to the PR is included even when the local worktree +# branch is stale) - works after spawn, at Validate time - the second, +# binding checkpoint against the real diff +# Exit codes: +# 0 no risk signal found +# 1 a RISK hit fired (one "RISK: " line per hit is printed) +# 2 could not check: a malformed invocation, neither a brief nor a usable +# worktree/project exists yet (nothing to check), or the worktree/project +# exist but their diff base could not be resolved, so the binding diff +# checkpoint could not run (warned to stderr, never a silent clean pass) +# Distinct codes matter so a caller branching on $? cannot mistake a malformed +# invocation for a real risk hit. A hit means: floor this task's model/effort to +# the safety-critical profile (opus/xhigh, ultracode independent-review) +# regardless of which rule the natural-language dispatch match picked, per +# AGENTS.md section 4's risk floor. +# +# This independently resolves the same PR-head base bin/fm-review-diff.sh uses +# (see resolve_pr_head below) because that script only exposes --stat, not a +# --name-only path list this scan needs - good enough for a keyword scan, not a +# substitute for that script's authoritative diff. +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" +STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" +DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" + +# shellcheck source=bin/fm-tangle-lib.sh +. "$SCRIPT_DIR/fm-tangle-lib.sh" + +usage() { + echo "usage: fm-risk-tripwire.sh " >&2 +} + +if [ "${1:-}" = "-h" ] || [ "${1:-}" = "--help" ]; then + usage + exit 0 +fi + +ID=${1:-} +[ -n "$ID" ] || { usage; exit 2; } +[ $# -le 1 ] || { usage; exit 2; } + +# Case-insensitive, word-bounded match against the brief's prose, tolerating the +# inflections and affixes real task phrasing uses. Word bounding stops substring +# false positives like "auth" inside "authoritative", but three things must still +# reach the scan or a genuinely risky task slips to a cheap tier (the dangerous +# direction for a safety floor): +# - verb forms need their own stems: appending the outer (s|ed|ing|ion|...) +# suffix to a bare NOUN literal cannot reach the verb, so "migration" alone +# could only become "migrations", never "migrate"/"migrating"/"migrated", +# and the bare "auth" branch never reaches "authorize"/"authenticate". +# - the un/re/de/pre prefixes ("unauthorized"/"unauthenticated" - the most +# common access-control phrasing) glue a letter onto the stem's front, so +# the auth stems carry an explicit optional prefix. +# - a bare "auth"-anything stem is still avoided, so "authoritative"/"author" +# stay shut. +# Word bounding is done by splitting the body into whole tokens (below), NOT by +# grep's \< / \> anchors: those are a GNU extension BSD grep does not honor, so +# on macOS they would silently match nothing and let every risky brief through - +# a false negative in the dangerous direction. Whole-token matching is identical +# on GNU and BSD grep and, unlike a boundary-consuming grep -o pattern, still +# reports BOTH words of an adjacent risk pair like "session token" (a consumed +# shared delimiter would drop the second). Single-word keywords go in WORD_REGEX; +# the multi-word phrases have no lone token and are matched on the +# space-normalized stream via PHRASE_REGEX. +WORD_KEYWORDS='auth|authentication|authorization|(un|re|de|pre)?authoriz(e|ed|es|ing|ation|ations|er|ers)?|(un|re|de|pre)?authenticat(e|ed|es|ing|ion|ions|or|ors)?|session|credential|password|secret|token|payment|billing|migrat(e|ed|es|ing|ion|ions)|schema|security|encrypt|decrypt|permission' +WORD_REGEX="(${WORD_KEYWORDS})(s|es|ed|ing|ion|ions)?" +PHRASE_REGEX='(access[[:space:]]+control|data[[:space:]]+deletion|bulk[[:space:]]+mutation|public[[:space:]]+exposure|breaking[[:space:]]+change)(s|es|ed|ing|ion|ions)?' + +# Scan only the task-specific body of the brief (the # Task section up to the +# next scaffold section heading), never the fixed scaffold boilerplate +# bin/fm-brief.sh writes into every brief - that boilerplate contains benign +# words like "authoritative" and "future session" that would otherwise trip the +# wire on every task. The Herdr block bin/fm-brief.sh injects immediately after +# # Task (either the # Herdr isolation ... hard-safety contract, whose text is +# dense with "session"/"--session", or the # Herdr lifecycle declaration ... +# not-enabled stub) is scaffold boilerplate too, so its heading is a boundary as +# well; without it every --herdr-lab brief would trip on the contract's own +# "session" wording rather than the real task text. The section boundary is one +# of the scaffold's own known headings (# Herdr isolation .../# Herdr lifecycle +# declaration .../# Setup/# Rules/# Project memory/# Definition of done), not any +# column-0 "# " line: a shell/code comment embedded in the task body (e.g. +# "# then run the schema migration") must NOT end the scan, or the risk words +# after it are silently dropped - the dangerous direction for a safety floor. +# A boundary heading must also be blank-line-preceded, as every real scaffold +# heading is (bin/fm-brief.sh emits a blank line before each), so a task body +# that itself quotes a bare "# Setup" line inline does not cut the scan short. +# Lines inside a fenced code block are likewise never treated as a boundary. +# Falls back to the whole brief when there is no # Task section (a non-standard +# or hand-written brief), keeping the permissive safety bias. +brief_task_body() { + local body + body=$(awk ' + { blank = ($0 ~ /^[[:space:]]*$/) } + /^```/ { fence = !fence; prevblank = blank; next } + !fence && /^# Task[[:space:]]*$/ { intask=1; prevblank = blank; next } + intask && !fence && prevblank && /^# (Herdr isolation.*|Herdr lifecycle declaration.*|Setup|Rules|Project memory|Definition of done)[[:space:]]*$/ { intask=0 } + intask { print } + { prevblank = blank } + ' "$1") + if [ -n "$body" ]; then + printf '%s\n' "$body" + else + cat "$1" + fi +} + +# Component/token match against one changed file's path (already lowercased by +# the caller). Deliberately NOT a blanket bin/* match: firstmate's own +# supervision backbone lives under bin/ and is routed by an explicit dispatch +# rule (see docs/examples/crew-dispatch.json), so flooring every bin/ change +# here would override that rule and defeat the finer tiers. +# +# The same word-boundary discipline as the brief-text scan applies here, so a +# risk word must be a real path component or delimited token, not a bare +# substring: "strong" words (auth/migrat/schema/... families) match anywhere +# they appear as a /, -, _, or . delimited token, catching bin/auth-setup.sh, +# bin/run-migration.sh, and db/schema.sql; the weaker "session" matches only as +# a whole path component or filename base (lib/auth/session.rb, +# app/models/session.rb), never as a hyphen fragment - otherwise the +# supervision backbone bin/fm-session-start.sh would over-match. Bare-substring +# false positives like "auth" inside AUTHORS/docs/authors.md no longer trip. +# The flip side is intentional: a risk word glued into a compound component with +# NO delimiter (e.g. "authsetup.rb") is not matched, because catching it would +# require substring matching again and reopen exactly those false positives; the +# brief-text scan and the delimiter tokenization below cover the realistic cases. +PATH_STRONG_REGEX='^(auth|authn|authz|authoriz(e|ed|es|ing|ation|ations|er|ers)?|authenticat(e|ed|es|ing|ion|ions|or|ors)?|migrat(e|ed|es|ing|ion|ions)|schema|schemas|secret|secrets|credential|credentials|payment|payments|billing|security|password|passwords|token|tokens|permission|permissions|encrypt(s|ed|ing|ion|ions)?|decrypt(s|ed|ing|ion|ions)?)$' +PATH_WEAK_REGEX='^(session|sessions)$' +path_is_risky() { + local path=$1 comp base + case "$path" in + .github/workflows/*|*/.github/workflows/*) return 0 ;; + *dockerfile*|*docker-compose*) return 0 ;; + esac + while IFS= read -r comp; do + [ -n "$comp" ] || continue + base=${comp%.*} + # Whole path component or filename base equal to a strong or weak word. + if printf '%s\n%s\n' "$comp" "$base" | grep -Eiq "$PATH_STRONG_REGEX|$PATH_WEAK_REGEX"; then + return 0 + fi + # Strong words also match as a ., -, or _ delimited token inside a compound + # name (e.g. config.schema.json, run-migration.sh), the fail-safe direction. + if printf '%s\n' "$base" | tr '._-' '\n' | grep -Eiq "$PATH_STRONG_REGEX"; then + return 0 + fi + done < <(printf '%s\n' "$path" | tr '/' '\n') + return 1 +} + +pr_number_from_target() { + local target=$1 n + case "$target" in + '') return 1 ;; + *"/pull/"*) + n=${target##*/pull/} + n=${n%%[!0-9]*} + ;; + [0-9]*) + n=${target%%[!0-9]*} + ;; + *) return 1 ;; + esac + [ -n "$n" ] || return 1 + printf '%s' "$n" +} + +resolve_pr_head() { + local pr_url=$1 recorded_head=$2 n resolved + if [ -n "$recorded_head" ] \ + && git -C "$WT" cat-file -e "$recorded_head^{commit}" 2>/dev/null; then + printf '%s' "$recorded_head" + return 0 + fi + n=$(pr_number_from_target "$pr_url") || return 1 + git -C "$WT" remote get-url origin >/dev/null 2>&1 || return 1 + git -C "$WT" fetch --quiet origin "refs/pull/$n/head" >/dev/null 2>&1 || return 1 + resolved=$(git -C "$WT" rev-parse --verify 'FETCH_HEAD^{commit}' 2>/dev/null) || return 1 + [ -n "$resolved" ] || return 1 + printf '%s' "$resolved" +} + +FOUND=0 +CHECKED=0 +DIFF_UNRESOLVED=0 + +BRIEF="$DATA/$ID/brief.md" +if [ -f "$BRIEF" ]; then + CHECKED=1 + lc=$(brief_task_body "$BRIEF" | tr '[:upper:]' '[:lower:]') + # Split on every non-alnum char so each word is its own token (this also + # splits snake_case identifiers like run_schema_migration), then match whole + # tokens; phrases keep their inter-word gap on the space-normalized stream. + word_hits=$(printf '%s\n' "$lc" | tr -c '[:alnum:]' '\n' | grep -xE "$WORD_REGEX" || true) + phrase_hits=$(printf '%s\n' "$lc" | tr -c '[:alnum:]' ' ' | tr -s ' ' | grep -oE "$PHRASE_REGEX" || true) + hit=$(printf '%s\n%s\n' "$word_hits" "$phrase_hits" | sed '/^$/d' | sort -u | tr '\n' ',' | sed 's/,$//') + if [ -n "$hit" ]; then + echo "RISK: brief for $ID mentions risk-adjacent term(s): $hit" + FOUND=1 + fi +fi + +META="$STATE/$ID.meta" +if [ -f "$META" ]; then + WT=$(grep '^worktree=' "$META" | tail -1 | cut -d= -f2- || true) + PROJ=$(grep '^project=' "$META" | tail -1 | cut -d= -f2- || true) + if [ -n "$WT" ] && [ -n "$PROJ" ] && [ -d "$WT" ] && [ -d "$PROJ" ]; then + CHECKED=1 + diff_base_resolved=0 + compare_ref=HEAD + PR_URL=$(grep '^pr=' "$META" | tail -1 | cut -d= -f2- || true) + PR_HEAD_RECORDED=$(grep '^pr_head=' "$META" | tail -1 | cut -d= -f2- || true) + if [ -n "$PR_URL$PR_HEAD_RECORDED" ]; then + if PR_HEAD=$(resolve_pr_head "$PR_URL" "$PR_HEAD_RECORDED"); then + compare_ref=$PR_HEAD + elif [ -n "$PR_URL" ]; then + echo "warning: PR head unavailable for $ID; risk path scan may lag the open PR (using local HEAD)" >&2 + fi + fi + # The base resolution below duplicates bin/fm-review-diff.sh's origin fetch + # and origin/ verification on purpose: this scan needs the changed + # PATH LIST, but fm-review-diff.sh only exposes --stat (a diffstat), not a + # --name-only path list, so there is nothing to reuse for a name scan. + # Adding a --name-only mode to that script is the clean fix and is left as a + # follow-up rather than expanding this task's scope into another owner. + DEFAULT=$(fm_default_branch "$PROJ" 2>/dev/null || true) + if [ -n "$DEFAULT" ]; then + BASE="$DEFAULT" + if git -C "$PROJ" remote get-url origin >/dev/null 2>&1; then + git -C "$WT" fetch origin "+refs/heads/$DEFAULT:refs/remotes/origin/$DEFAULT" --quiet 2>/dev/null || true + git -C "$WT" rev-parse --verify --quiet "refs/remotes/origin/$DEFAULT^{commit}" >/dev/null 2>&1 && BASE="origin/$DEFAULT" + fi + if git -C "$WT" rev-parse --verify --quiet "$BASE^{commit}" >/dev/null 2>&1; then + diff_base_resolved=1 + diff_paths= + diff_rc=0 + diff_paths=$(git -C "$WT" diff --name-only "$BASE...$compare_ref" -- 2>/dev/null) || diff_rc=$? + if [ "$diff_rc" -ne 0 ]; then + echo "warning: could not read changed paths for $ID (project $PROJ, base '$BASE', compare '$compare_ref'); the diff checkpoint did not run - reporting not-checkable (exit 2), not a clean pass" >&2 + DIFF_UNRESOLVED=1 + else + risky_paths= + if [ -n "$diff_paths" ]; then + while IFS= read -r path; do + [ -n "$path" ] || continue + lower_path=$(printf '%s' "$path" | tr '[:upper:]' '[:lower:]') + if path_is_risky "$lower_path"; then + risky_paths="${risky_paths}${risky_paths:+, }$path" + fi + done <<< "$diff_paths" + fi + if [ -n "$risky_paths" ]; then + echo "RISK: diff for $ID touches risk-adjacent path(s): $risky_paths" + FOUND=1 + fi + fi + fi + fi + # The worktree/project exist, so the binding diff checkpoint was expected to + # run - but the diff base could not be resolved (no default branch, or the + # base commit does not verify in the worktree). Do not let that read as a + # clean pass: warn loudly, like the sibling bin/fm-review-diff.sh, and mark + # the run not-checkable so a caller branching on $? floors the task rather + # than trusting a silent 0 - the safe direction for this guardrail. + if [ "$diff_base_resolved" -eq 0 ]; then + echo "warning: could not resolve a diff base for $ID (project $PROJ, default branch '${DEFAULT:-unresolved}'); the diff checkpoint did not run - reporting not-checkable (exit 2), not a clean pass" >&2 + DIFF_UNRESOLVED=1 + fi + else + echo "warning: unusable worktree/project for $ID (worktree '${WT:-unresolved}', project '${PROJ:-unresolved}'); the diff checkpoint did not run - reporting not-checkable (exit 2), not a clean pass" >&2 + DIFF_UNRESOLVED=1 + fi +fi + +if [ "$CHECKED" -eq 0 ] && [ "$DIFF_UNRESOLVED" -eq 0 ]; then + echo "error: neither $BRIEF nor a usable worktree/project in $META was found for $ID; nothing to check" >&2 + exit 2 +fi + +# A real risk hit (exit 1) always wins over an unresolvable diff base; only when +# nothing tripped does an unresolvable base downgrade the run to could-not-check. +if [ "$FOUND" -eq 0 ] && [ "$DIFF_UNRESOLVED" -eq 1 ]; then + exit 2 +fi + +exit "$FOUND" diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index a4ec0890c..1be7d491a 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -79,6 +79,19 @@ # On success prints: spawned harness= kind= mode= yolo= window= worktree= # mode/yolo are resolved per-project from data/projects.md for ship/scout tasks; # secondmate spawns record mode=secondmate, yolo=off, home=, and projects=. +# Ship/scout spawns also derive a Docker-Compose-safe project name from the +# worktree's own pool-slot path, write it to /.treehouse-compose-project +# (excluded from git via .git/info/exclude, like the per-harness turn-end hook +# files below), and record it in meta as compose_project=, so two +# worktrees of the same project never collide on Compose's default +# project-naming (fm-brief.sh documents how a crewmate consumes it). +# claude launches (crewmate, scout, and secondmate) carry +# CLAUDE_CODE_AUTO_COMPACT_WINDOW=300000 so they auto-compact the way the primary +# interactive session already does via its own .claude/settings.local.json, which a +# separately-launched claude process never inherits; see the comment on the claude +# case in launch_template() below for why 300000 (not the primary's +# WINDOW=1000000 + PCT_OVERRIDE=20 pair) is safe to hardcode: it is capped at the +# real context window, so it is a no-op below that window and takes effect above it. set -eu SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -190,6 +203,8 @@ fi ORCA_ABORT_CLEANUP=0 ORCA_WORKTREE_ID= ORCA_TERMINAL= +SPAWN_TASK_TMP_CLEANUP=0 +TASK_TMP= parse_orca_worktree_result() { local raw=$1 rest @@ -209,7 +224,7 @@ parse_orca_worktree_result() { } orca_spawn_abort_cleanup() { - local status=$? + local status=${1:-$?} [ "$ORCA_ABORT_CLEANUP" = 1 ] || return "$status" ORCA_ABORT_CLEANUP=0 if [ -n "${ORCA_TERMINAL:-}" ]; then @@ -219,7 +234,7 @@ orca_spawn_abort_cleanup() { if ! fm_backend_remove_worktree orca "$ORCA_WORKTREE_ID" 2>/dev/null; then mkdir -p "$STATE" 2>/dev/null || true if [ -d "$STATE" ]; then - { + if { echo "window=$W" echo "worktree=${WT:-}" echo "project=$PROJ_ABS" @@ -233,13 +248,31 @@ orca_spawn_abort_cleanup() { echo "backend=orca" echo "orca_worktree_id=$ORCA_WORKTREE_ID" [ -z "${ORCA_TERMINAL:-}" ] || echo "terminal=$ORCA_TERMINAL" - } > "$STATE/$ID.meta" 2>/dev/null || true + } > "$STATE/$ID.meta" 2>/dev/null; then + SPAWN_TASK_TMP_CLEANUP=0 + fi fi fi fi return "$status" } -trap orca_spawn_abort_cleanup EXIT + +cleanup_task_tmp_on_spawn_error() { + local status=${1:-$?} + if [ "$SPAWN_TASK_TMP_CLEANUP" = 1 ] && [ -n "${TASK_TMP:-}" ]; then + rm -rf -- "$TASK_TMP" + SPAWN_TASK_TMP_CLEANUP=0 + fi + return "$status" +} + +spawn_exit_cleanup() { + local status=$? + orca_spawn_abort_cleanup "$status" + status=$? + cleanup_task_tmp_on_spawn_error "$status" +} +trap spawn_exit_cleanup EXIT # Batch dispatch (see header): when the first positional is an `id=repo` pair, treat every # positional as one and spawn each by re-execing this script in single-task mode. We use @@ -321,7 +354,25 @@ launch_template() { # does NOT suppress the interactive ghost text (verified empirically), so the env # var is the correct control. The dim-aware composer reader in fm-tmux-lib.sh is # the defense-in-depth backstop for any pane this flag cannot reach. - claude) printf '%s' 'CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false claude --dangerously-skip-permissions __MODELFLAG____EFFORTFLAG__"$(cat __BRIEF__)"' ;; + # + # CLAUDE_CODE_AUTO_COMPACT_WINDOW=300000 gives claude crewmates and secondmates + # a higher auto-compaction ceiling than the plain-window default, the same way + # the primary interactive session already has via its own + # .claude/settings.local.json, which these separately-launched processes never + # inherit. The var is read once at startup and is CAPPED at the model's actual + # context window, so it can only lower the effective trigger, never raise it past + # the real ceiling: min(300000, window) is 300000 once window exceeds it, but a + # no-op (stays at window) below that. This is not a universal guarantee for every + # possible future model choice - a differently-configured account/model capped at + # the plain ~200k window would see no change - but it is true for the window tier + # crew actually runs on today: as verified for --model sonnet on this account + # (the standard crewmate spawn), the window resolves to the ~1M-context beta, + # well above 300000. + # Do not copy the primary's two-var form (WINDOW=1000000 + PCT_OVERRIDE=20): + # that pair assumes a 1M window and is not safe for a harness/account + # combination with a smaller one. See the harness-adapters skill's claude + # section for the dated /context verification. + claude) printf '%s' 'CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false CLAUDE_CODE_AUTO_COMPACT_WINDOW=300000 claude --dangerously-skip-permissions __MODELFLAG____EFFORTFLAG__"$(cat __BRIEF__)"' ;; codex) if [ "$kind" = secondmate ]; then printf '%s' 'codex __MODELFLAG____EFFORTFLAG__--dangerously-bypass-approvals-and-sandbox "$(cat __BRIEF__)"' @@ -428,6 +479,50 @@ shell_quote() { printf "'" } +# Appends a path (relative to a worktree root) to that worktree's own +# .git/info/exclude, so a firstmate-owned pointer/marker file never blocks +# teardown's dirty-worktree check or leaks into a crewmate's commit. Shared by +# every per-harness turn-end hook file below and the docker-compose isolation +# marker. Operates on $WT, which callers must have already resolved. +exclude_path() { + local rel=$1 EXCL + EXCL=$(git -C "$WT" rev-parse --git-path info/exclude 2>/dev/null || true) + [ -n "$EXCL" ] || return 0 + mkdir -p "$(dirname "$EXCL")" + grep -qxF "$rel" "$EXCL" 2>/dev/null || echo "$rel" >> "$EXCL" +} + +# Derives a unique, Docker-Compose-safe project name from a worktree's own +# pool-slot path (AGENTS.md "Layout and state"; a treehouse pool worktree is +# .../-//), so two worktrees of the same +# project - which always share the same leaf directory basename, e.g. +# "wealthsync" - never collide on Compose's default project-naming (derived +# from cwd basename) for container/volume/network names. Stable across +# respawns into the same pool slot, so a stale stack left behind by a prior +# task in that slot is superseded rather than orphaned under a new name; +# distinct across the fleet because no two live tasks occupy the same slot at +# once. Falls back to the worktree's own leaf name, then the task id, when the +# path does not have two parent directories (e.g. a non-treehouse backend). +fm_compose_project_name() { # + local wt=$1 fallback=$2 slot pool raw name + slot=$(basename "$(dirname "$wt")" 2>/dev/null || true) + pool=$(basename "$(dirname "$(dirname "$wt")")" 2>/dev/null || true) + if [ -n "$slot" ] && [ "$slot" != / ] && [ -n "$pool" ] && [ "$pool" != / ]; then + raw="$pool-$slot" + else + raw=$(basename "$wt") + fi + name=$(printf '%s' "$raw" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9-]/-/g' | tr -s '-' | sed 's/^-//; s/-$//') + if [ -z "$name" ]; then + name=$(printf '%s' "wt-$fallback" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9-]/-/g' | tr -s '-' | sed 's/^-//; s/-$//') + fi + case "$name" in + [a-z0-9]*) : ;; + *) name="wt-$name" ;; + esac + printf '%s\n' "$name" +} + model_flag_for_harness() { local harness=$1 model=$2 [ -n "$model" ] && [ "$model" != default ] || return 0 @@ -861,12 +956,24 @@ if [ "$KIND" != secondmate ] && [ "$BACKEND" != orca ]; then validate_spawn_worktree "treehouse get" "$T" fi -# Per-task temp root: /tmp/fm-/ with Go's build temp nested at gotmp/. Go won't -# create GOTMPDIR, so mkdir before it is used; fm-teardown removes the whole root. -# Nested (not a bare /tmp/fm-/gotmp) so other per-task temp can live alongside -# later, and teardown cleans one deterministic path. GOTMPDIR (not TMPDIR) is the -# targeted knob: TMPDIR is too broad (affects every program's temp, not just Go's). -TASK_TMP="/tmp/fm-$ID" +# Docker Compose isolation marker (ship/scout only; a secondmate operates in +# its own firstmate home, not a per-task pool worktree). Two treehouse +# worktrees of the same project share a leaf directory basename, so Compose's +# default project-naming collides across them; drop a stable, worktree-scoped +# name here that the crewmate's brief (fm-brief.sh) tells it to pass on every +# docker/docker-compose invocation. +if [ "$KIND" != secondmate ]; then + COMPOSE_PROJECT=$(fm_compose_project_name "$WT" "$ID") + printf '%s\n' "$COMPOSE_PROJECT" > "$WT/.treehouse-compose-project" + exclude_path '.treehouse-compose-project' +fi + +# Per-task temp root: /tmp/fm-./ with Go's build temp nested at gotmp/. +# Go won't create GOTMPDIR, so mkdir before it is used; fm-teardown removes the +# whole root. GOTMPDIR (not TMPDIR) is the targeted knob: TMPDIR is too broad +# (affects every program's temp, not just Go's). +TASK_TMP=$(mktemp -d "/tmp/fm-$ID.XXXXXX") || { echo "error: failed to create task temp root" >&2; exit 1; } +SPAWN_TASK_TMP_CLEANUP=1 mkdir -p "$TASK_TMP/gotmp" # Per-harness turn-end hook: a file that touches state/.turn-ended when the @@ -875,13 +982,6 @@ mkdir -p "$TASK_TMP/gotmp" mkdir -p "$STATE" STATE_REAL=$(cd "$STATE" && pwd -P) TURNEND="$STATE_REAL/$ID.turn-ended" -exclude_path() { - local rel=$1 EXCL - EXCL=$(git -C "$WT" rev-parse --git-path info/exclude 2>/dev/null || true) - [ -n "$EXCL" ] || return 0 - mkdir -p "$(dirname "$EXCL")" - grep -qxF "$rel" "$EXCL" 2>/dev/null || echo "$rel" >> "$EXCL" -} if [ "$KIND" != secondmate ]; then case "$HARNESS" in claude*) @@ -1001,6 +1101,7 @@ META_WINDOW=$T echo "tasktmp=$TASK_TMP" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" + [ "$KIND" = secondmate ] || echo "compose_project=$COMPOSE_PROJECT" # backend= is written only for a non-default (non-tmux) backend, so the # default path's meta stays byte-identical (absent backend= means tmux; # data/fm-backend-design-d7's P1 compatibility contract). @@ -1030,6 +1131,7 @@ META_WINDOW=$T fi } > "$STATE/$ID.meta" [ "$BACKEND" = orca ] && ORCA_ABORT_CLEANUP=0 +SPAWN_TASK_TMP_CLEANUP=0 sq_brief=$(shell_quote "$BRIEF") sq_turnend=$(shell_quote "$TURNEND") diff --git a/bin/fm-teardown.sh b/bin/fm-teardown.sh index 2ee4f35d7..ebeb980e5 100755 --- a/bin/fm-teardown.sh +++ b/bin/fm-teardown.sh @@ -112,7 +112,7 @@ fi HOME_PATH=$(grep '^home=' "$META" | cut -d= -f2- || true) PR_URL=$(grep '^pr=' "$META" | tail -1 | cut -d= -f2- || true) # tasktmp is recorded by fm-spawn for tasks that set up a per-task temp root -# (/tmp/fm-/); absent for tasks spawned before that change, so tolerate empty. +# (/tmp/fm-./); absent for tasks spawned before that change, so tolerate empty. TASK_TMP=$(grep '^tasktmp=' "$META" | cut -d= -f2- || true) ORCA_WORKTREE_ID=$(fm_meta_get "$META" orca_worktree_id) ORCA_PATH_MATCH_VERIFIED=0 @@ -121,6 +121,38 @@ KIND=$(grep '^kind=' "$META" | cut -d= -f2- || true) [ -n "$KIND" ] || KIND=ship MODE=$(grep '^mode=' "$META" | cut -d= -f2- || true) [ -n "$MODE" ] || MODE=no-mistakes +NAS_DEPLOY_SYNC_ELIGIBLE=0 + +task_tmp_is_safe() { + local path=$1 base prefix suffix + [ -n "$path" ] || return 1 + case "$path" in + /tmp/*) base=${path#/tmp/} ;; + *) return 1 ;; + esac + case "$base" in + */*|'') return 1 ;; + esac + [ "$base" = "fm-$ID" ] && return 0 + prefix="fm-$ID." + case "$base" in + "$prefix"*) suffix=${base#"$prefix"} ;; + *) return 1 ;; + esac + [ -n "$suffix" ] || return 1 + case "$suffix" in + *[!abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._-]*) return 1 ;; + esac +} + +remove_task_tmp() { + [ -n "$TASK_TMP" ] || return 0 + if ! task_tmp_is_safe "$TASK_TMP"; then + echo "teardown: skipped unsafe tasktmp path $TASK_TMP" >&2 + return 0 + fi + rm -rf -- "$TASK_TMP" +} default_branch() { local ref branch @@ -275,29 +307,51 @@ pr_is_merged() { unpushed_patches_are_in_pr_head "$head" } -# Is the branch's content already present in the up-to-date default branch? Fetches -# first, then 3-way merges the default branch with HEAD: when HEAD introduces nothing -# the default branch does not already contain (e.g. its change landed via squash) the -# merged tree equals the default branch's tree. This isolates branch-only changes, so -# unrelated commits the default branch gained past the merge-base do not count as -# "added". Returns non-zero when inconclusive (no default ref, or a merge conflict), -# so the caller refuses rather than guesses. +# Does the given ref's tree already contain everything HEAD introduces? 3-way merges +# the ref (ours) with HEAD (theirs) off their merge base into a throwaway index: when +# HEAD introduces nothing the ref does not already contain (e.g. its change landed via +# squash), the merged tree equals the ref's tree. read-tree + write-tree is used +# instead of `git merge-tree --write-tree` (git >= 2.38 only) so this stays portable to +# older git; a real merge conflict leaves unmerged index entries, write-tree fails, and +# the check stays inconclusive so the caller refuses rather than guesses. +content_matches_ref() { + local ref=$1 default_tree base merged_tree tmpdir rc + default_tree=$(git -C "$WT" rev-parse --quiet --verify "$ref^{tree}" 2>/dev/null) || return 1 + [ -n "$default_tree" ] || return 1 + base=$(git -C "$WT" merge-base "$ref" HEAD 2>/dev/null) || return 1 + tmpdir=$(mktemp -d 2>/dev/null) || return 1 + GIT_INDEX_FILE="$tmpdir/index" git -C "$WT" read-tree -im --aggressive \ + "$base^{tree}" "$ref^{tree}" "HEAD^{tree}" 2>/dev/null \ + && merged_tree=$(GIT_INDEX_FILE="$tmpdir/index" git -C "$WT" write-tree 2>/dev/null) + rc=$? + rm -rf "$tmpdir" + [ "$rc" -eq 0 ] || return 1 + [ -n "$merged_tree" ] || return 1 + [ "$merged_tree" = "$default_tree" ] +} + +# Is the branch's content already present in the up-to-date default branch? Checks +# two candidate refs, succeeding if either already contains HEAD's content: the +# fetched remote-tracking ref refs/remotes/origin/$name (the normal PR-merged case), +# and the LOCAL refs/heads/$name (a worktree and its project's primary checkout are +# linked worktrees of the same repository, so this local ref already reflects a +# firstmate-performed local fast-forward merge, even for a project whose delivery +# mode is not local-only - e.g. a PR closed unmerged upstream but the branch was +# landed into local main by hand on the captain's approval). A fetch failure only +# rules out the remote-tracking path; the local ref is still tried. Returns +# non-zero only when neither candidate ref exists or matches. content_in_default() { - local name ref default_tree merged_tree + local name name=$(default_branch) || return 1 if git -C "$WT" remote get-url origin >/dev/null 2>&1; then - git -C "$WT" fetch --quiet origin "+refs/heads/$name:refs/remotes/origin/$name" >/dev/null 2>&1 || return 1 - ref="refs/remotes/origin/$name" - elif git -C "$WT" rev-parse --quiet --verify "refs/heads/$name" >/dev/null 2>&1; then - ref="refs/heads/$name" - else - return 1 + if git -C "$WT" fetch --quiet origin "+refs/heads/$name:refs/remotes/origin/$name" >/dev/null 2>&1; then + content_matches_ref "refs/remotes/origin/$name" && return 0 + fi fi - default_tree=$(git -C "$WT" rev-parse --quiet --verify "$ref^{tree}" 2>/dev/null) || return 1 - [ -n "$default_tree" ] || return 1 - merged_tree=$(git -C "$WT" merge-tree --write-tree "$ref" HEAD 2>/dev/null) || return 1 - merged_tree=$(printf '%s\n' "$merged_tree" | head -1) - [ "$merged_tree" = "$default_tree" ] + if git -C "$WT" rev-parse --quiet --verify "refs/heads/$name" >/dev/null 2>&1; then + content_matches_ref "refs/heads/$name" && return 0 + fi + return 1 } # Has the worktree's committed work actually LANDED, though its commits are not @@ -311,6 +365,16 @@ work_is_landed() { content_in_default } +mark_nas_deploy_sync_eligible() { + NAS_DEPLOY_SYNC_ELIGIBLE=0 + [ "$FORCE" != "--force" ] || return 0 + case "$KIND" in + scout|secondmate) return 0 ;; + esac + content_in_default && NAS_DEPLOY_SYNC_ELIGIBLE=1 + return 0 +} + backlog_refresh_reminder() { local pr done_cmd report_path [ "$KIND" = secondmate ] && return 0 @@ -620,6 +684,7 @@ validate_worktree_teardown_safety() { return 1 fi fi + mark_nas_deploy_sync_eligible } require_orca_worktree_path_match() { @@ -1047,12 +1112,21 @@ if [ "$KIND" = secondmate ]; then fi remove_grok_turnend_auth "$STATE" "$ID" fm_backend_clear_transition "$BACKEND" "$STATE" "$T" || true -# Remove the per-task temp root (/tmp/fm-/, incl. its gotmp/) recorded by spawn. +# Remove the per-task temp root (/tmp/fm-./, incl. its gotmp/) recorded by spawn. # Read before the state-file rm below; empty (pre-fix tasks without tasktmp=) is a no-op. -[ -n "$TASK_TMP" ] && rm -rf "$TASK_TMP" -rm -f "$STATE/$ID.status" "$STATE/$ID.turn-ended" "$STATE/$ID.check.sh" "$STATE/$ID.meta" "$STATE/$ID.pi-ext.ts" "$STATE/$ID.grok-turnend-token" +remove_task_tmp +rm -f "$STATE/$ID.status" "$STATE/$ID.turn-ended" "$STATE/$ID.check.sh" "$STATE/$ID.meta" "$STATE/$ID.pi-ext.ts" "$STATE/$ID.grok-turnend-token" "$STATE/$ID.ultracode" if [ "$KIND" != scout ] && [ "$KIND" != secondmate ] && [ "$MODE" != local-only ]; then "$FM_ROOT/bin/fm-fleet-sync.sh" "$PROJ" || true fi +# Best-effort, non-blocking: a landed ship task (PR merged, or a local-only merge +# to local main) may have a live NAS deployment that also needs pulling and +# restarting - see bin/fm-nas-deploy-sync.sh's header. Scouts and secondmates +# never land a project change, so they are excluded the same as the fleet-sync +# call above; unlike that call, local-only IS included here, since a local-only +# merge is just as much a landed change as a PR merge. +if [ "$KIND" != scout ] && [ "$KIND" != secondmate ] && [ "$NAS_DEPLOY_SYNC_ELIGIBLE" = 1 ]; then + "$FM_ROOT/bin/fm-nas-deploy-sync.sh" "$(basename "$PROJ")" || true +fi echo "teardown $ID complete (window $T, worktree $WT)" backlog_refresh_reminder diff --git a/bin/fm-tier-guard.sh b/bin/fm-tier-guard.sh new file mode 100755 index 000000000..934a5bfb4 --- /dev/null +++ b/bin/fm-tier-guard.sh @@ -0,0 +1,152 @@ +#!/usr/bin/env bash +# Escalation-trigger guard for the trivial (Haiku/low) dispatch tier: mechanically +# checks whether a task's actual diff size or elapsed time has outgrown the +# resource envelope its assigned model/effort tier implies, so under-resourcing +# is never caught by the crewmate's own self-report alone (guardrail #1, +# data/research-resource-tiering-synthesis.md). Read-only: never edits meta, +# the worktree, or the branch. +# +# Usage: fm-tier-guard.sh +# +# Reads state/.meta for model=/effort=, and that file's own mtime as a +# best-effort spawn-time proxy for the age check - NOT authoritative. A later +# append to meta (bin/fm-pr-check.sh recording pr=/pr_head=, or fm-x-link +# recording X-mode fields) resets the mtime, so a long-running task can read as +# younger than it is and slip the age ceiling; the diff size/file ceilings are +# the primary signal and are unaffected by that reset. Reuses +# bin/fm-review-diff.sh --stat for the actual diff size instead of re-deriving +# the authoritative base/PR-head resolution here. +# +# Prints one "ESCALATE: " line per triggered condition. Exit codes: +# 0 within the tier's envelope (silent) +# 1 at least one escalation fired +# 2 usage or setup error (bad args, missing meta, or fm-review-diff failed) +# Distinct codes let a caller branching on $? tell a real escalation from a +# malformed invocation, matching the sibling guardrail bin/fm-risk-tripwire.sh. +# Only the trivial (Haiku/low) tier has a size/age ceiling checked here; every +# other tier is covered only by the general, tier-independent ceiling below. +# On an escalation, bump the task's model/effort in place per AGENTS.md +# section 7's Validate step; never silently de-escalate afterward. +# +# Env overrides (starting points from +# data/scout-nomistakes-tiering-t8/report.md section 3, not yet calibrated +# against this fleet's own diff-size history): +# FM_TIER_TRIVIAL_MAX_LINES default 30 +# FM_TIER_TRIVIAL_MAX_FILES default 2 +# FM_TIER_TRIVIAL_MAX_AGE_SECONDS default 1800 (30 minutes) +# FM_TIER_HEAVY_MIN_LINES default 400 +# FM_TIER_HEAVY_MIN_FILES default 8 +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" +STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" + +TRIVIAL_MAX_LINES=${FM_TIER_TRIVIAL_MAX_LINES:-30} +TRIVIAL_MAX_FILES=${FM_TIER_TRIVIAL_MAX_FILES:-2} +TRIVIAL_MAX_AGE=${FM_TIER_TRIVIAL_MAX_AGE_SECONDS:-1800} +HEAVY_MIN_LINES=${FM_TIER_HEAVY_MIN_LINES:-400} +HEAVY_MIN_FILES=${FM_TIER_HEAVY_MIN_FILES:-8} + +usage() { + echo "usage: fm-tier-guard.sh " >&2 +} + +if [ "${1:-}" = "-h" ] || [ "${1:-}" = "--help" ]; then + usage + exit 0 +fi + +ID=${1:-} +[ -n "$ID" ] || { usage; exit 2; } +[ $# -le 1 ] || { usage; exit 2; } + +META="$STATE/$ID.meta" +[ -f "$META" ] || { echo "error: no meta for task $ID at $META" >&2; exit 2; } + +MODEL=$(grep '^model=' "$META" | tail -1 | cut -d= -f2- || true) +EFFORT=$(grep '^effort=' "$META" | tail -1 | cut -d= -f2- || true) + +# Portable mtime; Linux stat lacks -f, macOS stat lacks -c (mirrors +# bin/fm-supervision-lib.sh's fm_sup_stat_mtime; not sourced from there to keep +# this script's only dependency on that library's actual contract-bearing +# function, fm_supervision_status, out of scope here). +tier_guard_stat_mtime() { + if [ "$(uname)" = Darwin ]; then + stat -f %m "$1" 2>/dev/null + else + stat -c %Y "$1" 2>/dev/null + fi +} + +is_trivial_tier() { + case "$MODEL" in + *haiku*) + if [ "$EFFORT" = low ] || [ "$EFFORT" = default ]; then + return 0 + fi + ;; + esac + return 1 +} + +STAT_ERR=$(mktemp) +STAT_OUT=$("$SCRIPT_DIR/fm-review-diff.sh" "$ID" --stat 2>"$STAT_ERR") || { + echo "error: fm-review-diff.sh failed for $ID" >&2 + rm -f "$STAT_ERR" + exit 2 +} +# fm-review-diff.sh still exits 0 but warns when it cannot resolve an open PR's +# head and falls back to the possibly-stale local branch, so the size measured +# below may under-count the real PR diff. Blanket-discarding that warning (the +# 2>/dev/null this replaces) made a degraded read indistinguishable from an +# authoritative one - the dangerous under-report direction for this guardrail. +# Detect only that specific marker (not all of fm-review-diff.sh's stderr, which +# also carries fm-guard.sh's supervision banner) and surface it below. +DIFF_MAY_LAG_PR=0 +if grep -q 'PR head unavailable' "$STAT_ERR"; then + DIFF_MAY_LAG_PR=1 +fi +rm -f "$STAT_ERR" + +FILES=0 +LINES=0 +if ! printf '%s\n' "$STAT_OUT" | grep -q '^no changes vs'; then + FILES=$(printf '%s\n' "$STAT_OUT" | grep -oE '[0-9]+ files? changed' | grep -oE '^[0-9]+' || true) + INS=$(printf '%s\n' "$STAT_OUT" | grep -oE '[0-9]+ insertions?\(\+\)' | grep -oE '^[0-9]+' || true) + DEL=$(printf '%s\n' "$STAT_OUT" | grep -oE '[0-9]+ deletions?\(-\)' | grep -oE '^[0-9]+' || true) + FILES=${FILES:-0} + INS=${INS:-0} + DEL=${DEL:-0} + LINES=$((INS + DEL)) +fi + +FOUND=0 + +if is_trivial_tier; then + if [ "$FILES" -gt "$TRIVIAL_MAX_FILES" ] || [ "$LINES" -gt "$TRIVIAL_MAX_LINES" ]; then + echo "ESCALATE: trivial (haiku/low) task $ID has grown to $FILES files / $LINES changed lines, past the trivial envelope ($TRIVIAL_MAX_FILES files / $TRIVIAL_MAX_LINES lines) - bump to at least sonnet/high" + FOUND=1 + fi + AGE_SRC=$(tier_guard_stat_mtime "$META" || true) + if [ -n "$AGE_SRC" ]; then + NOW=$(date +%s) + AGE=$((NOW - AGE_SRC)) + if [ "$AGE" -gt "$TRIVIAL_MAX_AGE" ]; then + echo "ESCALATE: trivial (haiku/low) task $ID has run ${AGE}s, past the trivial tier's ${TRIVIAL_MAX_AGE}s ceiling - bump to at least sonnet/high" + FOUND=1 + fi + fi +fi + +if [ "$FILES" -gt "$HEAVY_MIN_FILES" ] || [ "$LINES" -gt "$HEAVY_MIN_LINES" ]; then + echo "ESCALATE: task $ID's diff ($FILES files / $LINES changed lines) exceeds the heavy-scale ceiling ($HEAVY_MIN_FILES files / $HEAVY_MIN_LINES lines) regardless of its assigned tier - re-check whether it still matches its dispatched rule" + FOUND=1 +fi + +if [ "$DIFF_MAY_LAG_PR" -eq 1 ]; then + echo "warning: the diff for $ID was sized against a possibly-stale local branch because the open PR head could not be resolved; the ceilings above may under-count the real PR diff - re-check against the current PR before trusting a within-envelope result" >&2 +fi + +exit "$FOUND" diff --git a/bin/fm-ultracode-guard.sh b/bin/fm-ultracode-guard.sh new file mode 100755 index 000000000..f80d7bf69 --- /dev/null +++ b/bin/fm-ultracode-guard.sh @@ -0,0 +1,110 @@ +#!/usr/bin/env bash +# Ultracode enforcement (guardrail #3): mechanically confirms a genuinely +# independent second pass ran on an ultracode-flagged task's finished diff +# before it can go PR-ready - not a sub-task the same crewmate spawned itself +# (data/research-resource-tiering-synthesis.md). Tracks state via a plain +# marker file, state/.ultracode, the same convention as other +# firstmate state/ markers (state/.afk, state/.turn-ended); it never +# touches fm-spawn.sh or the task's own meta. +# +# Usage: +# fm-ultracode-guard.sh flag [] +# Records that was dispatched under a crew-dispatch rule whose +# resolved profile set ultracode=true. defaults to +# "independent-review" (the only role this fleet's rules use today; see +# docs/examples/crew-dispatch.json). Firstmate runs this right after +# spawning the task. +# fm-ultracode-guard.sh reviewed +# Records that - a distinct, separately dispatched task +# (its own state/.meta must exist) - independently +# reviewed 's finished diff and its findings were addressed. +# Refuses if equals or has no recorded meta, +# so a sub-task the same crewmate spawned itself cannot satisfy this. +# fm-ultracode-guard.sh check +# Exits 0 if was never flagged, or was flagged and reviewed. +# Exits 1 with an explanatory message if flagged but not yet reviewed - +# firstmate runs this before treating an ultracode-flagged task as +# PR-ready (AGENTS.md section 7's Validate step). +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" +STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" + +usage() { + echo "usage: fm-ultracode-guard.sh flag []" >&2 + echo " fm-ultracode-guard.sh reviewed " >&2 + echo " fm-ultracode-guard.sh check " >&2 +} + +safe_token() { + case "$1" in + ''|*[!A-Za-z0-9._-]*) return 1 ;; + *) return 0 ;; + esac +} + +if [ "${1:-}" = "-h" ] || [ "${1:-}" = "--help" ]; then + usage + exit 0 +fi + +CMD=${1:-} +ID=${2:-} +if [ -z "$CMD" ] || [ -z "$ID" ]; then + usage + exit 1 +fi + +MARKER="$STATE/$ID.ultracode" + +cmd_flag() { + local role=${3:-independent-review} + [ $# -le 3 ] || { usage; exit 1; } + safe_token "$role" || { echo "error: role must be a safe token" >&2; exit 1; } + mkdir -p "$STATE" + # Overwrites any existing marker: re-flagging (e.g. after an escalation) + # deliberately clears a prior reviewed_by=, since that review was against an + # earlier version of the diff and the requirement starts over. + printf 'role=%s\n' "$role" > "$MARKER" + echo "flagged $ID ultracode role=$role" +} + +cmd_reviewed() { + local reviewer=${3:-} + [ $# -eq 3 ] || { usage; exit 1; } + [ -n "$reviewer" ] || { echo "error: reviewed requires a reviewer-task-id" >&2; exit 1; } + safe_token "$reviewer" || { echo "error: reviewer-task-id must be a safe token" >&2; exit 1; } + [ -f "$MARKER" ] || { echo "error: $ID is not ultracode-flagged (no $MARKER); nothing to mark reviewed" >&2; exit 1; } + if [ "$reviewer" = "$ID" ]; then + echo "error: reviewer-task-id must be a task distinct from $ID - a task cannot independently review itself" >&2 + exit 1 + fi + if [ ! -f "$STATE/$reviewer.meta" ]; then + echo "error: $reviewer has no recorded state/$reviewer.meta - it must be a genuinely, separately dispatched task, not a made-up id or a sub-task $ID spawned itself" >&2 + exit 1 + fi + grep -Fqx "reviewed_by=$reviewer" "$MARKER" 2>/dev/null || printf 'reviewed_by=%s\n' "$reviewer" >> "$MARKER" + echo "recorded $reviewer as the independent review of $ID" +} + +cmd_check() { + [ $# -eq 2 ] || { usage; exit 1; } + if [ ! -f "$MARKER" ]; then + exit 0 + fi + if grep -q '^reviewed_by=' "$MARKER" 2>/dev/null; then + exit 0 + fi + role=$(grep '^role=' "$MARKER" | tail -1 | cut -d= -f2- || true) + echo "error: $ID is ultracode-flagged (role=${role:-independent-review}) but has no recorded independent review yet - dispatch a genuinely separate task to review the finished diff, then run: fm-ultracode-guard.sh reviewed $ID " >&2 + exit 1 +} + +case "$CMD" in + flag) cmd_flag "$@" ;; + reviewed) cmd_reviewed "$@" ;; + check) cmd_check "$@" ;; + *) usage; exit 1 ;; +esac diff --git a/docs/architecture.md b/docs/architecture.md index f2c590f9c..6085fe28e 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -95,6 +95,7 @@ Codex App support is recorded in `docs/codex-app-backend.md`; it is not selectab Crewmates never intentionally touch your project clone; [treehouse](https://github.com/kunchenguid/treehouse) pools clean worktrees for tmux, herdr, zellij, and cmux tasks, while Orca creates its own worktrees for `backend=orca`. For ship and scout work, `fm-spawn.sh` refuses to launch unless the resolved task path is a real git worktree root that is distinct from the project primary checkout. +Pooled worktrees of the same project share a leaf directory basename, which collides on Docker Compose's default project-naming; see `docs/configuration.md` "Docker Compose project isolation" for the per-worktree `compose_project=` name `fm-spawn.sh` derives to avoid it. The firstmate repo has one extra exposure because it can dispatch crewmates to work on itself. Its operating checkout (`FM_ROOT`) and the disposable crewmate worktrees are all linked git worktrees of the same repository, so the valid discriminator is branch state, not whether the checkout is linked. @@ -129,6 +130,13 @@ Secondmate launches are exempt because they resolve the secondmate harness and a Unsupported effort values are still recorded in task meta when passed to `fm-spawn.sh`, but the launch template omits any effort flag that the selected harness does not accept. That keeps spawn launch compatible across claude, codex, grok, pi, and opencode while preserving the requested profile for later audit. +A crew-dispatch profile can also carry `ultracode: true` with an optional `ultracode_role` (today only `independent-review`): a task matched to that profile must get a genuinely independent second pass on its finished diff, dispatched as its own separate task rather than a sub-task the implementing crewmate spawns, before it can go PR-ready. +Three mechanical guardrails back this tiering, each a structurally separate check from the natural-language rule match so a misclassified task cannot slip a cheaper tier on one judgment call. +`bin/fm-risk-tripwire.sh` greps a task's brief text and, once code exists, its changed file paths for migration, auth, schema, and security signals; a hit floors the task to the safety-critical profile (`opus`/`xhigh`, ultracode `independent-review`) regardless of which rule was matched. +`bin/fm-tier-guard.sh` is a read-only Validate-time check that escalates a trivial (`haiku`/`low`) task in place once its actual diff size or elapsed time outgrows that tier's envelope, plus a general heavy-scale ceiling that escalates any tier once its diff crosses it; it reuses `bin/fm-review-diff.sh --stat` for the authoritative diff size. +`bin/fm-ultracode-guard.sh` tracks the ultracode requirement through a plain `state/.ultracode` marker (`role=`, then `reviewed_by=`), the same marker convention as `state/.afk`: it flags the requirement right after spawn (also on a risk-tripwire hit), records a distinct separately-dispatched reviewer task as the independent review, and refuses to let an ultracode-flagged task go PR-ready until that review is recorded. +None of the three modify `fm-spawn.sh` or task meta. + ## Optional secondmates `data/secondmates.md` records persistent domain supervisors with natural-language scopes, project clone lists, and home paths. @@ -228,6 +236,11 @@ Fetches blocked by an orphaned `.git/packed-refs.lock` use bounded retries and r Local-only projects, clones without an origin remote, and fetch failures remain benign skips. The refresh also prunes local branches whose remote is gone and that no worktree still needs. +A landed ship task's `projects/` dev clone is not the same checkout as a live deployment of that project running elsewhere on the host. +When a project has a row in `data/nas-deployments.md`, non-scout, non-secondmate teardown also best-effort fast-forwards that project's separate live NAS checkout and restarts its recorded pm2 process(es), verifying each comes back online (`bin/fm-nas-deploy-sync.sh`). +It mirrors the same clean/fast-forward-only safety as the clone refresh above, including the bounded orphaned-packed-refs-lock recovery, and reports a diverged or dirty checkout as `STUCK:` rather than touching it; a project absent from the mapping is a silent no-op. +Every filesystem/git touch of the NAS checkout is time-bounded so an unreachable mount cannot hang the teardown call that triggered it. + ## Self-updates stay safe `/updatefirstmate` fast-forwards the running firstmate repo and registered secondmate homes from `origin`, then re-reads updated instructions and nudges updated secondmates without touching project clones. diff --git a/docs/cmux-backend.md b/docs/cmux-backend.md index 5859e7f2d..305a582e6 100644 --- a/docs/cmux-backend.md +++ b/docs/cmux-backend.md @@ -173,7 +173,7 @@ No session field is needed - unlike herdr/zellij there is no session layer to re | Duplicate task check | `cmux workspace list --json --id-format uuids`, match by home-scoped `.title` | cmux enforces NO title uniqueness for workspaces OR surfaces/tabs - verified live: two workspaces, and two surfaces within one workspace, all created successfully sharing one title. The adapter's own duplicate check is required, mirroring herdr/zellij, and it checks the scoped title such as `fm-firstmate-<8hex>-`. | | Create task workspace | `cmux new-workspace --name --cwd --focus false --id-format uuids` | Creates a workspace with exactly one default surface. `--focus` verified to already default to `false` for workspace/surface/pane creation - no focus-restore dance needed, unlike zellij. The caller passes `fm-`, but the adapter creates `fm--`. | | Workspace/surface id resolution | `cmux workspace list --json --id-format uuids` (find by home-scoped title), then `cmux list-panes --workspace --json --id-format uuids` (`.panes[0].selected_surface_id`) | A freshly created workspace already has exactly one surface, so no separate `new-surface` call is needed. `--id-format uuids` (or `both`) is required to get a bare `id` field in JSON; the default JSON shape returns only short `ref` strings like `"workspace:2"`. | -| Liveness / target readiness | `cmux list-panes --workspace --json --id-format uuids`, checking the surface id appears in `.panes[].surface_ids` | Structural existence check, NOT a content read - see "read-screen fails on a genuinely fresh surface" below for why `read-screen` cannot be used here. Verified reliable on a completely untouched fresh surface, unlike `read-screen`. | +| Liveness / target readiness | `cmux list-panes --workspace --json --id-format uuids`, checking the surface id appears in `.panes[].surface_ids` | Structural existence check, NOT a content read - see "read-screen fails on a genuinely fresh surface" below for why `read-screen` cannot be used here. Verified reliable on a completely untouched fresh surface, unlike `read-screen`. `fm_backend_cmux_surface_exists` captures `list-panes`' own output and checks ITS exit status before piping into `jq -e`: on jq 1.6, `jq -e` over truly empty stdin exits 0, so piping a failed/empty `list-panes` call straight into `jq` would misread a dead target as a live one. | | Send literal (unsubmitted) | `cmux send --workspace --surface -- ` | Verified live: does NOT auto-submit - text sits at the prompt, unexecuted, until a separate Enter. Matches every other backend's "literal-then-separate-Enter" contract. The `--` separator keeps option-shaped text such as `--help` literal. | | Send key | `cmux send-key --workspace --surface ` | Verified names: `enter`, `escape`, `ctrl-c` all work directly (lowercase, hyphenated). Escape is natively supported (unlike Orca); Ctrl-C correctly interrupted a running `sleep 100` in a live test. cmux's own key vocabulary is richer still (`ctrl-d`/`ctrl-z`/`ctrl-\\`, semantic aliases `sigint`/`sigtstp`/`sigquit`), but firstmate's shared vocabulary only needs these three today. | | Send + submit, composed | `send` then `send-key enter` | cmux has no single-call atomic "type and submit" primitive (unlike tmux's `send-keys ... Enter` or herdr's `pane run`); `fm_backend_cmux_send_text_line` composes the two calls, mirroring zellij's equivalent composition. | diff --git a/docs/configuration.md b/docs/configuration.md index d289c4c15..2710d535a 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -80,6 +80,15 @@ The caller-facing label remains `fm-`, but the actual cmux workspace title i Test cleanup must use the guarded path described in [`docs/cmux-backend.md`](cmux-backend.md)'s "Test safety" section, never enumerate-and-close every workspace. The `config/backend` file is not inherited by secondmate homes. +## Docker Compose project isolation (per-worktree) + +Two treehouse worktrees of the same project always share the same leaf directory basename (e.g. `wealthsync`), so Docker Compose's default project-naming, derived from the current directory's basename, collides across them on container, volume, and network names. +For every ship/scout spawn, regardless of runtime backend, `fm-spawn.sh` derives a stable, Compose-safe project name from the worktree's own pool-slot path once it is resolved (a treehouse pool worktree is `.../-//`; the name is `-`, lowercased, with non-`[a-z0-9-]` characters replaced by `-` and repeats collapsed), falling back to the worktree's own leaf name or the task id when the path lacks that pool-slot shape (e.g. a non-treehouse backend). +The name is written to `.treehouse-compose-project` in the worktree root, excluded from git via that worktree's own `.git/info/exclude` so it never blocks teardown's dirty-worktree check or leaks into a crewmate's commit, and recorded in `state/.meta` as `compose_project=`. +It is stable across respawns into the same pool slot, not per task id, so a stale Compose stack left by a prior task in that slot is naturally superseded rather than accumulating orphaned volumes under a new name every respawn. +Secondmates never get this marker or meta field; they run in their own firstmate home, not a per-task pool worktree. +`fm-brief.sh` tells the crewmate, in both ship and scout briefs, to read `.treehouse-compose-project` and pass it explicitly on every `docker compose` invocation (e.g. `docker compose -p "$(cat .treehouse-compose-project)" up -d`), since exported env vars do not reliably persist across separate tool-call invocations; it also flags fixed host ports as a related collision vector and recommends `docker compose port ` or another deterministic per-worktree-derived port instead of a hard-coded one. + ## Away-mode supervisor backend (FM_SUPERVISOR_BACKEND / FM_SUPERVISOR_TARGET) The `/afk` sub-supervisor injects escalation digests into firstmate's own pane independently of where new task endpoints are spawned. @@ -101,6 +110,24 @@ An absent file means `auto`, i.e. default-on on macOS: the alarm exists precisel A missing or failing channel logs and falls through to the next, never crashing the daemon. See [`wedge-alarm.md`](wedge-alarm.md) for the channel reference and macOS verification evidence, and [`examples/wedge-alarm`](examples/wedge-alarm) for a copyable config. +## Critical service watch (config/critical-services) + +`config/critical-services` (local, gitignored) lists systemd unit names, one per non-empty, non-comment line, that firstmate watches for the `failed` state. +At every session start bootstrap runs `systemctl is-failed` (read-only, no root) on each listed unit and prints `SERVICE_FAILED: - failed since ` for any that are failed, dropping the `failed since` clause when the unit's `StateChangeTimestamp` is unavailable. +It exists so a silently-failed critical unit, the kind of boot-time oneshot whose failure can otherwise go unnoticed for days, surfaces the next time firstmate starts rather than never. +An absent or empty file is a no-op, and a host without `systemctl` (no systemd) silently does nothing. +The file is not inherited by secondmate homes, whose same-machine bootstrap would only re-check the same units. +See [`examples/critical-services`](examples/critical-services) for a copyable config; report a failed unit to the captain in plain language and never restart it, since a unit that failed may be unsafe to restart without knowing why. + +## Upstream template drift (UPSTREAM_DRIFT) + +firstmate's own repo tracks a read-only upstream template (`kunchenguid/firstmate`, the `upstream` remote after the remote swap; see `AGENTS.md` section 1) that this fleet can never push or merge into directly, so local `main` and `upstream/main` drift apart silently over time. +Every session, bootstrap prints one `UPSTREAM_DRIFT: local main is ahead / behind upstream/main, last reconciled d ago ()` FYI line by reading whatever `refs/remotes/upstream/main` currently holds, without fetching, so the report is safe in the read-only/no-lock detect path. +The wording escalates to `UPSTREAM_DRIFT: this repo's upstream sync needs attention - ...` once local `main` is more than 30 commits behind `upstream/main` or their merge-base is older than 10 days. +It only reports; reconciling upstream is always a deliberately dispatched, reviewed ship task, never automated. +No-op when there is no `upstream` remote or no `upstream/main` ref yet. +The network fetch that refreshes `refs/remotes/upstream/main` runs only in the locked, mutating fleet-sync sweep, bounded by `FM_UPSTREAM_FETCH_TIMEOUT` (default 20s); it requires a `timeout` or `gtimeout` binary, otherwise the fetch is skipped and the last-fetched ref is kept. + ## Gate defaults (.no-mistakes.yaml) The tracked `.no-mistakes.yaml` keeps test evidence outside the repo and defines `commands.test` so no-mistakes runs firstmate's bash behavior suite directly. @@ -192,7 +219,7 @@ This section is the single owner of the canonical schema and its per-field seman { "when": "", "use": [ - { "harness": "", "model": "", "effort": "" } + { "harness": "", "model": "", "effort": "", "ultracode": "", "ultracode_role": "" } ], "select": "", "why": "" @@ -205,6 +232,7 @@ This section is the single owner of the canonical schema and its per-field seman Per rule, `when` and `use` are required. `use` may be a single profile object or an ordered array of profile objects; the single-object form stays fully backward-compatible, and every profile needs `harness`. `use.model`, `use.effort`, and `why` are optional. +`use.ultracode` and `use.ultracode_role` are optional; `AGENTS.md` section 4 owns what they mean and the mechanical `bin/fm-ultracode-guard.sh` tracking they trigger. `select` is optional and currently supports `quota-balanced`. Absent `select` means use the first array element, or the only object in the single-object form; the first array element is the deterministic tie-break and the ultimate fallback. `default` is optional. @@ -385,9 +413,20 @@ FM_CLASSIFY_PAUSED_VERB=paused # leading status verb for a declared external FM_STALE_ESCALATE_SECS=240 # idle seconds before a provably-working stale pane escalates; stale panes whose crew is not provably working surface immediately unless they declare the pause verb FM_PAUSE_RESURFACE_SECS=3600 # seconds before an idle declared external wait re-surfaces for a recheck in the watcher or away-mode daemon FM_WEDGE_DEMAND_INSPECT_COUNT=3 # consecutive provably-working stale escalations on the same unchanged pane before demand-deep-inspection is added +FM_TIER_TRIVIAL_MAX_LINES=30 # fm-tier-guard.sh: changed-line count a trivial (haiku/low) task may reach before escalation fires +FM_TIER_TRIVIAL_MAX_FILES=2 # fm-tier-guard.sh: changed-file count a trivial (haiku/low) task may reach before escalation fires +FM_TIER_TRIVIAL_MAX_AGE_SECONDS=1800 # fm-tier-guard.sh: elapsed seconds a trivial (haiku/low) task may run before escalation fires (30 min) +FM_TIER_HEAVY_MIN_LINES=400 # fm-tier-guard.sh: changed-line count that escalates any tier via the general heavy-scale ceiling +FM_TIER_HEAVY_MIN_FILES=8 # fm-tier-guard.sh: changed-file count that escalates any tier via the general heavy-scale ceiling FM_WATCH_TRIAGE_LOG_MAX_BYTES=262144 # size cap for the watcher's absorbed-wake debug log FM_FLEET_SYNC_BOOTSTRAP_TIMEOUT= # optional seconds allowed for bootstrap's best-effort clone refresh; unset/blank defaults to max(20, 5 + 3 * origin-backed-project-count) FM_FLEET_PRUNE=1 # set to 0 to skip pruning local branches whose upstream is gone +FM_UPSTREAM_FETCH_TIMEOUT=20 # seconds allowed for the locked fleet-sync sweep to fetch upstream/main for the UPSTREAM_DRIFT report; requires timeout/gtimeout or the fetch is skipped +FM_NAS_DEPLOYMENTS_OVERRIDE= # alternate data/nas-deployments.md path for fm-nas-deploy-sync.sh, mainly for tests +FM_NAS_SYNC_TIMEOUT=15 # seconds allowed per filesystem/git touch of fm-nas-deploy-sync.sh's $NAS_PATH before it is killed, so an unreachable NAS mount cannot hang fm-teardown.sh's caller +FM_NAS_SYNC_PACKED_REFS_LOCK_RETRIES=3 # fetch retries after fm-nas-deploy-sync.sh hits the orphaned .git/packed-refs.lock signature on the shared NAS checkout (two landed teardowns for the same project can race it) +FM_NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS=1 # seconds fm-nas-deploy-sync.sh waits before each of those retries +FM_NAS_SYNC_PACKED_REFS_LOCK_AGE_SECS=30 # min mtime age before fm-nas-deploy-sync.sh treats a leftover packed-refs.lock as provably stale FM_STALE_WORKTREE_LOCK_AGE_SECS=30 # min mtime age before fm-teardown.sh treats a leftover worktree git index.lock as provably stale FM_TREEHOUSE_RETURN_LOCK_RETRIES=3 # retries after a treehouse return fails on the transient git index.lock signature FM_TREEHOUSE_RETURN_LOCK_RETRY_WAIT_SECS=1 # seconds fm-teardown.sh waits before each retry after that signature diff --git a/docs/examples/crew-dispatch.json b/docs/examples/crew-dispatch.json index 8aec19650..8f27118e6 100644 --- a/docs/examples/crew-dispatch.json +++ b/docs/examples/crew-dispatch.json @@ -18,6 +18,21 @@ ], "select": "quota-balanced", "why": "Use a strong coding profile for broad design and implementation work." + }, + { + "when": "The task changes this firstmate repo's own supervision backbone: bin/fm-watch.sh, bin/fm-guard.sh, a harness turnend-guard hook, or other logic the always-on watcher/guard rely on for correctness.", + "use": { "harness": "claude", "model": "sonnet", "effort": "xhigh" }, + "why": "A narrow set of scripts backs every other task's supervision; a subtle regression here is silent and fleet-wide, so it earns deeper reasoning than the general firstmate-repo default even though the model tier does not need to change." + }, + { + "when": "Safety-critical, security-sensitive, or hard-to-reverse work: schema or data migrations, auth/session/credential changes, payment or billing logic, or anything touching a frozen or safety-critical spec.", + "use": { "harness": "claude", "model": "opus", "effort": "xhigh", "ultracode": true, "ultracode_role": "independent-review" }, + "why": "A model reviewing its own irreversible change has anchoring bias; an independent pass and the strongest available model are both cheap relative to the cost of a bad migration or a broken safety-flow." + }, + { + "when": "A genuine architectural or product tradeoff with long-term consequences that the crewmate must reason through and justify, not just implement.", + "use": { "harness": "claude", "model": "opus", "effort": "xhigh" }, + "why": "The deliverable's entire value is the quality of the argued judgment call; this is low-volume, high-stakes work, so the cost delta of the top model is bounded and well spent." } ], "default": { "harness": "codex", "model": "gpt-5.5", "effort": "medium" } diff --git a/docs/examples/critical-services b/docs/examples/critical-services new file mode 100644 index 000000000..5b4cf5373 --- /dev/null +++ b/docs/examples/critical-services @@ -0,0 +1,19 @@ +# config/critical-services - systemd units firstmate watches for the failed +# state (bin/fm-bootstrap.sh's critical_services_check). Copy this into a local, +# gitignored config/critical-services and replace the placeholders with your own +# unit names. One systemd unit name per non-empty, non-comment line; blank lines +# and full-line "#" comments are ignored. +# +# At every session start, firstmate runs `systemctl is-failed` (read-only, no +# root) on each listed unit and surfaces a SERVICE_FAILED line for any that are +# failed, then reports it to the captain in plain language. It never restarts a +# unit: a unit that failed may be unsafe to restart without knowing why. +# +# An ABSENT or empty config/critical-services is a no-op. On a host without +# systemd (no systemctl) the check silently does nothing. +# +# The names below are illustrative placeholders - list the units whose silent +# failure would actually matter on this machine (the oneshot that runs a +# production app at boot, a backup timer, a sync daemon, and so on). +my-app.service +nightly-backup.timer diff --git a/docs/orca-backend.md b/docs/orca-backend.md index 6be2ad58f..b10e948e3 100644 --- a/docs/orca-backend.md +++ b/docs/orca-backend.md @@ -71,7 +71,7 @@ Spawn: 1. Ensure the project repo is registered in Orca, adding it with `orca repo add --path` when needed. 2. Create an independent Orca worktree with `orca worktree create --repo id: --name fm- --no-parent --setup skip`. 3. Reuse the terminal returned by Orca worktree creation only when it appears in the verified `result.terminal.handle` shape, or create a titled terminal in that worktree when Orca returns only the worktree. -4. Install firstmate's per-harness turn-end hooks in the Orca worktree. +4. Write the docker-compose isolation marker (`.treehouse-compose-project`) and install firstmate's per-harness turn-end hooks in the Orca worktree. 5. Write metadata, then send `GOTMPDIR` export and the selected harness launch through the recorded Orca terminal. Operation routing: diff --git a/docs/scripts.md b/docs/scripts.md index 8ed71b89f..a28c57fbd 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -16,7 +16,7 @@ The shared no-mistakes gate refusal used by `fm-spawn.sh`, `fm-send.sh`, and `fm | `fm-update.sh` | Fast-forward-only self-update of firstmate and secondmate homes from origin | | `fm-backlog-handoff.sh` | Validate and delegate queued backlog-item moves into a secondmate home | | `fm-decision-hold.sh` | Create, verify, complete, and resolve durable captain-held decisions | -| `fm-brief.sh` | Scaffold ship, scout, secondmate-charter, and Herdr-lab briefs | +| `fm-brief.sh` | Scaffold ship, scout, secondmate-charter, and Herdr-lab briefs, including the docker-compose isolation note for ship/scout | | `fm-herdr-lab.sh` | Provision and guardedly operate an isolated, never-default Herdr lab session | | `fm-ensure-agents-md.sh` | Ensure a project's real `AGENTS.md`, its `CLAUDE.md` symlink, and the canonical self-governance section | | `fm-guard.sh` | Warn on primary-checkout tangles, pending queued wakes, and stale watcher liveness | @@ -26,8 +26,11 @@ The shared no-mistakes gate refusal used by `fm-spawn.sh`, `fm-send.sh`, and `fm | `fm-arm-command-policy.mjs` | Semantic owner of the watcher-arm PreToolUse policy (docs/arm-pretool-check.md) | | `fm-supervision-instructions.sh` | Render the session-start primary-harness supervision block or the one-line repair instruction | | `fm-home-seed.sh` | Transactionally provision a secondmate home and maintain `data/secondmates.md` | -| `fm-spawn.sh` | Spawn crewmates, scouts, `id=repo` batches, and secondmates on the resolved harness and runtime backend | +| `fm-spawn.sh` | Spawn crewmates, scouts, `id=repo` batches, and secondmates on the resolved harness and runtime backend, recording a per-worktree docker-compose project name for ship/scout tasks (`compose_project=`; docs/configuration.md "Docker Compose project isolation") | | `fm-dispatch-select.sh` | Resolve a matched crew-dispatch rule to one concrete profile, owning `quota-balanced` selection | +| `fm-tier-guard.sh` | Mechanically check whether a dispatched task's actual diff size or elapsed time has outgrown the resource envelope its assigned model/effort tier implies (guardrail #1); read-only, never edits meta, the worktree, or the branch | +| `fm-risk-tripwire.sh` | Mechanically grep a task's brief text and, once code exists, its changed file paths for migration/auth/schema/security signals - a second, structurally different check from the natural-language crew-dispatch match, so a misclassified risky task cannot slip to a cheap tier (guardrail #2) | +| `fm-ultracode-guard.sh` | Mechanically confirm a genuinely independent second pass ran on an ultracode-flagged task's finished diff before it can go PR-ready, tracked via a plain `state/.ultracode` marker file (guardrail #3) | | `fm-backend.sh` | Runtime-backend selection, meta helpers, selector resolution, and operation dispatch | | `fm-backend-hometag-lib.sh` | Shared per-installation home-tag derivation for zellij tab and cmux workspace titles | | `fm-composer-lib.sh` | Single fleet-wide owner of composer-content classification for all backends | @@ -38,7 +41,7 @@ The shared no-mistakes gate refusal used by `fm-spawn.sh`, `fm-send.sh`, and `fm | `backends/cmux.sh` | Experimental cmux session-provider adapter | | `fm-config-push.sh` | Push declared inheritable local config to live secondmate homes mid-session | | `fm-project-mode.sh` | Resolve a project's delivery mode and `+yolo` flag from `data/projects.md` | -| `fm-merge-local.sh` | Fast-forward a `local-only` project's local default branch after approval | +| `fm-merge-local.sh` | Fast-forward a `local-only` project's local default branch after approval, best-effort pushing it to `origin` when a remote is present | | `fm-review-diff.sh` | Review a crewmate branch or recorded PR head against the authoritative base | | `fm-marker-lib.sh` | Shared from-firstmate request marker, detector, and idempotent transformation | | `fm-gate-refuse-lib.sh` | Shared no-mistakes gate-context refusal for fleet lifecycle entrypoints | @@ -67,6 +70,7 @@ The shared no-mistakes gate refusal used by `fm-spawn.sh`, `fm-send.sh`, and `fm | `fm-pr-merge.sh` | Record PR metadata, then merge a task's PR from its full GitHub URL | | `fm-promote.sh` | Promote a scout task in place to a protected ship task | | `fm-teardown.sh` | Fail-closed teardown: return landed ship worktrees, require completed scout deliverables, retire secondmate homes | +| `fm-nas-deploy-sync.sh` | Best-effort, non-fatal fast-forward sync and pm2 restart of a project's live NAS checkout after a landed ship-task teardown, mirroring `fm-fleet-sync.sh`'s dirty/diverged safety | | `fm-harness.sh` | Detect the running harness and resolve crew or secondmate harness, model, and effort | | `fm-lock.sh` | Per-home firstmate session lock | | `fm-x-lib.sh` | Shared X-mode config, relay, and reply-threading helpers | diff --git a/tests/fm-backend-herdr.test.sh b/tests/fm-backend-herdr.test.sh index 5186cca2c..77484a693 100755 --- a/tests/fm-backend-herdr.test.sh +++ b/tests/fm-backend-herdr.test.sh @@ -34,16 +34,33 @@ set -u LOG="${FM_HERDR_LOG:?}" RESP="${FM_HERDR_RESPONSES:?}" COUNT_FILE="$RESP/.count" -next=$(( $(cat "$COUNT_FILE" 2>/dev/null || echo 0) + 1 )) { printf 'HERDR_SESSION=%s' "${HERDR_SESSION:-}" for a in "$@"; do printf '\x1f%s' "$a"; done printf '\n' } >> "$LOG" +# `herdr server` is a fire-and-forget launch: server_ensure backgrounds it and +# discards its output, so its timing relative to the immediately-following status +# poll is non-deterministic. It must NOT consume a canned-response slot, or a lost +# race with that poll would steal a status response and desync the whole sequence. +# It is still logged above, so the "did it start the server" assertion holds. +if [ "${1:-}" = server ]; then + exit 0 +fi if [ "${1:-}" = status ] && [ "${2:-}" = --json ] && [ "${FM_HERDR_SCRIPT_STATUS:-0}" != 1 ]; then printf '{"client":{"version":"0.7.1","protocol":14},"server":{"running":true}}\n' exit 0 fi +# fm_backend_herdr_server_ensure launches `herdr server` fire-and-forget +# (backgrounded) so its poll loop can proceed concurrently. No test scripts a +# canned response for it, so it must never read/write the shared call-ordinal +# COUNT_FILE below - doing so races with whichever synchronous call the poll +# loop issues at the same moment, losing an update and permanently shifting +# every later call's response number. +if [ "${1:-}" = server ]; then + exit 0 +fi +next=$(( $(cat "$COUNT_FILE" 2>/dev/null || echo 0) + 1 )) n=$next echo "$n" > "$COUNT_FILE" if [ -f "$RESP/$n.exit" ]; then @@ -291,14 +308,15 @@ test_container_ensure_starts_server_and_workspace() { printf '{"client":{"version":"0.7.1","protocol":14}}\n' > "$resp/1.out" # 2: server_ensure's status --json check -> not running printf '{"server":{"running":false}}\n' > "$resp/2.out" - # 3: `herdr server` backgrounded launch - no meaningful output - # 4: server_ensure poll -> now running - printf '{"server":{"running":true}}\n' > "$resp/4.out" - # 5: workspace list -> empty (no "firstmate" workspace yet) - printf '{"result":{"workspaces":[]}}\n' > "$resp/5.out" - # 6: workspace create -> w1, seeding default tab w1:t9 (real herdr returns + # `herdr server` is backgrounded and consumes no response slot (see the fake), + # so the very next scripted response is the server_ensure poll. + # 3: server_ensure poll -> now running + printf '{"server":{"running":true}}\n' > "$resp/3.out" + # 4: workspace list -> empty (no "firstmate" workspace yet) + printf '{"result":{"workspaces":[]}}\n' > "$resp/4.out" + # 5: workspace create -> w1, seeding default tab w1:t9 (real herdr returns # the seeded tab/pane ids in the SAME response - verified empirically). - printf '{"result":{"workspace":{"workspace_id":"w1","label":"firstmate"},"tab":{"tab_id":"w1:t9"},"root_pane":{"pane_id":"w1:p9"}}}\n' > "$resp/6.out" + printf '{"result":{"workspace":{"workspace_id":"w1","label":"firstmate"},"tab":{"tab_id":"w1:t9"},"root_pane":{"pane_id":"w1:p9"}}}\n' > "$resp/5.out" fb=$(make_herdr_fakebin "$dir") out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" FM_HERDR_SCRIPT_STATUS=1 HERDR_SESSION=fmtest \ bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_container_ensure /tmp' "$ROOT" ) diff --git a/tests/fm-backend-orca.test.sh b/tests/fm-backend-orca.test.sh index 60cc0dca8..bf3b38550 100755 --- a/tests/fm-backend-orca.test.sh +++ b/tests/fm-backend-orca.test.sh @@ -470,7 +470,7 @@ test_spawn_preserves_orca_metadata_when_pathless_worktree_cleanup_fails() { } test_spawn_writes_orca_metadata_and_launches_harness() { - local proj wt data state config id out log + local proj wt data state config id out log task_tmp id="orcaspawnz1" proj="$TMP_ROOT/spawn-project" wt="$TMP_ROOT/spawn-wt" @@ -498,13 +498,19 @@ test_spawn_writes_orca_metadata_and_launches_harness() { assert_grep "terminal=term-spawn" "$state/$id.meta" "meta missing terminal handle" assert_grep "orca_worktree_id=wt-spawn" "$state/$id.meta" "meta missing Orca worktree id" assert_grep "worktree=$wt" "$state/$id.meta" "meta missing Orca worktree path" + task_tmp=$(grep '^tasktmp=' "$state/$id.meta" | cut -d= -f2-) + case "$task_tmp" in + /tmp/fm-$id.*) ;; + *) fail "meta tasktmp should be randomized under /tmp/fm-$id.*, got '$task_tmp'" ;; + esac + FM_TEST_CLEANUP_DIRS+=("$task_tmp") + assert_present "$task_tmp/gotmp" "spawn did not create GOTMPDIR under tasktmp" assert_not_contains "$(cat "$log")" $'orca\x1f''terminal'$'\x1f''create' \ "spawn should reuse the implicit terminal returned by Orca worktree creation" - assert_contains "$(cat "$log")" $'orca\x1f''terminal'$'\x1f''send'$'\x1f''--terminal'$'\x1f''term-spawn'$'\x1f''--text'$'\x1f''export GOTMPDIR=/tmp/fm-orcaspawnz1/gotmp'$'\x1f''--enter'$'\x1f''--json' \ + assert_contains "$(cat "$log")" $'orca\x1f''terminal'$'\x1f''send'$'\x1f''--terminal'$'\x1f''term-spawn'$'\x1f''--text'$'\x1f''export GOTMPDIR='"$task_tmp"'/gotmp'$'\x1f''--enter'$'\x1f''--json' \ "spawn did not export GOTMPDIR through the Orca terminal" - assert_contains "$(cat "$log")" "CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false claude --dangerously-skip-permissions" \ + assert_contains "$(cat "$log")" "CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false CLAUDE_CODE_AUTO_COMPACT_WINDOW=300000 claude --dangerously-skip-permissions" \ "spawn did not send the selected harness launch command through Orca" - rm -rf "/tmp/fm-$id" pass "fm-spawn.sh --backend orca: reuses implicit terminal, records metadata, launches harness" } diff --git a/tests/fm-backlog-handoff.test.sh b/tests/fm-backlog-handoff.test.sh index 9e10703e8..8000b8fc1 100755 --- a/tests/fm-backlog-handoff.test.sh +++ b/tests/fm-backlog-handoff.test.sh @@ -10,8 +10,11 @@ set -u . "$(dirname "${BASH_SOURCE[0]}")/secondmate-helpers.sh" # The move is delegated to `tasks-axi mv`, so this suite exercises the real -# binary. Skip cleanly when it is absent (matching the backend smoke suites). -command -v tasks-axi >/dev/null 2>&1 || { echo "skip: tasks-axi not found (required by the delegated handoff path)"; exit 0; } +# binary. Skip cleanly when it is absent or too old for atomic multi-ID moves +# (matching the lifecycle e2e's handoff phase). +# shellcheck source=bin/fm-tasks-axi-lib.sh disable=SC1091 +. "$ROOT/bin/fm-tasks-axi-lib.sh" +fm_tasks_axi_compatible || { echo "skip: tasks-axi not found or too old (required by the delegated handoff path)"; exit 0; } TMP_ROOT=$(fm_test_tmproot fm-backlog-handoff) diff --git a/tests/fm-bootstrap.test.sh b/tests/fm-bootstrap.test.sh index 2b3399e58..4eb5fb318 100755 --- a/tests/fm-bootstrap.test.sh +++ b/tests/fm-bootstrap.test.sh @@ -82,6 +82,42 @@ SH chmod +x "$fakebin/quota-axi" } +# Fake systemctl for the critical-services check. It shadows any real systemctl +# on PATH so the test is deterministic on a systemd host. FM_FAKE_FAILED_UNITS is +# a space-separated set of units reported as failed; FM_FAKE_SERVICE_TS is the +# StateChangeTimestamp value returned for `show` (empty = unavailable). is-failed +# prints its state to stdout (as the real one does without --quiet), so a case +# expecting silence also proves the production call redirects that stream. +add_fake_systemctl() { + local fakebin=$1 + cat > "$fakebin/systemctl" <<'SH' +#!/usr/bin/env bash +case "${1:-}" in + is-failed) + shift + unit= + for a in "$@"; do + case "$a" in --*) ;; *) unit=$a ;; esac + done + for f in ${FM_FAKE_FAILED_UNITS:-}; do + if [ "$f" = "$unit" ]; then + printf '%s\n' failed + exit 0 + fi + done + printf '%s\n' inactive + exit 1 + ;; + show) + printf '%s\n' "${FM_FAKE_SERVICE_TS:-}" + exit 0 + ;; +esac +exit 0 +SH + chmod +x "$fakebin/systemctl" +} + add_tasks_axi() { local fakebin=$1 version=$2 archive_body=${3:-yes} multi_id=${4:-yes} archive_line mv_usage archive_line="" @@ -692,12 +728,296 @@ empty array use is flagged^{"rules":[{"when":"big feature","use":[]}]}^exact^CRE array profile without harness is flagged^{"rules":[{"when":"big feature","use":[{"model":"gpt-5.5"}]}]}^exact^CREW_DISPATCH: invalid config/crew-dispatch.json - each use profile needs harness unknown select is flagged^{"rules":[{"when":"big feature","use":[{"harness":"claude"},{"harness":"codex"}],"select":"mystery"}]}^exact^CREW_DISPATCH: invalid config/crew-dispatch.json - unknown select: mystery array profile unsupported effort is flagged^{"rules":[{"when":"big feature","use":[{"harness":"codex","effort":"max"}]}]}^exact^CREW_DISPATCH: invalid config/crew-dispatch.json - invalid effort: codex:max +non-boolean ultracode is flagged^{"rules":[{"when":"safety work","use":{"harness":"claude","ultracode":"true"}}]}^exact^CREW_DISPATCH: invalid config/crew-dispatch.json - ultracode must be boolean +non-string ultracode role is flagged^{"rules":[{"when":"safety work","use":{"harness":"claude","ultracode":true,"ultracode_role":true}}]}^exact^CREW_DISPATCH: invalid config/crew-dispatch.json - ultracode_role must be a string +unsafe ultracode role is flagged^{"rules":[{"when":"safety work","use":{"harness":"claude","ultracode":true,"ultracode_role":"independent-review\nreviewed_by=task-x2"}}]}^exact^CREW_DISPATCH: invalid config/crew-dispatch.json - ultracode_role must be a safe token ROWS pass "bootstrap validates crew-dispatch.json and reports malformed or unverified configs" } +# A home whose backlog backend is manual (suppresses TASKS_AXI) with a fully +# present toolchain and a fake systemctl, so SERVICE_FAILED lines are the only +# possible bootstrap output. Prints the fakebin path. +setup_critical_case() { + local case_dir=$1 fakebin + mkdir -p "$case_dir/home/config" + printf '%s\n' manual > "$case_dir/home/config/backlog-backend" + fakebin=$(make_fake_toolchain "$case_dir") + add_fake_systemctl "$fakebin" + printf '%s\n' "$fakebin" +} + +run_critical_bootstrap() { + local home=$1 fakebin=$2 failed=$3 ts=$4 + PATH="$fakebin:$BASE_PATH" FM_HOME="$home" FM_ROOT_OVERRIDE="$home" \ + FM_FAKE_TREEHOUSE_LEASE_HELP=1 \ + FM_FAKE_FAILED_UNITS="$failed" FM_FAKE_SERVICE_TS="$ts" \ + "$ROOT/bin/fm-bootstrap.sh" +} + +test_critical_services_check() { + local case_dir home fakebin out ts expect bash_env + ts='Wed 2026-07-01 03:14:07 UTC' + + # A failed unit with a StateChangeTimestamp surfaces with the since clause. + case_dir="$TMP_ROOT/critical-failed-ts" + home="$case_dir/home" + fakebin=$(setup_critical_case "$case_dir") + printf '%s\n' 'my-app.service' > "$home/config/critical-services" + out=$(run_critical_bootstrap "$home" "$fakebin" 'my-app.service' "$ts") + [ "$out" = "SERVICE_FAILED: my-app.service - failed since $ts" ] \ + || fail "failed unit with timestamp: got: $out" + + # A failed unit with no StateChangeTimestamp drops the since clause. + case_dir="$TMP_ROOT/critical-failed-no-ts" + home="$case_dir/home" + fakebin=$(setup_critical_case "$case_dir") + printf '%s\n' 'my-app.service' > "$home/config/critical-services" + out=$(run_critical_bootstrap "$home" "$fakebin" 'my-app.service' '') + [ "$out" = "SERVICE_FAILED: my-app.service - in failed state" ] \ + || fail "failed unit without timestamp: got: $out" + + # A listed unit that is not failed stays silent (also proves is-failed's stdout + # is redirected: the fake would otherwise leak 'inactive' into the output). + case_dir="$TMP_ROOT/critical-not-failed" + home="$case_dir/home" + fakebin=$(setup_critical_case "$case_dir") + printf '%s\n' 'my-app.service' > "$home/config/critical-services" + out=$(run_critical_bootstrap "$home" "$fakebin" 'other.service' "$ts") + [ -z "$out" ] || fail "not-failed unit should be silent, got: $out" + + # Blank lines and full-line '#' comments are ignored, whitespace is trimmed, + # and only the listed units are reported, in file order. + case_dir="$TMP_ROOT/critical-comments" + home="$case_dir/home" + fakebin=$(setup_critical_case "$case_dir") + printf '%s\n' '# critical units' '' ' spaced.service ' 'ok.timer' '#skip.service' 'down.service' \ + > "$home/config/critical-services" + out=$(run_critical_bootstrap "$home" "$fakebin" 'spaced.service down.service' "$ts") + expect=$'SERVICE_FAILED: spaced.service - failed since '"$ts"$'\nSERVICE_FAILED: down.service - failed since '"$ts" + [ "$out" = "$expect" ] || fail "comment/whitespace handling: got: $out" + + # An absent config file is a no-op even with systemctl present. + case_dir="$TMP_ROOT/critical-absent" + home="$case_dir/home" + fakebin=$(setup_critical_case "$case_dir") + out=$(run_critical_bootstrap "$home" "$fakebin" 'my-app.service' "$ts") + [ -z "$out" ] || fail "absent config should be a no-op, got: $out" + + # Without systemctl on PATH (a non-systemd host) the check is a silent no-op + # even when the config lists a unit. Override `command -v systemctl` so the + # host's real systemctl cannot satisfy the guard. + case_dir="$TMP_ROOT/critical-no-systemctl" + home="$case_dir/home" + fakebin=$(setup_critical_case "$case_dir") + rm -f "$fakebin/systemctl" + printf '%s\n' 'my-app.service' > "$home/config/critical-services" + bash_env="$case_dir/no-systemctl.bash" + cat > "$bash_env" <<'SH' +command() { + if [ "${1:-}" = -v ] && [ "${2:-}" = systemctl ]; then + return 1 + fi + builtin command "$@" +} +SH + out=$(PATH="$fakebin:$BASE_PATH" BASH_ENV="$bash_env" FM_HOME="$home" FM_ROOT_OVERRIDE="$home" \ + FM_FAKE_TREEHOUSE_LEASE_HELP=1 FM_FAKE_FAILED_UNITS='my-app.service' FM_FAKE_SERVICE_TS="$ts" \ + "$ROOT/bin/fm-bootstrap.sh") + [ -z "$out" ] || fail "no systemctl should be a silent no-op, got: $out" + + pass "bootstrap surfaces failed critical services and stays silent otherwise" +} + +# --- UPSTREAM_DRIFT diagnostic ------------------------------------------------- +# The always-on drift report reads firstmate's OWN repo (FM_ROOT) and needs a real +# git repo with a `main` branch, an `upstream` remote, and a refs/remotes/upstream/main +# ref - so these cases build one per scenario instead of the fake toolchain's +# non-git home. It reads the ref WITHOUT fetching, so a dummy upstream URL is fine. +make_drift_repo() { # + local repo=$1 + git init -q -b main "$repo" + git -C "$repo" config user.email t@t.test + git -C "$repo" config user.name tester +} + +drift_commit() { # [] + local repo=$1 msg=$2 date=${3:-} + if [ -n "$date" ]; then + GIT_AUTHOR_DATE="$date" GIT_COMMITTER_DATE="$date" \ + git -C "$repo" commit -q --allow-empty -m "$msg" + else + git -C "$repo" commit -q --allow-empty -m "$msg" + fi +} + +# Run bootstrap in detect-only mode (the drift report runs there and never +# fetches) against a real git FM_ROOT, returning only the UPSTREAM_DRIFT line. +run_drift_bootstrap() { # + local repo=$1 fakebin=$2 + PATH="$fakebin:$BASE_PATH" FM_HOME="$repo" FM_ROOT_OVERRIDE="$repo" \ + FM_FAKE_TREEHOUSE_LEASE_HELP=1 FM_BOOTSTRAP_DETECT_ONLY=1 \ + "$ROOT/bin/fm-bootstrap.sh" 2>/dev/null | grep '^UPSTREAM_DRIFT:' || true +} + +test_upstream_drift_fetch_uses_gtimeout_when_timeout_absent() { + local case_dir repo fakebin real_git log bash_env out + case_dir="$TMP_ROOT/drift-fetch-gtimeout" + repo="$case_dir/repo" + mkdir -p "$case_dir" + make_drift_repo "$repo" + drift_commit "$repo" base + git -C "$repo" remote add upstream https://example.invalid/upstream.git + mkdir -p "$repo/config" + printf '%s\n' manual > "$repo/config/backlog-backend" + fakebin=$(make_fake_toolchain "$case_dir") + real_git=$(command -v git) + log="$case_dir/fetch.log" + cat > "$fakebin/git" <> '$log' + exit 42 +fi +exec '$real_git' ${repo:+-C "\$repo"} "\$@" +SH + chmod +x "$fakebin/git" + cat > "$fakebin/gtimeout" <> '$log' +exit 0 +SH + chmod +x "$fakebin/gtimeout" + bash_env="$case_dir/no-timeout.bash" + cat > "$bash_env" <<'SH' +command() { + if [ "${1:-}" = -v ] && [ "${2:-}" = timeout ]; then + return 1 + fi + builtin command "$@" +} +SH + + out=$(PATH="$fakebin:$BASE_PATH" BASH_ENV="$bash_env" FM_HOME="$repo" FM_ROOT_OVERRIDE="$repo" \ + FM_FAKE_TREEHOUSE_LEASE_HELP=1 FM_UPSTREAM_FETCH_TIMEOUT=7 "$ROOT/bin/fm-bootstrap.sh") + assert_contains "$(cat "$log" 2>/dev/null || true)" "gtimeout 7 git -C $repo fetch --quiet upstream" \ + "upstream drift fetch should use gtimeout when timeout is absent" + assert_not_contains "$(cat "$log" 2>/dev/null || true)" "raw-fetch" \ + "upstream drift fetch should not fall back to an unbounded raw git fetch when gtimeout exists" + assert_not_contains "$out" "MISSING:" "gtimeout fetch case should use the complete fake toolchain" + pass "bootstrap bounds upstream drift fetch with gtimeout when GNU timeout is absent" +} + +test_upstream_drift_fetch_skips_when_no_timeout_tool_exists() { + local case_dir repo fakebin real_git log bash_env out + case_dir="$TMP_ROOT/drift-fetch-no-timeout" + repo="$case_dir/repo" + mkdir -p "$case_dir" + make_drift_repo "$repo" + drift_commit "$repo" base + git -C "$repo" remote add upstream https://example.invalid/upstream.git + mkdir -p "$repo/config" + printf '%s\n' manual > "$repo/config/backlog-backend" + fakebin=$(make_fake_toolchain "$case_dir") + real_git=$(command -v git) + log="$case_dir/fetch.log" + cat > "$fakebin/git" <> '$log' + exit 42 +fi +exec '$real_git' ${repo:+-C "\$repo"} "\$@" +SH + chmod +x "$fakebin/git" + bash_env="$case_dir/no-timeout-tools.bash" + cat > "$bash_env" <<'SH' +command() { + if [ "${1:-}" = -v ] && { [ "${2:-}" = timeout ] || [ "${2:-}" = gtimeout ]; }; then + return 1 + fi + builtin command "$@" +} +SH + + out=$(PATH="$fakebin:$BASE_PATH" BASH_ENV="$bash_env" FM_HOME="$repo" FM_ROOT_OVERRIDE="$repo" \ + FM_FAKE_TREEHOUSE_LEASE_HELP=1 FM_UPSTREAM_FETCH_TIMEOUT=7 "$ROOT/bin/fm-bootstrap.sh") + [ ! -e "$log" ] || fail "upstream drift fetch ran without timeout/gtimeout: $(cat "$log")" + assert_not_contains "$out" "MISSING:" "no-timeout fetch case should use the complete fake toolchain" + pass "bootstrap skips upstream drift fetch instead of running it unbounded when no timeout tool exists" +} + +test_upstream_drift_report() { + local case_dir repo fakebin out base tip i + + # FYI variant: local main ahead, not behind, recent merge-base -> plain report. + case_dir="$TMP_ROOT/drift-fyi"; repo="$case_dir/repo" + mkdir -p "$case_dir"; make_drift_repo "$repo" + drift_commit "$repo" base + base=$(git -C "$repo" rev-parse HEAD) + git -C "$repo" remote add upstream https://example.invalid/upstream.git + git -C "$repo" update-ref refs/remotes/upstream/main "$base" + drift_commit "$repo" a1; drift_commit "$repo" a2; drift_commit "$repo" a3 + fakebin=$(make_fake_toolchain "$case_dir") + out=$(run_drift_bootstrap "$repo" "$fakebin") + assert_contains "$out" "UPSTREAM_DRIFT: local main is 3 ahead / 0 behind upstream/main" \ + "FYI drift line reports ahead/behind counts" + assert_not_contains "$out" "needs attention" "FYI variant must not escalate" + + # Escalation via a stale merge-base (older than the 10-day threshold). + case_dir="$TMP_ROOT/drift-stale"; repo="$case_dir/repo" + mkdir -p "$case_dir"; make_drift_repo "$repo" + drift_commit "$repo" base "2020-01-01T00:00:00" + base=$(git -C "$repo" rev-parse HEAD) + git -C "$repo" remote add upstream https://example.invalid/upstream.git + git -C "$repo" update-ref refs/remotes/upstream/main "$base" + drift_commit "$repo" recent + fakebin=$(make_fake_toolchain "$case_dir") + out=$(run_drift_bootstrap "$repo" "$fakebin") + assert_contains "$out" "UPSTREAM_DRIFT: this repo's upstream sync needs attention" \ + "a stale merge-base escalates the wording" + assert_contains "$out" "1 ahead" "stale case still reports the ahead count" + + # Escalation via being more than 30 commits behind upstream/main. + case_dir="$TMP_ROOT/drift-behind"; repo="$case_dir/repo" + mkdir -p "$case_dir"; make_drift_repo "$repo" + drift_commit "$repo" base + git -C "$repo" checkout -q -b upstream-sim + i=1; while [ "$i" -le 31 ]; do drift_commit "$repo" "u$i"; i=$((i + 1)); done + tip=$(git -C "$repo" rev-parse HEAD) + git -C "$repo" checkout -q main + git -C "$repo" remote add upstream https://example.invalid/upstream.git + git -C "$repo" update-ref refs/remotes/upstream/main "$tip" + git -C "$repo" branch -q -D upstream-sim + fakebin=$(make_fake_toolchain "$case_dir") + out=$(run_drift_bootstrap "$repo" "$fakebin") + assert_contains "$out" "needs attention" "being far behind upstream escalates the wording" + assert_contains "$out" "31 behind" "far-behind case reports the behind count" + + # No upstream remote -> silent no-op (no ref to count against). + case_dir="$TMP_ROOT/drift-none"; repo="$case_dir/repo" + mkdir -p "$case_dir"; make_drift_repo "$repo" + drift_commit "$repo" base + fakebin=$(make_fake_toolchain "$case_dir") + out=$(run_drift_bootstrap "$repo" "$fakebin") + [ -z "$out" ] || fail "no upstream remote must be silent, got: $out" + + pass "bootstrap reports upstream drift (FYI, stale, and far-behind escalations) and stays silent without an upstream" +} + test_bootstrap_reporting test_no_mistakes_min_version +test_upstream_drift_fetch_uses_gtimeout_when_timeout_absent +test_upstream_drift_fetch_skips_when_no_timeout_tool_exists +test_upstream_drift_report test_git_is_required_with_supported_install_instruction test_orca_backend_gates_orca_tool_only_when_selected test_session_provider_backends_do_not_require_tmux @@ -714,3 +1034,4 @@ test_fleet_sync_timeout_empty_override_uses_default test_fleet_sync_timeout_is_computed_before_launch test_crew_dispatch_active_rules_are_surfaced test_crew_dispatch_validation +test_critical_services_check diff --git a/tests/fm-fleet-sync.test.sh b/tests/fm-fleet-sync.test.sh index b1fcd0a38..7122f3fcf 100755 --- a/tests/fm-fleet-sync.test.sh +++ b/tests/fm-fleet-sync.test.sh @@ -124,6 +124,27 @@ build_packed_prunable() { plant_packed_refs_lock() { : > "$1/.git/packed-refs.lock"; } +# fetch_with_packed_refs_lock_guard only engages when the blocked prune makes +# `git fetch --prune` itself exit non-zero; git started propagating a failed +# prune into fetch's exit code in 2.37, while older git prints the +# packed-refs.lock error but exits 0, so the guard (correctly) never fires and +# the real-lock tests below cannot pass. Probe the REAL behavior rather than +# guessing a version cutoff, once, with a throwaway fixture. +fm_git_fetch_prune_failure_is_fatal() { + if [ -z "${FM_FETCH_PRUNE_FATAL:-}" ]; then + local home clone + home=$(new_home) + clone=$(build_packed_prunable "$home" prune-probe) + plant_packed_refs_lock "$clone" + if git -C "$clone" fetch origin --prune --quiet 2>/dev/null; then + FM_FETCH_PRUNE_FATAL=no + else + FM_FETCH_PRUNE_FATAL=yes + fi + fi + [ "$FM_FETCH_PRUNE_FATAL" = yes ] +} + # lsof shims mirror tests/fm-teardown.test.sh: no-holder (provably free), a live # holder, and an lsof error. Written into a per-home fakebin/ prepended to PATH. lsof_no_holder() { @@ -474,6 +495,10 @@ test_bootstrap_relays_recovered_and_stuck() { test_orphaned_stale_packed_refs_lock_recovers() { local home fakebin clone out err + fm_git_fetch_prune_failure_is_fatal || { + echo "skip: this git does not fail 'fetch --prune' on a blocked prune (needs git 2.37+)" + return 0 + } home=$(new_home) fakebin="$home/fb-lockstale"; rm -rf "$fakebin"; mkdir -p "$fakebin" clone=$(build_packed_prunable "$home" lockstale) @@ -503,6 +528,10 @@ test_orphaned_stale_packed_refs_lock_recovers() { test_live_packed_refs_lock_is_never_removed() { local home fakebin clone out err before + fm_git_fetch_prune_failure_is_fatal || { + echo "skip: this git does not fail 'fetch --prune' on a blocked prune (needs git 2.37+)" + return 0 + } home=$(new_home) fakebin="$home/fb-locklive"; rm -rf "$fakebin"; mkdir -p "$fakebin" clone=$(build_packed_prunable "$home" locklive) @@ -529,6 +558,10 @@ test_live_packed_refs_lock_is_never_removed() { test_live_git_cwd_in_clone_dir_blocks_removal() { local home fakebin clone out err before + fm_git_fetch_prune_failure_is_fatal || { + echo "skip: this git does not fail 'fetch --prune' on a blocked prune (needs git 2.37+)" + return 0 + } home=$(new_home) fakebin="$home/fb-lockcwd"; rm -rf "$fakebin"; mkdir -p "$fakebin" clone=$(build_packed_prunable "$home" lockcwd) diff --git a/tests/fm-gotmp.test.sh b/tests/fm-gotmp.test.sh index 6b6fd8aa1..fe3dc6bfc 100755 --- a/tests/fm-gotmp.test.sh +++ b/tests/fm-gotmp.test.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Behavior tests for per-task GOTMPDIR support (fm-gotmp). # -# fm-spawn gives each task a temp root /tmp/fm-/ with Go's build temp nested at +# fm-spawn gives each task a temp root /tmp/fm-./ with Go's build temp nested at # gotmp/, exports GOTMPDIR into the crewmate pane, and records tasktmp= in the task's # meta. fm-teardown reads tasktmp= and removes the whole root on cleanup. # @@ -31,8 +31,13 @@ pass() { } TMP_ROOT= +EXTRA_CLEANUP_DIRS=() cleanup() { + local d + for d in "${EXTRA_CLEANUP_DIRS[@]:-}"; do + [ -n "$d" ] && rm -rf "$d" + done if [ -n "${TMP_ROOT:-}" ]; then rm -rf "$TMP_ROOT" fi @@ -75,7 +80,14 @@ SH exit 0 SH chmod +x "$fake/bin/fm-fleet-sync.sh" - # fm-tasks-axi-lib.sh: stub (teardown sources it). Report no backend so + # fm-nas-deploy-sync.sh: stub (called for the same non-scout/non-local-only + # teardowns, right after fm-fleet-sync.sh). + cat > "$fake/bin/fm-nas-deploy-sync.sh" <<'SH' +#!/usr/bin/env bash +exit 0 +SH + chmod +x "$fake/bin/fm-nas-deploy-sync.sh" + # fm-tasks-axi-lib.sh: stub (teardown sources it). Report not-compatible so # backlog_refresh_reminder takes the plain-message path; no tasks-axi here. cat > "$fake/bin/fm-tasks-axi-lib.sh" <<'SH' fm_tasks_axi_backend_available() { return 1; } @@ -102,6 +114,16 @@ test_spawn_contract_and_mkdir_pattern() { # shellcheck disable=SC2016 # single quotes are deliberate: these are literal source strings grep -F 'mkdir -p "$TASK_TMP/gotmp"' "$SPAWN" >/dev/null \ || fail "fm-spawn missing: mkdir of gotmp under TASK_TMP" + # shellcheck disable=SC2016 + grep -F 'TASK_TMP=$(mktemp -d "/tmp/fm-$ID.XXXXXX")' "$SPAWN" >/dev/null \ + || fail "fm-spawn missing: unique mktemp task root" + # shellcheck disable=SC2016 + grep -F 'cleanup_task_tmp_on_spawn_error "$status"' "$SPAWN" >/dev/null \ + || fail "fm-spawn missing: temporary cleanup in spawn exit trap" + grep -F 'trap spawn_exit_cleanup EXIT' "$SPAWN" >/dev/null \ + || fail "fm-spawn missing: shared spawn exit cleanup trap" + grep -F 'SPAWN_TASK_TMP_CLEANUP=0' "$SPAWN" >/dev/null \ + || fail "fm-spawn missing: cleanup trap disarm after meta write" # shellcheck disable=SC2016 # single quotes are deliberate: literal source string grep -F 'echo "tasktmp=$TASK_TMP"' "$SPAWN" >/dev/null \ || fail "fm-spawn missing: tasktmp= line in meta write" @@ -111,10 +133,11 @@ test_spawn_contract_and_mkdir_pattern() { # a meta line whose value the teardown grep (tasktmp=, cut -d= -f2-) reads back whole. local id=spawn-sim-z1 local sim_root="$TMP_ROOT/$id-root" - local task_tmp="$sim_root/tmp/fm-$id" + local task_tmp mkdir -p "$sim_root/state" - # Replicate spawn's exact mkdir + meta-write lines. - TASK_TMP="$task_tmp" + TASK_TMP=$(mktemp -d "/tmp/fm-$id.XXXXXX") || fail "mktemp failed" + task_tmp=$TASK_TMP + EXTRA_CLEANUP_DIRS+=("$task_tmp") mkdir -p "$TASK_TMP/gotmp" { echo "tasktmp=$TASK_TMP" @@ -125,6 +148,10 @@ test_spawn_contract_and_mkdir_pattern() { read_back=$(grep '^tasktmp=' "$sim_root/state/$id.meta" | cut -d= -f2-) [ "$read_back" = "$task_tmp" ] \ || fail "tasktmp value not round-tripped by teardown's grep|cut (got '$read_back')" + case "$task_tmp" in + /tmp/fm-$id.*) ;; + *) fail "tasktmp did not include a unique /tmp/fm- prefix ($task_tmp)" ;; + esac pass "fm-spawn creates gotmp dir and records tasktmp in meta" } @@ -132,7 +159,9 @@ test_spawn_contract_and_mkdir_pattern() { test_teardown_removes_tasktmp_dir() { local id=td-rm-z2 - local task_tmp="$TMP_ROOT/fm-$id" + local task_tmp + task_tmp=$(mktemp -d "/tmp/fm-$id.XXXXXX") || fail "mktemp failed" + EXTRA_CLEANUP_DIRS+=("$task_tmp") mkdir -p "$task_tmp/gotmp" printf 'leftover\n' > "$task_tmp/gotmp/build-artifact" local fake @@ -171,6 +200,11 @@ SH exit 0 SH chmod +x "$fake/bin/fm-fleet-sync.sh" + cat > "$fake/bin/fm-nas-deploy-sync.sh" <<'SH' +#!/usr/bin/env bash +exit 0 +SH + chmod +x "$fake/bin/fm-nas-deploy-sync.sh" cat > "$fake/bin/fm-tasks-axi-lib.sh" <<'SH' fm_tasks_axi_backend_available() { return 1; } SH @@ -189,10 +223,26 @@ META pass "fm-teardown skips gracefully when tasktmp= is absent (backward compat)" } +test_teardown_removes_legacy_tasktmp_dir() { + local id=td-legacy-z6 + local task_tmp="/tmp/fm-$id" + rm -rf "$task_tmp" + mkdir -p "$task_tmp/gotmp" + EXTRA_CLEANUP_DIRS+=("$task_tmp") + local fake + fake=$(make_fake_root "$id" "$task_tmp") + + FM_HOME="$fake" bash "$fake/bin/fm-teardown.sh" "$id" >/dev/null 2>&1 \ + || fail "teardown exited non-zero with a legacy tasktmp" + [ ! -e "$task_tmp" ] \ + || fail "teardown did not remove the legacy deterministic tasktmp dir" + pass "fm-teardown removes legacy deterministic tasktmp roots" +} + test_teardown_skips_gracefully_when_dir_missing() { # tasktmp= points to a path that does not exist. Teardown must not error. local id=td-missing-z4 - local task_tmp="$TMP_ROOT/never-created-fm-$id" + local task_tmp="/tmp/fm-$id.missing" # Intentionally do NOT create $task_tmp. [ ! -e "$task_tmp" ] || fail "precondition: task_tmp should not exist yet" local fake @@ -203,7 +253,22 @@ test_teardown_skips_gracefully_when_dir_missing() { pass "fm-teardown skips gracefully when tasktmp= points to a nonexistent dir" } +test_teardown_skips_unsafe_tasktmp_path() { + local id=td-unsafe-z5 + local task_tmp="$TMP_ROOT/must-not-delete" + mkdir -p "$task_tmp" + printf 'keep\n' > "$task_tmp/sentinel" + local fake + fake=$(make_fake_root "$id" "$task_tmp") + FM_HOME="$fake" bash "$fake/bin/fm-teardown.sh" "$id" >/dev/null 2>&1 \ + || fail "teardown exited non-zero when tasktmp was unsafe" + [ -e "$task_tmp/sentinel" ] || fail "teardown removed an unsafe tasktmp path" + pass "fm-teardown skips unsafe tasktmp paths from mutable meta" +} + test_spawn_contract_and_mkdir_pattern test_teardown_removes_tasktmp_dir test_teardown_skips_gracefully_without_tasktmp +test_teardown_removes_legacy_tasktmp_dir test_teardown_skips_gracefully_when_dir_missing +test_teardown_skips_unsafe_tasktmp_path diff --git a/tests/fm-nas-deploy-sync.test.sh b/tests/fm-nas-deploy-sync.test.sh new file mode 100755 index 000000000..07f7ff7b5 --- /dev/null +++ b/tests/fm-nas-deploy-sync.test.sh @@ -0,0 +1,714 @@ +#!/usr/bin/env bash +# Behavior tests for fm-nas-deploy-sync.sh. +# +# A merged PR does not, by itself, reach the live site: every deployed app runs +# from its own separate checkout under /mnt/nas/experiments//, managed by +# pm2, independent from the projects/ clone firstmate/crewmates develop in +# (see the 2026-07-04 entry in data/learnings.md). fm-nas-deploy-sync.sh closes +# that gap: fast-forward the NAS checkout, restart and verify the pm2 +# process(es) that serve it - mirroring bin/fm-fleet-sync.sh's dirty/diverged +# safety exactly (never force, never touch a dirty or diverged checkout). +# +# This suite never touches a real /mnt/nas/experiments/ path or a real pm2 +# process: NAS checkouts are throwaway git repos under a per-test tmp home, and +# pm2 is a PATH-shimmed stub that records restarts and reports them online. +# +# Covers: +# - a clean, behind checkout fast-forwards and its pm2 process(es) restart and +# verify online +# - a dirty checkout is left completely untouched (reported, not fast-forwarded) +# - a diverged checkout is reported STUCK-style and left completely untouched +# - a project absent from data/nas-deployments.md (or the file itself absent) +# is a silent no-op: one skip line, exit 0 +# - bin/fm-teardown.sh's post-landing hook actually invokes the real script for +# a landed ship task, end to end (the NAS fixture fast-forwards and its pm2 +# process is restarted as a side effect of running teardown) +set -u + +# shellcheck source=tests/lib.sh disable=SC1091 +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" +fm_git_identity fmtest fmtest@example.invalid + +SYNC="$ROOT/bin/fm-nas-deploy-sync.sh" +TEARDOWN="$ROOT/bin/fm-teardown.sh" +TMP_ROOT=$(fm_test_tmproot fm-nas-deploy-sync-tests) +HOME_N=0 + +# --- generic fixtures -------------------------------------------------------- + +new_home() { + HOME_N=$((HOME_N + 1)) + local h="$TMP_ROOT/home-$HOME_N" + mkdir -p "$h/data" "$h/fakebin" + printf '%s\n' "$h" +} + +commit_file() { + local dir=$1 file=$2 content=$3 msg=$4 + printf '%s\n' "$content" > "$dir/$file" + git -C "$dir" add -- "$file" + git -C "$dir" commit -qm "$msg" +} + +# build_nas_pair : a bare "remote" origin plus a "work" repo wired to +# it (for advancing origin later) and a plain clone standing in for the NAS +# checkout, all with one commit on main. Echoes the NAS checkout path. +build_nas_pair() { + local home=$1 name=$2 work remote nas remote_abs + work="$home/work-$name" + remote="$home/remotes/$name.git" + nas="$home/nas-$name" + mkdir -p "$home/remotes" + + git init -q "$work" + git -C "$work" symbolic-ref HEAD refs/heads/main + commit_file "$work" file.txt v0 C0 + + git clone --quiet --bare "$work" "$remote" + remote_abs=$(cd "$remote" && pwd) + git -C "$work" remote add origin "file://$remote_abs" + git -C "$work" push -q -u origin main + + git clone --quiet "file://$remote_abs" "$nas" + printf '%s\n' "$nas" +} + +# advance_origin : push one more commit to 's origin via +# its work repo, so the NAS checkout (until synced) is behind origin/main. +advance_origin() { + local home=$1 name=$2 msg=$3 work + work="$home/work-$name" + commit_file "$work" file.txt "$msg" "$msg" + git -C "$work" push -q origin main +} + +head_sha() { git -C "$1" rev-parse HEAD; } + +# write_map : (re)write a one-row +# nas-deployments.md pipe table at $home/data/nas-deployments.md. +write_map() { + local home=$1 project=$2 nas_path=$3 procs=$4 + { + printf '| project | nas_repo_path | pm2_process(es) |\n' + printf '| --- | --- | --- |\n' + printf '| %s | %s | %s |\n' "$project" "$nas_path" "$procs" + } > "$home/data/nas-deployments.md" +} + +# write_pm2_stub : `pm2 restart ` appends to +# $restart_log and succeeds; `pm2 jlist` reports every logged name as online, so a +# process comes back "online" exactly when (and only when) it was restarted. +write_pm2_stub() { + local fakebin=$1 log=$2 + : > "$log" + cat > "$fakebin/pm2" <> "$log" + exit 0 + ;; + jlist) + printf '[' + first=1 + while IFS= read -r name; do + [ -n "\$name" ] || continue + [ "\$first" = 1 ] || printf ',' + first=0 + printf '{"name":"%s","pm2_env":{"status":"online"}}' "\$name" + done < "$log" + printf ']\n' + ;; + *) exit 0 ;; +esac +SH + chmod +x "$fakebin/pm2" +} + +# write_pm2_cluster_stub : like write_pm2_stub, but +# `pm2 jlist` always reports TWO instances of - one online, one stopped - +# regardless of what was restarted, simulating a clustered pm2 app where one +# instance in the cluster never comes back after a restart. +write_pm2_cluster_stub() { + local fakebin=$1 log=$2 proc=$3 + : > "$log" + cat > "$fakebin/pm2" <> "$log" + exit 0 + ;; + jlist) + printf '[{"name":"%s","pm2_env":{"status":"online"}},{"name":"%s","pm2_env":{"status":"stopped"}}]\n' "$proc" "$proc" + ;; + *) exit 0 ;; +esac +SH + chmod +x "$fakebin/pm2" +} + +write_pm2_hung_restart_stub() { + local fakebin=$1 log=$2 + : > "$log" + cat > "$fakebin/pm2" <> "$log" + exit 0 + ;; + jlist) + printf '[]\n' + ;; + *) exit 0 ;; +esac +SH + chmod +x "$fakebin/pm2" +} + +# run_sync [args...]: run the sync script against an isolated home. +run_sync() { + local home=$1 + shift + FM_HOME="$home" PATH="$home/fakebin:$PATH" "$SYNC" "$@" +} + +make_no_timeout_toolbin() { + local dir=$1 tb="$1/notimeoutbin" tool real + mkdir -p "$tb" + for tool in bash git awk sed grep jq sleep rm dirname; do + real=$(command -v "$tool" || true) + [ -n "$real" ] || fail "missing tool for no-timeout path: $tool" + ln -s "$real" "$tb/$tool" + done + printf '%s\n' "$tb" +} + +run_sync_no_timeout() { + local home=$1 toolbin + shift + toolbin=$(make_no_timeout_toolbin "$home") + FM_HOME="$home" PATH="$home/fakebin:$toolbin" "$SYNC" "$@" +} + +run_sync_without_tool() { + local home=$1 tool=$2 bash_env + shift 2 + bash_env="$home/no-$tool.bash" + cat > "$bash_env" <: any `fetch` call hangs indefinitely +# (well past FM_NAS_SYNC_TIMEOUT), so only the bounded wrapper's `timeout` kill +# can make the sync return promptly; every other call delegates to the real git. +write_git_hung_fetch_stub() { + local fakebin=$1 realgit + realgit=$(command -v git) + cat > "$fakebin/git" < : fail the FIRST +# `fetch` with the packed-refs.lock "File exists" signature, then delegate every +# later call - including the retried fetch - to the real git. Mirrors +# tests/fm-fleet-sync.test.sh's git_transient_packed_refs_lock, adapted to fake +# the stderr signature directly since fm-nas-deploy-sync.sh's fetch has no +# --prune to organically trigger a real packed-refs rewrite. +write_git_transient_packed_refs_lock_stub() { + local fakebin=$1 counter=$2 realgit + realgit=$(command -v git) + cat > "$fakebin/git" </dev/null || echo 0); n=\$(( n + 1 )) + printf '%s\n' "\$n" > "$counter" + if [ "\$n" -eq 1 ]; then + echo "error: could not delete reference refs/remotes/origin/feature: Unable to create '.git/packed-refs.lock': File exists." >&2 + exit 1 + fi +fi +exec "\$real" "\$@" +SH + chmod +x "$fakebin/git" +} + +write_git_status_failure_stub() { + local fakebin=$1 realgit + realgit=$(command -v git) + cat > "$fakebin/git" <&2 + exit 128 + fi +done +exec "\$real" "\$@" +SH + chmod +x "$fakebin/git" +} + +write_git_forbidden_stub() { + local fakebin=$1 log=$2 + cat > "$fakebin/git" <> "$log" +exit 99 +SH + chmod +x "$fakebin/git" +} + +# --- tests: standalone script ------------------------------------------------- + +test_clean_behind_fast_forwards_and_restarts() { + local home nas log out origin_head nas_head + home=$(new_home) + nas=$(build_nas_pair "$home" carscanner-test) + advance_origin "$home" carscanner-test C1 + write_map "$home" carscanner-test "$nas" carscanner-test + log="$home/restart.log" + write_pm2_stub "$home/fakebin" "$log" + + out=$(run_sync "$home" carscanner-test) + + assert_contains "$out" "carscanner-test: synced" "clean-behind: did not report a sync" + assert_contains "$out" "restarted carscanner-test and verified online" "clean-behind: did not report a verified restart" + origin_head=$(git -C "$home/work-carscanner-test" rev-parse main) + nas_head=$(head_sha "$nas") + [ "$origin_head" = "$nas_head" ] || fail "clean-behind: NAS checkout did not fast-forward to origin" + assert_grep carscanner-test "$log" "clean-behind: pm2 restart was not invoked" + pass "a clean, behind NAS checkout fast-forwards and its pm2 process restarts and verifies online" +} + +test_dirty_is_skipped_untouched() { + local home nas log out before after + home=$(new_home) + nas=$(build_nas_pair "$home" dirty-test) + advance_origin "$home" dirty-test C1 + before=$(head_sha "$nas") + printf 'uncommitted\n' >> "$nas/file.txt" + write_map "$home" dirty-test "$nas" dirty-test + log="$home/restart.log" + write_pm2_stub "$home/fakebin" "$log" + + out=$(run_sync "$home" dirty-test) + + assert_contains "$out" "STUCK" "dirty: did not report STUCK" + assert_contains "$out" "uncommitted changes" "dirty: did not name the dirty reason" + after=$(head_sha "$nas") + [ "$before" = "$after" ] || fail "dirty: NAS checkout HEAD moved despite being dirty" + git -C "$nas" diff --quiet -- file.txt && fail "dirty: uncommitted change was discarded" + [ ! -s "$log" ] || fail "dirty: pm2 restart was invoked on an untouched checkout" + pass "a dirty NAS checkout is reported STUCK and left completely untouched" +} + +test_diverged_is_stuck_untouched() { + local home nas log out before after + home=$(new_home) + nas=$(build_nas_pair "$home" diverged-test) + advance_origin "$home" diverged-test C1 + commit_file "$nas" local-only.txt local "local-only commit not on origin" + before=$(head_sha "$nas") + write_map "$home" diverged-test "$nas" diverged-test + log="$home/restart.log" + write_pm2_stub "$home/fakebin" "$log" + + out=$(run_sync "$home" diverged-test) + + assert_contains "$out" "STUCK" "diverged: did not report STUCK" + assert_contains "$out" "diverged" "diverged: did not name the diverged reason" + after=$(head_sha "$nas") + [ "$before" = "$after" ] || fail "diverged: NAS checkout HEAD moved despite diverging" + [ ! -s "$log" ] || fail "diverged: pm2 restart was invoked on an untouched checkout" + pass "a diverged NAS checkout is reported STUCK and left completely untouched" +} + +test_absent_project_is_silent_noop() { + local home nas out rc + home=$(new_home) + nas=$(build_nas_pair "$home" tracked-test) + write_map "$home" tracked-test "$nas" tracked-test + + set +e + out=$(run_sync "$home" untracked-project) + rc=$? + set -e + + expect_code 0 "$rc" "absent: sync should exit 0 for a project with no recorded deployment" + assert_contains "$out" "untracked-project: skipped: no recorded NAS deployment" "absent: did not report the expected skip line" + pass "a project absent from data/nas-deployments.md is a silent no-op" +} + +test_clustered_partial_restart_is_not_masked_online() { + local home nas log out + home=$(new_home) + nas=$(build_nas_pair "$home" cluster-test) + advance_origin "$home" cluster-test C1 + write_map "$home" cluster-test "$nas" cluster-test + log="$home/restart.log" + write_pm2_cluster_stub "$home/fakebin" "$log" cluster-test + + out=$(run_sync "$home" cluster-test) + + assert_contains "$out" "not online after restart" \ + "clustered: did not report the lagging cluster instance as a restart problem" + assert_contains "$out" "needs attention" "clustered: did not flag the deployment as needing attention" + case "$out" in + *"verified online"*) fail "clustered: a lagging cluster instance was masked as fully verified online" ;; + esac + pass "a clustered pm2 app with one lagging instance is reported as a restart problem, not masked as online" +} + +test_hung_fetch_is_bounded_by_timeout() { + local home nas out rc start elapsed + if ! command -v timeout >/dev/null 2>&1 && ! command -v gtimeout >/dev/null 2>&1; then + pass "SKIP (no timeout/gtimeout binary): hung-fetch timeout bound check" + return + fi + home=$(new_home) + nas=$(build_nas_pair "$home" hungfetch-test) + advance_origin "$home" hungfetch-test C1 + write_map "$home" hungfetch-test "$nas" hungfetch-test + write_pm2_stub "$home/fakebin" "$home/restart.log" + write_git_hung_fetch_stub "$home/fakebin" + + start=$SECONDS + set +e + out=$(FM_NAS_SYNC_TIMEOUT=1 run_sync "$home" hungfetch-test 2>/dev/null) + rc=$? + set -e + elapsed=$(( SECONDS - start )) + + expect_code 0 "$rc" "hung-fetch: sync should still exit 0 when the fetch hangs" + [ "$elapsed" -lt 10 ] || fail "hung-fetch: sync took ${elapsed}s - the timeout bound did not stop the hung fetch (stub sleeps 20s)" + assert_contains "$out" "hungfetch-test: skipped: fetch failed" "hung-fetch: did not report the bounded fetch as a failure" + pass "a hung NAS fetch is killed by FM_NAS_SYNC_TIMEOUT instead of blocking the caller" +} + +test_no_timeout_binary_skips_before_touching_nas() { + local home nas log out rc git_log + home=$(new_home) + nas=$(build_nas_pair "$home" notimeout-test) + advance_origin "$home" notimeout-test C1 + write_map "$home" notimeout-test "$nas" notimeout-test + log="$home/restart.log" + git_log="$home/git.log" + write_pm2_stub "$home/fakebin" "$log" + write_git_forbidden_stub "$home/fakebin" "$git_log" + + set +e + out=$(run_sync_no_timeout "$home" notimeout-test 2>/dev/null) + rc=$? + set -e + + expect_code 0 "$rc" "no-timeout: sync should still exit 0 when no timeout tool is available" + assert_contains "$out" "notimeout-test: skipped: no timeout/gtimeout binary available" "no-timeout: did not report the bounded-access skip" + [ ! -s "$git_log" ] || fail "no-timeout: touched the NAS checkout through git despite lacking a timeout tool" + [ ! -s "$log" ] || fail "no-timeout: pm2 restart was invoked despite lacking a timeout tool" + pass "fm-nas-deploy-sync skips before NAS access when no timeout binary is available" +} + +test_missing_pm2_skips_before_merge() { + local home nas log out before after + home=$(new_home) + nas=$(build_nas_pair "$home" missingpm2-test) + advance_origin "$home" missingpm2-test C1 + before=$(head_sha "$nas") + write_map "$home" missingpm2-test "$nas" missingpm2-test + log="$home/restart.log" + write_pm2_stub "$home/fakebin" "$log" + + out=$(run_sync_without_tool "$home" pm2 missingpm2-test) + + after=$(head_sha "$nas") + [ "$before" = "$after" ] || fail "missing-pm2: NAS checkout fast-forwarded before pm2 availability was proven" + [ ! -s "$log" ] || fail "missing-pm2: pm2 restart ran after missing pm2 prerequisite" + assert_contains "$out" "missingpm2-test: skipped: pm2 is unavailable" "missing-pm2: did not report the missing pm2 prerequisite" + pass "fm-nas-deploy-sync skips before merge when pm2 is unavailable" +} + +test_missing_jq_skips_before_merge() { + local home nas log out before after + home=$(new_home) + nas=$(build_nas_pair "$home" missingjq-test) + advance_origin "$home" missingjq-test C1 + before=$(head_sha "$nas") + write_map "$home" missingjq-test "$nas" missingjq-test + log="$home/restart.log" + write_pm2_stub "$home/fakebin" "$log" + + out=$(run_sync_without_tool "$home" jq missingjq-test) + + after=$(head_sha "$nas") + [ "$before" = "$after" ] || fail "missing-jq: NAS checkout fast-forwarded before jq availability was proven" + [ ! -s "$log" ] || fail "missing-jq: pm2 restart ran after missing jq prerequisite" + assert_contains "$out" "missingjq-test: skipped: jq is unavailable" "missing-jq: did not report the missing jq prerequisite" + pass "fm-nas-deploy-sync skips before merge when jq is unavailable" +} + +test_hung_pm2_restart_is_bounded_by_timeout() { + local home nas log out rc start elapsed + if ! command -v timeout >/dev/null 2>&1 && ! command -v gtimeout >/dev/null 2>&1; then + pass "SKIP (no timeout/gtimeout binary): hung-pm2 timeout bound check" + return + fi + home=$(new_home) + nas=$(build_nas_pair "$home" hungpm2-test) + advance_origin "$home" hungpm2-test C1 + write_map "$home" hungpm2-test "$nas" hungpm2-test + log="$home/restart.log" + rm -f "$home/fakebin/git" + write_pm2_hung_restart_stub "$home/fakebin" "$log" + + start=$SECONDS + set +e + out=$(FM_NAS_SYNC_TIMEOUT=1 run_sync "$home" hungpm2-test 2>/dev/null) + rc=$? + set -e + elapsed=$(( SECONDS - start )) + + expect_code 0 "$rc" "hung-pm2: sync should still exit 0 when pm2 restart hangs" + [ "$elapsed" -lt 3 ] || fail "hung-pm2: sync took ${elapsed}s - the timeout bound did not stop the hung pm2 restart" + assert_contains "$out" "hungpm2-test: synced" "hung-pm2: checkout should still fast-forward before bounded restart failure is reported" + assert_contains "$out" "restart failed" "hung-pm2: did not report the bounded restart failure" + pass "a hung pm2 restart is killed by FM_NAS_SYNC_TIMEOUT instead of blocking the caller" +} + +test_transient_packed_refs_lock_is_retried() { + local home nas out counter err + home=$(new_home) + nas=$(build_nas_pair "$home" locktrans-test) + advance_origin "$home" locktrans-test C1 + write_map "$home" locktrans-test "$nas" locktrans-test + write_pm2_stub "$home/fakebin" "$home/restart.log" + counter="$home/git-fetch-count"; : > "$counter" + write_git_transient_packed_refs_lock_stub "$home/fakebin" "$counter" + err="$home/err-locktrans" + + out=$(FM_NAS_SYNC_PACKED_REFS_LOCK_RETRIES=3 FM_NAS_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS=0 \ + run_sync "$home" locktrans-test 2>"$err") + + assert_contains "$out" "locktrans-test: synced" "transient lock: sync did not complete after the lock self-cleared" + assert_grep "cleared on its own" "$err" "transient lock: guard did not report the self-clear" + pass "a transient packed-refs.lock signature on the NAS fetch is retried instead of giving up immediately" +} + +test_status_failure_is_stuck_untouched() { + local home nas log out before after + home=$(new_home) + nas=$(build_nas_pair "$home" statusfail-test) + advance_origin "$home" statusfail-test C1 + before=$(head_sha "$nas") + write_map "$home" statusfail-test "$nas" statusfail-test + log="$home/restart.log" + write_pm2_stub "$home/fakebin" "$log" + write_git_status_failure_stub "$home/fakebin" + + out=$(run_sync "$home" statusfail-test 2>/dev/null) + + assert_contains "$out" "STUCK" "status-failure: did not report STUCK" + assert_contains "$out" "cannot inspect worktree status" "status-failure: did not name the inspection failure" + after=$(head_sha "$nas") + [ "$before" = "$after" ] || fail "status-failure: NAS checkout HEAD moved after status failed" + [ ! -s "$log" ] || fail "status-failure: pm2 restart was invoked after status failed" + pass "a NAS status inspection failure is reported STUCK and left untouched" +} + +test_missing_map_file_is_silent_noop() { + local home out rc + home=$(new_home) + rm -f "$home/data/nas-deployments.md" + + set +e + out=$(run_sync "$home" anything) + rc=$? + set -e + + expect_code 0 "$rc" "no-map: sync should exit 0 when data/nas-deployments.md does not exist" + assert_contains "$out" "anything: skipped: no recorded NAS deployment" "no-map: did not report the expected skip line" + pass "a project checked with no data/nas-deployments.md at all is a silent no-op" +} + +# --- tests: teardown integration --------------------------------------------- + +# make_teardown_case : a trimmed fixture mirroring tests/fm-teardown.test.sh +# make_case's ALLOW path (no-mistakes mode, task branch pushed to origin), plus a +# NAS-fixture pair and a nas-deployments.md row for the project so the real +# post-teardown fm-nas-deploy-sync.sh call has real work to do. Echoes the case dir. +make_teardown_case() { + local name=$1 case_dir fakebin nas + case_dir="$TMP_ROOT/teardown-$name" + fakebin="$case_dir/fakebin" + mkdir -p "$case_dir/state" "$case_dir/config" "$case_dir/data" "$fakebin" + + cat > "$fakebin/treehouse" <<'SH' +#!/usr/bin/env bash +exit 0 +SH + cat > "$fakebin/tmux" <<'SH' +#!/usr/bin/env bash +exit 0 +SH + cat > "$fakebin/gh-axi" <<'SH' +#!/usr/bin/env bash +case "${1:-} ${2:-}" in + "pr list") printf '%s\n' "count: 0 (showing first 0)" "pull_requests[]: []" ; exit 0 ;; + "pr view") echo "error: pull request not found" >&2 ; exit 1 ;; +esac +exit 0 +SH + cat > "$fakebin/gh" <<'SH' +#!/usr/bin/env bash +case "${1:-} ${2:-}" in + "pr view") echo "error: pull request not found" >&2 ; exit 1 ;; +esac +exit 0 +SH + chmod +x "$fakebin/treehouse" "$fakebin/tmux" "$fakebin/gh-axi" "$fakebin/gh" + + git init -q --bare "$case_dir/origin.git" + git -C "$case_dir/origin.git" symbolic-ref HEAD refs/heads/main + git clone -q "$case_dir/origin.git" "$case_dir/_seed" 2>/dev/null + git -C "$case_dir/_seed" commit -q --allow-empty -m "origin baseline" + git -C "$case_dir/_seed" push -q origin main + rm -rf "$case_dir/_seed" + git clone -q "$case_dir/origin.git" "$case_dir/project" + git -C "$case_dir/project" remote set-head origin main 2>/dev/null || true + git -C "$case_dir/project" worktree add -q -b fm/task-x1 "$case_dir/wt" main + + touch "$case_dir/state/.last-watcher-beat" + + fm_write_meta "$case_dir/state/task-x1.meta" \ + "window=fm-task-x1" \ + "worktree=$case_dir/wt" \ + "project=$case_dir/project" \ + "kind=ship" \ + "mode=no-mistakes" + + printf '%s\n' 'project change' > "$case_dir/wt/project-change.txt" + git -C "$case_dir/wt" add -- project-change.txt + git -C "$case_dir/wt" commit -q -m "shippable work" + git -C "$case_dir/wt" push -q origin fm/task-x1 + git -C "$case_dir/project" fetch -q origin + + nas=$(build_nas_pair "$case_dir" project-nas) + advance_origin "$case_dir" project-nas C1 + write_map "$case_dir" project "$nas" project-nas + write_pm2_stub "$fakebin" "$case_dir/restart.log" + + printf '%s\n' "$case_dir" +} + +run_teardown() { + local case_dir=$1 + shift + FM_ROOT_OVERRIDE="$ROOT" \ + FM_STATE_OVERRIDE="$case_dir/state" \ + FM_CONFIG_OVERRIDE="$case_dir/config" \ + FM_NAS_DEPLOYMENTS_OVERRIDE="$case_dir/data/nas-deployments.md" \ + PATH="$case_dir/fakebin:$PATH" \ + "$TEARDOWN" task-x1 "$@" +} + +test_teardown_invokes_nas_deploy_sync_for_landed_project() { + local case_dir rc origin_head nas_head + case_dir=$(make_teardown_case invokes) + git -C "$case_dir/wt" push -q origin HEAD:main + git -C "$case_dir/project" fetch -q origin + + set +e + run_teardown "$case_dir" > "$case_dir/stdout" 2> "$case_dir/stderr" + rc=$? + set -e + + expect_code 0 "$rc" "teardown-invokes: teardown should succeed for landed work" + ! grep -q REFUSED "$case_dir/stderr" || fail "teardown-invokes: teardown printed a REFUSED line" + + origin_head=$(git -C "$case_dir/work-project-nas" rev-parse main) + nas_head=$(head_sha "$case_dir/nas-project-nas") + [ "$origin_head" = "$nas_head" ] \ + || fail "teardown-invokes: post-teardown hook did not fast-forward the NAS fixture" + assert_grep project-nas "$case_dir/restart.log" \ + "teardown-invokes: post-teardown hook did not restart the NAS deployment's pm2 process" + pass "a landed ship-task teardown invokes fm-nas-deploy-sync.sh, which syncs and restarts the live NAS deployment" +} + +test_teardown_skips_nas_deploy_sync_for_remote_only_landing() { + local case_dir rc before after + case_dir=$(make_teardown_case remote-only) + before=$(head_sha "$case_dir/nas-project-nas") + + set +e + run_teardown "$case_dir" > "$case_dir/stdout" 2> "$case_dir/stderr" + rc=$? + set -e + + expect_code 0 "$rc" "teardown-remote-only: teardown should still succeed for remote-reachable work" + after=$(head_sha "$case_dir/nas-project-nas") + [ "$before" = "$after" ] \ + || fail "teardown-remote-only: NAS checkout changed for work only proven pushed to a task branch" + [ ! -s "$case_dir/restart.log" ] \ + || fail "teardown-remote-only: restarted NAS deployment for work not on the default branch" + pass "a remote-only landed teardown skips the NAS deployment sync" +} + +test_teardown_skips_nas_deploy_sync_for_force_discard() { + local case_dir rc before after + case_dir=$(make_teardown_case force-skip) + before=$(head_sha "$case_dir/nas-project-nas") + git -C "$case_dir/wt" commit -q --allow-empty -m "discarded local work" + + set +e + run_teardown "$case_dir" --force > "$case_dir/stdout" 2> "$case_dir/stderr" + rc=$? + set -e + + expect_code 0 "$rc" "teardown-force-skip: forced teardown should succeed" + after=$(head_sha "$case_dir/nas-project-nas") + [ "$before" = "$after" ] \ + || fail "teardown-force-skip: NAS checkout changed during forced discard teardown" + [ ! -s "$case_dir/restart.log" ] \ + || fail "teardown-force-skip: restarted NAS deployment during forced discard teardown" + pass "a forced teardown skips the NAS deployment sync" +} + +test_clean_behind_fast_forwards_and_restarts +test_dirty_is_skipped_untouched +test_diverged_is_stuck_untouched +test_clustered_partial_restart_is_not_masked_online +test_absent_project_is_silent_noop +test_hung_fetch_is_bounded_by_timeout +test_no_timeout_binary_skips_before_touching_nas +test_missing_pm2_skips_before_merge +test_missing_jq_skips_before_merge +test_hung_pm2_restart_is_bounded_by_timeout +test_transient_packed_refs_lock_is_retried +test_status_failure_is_stuck_untouched +test_missing_map_file_is_silent_noop +test_teardown_invokes_nas_deploy_sync_for_landed_project +test_teardown_skips_nas_deploy_sync_for_remote_only_landing +test_teardown_skips_nas_deploy_sync_for_force_discard diff --git a/tests/fm-risk-tripwire.test.sh b/tests/fm-risk-tripwire.test.sh new file mode 100755 index 000000000..371101278 --- /dev/null +++ b/tests/fm-risk-tripwire.test.sh @@ -0,0 +1,778 @@ +#!/usr/bin/env bash +# Tests for bin/fm-risk-tripwire.sh: a brief mentioning a risk-adjacent term, +# or a diff touching a risk-adjacent path, must trip the wire regardless of +# how the task's dispatch rule classified it - the mechanical, structurally +# different check behind AGENTS.md section 4's risk floor. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" +fm_git_identity fmtest fmtest@example.invalid + +TRIPWIRE="$ROOT/bin/fm-risk-tripwire.sh" +TMP_ROOT=$(fm_test_tmproot fm-risk-tripwire-tests) + +make_case() { + local name=$1 case_dir + case_dir="$TMP_ROOT/$name" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + + git init -q --bare "$case_dir/origin.git" + git -C "$case_dir/origin.git" symbolic-ref HEAD refs/heads/main + git clone -q "$case_dir/origin.git" "$case_dir/_seed" 2>/dev/null + printf 'base\n' > "$case_dir/_seed/feature.txt" + git -C "$case_dir/_seed" add feature.txt + git -C "$case_dir/_seed" -c user.email=t@t -c user.name=t commit -qm "origin baseline" + git -C "$case_dir/_seed" push -q origin main + rm -rf "$case_dir/_seed" + + git clone -q "$case_dir/origin.git" "$case_dir/project" + git -C "$case_dir/project" remote set-head origin main 2>/dev/null || true + git -C "$case_dir/project" worktree add -q -b "fm/$name" "$case_dir/wt" main + + printf '%s\n' "$case_dir" +} + +write_task_meta() { + local case_dir=$1 + fm_write_meta "$case_dir/state/task-x1.meta" \ + "window=fm-task-x1" \ + "worktree=$case_dir/wt" \ + "project=$case_dir/project" +} + +run_tripwire() { + local case_dir=$1 + FM_ROOT_OVERRIDE="$ROOT" \ + FM_STATE_OVERRIDE="$case_dir/state" \ + FM_DATA_OVERRIDE="$case_dir/data" \ + "$TRIPWIRE" task-x1 +} + +# Scaffold a real ship brief via bin/fm-brief.sh, then substitute the {TASK} +# placeholder with the given task text - exercising the actual scaffold +# boilerplate rather than a hand-written stand-in. Any extra args (e.g. +# --herdr-lab) are forwarded to fm-brief.sh so tests cover the real injected +# sections, not a stand-in. +scaffold_brief() { + local case_dir=$1 task=$2 brief + shift 2 + mkdir -p "$case_dir/state" + FM_ROOT_OVERRIDE="$ROOT" FM_DATA_OVERRIDE="$case_dir/data" FM_STATE_OVERRIDE="$case_dir/state" \ + "$ROOT/bin/fm-brief.sh" task-x1 someproject "$@" >/dev/null 2>&1 + brief="$case_dir/data/task-x1/brief.md" + sed "s|{TASK}|$task|" "$brief" > "$brief.tmp" && mv "$brief.tmp" "$brief" +} + +test_clean_brief_and_diff_passes() { + local case_dir out status + case_dir=$(make_case clean) + printf 'Add a --json flag to the status command.\n' > "$case_dir/data/task-x1/brief.md" + printf 'ordinary change\n' > "$case_dir/wt/feature.txt" + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "ordinary change" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 0 "$status" "clean: an ordinary brief and diff must not trip the wire" + [ -z "$out" ] || fail "clean: expected no RISK output, got: $out" + pass "fm-risk-tripwire passes a clean brief and diff" +} + +test_brief_keyword_trips_wire() { + local case_dir out status + case_dir=$(make_case brief-keyword) + printf 'Add a data migration for the new billing schema.\n' > "$case_dir/data/task-x1/brief.md" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 1 "$status" "brief-keyword: a risk-worded brief must trip the wire" + assert_contains "$out" "RISK: brief for task-x1 mentions risk-adjacent term(s)" "brief-keyword: should name the brief hit" + assert_contains "$out" "migration" "brief-keyword: should surface the matched term" + pass "fm-risk-tripwire trips on a brief that mentions risk-adjacent terms" +} + +test_diff_path_trips_wire() { + local case_dir out status + case_dir=$(make_case diff-path) + printf 'Fix a typo in the help text.\n' > "$case_dir/data/task-x1/brief.md" + mkdir -p "$case_dir/wt/lib/auth" + printf 'session handling\n' > "$case_dir/wt/lib/auth/session.rb" + git -C "$case_dir/wt" add lib/auth/session.rb + git -C "$case_dir/wt" commit -qm "touch auth session code" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 1 "$status" "diff-path: a diff touching an auth path must trip the wire" + assert_contains "$out" "RISK: diff for task-x1 touches risk-adjacent path(s)" "diff-path: should name the diff hit" + assert_contains "$out" "lib/auth/session.rb" "diff-path: should list the risky path" + pass "fm-risk-tripwire trips on a diff that touches an auth-adjacent path" +} + +test_brief_only_mode_before_worktree_exists() { + local case_dir out status + case_dir="$TMP_ROOT/brief-only" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf 'Rotate the payment provider credentials.\n' > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "brief-only: brief-only checkpoint must work before any meta/worktree exists" + assert_contains "$out" "RISK:" "brief-only: should still trip on the brief text alone" + pass "fm-risk-tripwire checks the brief alone before a task has been spawned" +} + +test_nothing_to_check_errors() { + local case_dir out status + case_dir="$TMP_ROOT/nothing-to-check" + mkdir -p "$case_dir/state" "$case_dir/data" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1 2>&1) + status=$? + set -e + + expect_code 2 "$status" "nothing-to-check: neither a brief nor meta must error distinctly" + assert_contains "$out" "nothing to check" "nothing-to-check: should explain there was nothing to check" + pass "fm-risk-tripwire errors distinctly when neither a brief nor a usable worktree exists" +} + +test_scaffolded_brief_boilerplate_does_not_trip() { + local case_dir out status + case_dir="$TMP_ROOT/scaffold-clean" + scaffold_brief "$case_dir" "Fix a typo in the CLI help text." + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1 2>&1) + status=$? + set -e + + expect_code 0 "$status" "scaffold-clean: a real scaffolded ship brief with a benign task must not trip" + [ -z "$out" ] || fail "scaffold-clean: expected no RISK output from scaffold boilerplate, got: $out" + pass "fm-risk-tripwire does not trip on fm-brief.sh scaffold boilerplate" +} + +test_scaffolded_brief_risky_task_still_trips() { + local case_dir out status + case_dir="$TMP_ROOT/scaffold-risky" + scaffold_brief "$case_dir" "Rotate the payment credentials and run the schema migration." + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "scaffold-risky: a risk-worded task body inside a real scaffold must still trip" + assert_contains "$out" "RISK: brief for task-x1" "scaffold-risky: should name the brief hit" + pass "fm-risk-tripwire still scans the task body of a scaffolded brief" +} + +test_herdr_lab_boilerplate_does_not_trip() { + # The --herdr-lab contract fm-brief.sh injects between # Task and # Setup is + # dense with "session"/"--session"; it is scaffold boilerplate, so a benign + # task must not trip on it. + local case_dir out status + case_dir="$TMP_ROOT/scaffold-herdr-clean" + scaffold_brief "$case_dir" "Fix a typo in the CLI help text." --herdr-lab + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1 2>&1) + status=$? + set -e + + expect_code 0 "$status" "scaffold-herdr-clean: the --herdr-lab contract's own 'session' text must not trip a benign task" + [ -z "$out" ] || fail "scaffold-herdr-clean: expected no RISK output from --herdr-lab boilerplate, got: $out" + pass "fm-risk-tripwire does not trip on --herdr-lab scaffold boilerplate" +} + +test_herdr_lab_risky_task_still_trips() { + # The Herdr block is now a scan boundary, so its "session" text is excluded - + # but the real task body between # Task and the Herdr heading must still be + # scanned, and the boilerplate's "session" must not leak into the hit list. + local case_dir out status + case_dir="$TMP_ROOT/scaffold-herdr-risky" + scaffold_brief "$case_dir" "Rotate the payment credentials and run the schema migration." --herdr-lab + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "scaffold-herdr-risky: a risk-worded task body must still trip under --herdr-lab" + assert_contains "$out" "RISK: brief for task-x1" "scaffold-herdr-risky: should name the brief hit" + assert_contains "$out" "payment" "scaffold-herdr-risky: should surface the real task-body term" + case "$out" in + *session*) fail "scaffold-herdr-risky: the Herdr boilerplate's 'session' must not leak into the hit list, got: $out" ;; + esac + pass "fm-risk-tripwire scans the task body but excludes the --herdr-lab Herdr block" +} + +test_word_boundary_avoids_substring_false_positive() { + local case_dir out status + case_dir="$TMP_ROOT/word-boundary" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nMake the config loader the authoritative source of truth.\n\n# Setup\nnothing risky here.\n' \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 0 "$status" "word-boundary: 'authoritative' must not match the 'auth' keyword" + [ -z "$out" ] || fail "word-boundary: expected no RISK output, got: $out" + pass "fm-risk-tripwire does not treat 'authoritative' as an auth keyword hit" +} + +test_inflected_keyword_still_trips() { + local case_dir out status + case_dir="$TMP_ROOT/inflected" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nRun the pending database migrations and rotate the tokens.\n\n# Setup\nx\n' \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "inflected: a plural risk word must still trip (no false negative)" + assert_contains "$out" "migration" "inflected: should surface the matched migration term" + pass "fm-risk-tripwire still trips on inflected/plural risk words" +} + +test_supervision_bin_path_does_not_trip() { + local case_dir out status + case_dir=$(make_case bin-path) + printf 'Tune the watcher poll cadence.\n' > "$case_dir/data/task-x1/brief.md" + mkdir -p "$case_dir/wt/bin" + printf 'watcher tweak\n' > "$case_dir/wt/bin/fm-watch.sh" + printf 'guard tweak\n' > "$case_dir/wt/bin/fm-guard.sh" + git -C "$case_dir/wt" add bin/fm-watch.sh bin/fm-guard.sh + git -C "$case_dir/wt" commit -qm "tweak supervision backbone" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 0 "$status" "bin-path: a supervision-backbone bin/ change alone must not trip the wire" + [ -z "$out" ] || fail "bin-path: expected no RISK output, got: $out" + pass "fm-risk-tripwire does not trip on a supervision-backbone bin/ path" +} + +test_usage_error_exit_code() { + local status + set +e + FM_ROOT_OVERRIDE="$ROOT" "$TRIPWIRE" >/dev/null 2>&1 + status=$? + set -e + expect_code 2 "$status" "usage-empty-id: a malformed invocation must exit 2, not 1 (the RISK code)" + + set +e + FM_ROOT_OVERRIDE="$ROOT" "$TRIPWIRE" one two >/dev/null 2>&1 + status=$? + set -e + expect_code 2 "$status" "usage-extra-args: extra args must exit 2, not 1 (the RISK code)" + pass "fm-risk-tripwire uses a distinct exit code for malformed invocations" +} + +test_embedded_comment_task_body_still_scanned() { + local case_dir out status + case_dir="$TMP_ROOT/embedded-comment" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + # A column-0 "# " line inside the task body (a shell comment in an example + # command) must NOT end the Task-section scan, or the risk words after it are + # silently dropped - the dangerous direction for a safety floor. + printf '# Task\nImplement the DB runner. Example invocation:\n# then run the schema migration and rotate the tokens\n./run up\n\n# Setup\nnothing risky here.\n' \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "embedded-comment: risk text after an embedded '# ' line must still be scanned" + assert_contains "$out" "schema" "embedded-comment: should surface the term on the embedded comment line" + assert_contains "$out" "migration" "embedded-comment: should surface the migration term after the comment line" + pass "fm-risk-tripwire keeps scanning the task body past an embedded '# ' comment line" +} + +test_auth_verbs_trip_wire() { + local verb case_dir out status i=0 + for verb in authorize authorized authorizing authenticate authenticated; do + i=$((i + 1)) + case_dir="$TMP_ROOT/auth-verb-$i" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nAdd middleware to %s admin requests.\n\n# Setup\nx\n' "$verb" \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "auth-verb: '$verb' must trip the wire" + assert_contains "$out" "RISK: brief for task-x1" "auth-verb: '$verb' should name the brief hit" + done + pass "fm-risk-tripwire trips on auth verbs (authorize/authenticate families)" +} + +test_auth_nouns_do_not_false_positive() { + local word case_dir out status i=0 + for word in authoritative author; do + i=$((i + 1)) + case_dir="$TMP_ROOT/auth-noun-$i" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nMake the loader the %s source of config.\n\n# Setup\nx\n' "$word" \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 0 "$status" "auth-noun: '$word' must not trip the wire" + [ -z "$out" ] || fail "auth-noun: '$word' expected no RISK output, got: $out" + done + pass "fm-risk-tripwire does not treat 'authoritative'/'author' as auth hits" +} + +test_session_start_bin_path_does_not_trip() { + local case_dir out status + case_dir=$(make_case session-start) + printf 'Tune the digest ordering.\n' > "$case_dir/data/task-x1/brief.md" + mkdir -p "$case_dir/wt/bin" + printf 'digest tweak\n' > "$case_dir/wt/bin/fm-session-start.sh" + git -C "$case_dir/wt" add bin/fm-session-start.sh + git -C "$case_dir/wt" commit -qm "tweak session-start" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 0 "$status" "session-start: 'session' as a hyphen fragment of a supervision script must not trip" + [ -z "$out" ] || fail "session-start: expected no RISK output, got: $out" + pass "fm-risk-tripwire does not trip on bin/fm-session-start.sh" +} + +test_auth_setup_bin_path_trips() { + local case_dir out status + case_dir=$(make_case auth-setup) + printf 'Wire up the setup helper.\n' > "$case_dir/data/task-x1/brief.md" + mkdir -p "$case_dir/wt/bin" + printf 'setup\n' > "$case_dir/wt/bin/auth-setup.sh" + git -C "$case_dir/wt" add bin/auth-setup.sh + git -C "$case_dir/wt" commit -qm "add auth setup" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 1 "$status" "auth-setup: 'auth' as a real hyphen token must still trip under bin/" + assert_contains "$out" "bin/auth-setup.sh" "auth-setup: should list the risky path" + pass "fm-risk-tripwire still trips on bin/auth-setup.sh via its auth token" +} + +test_dot_delimited_strong_token_trips() { + local case_dir out status + case_dir=$(make_case dot-token) + printf 'Update the generated config file.\n' > "$case_dir/data/task-x1/brief.md" + mkdir -p "$case_dir/wt/config" + printf 'x\n' > "$case_dir/wt/config/db.schema.json" + git -C "$case_dir/wt" add config/db.schema.json + git -C "$case_dir/wt" commit -qm "add db schema config" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 1 "$status" "dot-token: 'schema' as an interior dot token must trip" + assert_contains "$out" "config/db.schema.json" "dot-token: should list the risky path" + pass "fm-risk-tripwire trips on a strong risk word as a dot-delimited token" +} + +test_authors_doc_path_does_not_trip() { + local case_dir out status + case_dir=$(make_case authors-doc) + printf 'Add a contributors list.\n' > "$case_dir/data/task-x1/brief.md" + mkdir -p "$case_dir/wt/docs" + printf 'names\n' > "$case_dir/wt/docs/authors.md" + git -C "$case_dir/wt" add docs/authors.md + git -C "$case_dir/wt" commit -qm "add authors doc" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 0 "$status" "authors-doc: 'authors' is not the 'auth' token, must not trip" + [ -z "$out" ] || fail "authors-doc: expected no RISK output, got: $out" + pass "fm-risk-tripwire does not trip on docs/authors.md" +} + +test_migrate_verbs_trip_wire() { + local verb case_dir out status i=0 + for verb in migrate migrating migrated; do + i=$((i + 1)) + case_dir="$TMP_ROOT/migrate-verb-$i" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\n%s the customers table to the new engine.\n\n# Setup\nx\n' "$verb" \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "migrate-verb: '$verb' must trip the wire" + assert_contains "$out" "RISK: brief for task-x1" "migrate-verb: '$verb' should name the brief hit" + done + pass "fm-risk-tripwire trips on migrate verb forms (migrate/migrating/migrated)" +} + +test_auth_prefix_forms_trip_wire() { + local word case_dir out status i=0 + for word in unauthorized unauthenticated reauthenticate deauthorize; do + i=$((i + 1)) + case_dir="$TMP_ROOT/auth-prefix-$i" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nReject %s requests at the gateway.\n\n# Setup\nx\n' "$word" \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "auth-prefix: '$word' must trip the wire" + done + pass "fm-risk-tripwire trips on prefixed auth forms (unauthorized/unauthenticated/...)" +} + +test_authenticator_noun_trips_wire() { + local word case_dir out status i=0 + for word in authenticator authenticators; do + i=$((i + 1)) + case_dir="$TMP_ROOT/authenticator-$i" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nAdd support for hardware %s at login.\n\n# Setup\nx\n' "$word" \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "authenticator: '$word' must trip the wire" + done + pass "fm-risk-tripwire trips on authenticator/authenticators nouns" +} + +test_snake_case_risk_word_trips() { + local case_dir out status + case_dir="$TMP_ROOT/snake-case" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nImplement the runner. Call run_schema_migration_now to apply it.\n\n# Setup\nx\n' \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "snake-case: a risk word inside a snake_case identifier must trip" + assert_contains "$out" "schema" "snake-case: should surface the schema token" + assert_contains "$out" "migration" "snake-case: should surface the migration token" + pass "fm-risk-tripwire splits snake_case identifiers so embedded risk words trip" +} + +test_task_body_inline_heading_still_scanned() { + local case_dir out status + case_dir="$TMP_ROOT/inline-heading" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + # A bare "# Setup" quoted inline in the task body (not blank-line-preceded, as + # the real scaffold heading always is) must NOT terminate the scan early. + printf '# Task\nDo the thing. Configuration follows:\n# Setup\nRotate the credentials and run the migration.\n\n# Setup\nbenign boilerplate goes here.\n' \ + > "$case_dir/data/task-x1/brief.md" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + + expect_code 1 "$status" "inline-heading: risk text after an inline (non-blank-preceded) '# Setup' must still be scanned" + assert_contains "$out" "credential" "inline-heading: should surface the credentials term" + assert_contains "$out" "migration" "inline-heading: should surface the migration term" + pass "fm-risk-tripwire keeps scanning past an inline heading that is not the scaffold boundary" +} + +test_authorizer_path_trips() { + local case_dir out status + case_dir=$(make_case authorizer-path) + printf 'Refactor the request pipeline.\n' > "$case_dir/data/task-x1/brief.md" + mkdir -p "$case_dir/wt/app/authorizers" + printf 'x\n' > "$case_dir/wt/app/authorizers/user_authorizer.rb" + git -C "$case_dir/wt" add app/authorizers/user_authorizer.rb + git -C "$case_dir/wt" commit -qm "add user authorizer" + write_task_meta "$case_dir" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 1 "$status" "authorizer-path: an 'authorizer' component must trip (path authoriz stem parity)" + assert_contains "$out" "app/authorizers/user_authorizer.rb" "authorizer-path: should list the risky path" + pass "fm-risk-tripwire trips on authorizer paths" +} + +test_bare_auth_matches_but_authoritative_does_not() { + # Guards the portable word boundary against silently no-op'ing on BSD grep: a + # no-op that matches nothing would miss the bare 'auth' (part b), and a no-op + # that matches substrings would trip on 'authoritative' (part a). Both asserted. + local case_dir out status + # (a) 'authoritative' alone must not match the 'auth' keyword. + case_dir="$TMP_ROOT/authoritative-only" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nMake the loader the authoritative config source only.\n\n# Setup\nx\n' \ + > "$case_dir/data/task-x1/brief.md" + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + expect_code 0 "$status" "authoritative-only: 'authoritative' must not match the auth keyword" + [ -z "$out" ] || fail "authoritative-only: expected no RISK output, got: $out" + + # (b) a bare 'auth' word appearing mid-line (not at the string edges) must trip. + case_dir="$TMP_ROOT/bare-auth-midline" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nPlease auth the request before the handler runs.\n\n# Setup\nx\n' \ + > "$case_dir/data/task-x1/brief.md" + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + expect_code 1 "$status" "bare-auth-midline: a standalone mid-line 'auth' must trip the wire" + assert_contains "$out" "auth" "bare-auth-midline: should surface the auth token" + pass "fm-risk-tripwire matches a bare mid-line 'auth' but never 'authoritative'" +} + +test_adjacent_keywords_both_reported() { + # A boundary-consuming grep -o pattern drops the second word of an adjacent + # pair (the shared delimiter is eaten); whole-token matching reports both. + local case_dir out status + case_dir="$TMP_ROOT/adjacent-pair" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nRotate the session token on every login.\n\n# Setup\nx\n' \ + > "$case_dir/data/task-x1/brief.md" + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + expect_code 1 "$status" "adjacent-pair: adjacent risk words must trip" + assert_contains "$out" "session" "adjacent-pair: should surface session" + assert_contains "$out" "token" "adjacent-pair: should surface the adjacent token too" + pass "fm-risk-tripwire reports both words of an adjacent risk pair" +} + +test_unresolvable_diff_base_is_not_a_clean_pass() { + # A meta with a real worktree/project but an unresolvable diff base (no default + # branch, no origin) must NOT silently read as a clean pass (exit 0). The + # binding second checkpoint could not run, so it must warn and report + # could-not-check (2), matching the sibling fm-tier-guard.sh/fm-review-diff.sh. + local case_dir out status + case_dir="$TMP_ROOT/unresolvable-base" + mkdir -p "$case_dir/state" "$case_dir/wt" + git init -q "$case_dir/project" + + fm_write_meta "$case_dir/state/task-x1.meta" \ + "window=fm-task-x1" \ + "worktree=$case_dir/wt" \ + "project=$case_dir/project" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1 2>&1) + status=$? + set -e + + expect_code 2 "$status" "unresolvable-base: an unresolvable diff base must read as could-not-check (2), not a clean pass (0)" + assert_contains "$out" "could not resolve a diff base" "unresolvable-base: should warn that the diff checkpoint did not run" + pass "fm-risk-tripwire reports could-not-check when the diff base is unresolvable, not a clean pass" +} + +test_unresolvable_diff_base_still_reports_brief_hit() { + # A brief risk hit must still win (exit 1) even when the diff base is + # unresolvable - the risk floor beats the could-not-check downgrade. + local case_dir out status + case_dir="$TMP_ROOT/unresolvable-base-brief-hit" + mkdir -p "$case_dir/state" "$case_dir/wt" "$case_dir/data/task-x1" + git init -q "$case_dir/project" + printf 'Add a data migration for the new billing schema.\n' > "$case_dir/data/task-x1/brief.md" + + fm_write_meta "$case_dir/state/task-x1.meta" \ + "window=fm-task-x1" \ + "worktree=$case_dir/wt" \ + "project=$case_dir/project" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1 2>/dev/null) + status=$? + set -e + + expect_code 1 "$status" "unresolvable-base-brief-hit: a brief risk hit must still win (1) even when the diff base is unresolvable" + assert_contains "$out" "RISK: brief for task-x1" "unresolvable-base-brief-hit: should still name the brief hit" + pass "fm-risk-tripwire still reports a brief hit (1) when the diff base is unresolvable" +} + +test_unusable_meta_with_clean_brief_is_not_a_clean_pass() { + local case_dir out status + case_dir="$TMP_ROOT/unusable-meta-clean-brief" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf 'Add a --json flag to the status command.\n' > "$case_dir/data/task-x1/brief.md" + + fm_write_meta "$case_dir/state/task-x1.meta" \ + "window=fm-task-x1" \ + "worktree=$case_dir/missing-wt" \ + "project=$case_dir/missing-project" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1 2>&1) + status=$? + set -e + + expect_code 2 "$status" "unusable-meta-clean-brief: unusable meta must read as could-not-check (2), not a clean pass (0)" + assert_contains "$out" "unusable worktree/project" "unusable-meta-clean-brief: should warn that the diff checkpoint did not run" + pass "fm-risk-tripwire reports could-not-check when meta paths are unusable despite a clean brief" +} + +test_diff_name_scan_failure_is_not_a_clean_pass() { + local case_dir fakebin realgit out status + case_dir=$(make_case diff-failure) + fakebin="$case_dir/fakebin" + mkdir -p "$fakebin" + realgit=$(command -v git) + printf 'Fix a typo in the help text.\n' > "$case_dir/data/task-x1/brief.md" + write_task_meta "$case_dir" + + cat > "$fakebin/git" <&2 + exit 128 +fi +exec "\$real" "\$@" +SH + chmod +x "$fakebin/git" + + set +e + out=$(PATH="$fakebin:$PATH" run_tripwire "$case_dir" 2>&1) + status=$? + set -e + + expect_code 2 "$status" "diff-failure: a failed name-only diff must read as could-not-check (2), not a clean pass (0)" + assert_contains "$out" "could not read changed paths" "diff-failure: should warn that the diff checkpoint failed" + pass "fm-risk-tripwire reports could-not-check when the diff path scan fails" +} + +test_multiword_phrase_keyword_trips() { + local case_dir out status + case_dir="$TMP_ROOT/phrase-keyword" + mkdir -p "$case_dir/state" "$case_dir/data/task-x1" + printf '# Task\nEnforce access control and handle data deletion on the endpoint.\n\n# Setup\nx\n' \ + > "$case_dir/data/task-x1/brief.md" + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" FM_DATA_OVERRIDE="$case_dir/data" "$TRIPWIRE" task-x1) + status=$? + set -e + expect_code 1 "$status" "phrase-keyword: a multi-word risk phrase must still trip" + assert_contains "$out" "access control" "phrase-keyword: should surface the access control phrase" + pass "fm-risk-tripwire still trips on multi-word risk phrases" +} + +test_diff_uses_recorded_pr_head_when_present() { + local case_dir pr_head out status + case_dir=$(make_case pr-head) + printf 'Update generated docs.\n' > "$case_dir/data/task-x1/brief.md" + printf 'ordinary local change\n' > "$case_dir/wt/feature.txt" + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "ordinary local change" + mkdir -p "$case_dir/wt/lib/auth" + printf 'auth change\n' > "$case_dir/wt/lib/auth/token.rb" + git -C "$case_dir/wt" add lib/auth/token.rb + git -C "$case_dir/wt" commit -qm "remote pr head touches auth" + pr_head=$(git -C "$case_dir/wt" rev-parse HEAD) + git -C "$case_dir/wt" reset -q --hard HEAD~1 + write_task_meta "$case_dir" + printf '%s\n' "pr_head=$pr_head" >> "$case_dir/state/task-x1.meta" + + set +e + out=$(run_tripwire "$case_dir") + status=$? + set -e + + expect_code 1 "$status" "pr-head: risk-adjacent paths present only in recorded pr_head must trip" + assert_contains "$out" "lib/auth/token.rb" "pr-head: should scan paths through recorded pr_head" + pass "fm-risk-tripwire scans the recorded PR head when it is ahead of local HEAD" +} + +test_clean_brief_and_diff_passes +test_bare_auth_matches_but_authoritative_does_not +test_adjacent_keywords_both_reported +test_unresolvable_diff_base_is_not_a_clean_pass +test_unresolvable_diff_base_still_reports_brief_hit +test_unusable_meta_with_clean_brief_is_not_a_clean_pass +test_diff_name_scan_failure_is_not_a_clean_pass +test_multiword_phrase_keyword_trips +test_diff_uses_recorded_pr_head_when_present +test_brief_keyword_trips_wire +test_diff_path_trips_wire +test_brief_only_mode_before_worktree_exists +test_nothing_to_check_errors +test_scaffolded_brief_boilerplate_does_not_trip +test_scaffolded_brief_risky_task_still_trips +test_herdr_lab_boilerplate_does_not_trip +test_herdr_lab_risky_task_still_trips +test_word_boundary_avoids_substring_false_positive +test_inflected_keyword_still_trips +test_supervision_bin_path_does_not_trip +test_usage_error_exit_code +test_embedded_comment_task_body_still_scanned +test_auth_verbs_trip_wire +test_auth_nouns_do_not_false_positive +test_session_start_bin_path_does_not_trip +test_auth_setup_bin_path_trips +test_dot_delimited_strong_token_trips +test_authors_doc_path_does_not_trip +test_migrate_verbs_trip_wire +test_auth_prefix_forms_trip_wire +test_authenticator_noun_trips_wire +test_snake_case_risk_word_trips +test_task_body_inline_heading_still_scanned +test_authorizer_path_trips + +echo "# all fm-risk-tripwire tests passed" diff --git a/tests/fm-secondmate-lifecycle-e2e.test.sh b/tests/fm-secondmate-lifecycle-e2e.test.sh index fd71fdb35..3c7befc0b 100755 --- a/tests/fm-secondmate-lifecycle-e2e.test.sh +++ b/tests/fm-secondmate-lifecycle-e2e.test.sh @@ -27,6 +27,9 @@ set -u # shellcheck source=tests/secondmate-helpers.sh disable=SC1091 . "$(dirname "${BASH_SOURCE[0]}")/secondmate-helpers.sh" +# Reclaim any leftover root from a prior run killed abnormally (e.g. by the OOM +# killer, which SIGKILLs and so bypasses every EXIT trap) before adding another. +fm_test_sweep_stale_tmproots fm-secondmate-lifecycle TMP_ROOT=$(fm_test_tmproot fm-secondmate-lifecycle) export FM_BACKEND=tmux @@ -130,6 +133,10 @@ phase_spawn() { assert_no_grep 'notify=' "$LOG" "secondmate codex launch included the parent turn-end notify hook" assert_no_grep 'turn-ended' "$LOG" "secondmate codex launch referenced a parent turn-ended signal" assert_no_grep 'treehouse get' "$LOG" "secondmate spawn ran a project treehouse get" + # A secondmate operates in its own persistent home, not a per-task pool + # worktree, so it never gets the docker-compose isolation marker. + assert_absent "$SUB/.treehouse-compose-project" "secondmate spawn wrongly wrote a compose-project marker into its home" + assert_no_grep 'compose_project=' "$meta" "secondmate spawn meta wrongly recorded compose_project=" pass "spawn: launches in the subhome with persistent charter, records routing meta" } @@ -151,10 +158,13 @@ phase_send() { } phase_handoff() { - # The move is delegated to `tasks-axi mv`; skip cleanly when it is absent (the - # downstream recovery and teardown phases do not depend on this phase). - if ! command -v tasks-axi >/dev/null 2>&1; then - echo "skip: tasks-axi not found (backlog handoff delegates to it)" + # The move is delegated to `tasks-axi mv`; skip cleanly when it is absent or + # too old for atomic multi-ID moves (the downstream recovery and teardown + # phases do not depend on this phase). + # shellcheck source=bin/fm-tasks-axi-lib.sh disable=SC1091 + . "$ROOT/bin/fm-tasks-axi-lib.sh" + if ! fm_tasks_axi_compatible; then + echo "skip: tasks-axi not found or too old (backlog handoff delegates to it)" return 0 fi cat > "$HOME_DIR/data/backlog.md" <<'EOF' diff --git a/tests/fm-secondmate-safety.test.sh b/tests/fm-secondmate-safety.test.sh index 2c8a96d78..9d00eb9ee 100755 --- a/tests/fm-secondmate-safety.test.sh +++ b/tests/fm-secondmate-safety.test.sh @@ -10,6 +10,11 @@ set -u # shellcheck source=tests/secondmate-helpers.sh disable=SC1091 . "$(dirname "${BASH_SOURCE[0]}")/secondmate-helpers.sh" +# This suite clones the full firstmate repo (git clone --quiet "$ROOT") into +# several fixtures, so a leftover root from a prior run killed abnormally (e.g. +# by the OOM killer, which SIGKILLs and so bypasses every EXIT trap) is worth +# reclaiming before adding another one of our own. +fm_test_sweep_stale_tmproots fm-secondmate-safety TMP_ROOT=$(fm_test_tmproot fm-secondmate-safety) export FM_BACKEND=tmux diff --git a/tests/fm-session-start.test.sh b/tests/fm-session-start.test.sh index 6d3b0d970..86aeda132 100755 --- a/tests/fm-session-start.test.sh +++ b/tests/fm-session-start.test.sh @@ -350,7 +350,10 @@ EOF make_fake_toolchain "$fakebin" make_fake_ps_claude "$fakebin" # Force a MISSING diagnostic line so the bootstrap section is non-trivial. - rm -f "$fakebin/node" + # Uses chrome-devtools-axi rather than node/tmux/gh: those are common system + # packages that can shadow the fakebin removal via $BASE_PATH's real system + # directories, making the diagnostic flakily disappear depending on the host. + rm -f "$fakebin/chrome-devtools-axi" printf 'window=fm-sess:w1\nkind=ship\n' > "$home/state/task-a.meta" @@ -373,7 +376,7 @@ EOF [ "$context_line" -lt "$fleet_line" ] || fail "CONTEXT did not precede FLEET STATE" [ "$fleet_line" -lt "$next_line" ] || fail "FLEET STATE did not precede NEXT STEP" - missing_line=$(printf '%s\n' "$out" | grep -n 'MISSING: node' | head -1 | cut -d: -f1) + missing_line=$(printf '%s\n' "$out" | grep -n 'MISSING: chrome-devtools-axi' | head -1 | cut -d: -f1) [ -n "$missing_line" ] || fail "MISSING diagnostic did not appear at all" [ "$missing_line" -lt "$fleet_line" ] || fail "actionable MISSING diagnostic was buried after the bulk fleet-state digest" @@ -533,7 +536,9 @@ $rec EOF make_fake_toolchain "$fakebin" make_fake_ps_claude "$fakebin" - rm -f "$fakebin/node" + # See test_output_ordering_diagnostics_lead: chrome-devtools-axi rather than + # node avoids a real system package shadowing the forced-missing tool. + rm -f "$fakebin/chrome-devtools-axi" append_wake "$home/state" signal task-z "needs-decision: pick a library" @@ -542,7 +547,7 @@ EOF # fm-lock.sh's own exact success text. assert_contains "$out" "lock acquired: harness pid" "fm-lock.sh's real output did not appear (composition, not reimplementation)" # fm-bootstrap.sh's own exact MISSING-tool line format. - assert_contains "$out" "MISSING: node (install:" "fm-bootstrap.sh's real detect line did not appear verbatim" + assert_contains "$out" "MISSING: chrome-devtools-axi (install:" "fm-bootstrap.sh's real detect line did not appear verbatim" # fm-wake-drain.sh's real drained record (raw tab-separated queue line). assert_contains "$out" "$(printf 'signal\ttask-z\tneeds-decision: pick a library')" "fm-wake-drain.sh's real drained record did not appear" diff --git a/tests/fm-spawn-compose-isolation.test.sh b/tests/fm-spawn-compose-isolation.test.sh new file mode 100755 index 000000000..496c998ea --- /dev/null +++ b/tests/fm-spawn-compose-isolation.test.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# tests/fm-spawn-compose-isolation.test.sh - behavior tests for the +# docker-compose project-name isolation marker (AGENTS.md "Layout and state", +# bin/fm-spawn.sh's fm_compose_project_name). +# +# Two treehouse worktrees of the SAME project always share the same leaf +# directory basename (e.g. both are checked out at ...///wealthsync), +# so Compose's default project-naming (derived from cwd basename) collides +# across them on container/volume/network names. fm-spawn.sh now derives a +# stable name from the worktree's own pool-slot path and drops it at +# /.treehouse-compose-project, git-excluded so it never pollutes a +# crewmate's dirty-worktree check or commits, and records it in the task's +# meta as compose_project=. +# +# Drives the REAL bin/fm-spawn.sh end to end, exactly like +# tests/fm-tangle-guard.test.sh's isolation-abort case: a fake tmux reports +# FM_FAKE_PANE_PATH as the post-`treehouse get` pane cwd, and a real git +# worktree is added at a path shaped like a genuine treehouse pool slot +# (.../-//) so fm_compose_project_name sees +# realistic input. `treehouse` itself is stubbed to exit 0 (unused: the fake +# tmux answers the cwd query directly). +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +fm_git_identity fmtest fmtest@example.invalid + +# A fake tmux that reports FM_FAKE_PANE_PATH as the pane's cwd after +# `treehouse get`, names the session on '#S', and swallows window ops. +make_spawn_fakebin() { + local dir=$1 fakebin + fakebin=$(fm_fakebin "$dir") + cat > "$fakebin/tmux" <<'SH' +#!/usr/bin/env bash +set -u +case "$*" in + *"#{pane_current_path}"*) printf '%s\n' "${FM_FAKE_PANE_PATH:-}"; exit 0 ;; +esac +case "${1:-}" in + display-message) printf 'firstmate\n'; exit 0 ;; + list-windows) exit 0 ;; + has-session|new-session|new-window|send-keys) exit 0 ;; +esac +exit 0 +SH + chmod +x "$fakebin/tmux" + fm_fake_exit0 "$fakebin" treehouse + printf '%s\n' "$fakebin" +} + +run_spawn() { # [extra-args...] + local home=$1 id=$2 proj=$3 pane=$4 fakebin=$5 + shift 5 + mkdir -p "$home/data/$id" + printf 'brief\n' > "$home/data/$id/brief.md" + FM_ROOT_OVERRIDE='' FM_HOME="$home" \ + FM_STATE_OVERRIDE="$home/state" FM_DATA_OVERRIDE="$home/data" \ + FM_PROJECTS_OVERRIDE="$home/projects" FM_CONFIG_OVERRIDE="$home/config" \ + FM_SPAWN_NO_GUARD=1 FM_FAKE_PANE_PATH="$pane" TMUX="fake,1,0" \ + PATH="$fakebin:$PATH" \ + "$ROOT/bin/fm-spawn.sh" "$id" "$proj" codex "$@" 2>&1 +} + +# --- two worktrees of the same project, same leaf basename, different pool +# slots: this is the exact near-miss from the wealthsync incident this fix +# targets. Each must get its own compose_project name. -------------------- + +test_two_worktrees_same_leaf_get_distinct_compose_names() { + local tmp proj fakebin wt_a wt_b out_a out_b + tmp=$(fm_test_tmproot fm-spawn-compose) + proj=$(fm_git_init_commit "$tmp/proj" && printf '%s\n' "$tmp/proj") + fakebin=$(make_spawn_fakebin "$tmp/fake") + + # Two pool-slot worktrees of the SAME project, sharing the leaf name + # "wealthsync" - mirrors a real treehouse pool: -//. + wt_a="$tmp/wealthsync-323ec0/2/wealthsync" + wt_b="$tmp/wealthsync-323ec0/7/wealthsync" + git -C "$proj" worktree add -q --detach "$wt_a" || fail "setup: worktree add (slot 2) failed" + git -C "$proj" worktree add -q --detach "$wt_b" || fail "setup: worktree add (slot 7) failed" + + out_a=$(run_spawn "$tmp/home" wealthsync-task-a "$proj" "$wt_a" "$fakebin") + expect_code 0 "$?" "spawn into pool slot 2 should succeed" + assert_contains "$out_a" "spawned wealthsync-task-a" "slot-2 spawn did not report success" + + out_b=$(run_spawn "$tmp/home" wealthsync-task-b "$proj" "$wt_b" "$fakebin") + expect_code 0 "$?" "spawn into pool slot 7 should succeed" + assert_contains "$out_b" "spawned wealthsync-task-b" "slot-7 spawn did not report success" + + assert_present "$wt_a/.treehouse-compose-project" "slot-2 worktree missing the compose-project marker" + assert_present "$wt_b/.treehouse-compose-project" "slot-7 worktree missing the compose-project marker" + + local name_a name_b + name_a=$(cat "$wt_a/.treehouse-compose-project") + name_b=$(cat "$wt_b/.treehouse-compose-project") + + [ "$name_a" = "wealthsync-323ec0-2" ] || fail "slot-2 compose project name: expected 'wealthsync-323ec0-2', got '$name_a'" + [ "$name_b" = "wealthsync-323ec0-7" ] || fail "slot-7 compose project name: expected 'wealthsync-323ec0-7', got '$name_b'" + [ "$name_a" != "$name_b" ] || fail "two worktrees of the same project must not collide on the same compose project name" + + assert_grep "compose_project=$name_a" "$tmp/home/state/wealthsync-task-a.meta" "task-a meta missing compose_project=" + assert_grep "compose_project=$name_b" "$tmp/home/state/wealthsync-task-b.meta" "task-b meta missing compose_project=" + + pass "fm-spawn: two worktrees of the same project (same leaf basename, different pool slots) get distinct, stable compose_project names" +} + +# --- the marker must be invisible to the worktree's own git status, so it +# never trips teardown's dirty-worktree check or leaks into a crewmate commit. + +test_marker_is_git_excluded() { + local tmp proj fakebin wt status_out + tmp=$(fm_test_tmproot fm-spawn-compose-excl) + proj=$(fm_git_init_commit "$tmp/proj" && printf '%s\n' "$tmp/proj") + fakebin=$(make_spawn_fakebin "$tmp/fake") + + wt="$tmp/excltest-9ab1/3/excltest" + git -C "$proj" worktree add -q --detach "$wt" || fail "setup: worktree add failed" + + run_spawn "$tmp/home" excl-task-c "$proj" "$wt" "$fakebin" >/dev/null + expect_code 0 "$?" "spawn should succeed" + assert_present "$wt/.treehouse-compose-project" "worktree missing the compose-project marker" + + status_out=$(git -C "$wt" status --porcelain) + assert_not_contains "$status_out" ".treehouse-compose-project" "compose-project marker must be git-excluded, not shown as untracked" + + pass "fm-spawn: the compose-project marker is excluded from the worktree's own git status" +} + +# Secondmate spawns never write the marker (a secondmate runs in its own +# persistent firstmate home, not a per-task pool worktree). Setting up a real +# secondmate home's full validation contract (SUB_HOME_MARKER, AGENTS.md, +# bin/, registry) duplicates tests/fm-secondmate-lifecycle-e2e.test.sh's own +# fixture, so that suite's phase_spawn carries the corresponding assertions +# instead of re-deriving the fixture here. + +test_two_worktrees_same_leaf_get_distinct_compose_names +test_marker_is_git_excluded diff --git a/tests/fm-spawn-dispatch-profile.test.sh b/tests/fm-spawn-dispatch-profile.test.sh index d549c9bce..68717c985 100755 --- a/tests/fm-spawn-dispatch-profile.test.sh +++ b/tests/fm-spawn-dispatch-profile.test.sh @@ -118,7 +118,7 @@ test_no_profile_keeps_claude_launch_unchanged() { assert_meta_profile "$HOME_DIR/state/$id.meta" claude default default launch=$(cat "$LAUNCH_LOG") - expected="CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false claude --dangerously-skip-permissions \"\$(cat '$HOME_DIR/data/$id/brief.md')\"" + expected="CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false CLAUDE_CODE_AUTO_COMPACT_WINDOW=300000 claude --dangerously-skip-permissions \"\$(cat '$HOME_DIR/data/$id/brief.md')\"" [ "$launch" = "$expected" ] || fail "no-profile claude launch changed"$'\n'"expected: $expected"$'\n'"actual: $launch" pass "no --model/--effort records defaults and keeps the claude launch byte-identical" } @@ -221,9 +221,31 @@ test_claude_threads_model_and_effort() { launch=$(cat "$LAUNCH_LOG") assert_contains "$launch" "claude --dangerously-skip-permissions --model 'sonnet' --effort 'high'" \ "claude launch did not thread model and effort flags" + assert_contains "$launch" "CLAUDE_CODE_AUTO_COMPACT_WINDOW=300000" \ + "claude launch must carry the auto-compaction window env var regardless of model/effort flags" pass "claude receives --model and --effort profile flags" } +# Only claude launches carry the auto-compaction env var; codex, opencode, and pi +# ignore Claude Code env vars harmlessly, but firstmate should not set it for them +# (grok is intentionally out of scope too - unverified whether its binary honors it). +test_only_claude_launch_carries_autocompact_window() { + local rec id out status launch harness + for harness in codex opencode pi grok; do + id="profile-autocompact-$harness-z17" + rec=$(make_spawn_case "profile-autocompact-$harness" "$harness" "$id") + read_case_record "$rec" + + out=$(run_spawn "$HOME_DIR" "$WT_DIR" "$FAKEBIN_DIR" "$LAUNCH_LOG" "$id" "$PROJ_DIR") + status=$? + expect_code 0 "$status" "$harness spawn without profile flags should succeed" + launch=$(cat "$LAUNCH_LOG") + assert_not_contains "$launch" "CLAUDE_CODE_AUTO_COMPACT_WINDOW" \ + "$harness launch must not carry claude's auto-compaction env var" + done + pass "codex, opencode, pi, and grok launches never carry CLAUDE_CODE_AUTO_COMPACT_WINDOW" +} + test_codex_threads_model_and_effort() { local rec id out status launch id=profile-codex-z3 @@ -391,6 +413,7 @@ test_active_dispatch_profile_allows_explicit_harness test_active_dispatch_profile_allows_positional_harness test_active_dispatch_profile_allows_raw_launch_command test_claude_threads_model_and_effort +test_only_claude_launch_carries_autocompact_window test_codex_threads_model_and_effort test_codex_omits_invalid_max_effort test_grok_threads_model_and_reasoning_effort diff --git a/tests/fm-teardown.test.sh b/tests/fm-teardown.test.sh index 95b7c006c..b49a5d778 100755 --- a/tests/fm-teardown.test.sh +++ b/tests/fm-teardown.test.sh @@ -38,6 +38,8 @@ # (o) fm-pr-check rerun after HEAD moved -> no stale pr_head # (p) fm-pr-check when local HEAD lags -> record remote PR head # (q) no-mistakes + NO pr= recorded, PR discovered by branch -> ALLOW (yolo/no-CI merge) +# (z) no-mistakes + PR closed unmerged, never pushed, merged +# into LOCAL default branch by hand -> ALLOW (local-land fix) # # Also covers backlog teardown-lock-race: a git index.lock left in the worktree by a # killed crew process (bin/fm-teardown.sh's teardown_treehouse_return). @@ -236,6 +238,37 @@ SH chmod +x "$case_dir/fakebin/gh-axi" "$case_dir/fakebin/gh" } +# Override GitHub lookups to report PR 7 as closed (unmerged) with the current +# worktree HEAD as its head - simulates a PR deliberately closed without merging +# (e.g. upstream repo not owned by this user). +add_gh_pr_closed_for_head() { + local case_dir=$1 head=$2 + cat > "$case_dir/fakebin/gh-axi" <<'SH' +#!/usr/bin/env bash +case "${1:-} ${2:-}" in + "pr list") + printf '%s\n' "count: 1 (showing first 1)" "pull_requests[1]{number,state}:" " 7,closed" ; exit 0 ;; + "pr view") + printf '%s\n' "pull_request:" " number: 7" " state: closed" ; exit 0 ;; +esac +exit 0 +SH + cat > "$case_dir/fakebin/gh" <&2 +exit 1 +SH + chmod +x "$case_dir/fakebin/gh-axi" "$case_dir/fakebin/gh" +} + append_pr_meta_for_current_head() { local case_dir=$1 head head=$(git -C "$case_dir/wt" rev-parse HEAD) @@ -493,6 +526,7 @@ run_teardown() { FM_ROOT_OVERRIDE="$ROOT" \ FM_STATE_OVERRIDE="$case_dir/state" \ FM_CONFIG_OVERRIDE="$case_dir/config" \ + FM_NAS_DEPLOYMENTS_OVERRIDE="$case_dir/data/nas-deployments.md" \ PATH="$case_dir/fakebin:$PATH" \ "$TEARDOWN" task-x1 "$@" } @@ -609,6 +643,36 @@ test_no_mistakes_origin_remote_allows() { pass "no-mistakes worktree with HEAD on origin is torn down (no regression)" } +test_teardown_removes_per_task_state_markers() { + local case_dir rc + case_dir=$(make_case state-markers) + write_meta "$case_dir" no-mistakes ship + wt_commit "$case_dir" "shippable work" + git -C "$case_dir/wt" push -q origin fm/task-x1 + git -C "$case_dir/project" fetch -q origin + # Every per-task marker a teardown is responsible for reaping, including the + # ultracode flag written by fm-ultracode-guard.sh. + printf '%s\n' 'working: x' > "$case_dir/state/task-x1.status" + touch "$case_dir/state/task-x1.turn-ended" + printf '%s\n' 'exit 0' > "$case_dir/state/task-x1.check.sh" + printf '%s\n' 'ts' > "$case_dir/state/task-x1.pi-ext.ts" + printf '%s\n' 'tok' > "$case_dir/state/task-x1.grok-turnend-token" + printf '%s\n' 'role=independent-review' > "$case_dir/state/task-x1.ultracode" + + set +e + run_teardown "$case_dir" > "$case_dir/stdout" 2> "$case_dir/stderr" + rc=$? + set -e + + expect_code 0 "$rc" "state-markers: teardown should succeed when HEAD is on origin" + local marker + for marker in status turn-ended check.sh meta pi-ext.ts grok-turnend-token ultracode; do + [ ! -e "$case_dir/state/task-x1.$marker" ] \ + || fail "state-markers: teardown left orphan state/task-x1.$marker" + done + pass "teardown removes every per-task state marker including .ultracode (no leak)" +} + test_no_mistakes_truly_unpushed_refuses() { local case_dir rc case_dir=$(make_case nm-unpushed) @@ -627,6 +691,37 @@ test_no_mistakes_truly_unpushed_refuses() { pass "no-mistakes worktree with genuinely unlanded work is refused (safety preserved)" } +# Reproduces the local-land false-negative: a project registered under the normal +# no-mistakes delivery mode, whose task branch was never pushed to any remote and +# whose PR was deliberately closed unmerged (e.g. the upstream repo is not owned by +# this user), but which firstmate itself fast-forward-merged into the LOCAL default +# branch by hand, on explicit prior captain approval. content_in_default() used to +# only ever diff against the fetched refs/remotes/origin/$name, so it never saw this +# local-only landing outside of mode=local-only; it must now also accept containment +# in the LOCAL refs/heads/$name. +test_no_mistakes_merged_to_local_main_allows() { + local case_dir rc pr_head + case_dir=$(make_case nm-local-land) + write_meta "$case_dir" no-mistakes ship + wt_commit_file "$case_dir" feature.txt hello "shippable work" + pr_head=$(git -C "$case_dir/wt" rev-parse HEAD) + append_pr_meta_url "$case_dir" + add_gh_pr_closed_for_head "$case_dir" "$pr_head" + # Never pushed to origin or any fork. Instead, firstmate fast-forwards the + # project's LOCAL main to the worktree's HEAD, exactly as bin/fm-merge-local.sh + # would for an explicitly approved local landing. + git -C "$case_dir/project" update-ref refs/heads/main "$pr_head" + + set +e + run_teardown "$case_dir" > "$case_dir/stdout" 2> "$case_dir/stderr" + rc=$? + set -e + + expect_code 0 "$rc" "nm-local-land: teardown should succeed when work is merged into local main" + ! grep -q REFUSED "$case_dir/stderr" || fail "nm-local-land: teardown printed a REFUSED line" + pass "no-mistakes worktree with a closed PR but work merged into local main is torn down (local-land fix)" +} + test_squash_merged_branch_deleted_allows() { local case_dir rc pr_head case_dir=$(make_case squash-merged) @@ -1269,7 +1364,9 @@ test_teardown_manual_backend_prompts_hand_edit_even_when_tasks_axi_present test_local_only_truly_unpushed_refuses test_local_only_merged_to_local_main_allows test_no_mistakes_origin_remote_allows +test_teardown_removes_per_task_state_markers test_no_mistakes_truly_unpushed_refuses +test_no_mistakes_merged_to_local_main_allows test_local_only_force_overrides_unpushed test_herdr_teardown_clears_escalation_marker test_squash_merged_branch_deleted_allows diff --git a/tests/fm-tier-guard.test.sh b/tests/fm-tier-guard.test.sh new file mode 100755 index 000000000..0df6a3b0b --- /dev/null +++ b/tests/fm-tier-guard.test.sh @@ -0,0 +1,244 @@ +#!/usr/bin/env bash +# Tests for bin/fm-tier-guard.sh: the trivial (haiku/low) dispatch tier must +# escalate when its actual diff or elapsed time outgrows its assigned envelope, +# and any tier must escalate once its diff crosses the general heavy-scale +# ceiling, regardless of model/effort. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" +fm_git_identity fmtest fmtest@example.invalid + +TIER_GUARD="$ROOT/bin/fm-tier-guard.sh" +TMP_ROOT=$(fm_test_tmproot fm-tier-guard-tests) + +make_case() { + local name=$1 case_dir + case_dir="$TMP_ROOT/$name" + mkdir -p "$case_dir/state" + + git init -q --bare "$case_dir/origin.git" + git -C "$case_dir/origin.git" symbolic-ref HEAD refs/heads/main + git clone -q "$case_dir/origin.git" "$case_dir/_seed" 2>/dev/null + printf 'base\n' > "$case_dir/_seed/feature.txt" + git -C "$case_dir/_seed" add feature.txt + git -C "$case_dir/_seed" -c user.email=t@t -c user.name=t commit -qm "origin baseline" + git -C "$case_dir/_seed" push -q origin main + rm -rf "$case_dir/_seed" + + git clone -q "$case_dir/origin.git" "$case_dir/project" + git -C "$case_dir/project" remote set-head origin main 2>/dev/null || true + git -C "$case_dir/project" worktree add -q -b "fm/$name" "$case_dir/wt" main + + printf '%s\n' "$case_dir" +} + +write_task_meta() { + local case_dir=$1 id=$2 + shift 2 + fm_write_meta "$case_dir/state/$id.meta" \ + "window=fm-$id" \ + "worktree=$case_dir/wt" \ + "project=$case_dir/project" \ + "$@" +} + +run_tier_guard() { + local case_dir=$1 id=$2 + FM_ROOT_OVERRIDE="$ROOT" \ + FM_STATE_OVERRIDE="$case_dir/state" \ + "$TIER_GUARD" "$id" +} + +test_trivial_tier_within_envelope_passes() { + local case_dir out status + case_dir=$(make_case within-envelope) + printf 'base\nsmall change\n' > "$case_dir/wt/feature.txt" + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "small edit" + write_task_meta "$case_dir" task-x1 "model=claude-haiku-4-5" "effort=low" + + set +e + out=$(run_tier_guard "$case_dir" task-x1) + status=$? + set -e + + expect_code 0 "$status" "within-envelope: trivial task inside its size/age ceiling must pass" + [ -z "$out" ] || fail "within-envelope: expected no ESCALATE output, got: $out" + pass "fm-tier-guard passes a trivial task within its size and age envelope" +} + +test_trivial_tier_exceeds_line_ceiling_escalates() { + local case_dir out status i + case_dir=$(make_case line-ceiling) + : > "$case_dir/wt/feature.txt" + for i in $(seq 1 40); do echo "line $i" >> "$case_dir/wt/feature.txt"; done + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "big edit" + write_task_meta "$case_dir" task-x1 "model=claude-haiku-4-5" "effort=low" + + set +e + out=$(run_tier_guard "$case_dir" task-x1) + status=$? + set -e + + expect_code 1 "$status" "line-ceiling: trivial task past the line ceiling must escalate" + assert_contains "$out" "ESCALATE: trivial (haiku/low) task task-x1" "line-ceiling: should name the trivial-tier escalation" + assert_contains "$out" "bump to at least sonnet/high" "line-ceiling: should say what to bump to" + pass "fm-tier-guard escalates a trivial task once its diff exceeds the line ceiling" +} + +test_trivial_tier_exceeds_file_ceiling_escalates() { + local case_dir out status + case_dir=$(make_case file-ceiling) + printf 'a\n' > "$case_dir/wt/a.txt" + printf 'b\n' > "$case_dir/wt/b.txt" + printf 'c\n' > "$case_dir/wt/c.txt" + git -C "$case_dir/wt" add a.txt b.txt c.txt + git -C "$case_dir/wt" commit -qm "three new files" + write_task_meta "$case_dir" task-x1 "model=claude-haiku-4-5" "effort=low" + + set +e + out=$(run_tier_guard "$case_dir" task-x1) + status=$? + set -e + + expect_code 1 "$status" "file-ceiling: trivial task past the file ceiling must escalate" + assert_contains "$out" "ESCALATE: trivial (haiku/low) task task-x1" "file-ceiling: should name the trivial-tier escalation" + pass "fm-tier-guard escalates a trivial task once its diff exceeds the file ceiling" +} + +test_trivial_tier_stale_age_escalates() { + local case_dir out status + case_dir=$(make_case stale-age) + printf 'base\nsmall change\n' > "$case_dir/wt/feature.txt" + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "small edit" + write_task_meta "$case_dir" task-x1 "model=claude-haiku-4-5" "effort=low" + touch -t 202001010000 "$case_dir/state/task-x1.meta" + + set +e + out=$(run_tier_guard "$case_dir" task-x1) + status=$? + set -e + + expect_code 1 "$status" "stale-age: trivial task past the age ceiling must escalate" + assert_contains "$out" "has run" "stale-age: should name the age escalation" + assert_contains "$out" "bump to at least sonnet/high" "stale-age: should say what to bump to" + pass "fm-tier-guard escalates a trivial task that has run past its age ceiling" +} + +test_non_trivial_tier_small_diff_no_ceiling() { + local case_dir out status i + case_dir=$(make_case non-trivial-ok) + : > "$case_dir/wt/feature.txt" + for i in $(seq 1 40); do echo "line $i" >> "$case_dir/wt/feature.txt"; done + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "ordinary sonnet-tier edit" + write_task_meta "$case_dir" task-x1 "model=claude-sonnet-5" "effort=high" + + set +e + out=$(run_tier_guard "$case_dir" task-x1) + status=$? + set -e + + expect_code 0 "$status" "non-trivial-ok: sonnet/high has no trivial-tier ceiling to trip" + [ -z "$out" ] || fail "non-trivial-ok: expected no ESCALATE output, got: $out" + pass "fm-tier-guard does not apply the trivial-tier ceiling to a non-trivial task" +} + +test_any_tier_past_heavy_scale_escalates() { + local case_dir out status i + case_dir=$(make_case heavy-scale) + : > "$case_dir/wt/feature.txt" + for i in $(seq 1 6); do echo "line $i" >> "$case_dir/wt/feature.txt"; done + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "sonnet-tier edit that turned out large" + write_task_meta "$case_dir" task-x1 "model=claude-sonnet-5" "effort=high" + + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" \ + FM_TIER_HEAVY_MIN_LINES=5 FM_TIER_HEAVY_MIN_FILES=5 "$TIER_GUARD" task-x1) + status=$? + set -e + + expect_code 1 "$status" "heavy-scale: any tier past the heavy-scale ceiling must escalate" + assert_contains "$out" "exceeds the heavy-scale ceiling" "heavy-scale: should name the general heavy-scale escalation" + pass "fm-tier-guard escalates any tier once its diff crosses the general heavy-scale ceiling" +} + +test_unresolvable_pr_head_surfaces_degraded_diff_warning() { + # When state records pr= but the PR head cannot be resolved, fm-review-diff.sh + # falls back to the possibly-stale local branch and warns (still exit 0). The + # guard must NOT swallow that: the size it measured may under-count the real PR + # diff, so the degradation has to stay visible instead of reading as a clean, + # authoritative within-envelope pass - the same silent-reads-clean class as the + # sibling fm-risk-tripwire.sh unresolvable-base guard. + local case_dir out err status + case_dir=$(make_case degraded-diff) + printf 'base\nsmall change\n' > "$case_dir/wt/feature.txt" + git -C "$case_dir/wt" add feature.txt + git -C "$case_dir/wt" commit -qm "small local edit" + # pr= recorded, but pr_head is a bogus sha and origin has no matching PR ref, + # so resolve_pr_head fails and fm-review-diff.sh takes the stale-branch path. + write_task_meta "$case_dir" task-x1 "model=claude-haiku-4-5" "effort=low" \ + "pr=https://github.com/o/r/pull/7" \ + "pr_head=0000000000000000000000000000000000000000" + + local errfile; errfile="$case_dir/stderr.txt" + set +e + out=$(FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$case_dir/state" "$TIER_GUARD" task-x1 2>"$errfile") + status=$? + set -e + err=$(cat "$errfile") + + expect_code 0 "$status" "degraded-diff: a small stale-branch diff still measures within envelope (exit 0)" + [ -z "$out" ] || fail "degraded-diff: expected no ESCALATE on stdout, got: $out" + assert_contains "$err" "may under-count the real PR diff" "degraded-diff: the degraded measurement must be surfaced, not silently swallowed" + case "$err" in + *"WATCHER DOWN"*) fail "degraded-diff: must not leak fm-guard.sh's supervision banner from fm-review-diff.sh stderr, got: $err" ;; + esac + pass "fm-tier-guard surfaces a degraded (PR-head-unavailable) diff measurement instead of swallowing it" +} + +test_missing_meta_errors() { + local case_dir out status + case_dir=$(make_case missing-meta) + + set +e + out=$(run_tier_guard "$case_dir" task-x1 2>&1) + status=$? + set -e + + expect_code 2 "$status" "missing-meta: no meta file must exit 2 (setup error), not 1 (an escalation)" + assert_contains "$out" "no meta for task" "missing-meta: should report the missing meta file" + pass "fm-tier-guard errors distinctly (exit 2) when the task has no recorded meta" +} + +test_usage_error_exit_code() { + local status + set +e + FM_ROOT_OVERRIDE="$ROOT" "$TIER_GUARD" >/dev/null 2>&1 + status=$? + set -e + expect_code 2 "$status" "usage-empty-id: a malformed invocation must exit 2, not 1 (the escalation code)" + + set +e + FM_ROOT_OVERRIDE="$ROOT" "$TIER_GUARD" one two >/dev/null 2>&1 + status=$? + set -e + expect_code 2 "$status" "usage-extra-args: extra args must exit 2, not 1 (the escalation code)" + pass "fm-tier-guard uses a distinct exit code for malformed invocations" +} + +test_trivial_tier_within_envelope_passes +test_trivial_tier_exceeds_line_ceiling_escalates +test_trivial_tier_exceeds_file_ceiling_escalates +test_trivial_tier_stale_age_escalates +test_non_trivial_tier_small_diff_no_ceiling +test_any_tier_past_heavy_scale_escalates +test_unresolvable_pr_head_surfaces_degraded_diff_warning +test_missing_meta_errors +test_usage_error_exit_code + +echo "# all fm-tier-guard tests passed" diff --git a/tests/fm-ultracode-guard.test.sh b/tests/fm-ultracode-guard.test.sh new file mode 100755 index 000000000..bae18e383 --- /dev/null +++ b/tests/fm-ultracode-guard.test.sh @@ -0,0 +1,206 @@ +#!/usr/bin/env bash +# Tests for bin/fm-ultracode-guard.sh: an ultracode-flagged task must not reach +# PR-ready until a genuinely separate, independently-dispatched task recorded +# itself as having reviewed the finished diff - never a self-reference or a +# made-up id. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +GUARD="$ROOT/bin/fm-ultracode-guard.sh" +TMP_ROOT=$(fm_test_tmproot fm-ultracode-guard-tests) + +new_state_dir() { + local name=$1 dir + dir="$TMP_ROOT/$name/state" + mkdir -p "$dir" + printf '%s\n' "$dir" +} + +run_guard() { + local state=$1 + shift + FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$state" "$GUARD" "$@" +} + +test_check_passes_when_never_flagged() { + local state status + state=$(new_state_dir never-flagged) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + + set +e + run_guard "$state" check task-x1 >/dev/null 2>&1 + status=$? + set -e + + expect_code 0 "$status" "never-flagged: an unflagged task must pass check" + pass "fm-ultracode-guard check passes a task that was never ultracode-flagged" +} + +test_flag_then_check_fails_until_reviewed() { + local state out status + state=$(new_state_dir flag-then-check) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + + run_guard "$state" flag task-x1 >/dev/null + + set +e + out=$(run_guard "$state" check task-x1 2>&1) + status=$? + set -e + + expect_code 1 "$status" "flag-then-check: a flagged, unreviewed task must fail check" + assert_contains "$out" "is ultracode-flagged" "flag-then-check: should explain why it is blocked" + assert_contains "$out" "role=independent-review" "flag-then-check: should report the default role" + pass "fm-ultracode-guard check refuses a flagged task with no recorded review" +} + +test_flag_custom_role_reported_in_check() { + local state out + state=$(new_state_dir custom-role) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + + run_guard "$state" flag task-x1 breadth-fanout >/dev/null + + set +e + out=$(run_guard "$state" check task-x1 2>&1) + set -e + + assert_contains "$out" "role=breadth-fanout" "custom-role: check should report the custom role" + pass "fm-ultracode-guard flag records a custom role and check reports it" +} + +test_flag_rejects_unsafe_role_text() { + local state out status + state=$(new_state_dir unsafe-role) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + + set +e + out=$(run_guard "$state" flag task-x1 $'independent-review\nreviewed_by=task-x2' 2>&1) + status=$? + set -e + + expect_code 1 "$status" "unsafe-role: role text with a newline must be refused" + assert_contains "$out" "role must be a safe token" "unsafe-role: should explain the refusal" + assert_absent "$state/task-x1.ultracode" "unsafe-role: refused role must not create a marker" + pass "fm-ultracode-guard flag refuses role text that could inject marker lines" +} + +test_reviewed_by_self_is_refused() { + local state out status + state=$(new_state_dir self-review) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + run_guard "$state" flag task-x1 >/dev/null + + set +e + out=$(run_guard "$state" reviewed task-x1 task-x1 2>&1) + status=$? + set -e + + expect_code 1 "$status" "self-review: a task cannot review itself" + assert_contains "$out" "distinct from task-x1" "self-review: should explain the refusal" + pass "fm-ultracode-guard reviewed refuses a task naming itself as its own reviewer" +} + +test_reviewed_by_unknown_task_is_refused() { + local state out status + state=$(new_state_dir unknown-reviewer) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + run_guard "$state" flag task-x1 >/dev/null + + set +e + out=$(run_guard "$state" reviewed task-x1 made-up-id 2>&1) + status=$? + set -e + + expect_code 1 "$status" "unknown-reviewer: a made-up reviewer id must be refused" + assert_contains "$out" "no recorded state/made-up-id.meta" "unknown-reviewer: should explain the refusal" + pass "fm-ultracode-guard reviewed refuses a reviewer id with no recorded meta" +} + +test_reviewed_by_unsafe_task_id_is_refused() { + local state out status unsafe_reviewer + state=$(new_state_dir unsafe-reviewer) + unsafe_reviewer=$'task-x2\nrole=independent-review' + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + fm_write_meta "$state/$unsafe_reviewer.meta" "worktree=/tmp/y" "project=/tmp/y" + run_guard "$state" flag task-x1 >/dev/null + + set +e + out=$(run_guard "$state" reviewed task-x1 "$unsafe_reviewer" 2>&1) + status=$? + set -e + + expect_code 1 "$status" "unsafe-reviewer: reviewer id with a newline must be refused" + assert_contains "$out" "reviewer-task-id must be a safe token" "unsafe-reviewer: should explain the refusal" + [ "$(cat "$state/task-x1.ultracode")" = "role=independent-review" ] \ + || fail "unsafe-reviewer: refused reviewer must not append marker lines" + pass "fm-ultracode-guard reviewed refuses reviewer ids that could inject marker lines" +} + +test_reviewed_by_distinct_dispatched_task_passes_check() { + local state status + state=$(new_state_dir distinct-reviewer) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + fm_write_meta "$state/task-x2.meta" "worktree=/tmp/y" "project=/tmp/y" + run_guard "$state" flag task-x1 >/dev/null + + run_guard "$state" reviewed task-x1 task-x2 >/dev/null + + set +e + run_guard "$state" check task-x1 >/dev/null 2>&1 + status=$? + set -e + + expect_code 0 "$status" "distinct-reviewer: a genuinely separate reviewer must satisfy check" + pass "fm-ultracode-guard check passes once a distinct dispatched task recorded the review" +} + +test_reviewed_without_flag_is_refused() { + local state out status + state=$(new_state_dir no-flag) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + fm_write_meta "$state/task-x2.meta" "worktree=/tmp/y" "project=/tmp/y" + + set +e + out=$(run_guard "$state" reviewed task-x1 task-x2 2>&1) + status=$? + set -e + + expect_code 1 "$status" "no-flag: reviewed must refuse a task that was never flagged" + assert_contains "$out" "not ultracode-flagged" "no-flag: should explain the refusal" + pass "fm-ultracode-guard reviewed refuses to mark an unflagged task as reviewed" +} + +test_reflag_clears_prior_review() { + local state out status + state=$(new_state_dir reflag-clears) + fm_write_meta "$state/task-x1.meta" "worktree=/tmp/x" "project=/tmp/x" + fm_write_meta "$state/task-x2.meta" "worktree=/tmp/y" "project=/tmp/y" + run_guard "$state" flag task-x1 >/dev/null + run_guard "$state" reviewed task-x1 task-x2 >/dev/null + run_guard "$state" flag task-x1 >/dev/null + + set +e + out=$(run_guard "$state" check task-x1 2>&1) + status=$? + set -e + + expect_code 1 "$status" "reflag-clears: re-flagging must clear the prior review" + assert_contains "$out" "is ultracode-flagged" "reflag-clears: should be blocked again" + pass "fm-ultracode-guard re-flagging starts the review requirement over" +} + +test_check_passes_when_never_flagged +test_flag_then_check_fails_until_reviewed +test_flag_custom_role_reported_in_check +test_flag_rejects_unsafe_role_text +test_reviewed_by_self_is_refused +test_reviewed_by_unknown_task_is_refused +test_reviewed_by_unsafe_task_id_is_refused +test_reviewed_by_distinct_dispatched_task_passes_check +test_reviewed_without_flag_is_refused +test_reflag_clears_prior_review + +echo "# all fm-ultracode-guard tests passed" diff --git a/tests/lib.sh b/tests/lib.sh index add7f2904..c2eb544b5 100644 --- a/tests/lib.sh +++ b/tests/lib.sh @@ -53,29 +53,111 @@ pass() { # --- self-cleaning temp root ------------------------------------------------ # # fm_test_tmproot echoes a fresh temp dir and registers it for removal -# on EXIT. The first call installs the cleanup trap. A test file that needs -# extra teardown (e.g. killing a daemon) should define its own EXIT trap and -# call fm_test_cleanup from inside it so registered dirs are still removed. +# on EXIT. A test file that needs extra teardown (e.g. killing a daemon) should +# define its own EXIT trap and call fm_test_cleanup from inside it so registered +# dirs are still removed. +# +# Registration goes through an on-disk manifest, not just the FM_TEST_CLEANUP_DIRS +# array: every call site uses `TMP_ROOT=$(fm_test_tmproot prefix)`, and bash always +# runs a command substitution in a subshell, so a trap installed - or an array +# entry appended - inside fm_test_tmproot itself would apply only to that +# short-lived subshell and vanish the instant it exits. Confirmed empirically: an +# EXIT trap set there fires immediately on subshell exit, deleting the very +# directory just created, before control ever returns to the caller; the caller's +# own shell never gets a trap installed at all, so nothing then protects the +# directory a later `mkdir -p` silently recreates. Writing the path to a manifest +# file survives the subshell boundary because subshells share the real +# filesystem even though they don't share shell state; the real top-level trap, +# installed once below when this library is sourced (never inside a subshell, +# since sourcing always runs in the current shell), reads the manifest at the +# actual script's exit. FM_TEST_CLEANUP_DIRS keeps working unchanged for a +# caller that appends to it directly at its own top level (not through a +# command substitution), such as a helper that creates its temp dir with a bare +# `mktemp -d` assignment. +# +# Directory names embed the owning PID (..XXXXXX) so a later +# invocation's startup sweep (fm_test_sweep_stale_tmproots) can tell a leftover +# from a still-running test apart without relying on age alone. The manifest +# itself is mktemp-created at source time: a predictable name under a shared +# TMPDIR could be pre-seeded by another local user with arbitrary paths for the +# trap's `rm -rf` to chew through, so the name gets a random suffix, and it +# embeds the owning PID so the startup sweep can reclaim a SIGKILL'd run's +# manifest (and the roots it lists) once that PID is gone. +# +# None of this saves a directory from a SIGKILL (e.g. the OOM killer): no trap, +# EXIT or otherwise, ever runs for that signal, so a killed run's temp root is +# unavoidably left behind. fm_test_sweep_stale_tmproots is the actual safety net +# for that case - it runs at the start of the next invocation. FM_TEST_CLEANUP_DIRS=() +FM_TEST_CLEANUP_MANIFEST=$(mktemp "${TMPDIR:-/tmp}/.fm-test-cleanup.$$.XXXXXX") fm_test_cleanup() { local d for d in "${FM_TEST_CLEANUP_DIRS[@]:-}"; do - [ -n "$d" ] && rm -rf "$d" + [ -n "$d" ] && rm -rf -- "$d" done + if [ -f "$FM_TEST_CLEANUP_MANIFEST" ]; then + while IFS= read -r d; do + [ -n "$d" ] && rm -rf -- "$d" + done < "$FM_TEST_CLEANUP_MANIFEST" + rm -f -- "$FM_TEST_CLEANUP_MANIFEST" + fi } +trap fm_test_cleanup EXIT fm_test_tmproot() { local prefix=${1:-fm-test} root - root=$(mktemp -d "${TMPDIR:-/tmp}/${prefix}.XXXXXX") - if [ "${#FM_TEST_CLEANUP_DIRS[@]}" -eq 0 ]; then - trap fm_test_cleanup EXIT - fi - FM_TEST_CLEANUP_DIRS+=("$root") + root=$(mktemp -d "${TMPDIR:-/tmp}/${prefix}.$$.XXXXXX") + printf '%s\n' "$root" >> "$FM_TEST_CLEANUP_MANIFEST" printf '%s\n' "$root" } +# fm_test_sweep_stale_tmproots removes any leftover +# ..XXXXXX directory under TMPDIR whose owning PID is no longer +# alive - the actual safety net for a SIGKILL-terminated prior run, which no +# EXIT trap (see above) can ever run for. A directory whose PID is still alive +# is left untouched, so a genuinely concurrent run of the same test on this host +# is never swept. It also reclaims dead-owner cleanup manifests: each +# .fm-test-cleanup..XXXXXX file whose PID is gone has its listed roots +# removed (covering roots registered by suites that never call this sweep) and +# is then deleted. A manifest not owned by the current user, or reached through +# a symlink, is left alone so a planted file can never direct the `rm -rf`. +# Call it once at the top of a test file, before that file's own +# fm_test_tmproot call, for any prefix known to create large fixtures (e.g. a +# full repo clone) worth reclaiming promptly rather than waiting on the next +# `rm -rf` of an already-registered root. +fm_test_sweep_stale_tmproots() { + local prefix=$1 dir base pid manifest line + for dir in "${TMPDIR:-/tmp}/${prefix}".*; do + [ -d "$dir" ] || continue + base=$(basename "$dir") + pid=${base#"$prefix".} + pid=${pid%%.*} + case "$pid" in + ''|*[!0-9]*) continue ;; # not the ..XXXXXX scheme; leave it alone + esac + kill -0 "$pid" 2>/dev/null && continue # owner still alive: a genuine concurrent run + rm -rf -- "$dir" + done + for manifest in "${TMPDIR:-/tmp}"/.fm-test-cleanup.*; do + [ -f "$manifest" ] || continue + [ -L "$manifest" ] && continue + [ -O "$manifest" ] || continue + base=$(basename "$manifest") + pid=${base#.fm-test-cleanup.} + pid=${pid%%.*} + case "$pid" in + ''|*[!0-9]*) continue ;; + esac + kill -0 "$pid" 2>/dev/null && continue + while IFS= read -r line; do + [ -n "$line" ] && rm -rf -- "$line" + done < "$manifest" + rm -f -- "$manifest" + done +} + # --- fakebin / PATH shims --------------------------------------------------- # # fm_fakebin creates /fakebin and echoes it; prepend it to PATH to diff --git a/tests/wake-helpers.sh b/tests/wake-helpers.sh index a224f0c9d..42c6d1c5b 100644 --- a/tests/wake-helpers.sh +++ b/tests/wake-helpers.sh @@ -32,13 +32,7 @@ fi # that channel, to exercise graceful degradation. Suites that do not source this # harness still cannot fire a real notification: the daemon defaults the seam to # "discard" whenever it is sourced (its library-mode guard). -# Create the recorder dir with mktemp directly (not fm_test_tmproot, whose -# first call installs an EXIT trap that, invoked inside a command-substitution -# subshell, would delete the dir on subshell exit). Register it for the same -# cleanup and install the trap in THIS shell if it is the first registration. -_fm_wedge_rec_dir=$(mktemp -d "${TMPDIR:-/tmp}/fm-wedge-rec.XXXXXX") -if [ "${#FM_TEST_CLEANUP_DIRS[@]}" -eq 0 ]; then trap fm_test_cleanup EXIT; fi -FM_TEST_CLEANUP_DIRS+=("$_fm_wedge_rec_dir") +_fm_wedge_rec_dir=$(fm_test_tmproot fm-wedge-rec) cat > "$_fm_wedge_rec_dir/rec" <<'REC' #!/usr/bin/env bash printf '%s\t%s\n' "${1:-}" "${2:-}" >> "${FM_WEDGE_ALARM_LOG:-/dev/null}"