Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions bin/backends/herdr.sh
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}"
FM_BACKEND_HERDR_MIN_PROTOCOL=14
# events.subscribe (the native pane.agent_status_changed push stream) and its
# subscription_event schema first shipped at protocol 16 (verified: herdr
# 0.7.3). Below this, or with the events surface absent from `herdr api schema`,
# the event fast-path fails closed to the watcher's poll loop
# 0.7.3). Below this, or with either required capability marker absent from
# `herdr api schema`, the event fast-path fails closed to the watcher's poll loop
# (fm_backend_herdr_events_capable). Distinct from FM_BACKEND_HERDR_MIN_PROTOCOL
# (14): the adapter's spawn/capture/send primitives work on 14, only the push
# subscriber needs 16.
Expand Down Expand Up @@ -1225,8 +1225,10 @@ fm_backend_herdr_socket_path() { # <session>
# and both `events.subscribe` and `pane.agent_status_changed` present in `herdr
# api schema`. FM_BACKEND_HERDR_EVENTS_FORCE overrides the whole verdict for
# tests (1 = capable, 0 = incapable) without touching the real binary. The
# `api schema` read is ~220KB, so callers (the watcher) memoize this per session
# for a process lifetime rather than probing every poll.
# ~220KB `api schema` is captured exactly once and both markers are matched in
# that value, avoiding an early-closing pipeline that can SIGPIPE its producer
# under `pipefail`. Callers (the watcher) memoize this per session for a process
# lifetime rather than probing every poll.
fm_backend_herdr_events_capable() { # <session>
local session=$1 protocol schema
case "${FM_BACKEND_HERDR_EVENTS_FORCE:-}" in
Expand All @@ -1241,8 +1243,8 @@ fm_backend_herdr_events_capable() { # <session>
case "$protocol" in ''|*[!0-9]*) return 1 ;; esac
[ "$protocol" -ge "$FM_BACKEND_HERDR_MIN_EVENTS_PROTOCOL" ] || return 1
schema=$(herdr api schema --json 2>/dev/null) || return 1
printf '%s' "$schema" | grep -Fq 'events.subscribe' || return 1
printf '%s' "$schema" | grep -Fq 'pane.agent_status_changed' || return 1
case "$schema" in *events.subscribe*) ;; *) return 1 ;; esac
case "$schema" in *pane.agent_status_changed*) ;; *) return 1 ;; esac
return 0
}

Expand Down
5 changes: 3 additions & 2 deletions docs/herdr-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -798,7 +798,8 @@ There is no second watcher process: the reader is a short-lived subprocess of th

**Polling is the permanent fail-closed backstop.**
The watcher's poll loop runs every cycle regardless, so the event path only ever shortens latency and can never drop an escalation.
Three documented triggers fall back to pure polling (`fm_backend_herdr_events_capable` and the watcher's runtime-disable counter): a build below protocol 16 or missing the events surface in `herdr api schema`; a connect/subscribe failure; and repeated runtime failures, which disable the fast path for the rest of that watcher process (a restart re-probes).
Three documented triggers fall back to pure polling (`fm_backend_herdr_events_capable` and the watcher's runtime-disable counter): a build below protocol 16 or a schema missing either required capability marker (`events.subscribe` or `pane.agent_status_changed`); a connect/subscribe failure; and repeated runtime failures, which disable the fast path for the rest of that watcher process (a restart re-probes).
The capability probe captures the roughly 220 KB schema exactly once and checks both markers in that value, avoiding an early-closing pipeline that can SIGPIPE its producer under `pipefail` and falsely disable the push path.

**Empirical evidence (2026-07-11, herdr 0.7.3, protocol 16, macOS aarch64 Darwin 25.5.0, python3 3.13, jq present).**
Capability, verified read-only:
Expand Down Expand Up @@ -826,7 +827,7 @@ ok - real herdr: the watcher fast-path enqueues a stale wake naming the task win
```

The subscriber returned the `blocked` transition in **0.129s** and the watcher fast-path enqueued a durable `stale` wake naming the task window - versus up to `FM_POLL` (15s) plus `FM_STALE_ESCALATE_SECS` (240s) on the poll path this shortcuts.
Dedupe (one wake per `->blocked` edge, marker cleared when the pane returns to `working`), subscribe-then-reconcile ordering (an already-blocked pane enqueued exactly once while newer edges buffer in the active stream), the `kind=secondmate`/`paused:` exemptions, and the three fail-closed fallbacks are covered by the fake-CLI unit tests in `tests/fm-backend-herdr.test.sh` (the `wait_transition`/`apply_transition` cases), `tests/fm-transition-lib.test.sh`, and `tests/fm-supervision-events.test.sh`.
Dedupe (one wake per `->blocked` edge, marker cleared when the pane returns to `working`), subscribe-then-reconcile ordering (an already-blocked pane enqueued exactly once while newer edges buffer in the active stream), the `kind=secondmate`/`paused:` exemptions, the three fail-closed fallbacks, and the large-schema capability gate under `pipefail` (both markers present, either marker absent, and exactly one schema read) are covered by the fake-CLI unit tests in `tests/fm-backend-herdr.test.sh` (the `events_capable`, `wait_transition`, and `apply_transition` cases), `tests/fm-transition-lib.test.sh`, and `tests/fm-supervision-events.test.sh`.

## Away-mode daemon terminal launch (2026-07-12, herdr 0.7.3, protocol 16, macOS aarch64)

Expand Down
64 changes: 64 additions & 0 deletions tests/fm-backend-herdr.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1740,6 +1740,68 @@ test_no_jq_reserved_keyword_arg_names() {
# that drives a live idle->blocked transition lives in
# tests/fm-backend-herdr-eventwait-smoke.test.sh.

write_large_events_schema() { # <path> <method-marker> <event-marker>
local path=$1 method=$2 event=$3
{
# Keep each capability marker on an early complete line like herdr's
# pretty-printed schema, with roughly 256KB still unwritten after grep -q
# can exit and close the pipe.
printf '{\n"methods":["%s"],\n"events":["%s"],\n"padding":"' "$method" "$event"
awk 'BEGIN { for (i = 0; i < 262144; i++) printf "x" }'
printf '"}\n'
} > "$path"
}

test_events_capable_accepts_large_schema_under_pipefail() {
local dir log resp fb err status schema_calls
dir="$TMP_ROOT/events-capable-large"; mkdir -p "$dir/responses"
log="$dir/log"; resp="$dir/responses"; err="$dir/stderr"; : > "$log"
printf '{"client":{"version":"0.7.3","protocol":16}}\n' > "$resp/1.out"
write_large_events_schema "$resp/2.out" "events.subscribe" "pane.agent_status_changed"
fb=$(make_herdr_fakebin "$dir")

PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" FM_HERDR_SCRIPT_STATUS=1 \
FM_BACKEND_HERDR_EVENT_READER=true \
/bin/bash -o pipefail -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_events_capable sess' "$ROOT" 2>"$err"
status=$?

expect_code 0 "$status" "events_capable should accept both markers in a large schema under pipefail"
[ ! -s "$err" ] || fail "events_capable wrote stderr while scanning a large capable schema: $(cat "$err")"
schema_calls=$(grep -c $'\x1f''api'$'\x1f''schema'$'\x1f''--json' "$log")
[ "$schema_calls" = 1 ] || fail "events_capable should load the schema exactly once, got $schema_calls calls"
pass "fm_backend_herdr_events_capable: a large schema detects both required markers under pipefail without stderr noise"
}

test_events_capable_large_schema_missing_markers_fails_closed() {
local missing dir log resp fb err status schema_calls
for missing in events.subscribe pane.agent_status_changed; do
dir="$TMP_ROOT/events-capable-missing-${missing%%.*}"; mkdir -p "$dir/responses"
log="$dir/log"; resp="$dir/responses"; err="$dir/stderr"; : > "$log"
printf '{"client":{"version":"0.7.3","protocol":16}}\n' > "$resp/1.out"
case "$missing" in
events.subscribe)
write_large_events_schema "$resp/2.out" "events.missing" "pane.agent_status_changed"
;;
pane.agent_status_changed)
write_large_events_schema "$resp/2.out" "events.subscribe" "pane.agent_status_missing"
;;
esac
fb=$(make_herdr_fakebin "$dir")

status=0
PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" FM_HERDR_SCRIPT_STATUS=1 \
FM_BACKEND_HERDR_EVENT_READER=true \
/bin/bash -o pipefail -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_events_capable sess' "$ROOT" 2>"$err" \
|| status=$?

[ "$status" -ne 0 ] || fail "events_capable should fail closed when $missing is absent"
[ ! -s "$err" ] || fail "events_capable wrote stderr while rejecting a schema missing $missing: $(cat "$err")"
schema_calls=$(grep -c $'\x1f''api'$'\x1f''schema'$'\x1f''--json' "$log")
[ "$schema_calls" = 1 ] || fail "events_capable should load a schema missing $missing exactly once, got $schema_calls calls"
done
pass "fm_backend_herdr_events_capable: each missing schema marker fails closed after one large-schema load"
}

# make_herdr_eventfake: a herdr stub answering exactly the calls the event path
# makes - `session list --json` (echoes one session, name FM_FAKE_SESSION_NAME,
# socket FM_FAKE_SOCKET), `status --json`, and `agent get <pane>` (per-pane
Expand Down Expand Up @@ -2125,6 +2187,8 @@ test_dispatch_routes_herdr_backend
test_dispatch_busy_state_unknown_for_tmux
test_dispatch_composer_state_routes_by_backend
test_scripts_route_explicit_target_through_meta_backend
test_events_capable_accepts_large_schema_under_pipefail
test_events_capable_large_schema_missing_markers_fails_closed
test_normalize_event_leaves_from_empty
test_escalation_marker_keys_like_watcher
test_apply_transition_blocked_requires_commit_to_dedupe
Expand Down
4 changes: 2 additions & 2 deletions tests/fm-turnend-guard.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -785,7 +785,7 @@ SH
exit 0
SH
chmod +x "$repo/bin/fm-turnend-guard.sh" "$repo/bin/fm-arm-pretool-check.sh"
out=$(PLUGIN="$ext" FM_HOME="$home" FM_GUARD_LOG="$log" node --input-type=module 2>&1 <<'EOF'
out=$(NODE_NO_WARNINGS=1 PLUGIN="$ext" FM_HOME="$home" FM_GUARD_LOG="$log" node --input-type=module 2>&1 <<'EOF'
import { readFileSync } from "node:fs";
import { pathToFileURL } from "node:url";

Expand Down Expand Up @@ -845,7 +845,7 @@ SH
exit 0
SH
chmod +x "$repo/bin/fm-turnend-guard.sh" "$repo/bin/fm-arm-pretool-check.sh"
out=$(PLUGIN="$ext" FM_HOME="$home" node --input-type=module 2>&1 <<'EOF'
out=$(NODE_NO_WARNINGS=1 PLUGIN="$ext" FM_HOME="$home" node --input-type=module 2>&1 <<'EOF'
import { pathToFileURL } from "node:url";

const handlers = new Map();
Expand Down
2 changes: 1 addition & 1 deletion tests/fm-watcher-lock.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ test_guard_warnings() {
mkdir -p "$dir/config"
printf 'project=x\n' > "$state/task.meta"
: > "$dir/config/x-mode.env"
FM_ROOT_OVERRIDE="$dir" FM_STATE_OVERRIDE="$state" FM_GUARD_GRACE=1 "$ROOT/bin/fm-guard.sh" 2> "$err" >/dev/null || fail "guard failed"
FM_ROOT_OVERRIDE="$dir" FM_STATE_OVERRIDE="$state" FM_CONFIG_OVERRIDE="$dir/config" FM_GUARD_GRACE=1 "$ROOT/bin/fm-guard.sh" 2> "$err" >/dev/null || fail "guard failed"
grep -F "source '$dir/config/x-mode.env' first" "$err" >/dev/null || fail "guard repair line did not source the X-mode cadence config"

# (2) fresh watcher, empty queue -> silence.
Expand Down