From ec30bc2af297b4b0f8332b217a90318543a887ff Mon Sep 17 00:00:00 2001 From: Zachary Landes Date: Thu, 16 Jul 2026 01:15:42 -0700 Subject: [PATCH 1/3] fix(herdr): match bare prompt glyphs by alternation, not a bracket class MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Under a byte-oriented locale (LC_CTYPE=C) grep walks a [❯›] bracket class byte-wise, so it matches any row whose first byte is 0xE2 - including the box-drawing composer borders ╭ ╰ │ - and misreads a bordered composer as a bare prompt, classifying it 'pending' instead of 'empty'. The (❯|›) alternation matches each full glyph sequence and is correct in any locale. * Surfaced deterministically in the no-mistakes gate's C-locale test run, where it failed the bare-prompt composer-state assertion in tests/fm-backend-herdr.test.sh. cmux is unaffected (it classifies via locale-safe shell case globs, not this grep). * Keep docs/configuration.md's documented default in sync and note the alternation-not-bracket-class requirement for any override. --- bin/backends/herdr.sh | 8 ++++++-- docs/configuration.md | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/bin/backends/herdr.sh b/bin/backends/herdr.sh index 260d2b666..50b575094 100644 --- a/bin/backends/herdr.sh +++ b/bin/backends/herdr.sh @@ -739,7 +739,11 @@ FM_BACKEND_HERDR_IDLE_RE=${FM_BACKEND_HERDR_IDLE_RE:-'^Type a message\.\.\.$'} # Known bare (unbordered) prompt glyphs a composer row may start with: ❯ # (claude) and › (codex) only. Generic shell-style glyphs > $ % # are still # recognized after a bordered composer row has already been structurally found. -FM_BACKEND_HERDR_BARE_PROMPT_RE=${FM_BACKEND_HERDR_BARE_PROMPT_RE:-'^[❯›]'} +# Alternation, not a [❯›] bracket class: under a byte-oriented locale (LC_CTYPE=C) +# grep walks a bracket class byte-wise, so [❯›] matches any row whose first byte is +# 0xE2 - including the box-drawing rows ╭ ╰ │ - and misreads a composer border as a +# bare prompt. The (❯|›) form matches each full glyph sequence, correct in any locale. +FM_BACKEND_HERDR_BARE_PROMPT_RE=${FM_BACKEND_HERDR_BARE_PROMPT_RE:-'^(❯|›)'} # Pi allows a multi-line composer between its horizontal separators. Bound the # structural candidate so two unrelated transcript rules with an arbitrarily # large region between them can never be promoted into a composer. @@ -903,7 +907,7 @@ EOF fi # Delegate the empty/pending/unknown decision to the shared owner. The bare # shape only ever starts with an AGENT glyph (FM_BACKEND_HERDR_BARE_PROMPT_RE - # is '^[❯›]'), so a bare shell prompt never reaches here - it stays 'unknown' + # is '^(❯|›)'), so a bare shell prompt never reaches here - it stays 'unknown' # via the no-composer-row path above, exactly as before. fm_composer_classify_content "$bordered" "$stripped" "$FM_BACKEND_HERDR_IDLE_RE" } diff --git a/docs/configuration.md b/docs/configuration.md index abdd1b281..ab0fd9c35 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -342,7 +342,7 @@ FM_BACKEND= # optional runtime backend override for new spawns; tmux HERDR_SESSION=default # herdr-only: named session for normal backend ops; not enough for destructive cleanup (docs/herdr-backend.md) FM_BACKEND_HERDR_COMPOSER_LINES=20 # herdr-only: tail lines scanned by composer-state guard/fallback paths; idle-baseline submit confirmation uses agent-state FM_BACKEND_HERDR_IDLE_RE='^Type a message\.\.\.$' # herdr-only: empty-composer placeholder regex after shared ghost extraction plus border and prompt stripping -FM_BACKEND_HERDR_BARE_PROMPT_RE='^[❯›]' # herdr-only: verified agent glyphs recognized as an UNBORDERED (bare) composer row, e.g. claude's ❯ or codex's ›; shell glyphs remain unknown rather than empty, and de-emphasised ghost/placeholder text (dim or dark-truecolor) after an agent prompt reads empty via the shared fm_composer_strip_ghost (docs/herdr-backend.md "Incident (2026-07-08)", "Incident (2026-07-10)") +FM_BACKEND_HERDR_BARE_PROMPT_RE='^(❯|›)' # herdr-only: verified agent glyphs recognized as an UNBORDERED (bare) composer row, e.g. claude's ❯ or codex's ›; shell glyphs remain unknown rather than empty, and de-emphasised ghost/placeholder text (dim or dark-truecolor) after an agent prompt reads empty via the shared fm_composer_strip_ghost (docs/herdr-backend.md "Incident (2026-07-08)", "Incident (2026-07-10)"); spell an override as an alternation, never a [❯›] bracket class, which a byte-oriented locale matches byte-wise and misreads composer borders through (see the default's comment in bin/backends/herdr.sh) FM_BACKEND_HERDR_PI_COMPOSER_MAX_LINES=8 # herdr-only: maximum rows admitted between Pi's native-identity-corroborated separator pair; taller or ambiguous candidates stay unknown (docs/herdr-backend.md "Incident (2026-07-14)") FM_BACKEND_HERDR_SUBMIT_POLLS=6 # herdr-only: agent-state samples spread across each Enter attempt's budget when confirming a submit (docs/herdr-backend.md "Native agent-state submit confirmation") FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP=0.6 # herdr-only: minimum per-Enter confirmation budget before polling agent-state after an idle baseline From 4c4fd699d68dbba6f2c776fc32c28145c674b4b7 Mon Sep 17 00:00:00 2001 From: Zachary Landes Date: Thu, 16 Jul 2026 10:42:10 -0700 Subject: [PATCH 2/3] no-mistakes(review): add C-locale composer regression tests and incident entry --- docs/herdr-backend.md | 42 +++++++++++++++++++ tests/fm-backend-herdr.test.sh | 73 ++++++++++++++++++++++++++++++++++ 2 files changed, 115 insertions(+) diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index 0684bb8f9..e95eae792 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -785,6 +785,48 @@ The luminance rule assumes a dark terminal theme (the fleet reality); the SGR-2 **Resolved: backend-independent wedge alarm.** The max-defer wedge alarm (`inject_wedge_alarm`, `bin/fm-supervise-daemon.sh`) formerly alarmed into the void because its only active signal was a tmux client status-line flash, skipped for herdr, leaving only the passive `state/.subsuper-inject-wedged` marker. It now also attempts a configurable active alert independent of the supervisor backend; [`wedge-alarm.md`](wedge-alarm.md) owns its channels and verification evidence. +## Incident (2026-07-16): in the C locale, a composer's own border read as a bare prompt + +`FM_BACKEND_HERDR_BARE_PROMPT_RE` was spelled as the bracket class `'^[❯›]'`. +A bracket class is a set of CHARACTERS only where grep knows what a character is. +Under a byte-oriented locale (`LC_CTYPE=C`) grep has no multibyte notion and walks the class BYTE-wise instead, so `[❯›]` decomposed into the six bytes of the two glyphs' UTF-8 encodings (`e2 9d af`, `e2 80 ba`) and matched any row whose first byte was one of them. +Every box-drawing glyph starts `0xE2`, so `╭`, `╰` and `│` all matched the "bare prompt" pattern. + +This is a heisenbug by construction: an interactive shell is UTF-8, where the bracket class is glyph-wise and correct, while the supervise daemon inherits the C locale, where it is not. +The classifier was therefore correct everywhere it was ever run by hand and wrong in the only place it mattered. + +**Effect.** `fm_backend_herdr_composer_state` scans rows top-down and lets the LOWEST match win (the live composer is the bottom-most one). +In a bordered composer the bottom-most row is the closing border, so the scan overwrote its own correct `shape=bordered` verdict with `shape=bare` and carried the BORDER row in as composer content, discarding the real composer row entirely. +Both verdict directions broke, and they are not equally bad: + +``` +# empty composer, light theme -> the border IS the content, so it reads as text +'╰──── Composer ─╯' -> pending # away-mode defers forever: the delivery wedge + +# real typed text, dark theme -> the kept border is dark-truecolor, so the ghost +# stripper drops it and the row reads blank, while '❯ hello captain' was discarded +'❯ hello captain' -> empty # away-mode injects OVER the captain's draft +``` + +The second is the reason this was worth a same-day fix rather than a backlog note. +A false `pending` only defers delivery (fail-safe, and the max-defer alarm surfaces it); a false `empty` tells the injector the composer is free and destroys unsubmitted human input. + +**Fix.** `bin/backends/herdr.sh` spells the default as the alternation `'^(❯|›)'`, which matches each full glyph SEQUENCE rather than a set of loose bytes, and so is correct in any locale. +The new pattern matches a strict subset of the old one's matches - it removes only false positives - so no previously-correct read changes. +`docs/configuration.md` carries the same warning on the override knob, since a user-supplied `FM_BACKEND_HERDR_BARE_PROMPT_RE` written as a bracket class reintroduces exactly this bug. + +**Regression coverage (`tests/fm-backend-herdr.test.sh`).** No composer case pinned a locale before this, and that gap is what let the regression ship. +Every existing case inherits the AMBIENT locale, so what it proves depends on where it runs: in a UTF-8 interactive shell the bracket class is glyph-wise and the suite is green on the broken pattern, while in a C-locale non-interactive context the same case fails. +The verdict was a property of the developer's terminal rather than of the code - so the fix pins the locale in the assertion instead of inheriting it. + +Three cases now run the real `fm_backend_herdr_composer_state` under an explicit `LC_ALL=C`. +Two are the regression pins, and both fail on `'^[❯›]'` and pass on `'^(❯|›)'`: an empty bordered composer reads `empty` (`test_composer_state_bare_prompt_is_empty_in_byte_locale`; `pending` before the fix, the wedge direction), and a dark-themed composer holding real typed text reads `pending` (`test_composer_state_dark_border_real_text_is_pending_in_byte_locale`; `empty` before the fix, the unsafe direction). +The third, unbordered `❯` / `›` prompt rows reading `empty` (`test_composer_state_unbordered_prompt_glyphs_are_empty_in_byte_locale`), passes on both patterns by design: it pins that the alternation still MATCHES each full glyph in the C locale, so that a pattern matching NOTHING could not satisfy the other two while silently breaking every unbordered composer read. + +**Deferred to a follow-up (captain's call, deliberately not in this change).** The composer pipeline's other text steps pin their own locale (`fm_composer_strip_ansi` uses `LC_ALL=C sed`, `fm_composer_strip_ghost` uses `LC_ALL=C awk`), but the two `grep` call sites - the bare-prompt match in `fm_backend_herdr_composer_state` and `fm_composer_idle_matches` (`bin/fm-composer-lib.sh`) - inherit the ambient locale. +The alternation makes the DEFAULT pattern locale-invariant, so the shipped behavior is correct, but an operator override of `FM_BACKEND_HERDR_BARE_PROMPT_RE` or `FM_BACKEND_HERDR_IDLE_RE` still carries locale-dependent semantics. +Pinning `LC_ALL=C` at those call sites (and auditing the shared lib for the same byte-vs-glyph hazard) is a separate change against `bin/fm-composer-lib.sh`, which both backends share. + ## Native `pane.agent_status_changed` push escalation (immediate blocked wake) Herdr exposes a native, push-based agent-state event stream, and firstmate folds it into the watcher so a crew entering `blocked` (waiting on the human at a permission/trust dialog, an interactive menu, or a wedged prompt) wakes its supervisor sub-second instead of after the ~240s stale-pane wedge timer. diff --git a/tests/fm-backend-herdr.test.sh b/tests/fm-backend-herdr.test.sh index 5186cca2c..a68bafca6 100755 --- a/tests/fm-backend-herdr.test.sh +++ b/tests/fm-backend-herdr.test.sh @@ -823,6 +823,76 @@ test_composer_state_bare_prompt_is_empty() { pass "fm_backend_herdr_composer_state: a bare '❯' composer row reads empty" } +# --- composer_state: C-locale (byte-oriented) classification ------------------ +# +# Incident (2026-07-16, docs/herdr-backend.md): FM_BACKEND_HERDR_BARE_PROMPT_RE +# was spelled as the bracket class '^[❯›]'. A bracket class is a set of +# CHARACTERS only where grep knows what a character IS: under a byte-oriented +# locale (LC_CTYPE=C - what the supervise daemon inherits, unlike an interactive +# UTF-8 shell) grep walks the class BYTE-wise, so [❯›] became the six bytes of +# the two glyphs' UTF-8 encodings and matched every row starting with 0xE2 - the +# first byte of every box-drawing glyph (╭ ╰ │). The row scan lets the LOWEST +# match win, so a composer's own closing border overwrote the correct +# shape=bordered verdict with shape=bare and carried the BORDER row in as +# content, discarding the real composer row entirely. +# +# These cases PIN the locale, which no other composer case does. The rest of the +# suite inherits the ambient one, so their verdict depends on where they run: a +# UTF-8 interactive shell (bracket class glyph-wise, bug invisible - how this +# shipped) or a C-locale non-interactive/daemon context (bug visible). Pinning +# takes that lottery out of the assertion. +# Cases 1 and 2 fail on '^[❯›]' and pass on '^(❯|›)'; case 3 passes on both by +# design - it exists so a pattern that matched NOTHING could not satisfy 1 and 2. + +# Direction 1 (the wedge): an EMPTY composer read 'pending' - the away-mode +# injector's "captain is mid-sentence" defer signal - so delivery stalled. +test_composer_state_bare_prompt_is_empty_in_byte_locale() { + local dir log resp fb out + dir="$TMP_ROOT/composer-bare-c-locale"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + printf ' ╭────────────────────────╮\n │ ❯ │\n ╰──────── Composer ─────╯\n\n Shift+Tab:mode\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + out=$( PATH="$fb:$PATH" LC_ALL=C FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = empty ] || fail "an empty bordered composer must read empty in the byte-oriented C locale, got '$out' (a '╰──── Composer ─╯' border row matched the bare-prompt pattern byte-wise and was classified as the composer)" + pass "fm_backend_herdr_composer_state: an empty bordered '❯' composer reads empty in the C locale, not pending" +} + +# Direction 2 (the unsafe one): with the border misread as the composer row, the +# REAL typed row was discarded and the dark-truecolor border it kept instead was +# dropped as ghost - so a composer holding the captain's unsubmitted draft read +# 'empty' and the injector would have typed over it. A false 'pending' only +# defers; a false 'empty' destroys human input. +test_composer_state_dark_border_real_text_is_pending_in_byte_locale() { + local dir log resp fb out + dir="$TMP_ROOT/composer-dark-border-c-locale"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + printf ' \x1b[38;2;86;82;110m\xe2\x95\xad\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x95\xae\x1b[39m\n \x1b[38;2;86;82;110m\xe2\x94\x82\x1b[38;2;224;222;244m \xe2\x9d\xaf hello captain \x1b[38;2;86;82;110m\xe2\x94\x82\x1b[39m\n \x1b[38;2;86;82;110m\xe2\x95\xb0\xe2\x94\x80\xe2\x94\x80 Composer \xe2\x94\x80\xe2\x95\xaf\x1b[39m\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + out=$( PATH="$fb:$PATH" LC_ALL=C FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = pending ] || fail "real typed text in a dark-themed bordered composer must read pending in the byte-oriented C locale, got '$out' (reading it empty lets the away-mode injector overwrite the captain's unsubmitted draft)" + pass "fm_backend_herdr_composer_state: a dark-themed composer holding real typed text reads pending in the C locale, never empty" +} + +# Direction 3: the alternation must still MATCH each full glyph sequence in the C +# locale, so an unbordered agent prompt is still recognized. Without this, a +# bare-prompt pattern that matched NOTHING would satisfy both cases above while +# silently breaking every unbordered composer read. +test_composer_state_unbordered_prompt_glyphs_are_empty_in_byte_locale() { + local dir log resp fb out glyph idx=1 + dir="$TMP_ROOT/composer-bare-glyphs-c-locale"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + for glyph in '❯' '›'; do + printf '%s\n' "$glyph" > "$resp/$idx.out" + idx=$((idx + 1)) + done + fb=$(make_herdr_fakebin "$dir") + for glyph in '❯' '›'; do + out=$( PATH="$fb:$PATH" LC_ALL=C FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = empty ] || fail "an unbordered agent prompt '$glyph' must still be recognized as an empty composer row in the byte-oriented C locale, got '$out'" + done + pass "fm_backend_herdr_composer_state: unbordered '❯' and '›' prompt rows still read empty in the C locale (the alternation matches each full glyph)" +} + test_composer_state_ghost_placeholder_is_empty() { local dir log resp fb out dir="$TMP_ROOT/composer-ghost"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" @@ -2084,6 +2154,9 @@ test_busy_state_working_maps_to_busy test_busy_state_done_and_blocked_map_to_idle test_busy_state_unknown_on_no_agent test_composer_state_bare_prompt_is_empty +test_composer_state_bare_prompt_is_empty_in_byte_locale +test_composer_state_dark_border_real_text_is_pending_in_byte_locale +test_composer_state_unbordered_prompt_glyphs_are_empty_in_byte_locale test_composer_state_ghost_placeholder_is_empty test_composer_state_real_text_is_pending test_composer_state_popup_placeholder_fill_is_pending From 67893c40e00cd351c8285c0307aa9890345bb793 Mon Sep 17 00:00:00 2001 From: Zachary Landes Date: Thu, 16 Jul 2026 11:10:31 -0700 Subject: [PATCH 3/3] no-mistakes(document): point locale warning at its owner, log deferred gap --- docs/configuration.md | 2 +- docs/herdr-backend.md | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/configuration.md b/docs/configuration.md index ab0fd9c35..b9adc14f6 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -342,7 +342,7 @@ FM_BACKEND= # optional runtime backend override for new spawns; tmux HERDR_SESSION=default # herdr-only: named session for normal backend ops; not enough for destructive cleanup (docs/herdr-backend.md) FM_BACKEND_HERDR_COMPOSER_LINES=20 # herdr-only: tail lines scanned by composer-state guard/fallback paths; idle-baseline submit confirmation uses agent-state FM_BACKEND_HERDR_IDLE_RE='^Type a message\.\.\.$' # herdr-only: empty-composer placeholder regex after shared ghost extraction plus border and prompt stripping -FM_BACKEND_HERDR_BARE_PROMPT_RE='^(❯|›)' # herdr-only: verified agent glyphs recognized as an UNBORDERED (bare) composer row, e.g. claude's ❯ or codex's ›; shell glyphs remain unknown rather than empty, and de-emphasised ghost/placeholder text (dim or dark-truecolor) after an agent prompt reads empty via the shared fm_composer_strip_ghost (docs/herdr-backend.md "Incident (2026-07-08)", "Incident (2026-07-10)"); spell an override as an alternation, never a [❯›] bracket class, which a byte-oriented locale matches byte-wise and misreads composer borders through (see the default's comment in bin/backends/herdr.sh) +FM_BACKEND_HERDR_BARE_PROMPT_RE='^(❯|›)' # herdr-only: verified agent glyphs recognized as an UNBORDERED (bare) composer row, e.g. claude's ❯ or codex's ›; shell glyphs remain unknown rather than empty, and de-emphasised ghost/placeholder text (dim or dark-truecolor) after an agent prompt reads empty via the shared fm_composer_strip_ghost (docs/herdr-backend.md "Incident (2026-07-08)", "Incident (2026-07-10)"); spell an override as an alternation, never a [❯›] bracket class, which a byte-oriented locale matches byte-wise and misreads composer borders through (docs/herdr-backend.md "Incident (2026-07-16)") FM_BACKEND_HERDR_PI_COMPOSER_MAX_LINES=8 # herdr-only: maximum rows admitted between Pi's native-identity-corroborated separator pair; taller or ambiguous candidates stay unknown (docs/herdr-backend.md "Incident (2026-07-14)") FM_BACKEND_HERDR_SUBMIT_POLLS=6 # herdr-only: agent-state samples spread across each Enter attempt's budget when confirming a submit (docs/herdr-backend.md "Native agent-state submit confirmation") FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP=0.6 # herdr-only: minimum per-Enter confirmation budget before polling agent-state after an idle baseline diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index e95eae792..a88491769 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -926,6 +926,8 @@ Covered by the unit cases in `tests/fm-afk-launch.test.sh` (clear-on-fresh-entry - **Ghost/placeholder suggestion handling depends on ANSI style.** See "Incident (2026-07-08)" and "Incident (2026-07-10)" above. Herdr 0.7.3 preserves the harness's own de-emphasis style (dim/faint and truecolor foreground) in `pane read --format ansi`, and `fm_backend_herdr_composer_state` extracts real typed content with the shared `fm_composer_strip_ghost` (`bin/fm-composer-lib.sh`), which drops dim/faint AND dark-truecolor runs to distinguish ghost suggestions/placeholders from real typed text. If a future herdr build strips ANSI style from `--format ansi`, the classifier loses its ghost signal and falls back to reading the suggestion text as `pending` - the fail-safe direction (it defers rather than risks overwriting a human draft), which the max-defer alarm then surfaces. +- **The composer's two `grep` call sites still inherit the ambient locale.** See "Incident (2026-07-16)" above, which owns the byte-vs-glyph account and why the pin was deferred. + The shipped `FM_BACKEND_HERDR_BARE_PROMPT_RE` and `FM_BACKEND_HERDR_IDLE_RE` defaults are locale-invariant, so fleet behavior is correct, but an operator override still carries locale-dependent matching semantics until `LC_ALL=C` is pinned at those call sites the way `fm_composer_strip_ansi` and `fm_composer_strip_ghost` already pin theirs. - **RESOLVED: a "paused / awaiting-external" crew state for the stale-wedge escalation.** Raised alongside the 2026-07-07 incident: an in-flight crew intentionally idling on a known external wait (a vendor rate limit, say) still tripped `bin/fm-supervise-daemon.sh`'s "stale persisted ... (possible wedge)" escalation exactly like a genuinely wedged crew, with no way to mark the wait as expected. Fixed by the `paused:` external-wait verb: a crew declares a deliberate wait, and both `bin/fm-watch.sh` and `bin/fm-supervise-daemon.sh` absorb its idle pane through the shared `bin/fm-classify-lib.sh` vocabulary (`status_is_paused`, `crew_absorb_class`, `FM_PAUSE_RESURFACE_SECS`), re-surfacing it for a recheck on a long cadence instead of a wedge escalation. See `AGENTS.md` section 8 and the crew-facing brief contract in `bin/fm-brief.sh`.