bug(runtime/hooks): orchestrator shell defeats the daemon PATH pin — `ao hooks` resolves to a foreign `ao`, breaking lifecycle hooks and steering to the wrong daemon

[AgentWrapper]

Bug

The ReverbCode-spawned orchestrator session's interactive shell resolves bare ao to a different, non-ReverbCode ao install. This (a) breaks ReverbCode's lifecycle hooks (ao hooks ... fails with error: unknown command 'hooks') and (b) steers any orchestration the agent runs through bare ao to the wrong daemon. Symptom observed live: Failed with non-blocking status code: error: unknown command 'hooks' on prompt submission in the orchestrator session.

The root issue is two-fold:

1. ReverbCode ships no namespaced ao on PATH. The only ao a user typically has on PATH is the legacy npm/Homebrew agent-orchestrator CLI (v0.9.2 / 0.29.4), which has no hooks subcommand.
2. The daemon PATH pin is silently defeated for the orchestrator's shell. The pin (HookPATH) prepends the daemon-binary dir to PATH at the sh -lc wrapper layer, but the wrapper finishes with exec $SHELL -i — an interactive login shell that re-sources the user's rc files, which re-prepend other dirs (e.g. nvm) ahead of the pinned daemon dir. Bare ao then resolves to the foreign CLI again, and every ao hooks ... callback hits it.

Source: triage session (no public URL) | Reported by: @Itrytoohard | Analyzed against: 6ac65d2
Confidence: High (exact code path traced, mechanism reproduced end-to-end)
Priority: High (lifecycle hooks dead → activity tracking stalls; orchestration can hit the wrong daemon)

Reproduction

Two ao installs on the same machine:

which ao          # /Users/<me>/.nvm/versions/node/v22.11.0/bin/ao  (legacy npm CLI, daemon :3000)
ao hooks          # error: unknown command 'hooks'        <-- no hooks subcommand

# ReverbCode's bundled daemon binary (port 3001) DOES have it:
RB_AO=".../Agent Orchestrator.app/Contents/Resources/daemon/ao"
"$RB_AO" status        # port: 3001
"$RB_AO" hooks --help  # Usage: ao hooks <agent> <event>

The orchestrator pane is launched (Zellij adapter) as:

$SHELL -lc "export PATH='<daemon-dir>:...:<nvm-dir>:...'; <agent argv>; exec $SHELL -i"

The pin works at the -lc layer, but exec $SHELL -i re-sources rc files. Proven locally:

PINNED="<daemon-dir>:/opt/homebrew/bin:/usr/bin:/bin:<nvm-dir>"

# At the -lc wrapper layer (no rc re-source): pin holds, correct ao
PATH="$PINNED" sh -c 'command -v ao'
# -> <daemon-dir>/ao              (ReverbCode, correct)

# Inside the exec'd interactive login shell (.zshrc runs nvm.sh, re-prepends nvm):
PATH="$PINNED" zsh -i -c 'command -v ao'
# -> <nvm-dir>/ao                 (legacy CLI, WRONG)

# Hook child shell spawned by the agent from that interactive shell inherits the flip:
PATH="$PINNED" zsh -i -c 'sh -c "command -v ao"'
# -> <nvm-dir>/ao                 (legacy CLI -> 'ao hooks ...' fails)

Claude Code installs its hooks as the bare string ao hooks claude-code user-prompt-submit and runs them via a child shell, so UserPromptSubmit → error: unknown command 'hooks' — exactly the observed failure.

Root Cause

• PATH pin: backend/internal/session_manager/manager.go:739 (runtimeEnv) → backend/internal/session_manager/manager.go:758 (HookPATH). The pin prepends the daemon executable's dir to PATH and is applied to both worker and orchestrator spawns (call sites manager.go:275 and manager.go:503). The pin requires the daemon binary be named ao (manager.go:50, hookBinaryName = "ao"); the bundled daemon binary is indeed named ao, so the pin is applied.
• Pin defeated by the launch wrapper: backend/internal/adapters/runtime/zellij/commands.go:119 (wrapLaunchCommandUnix) builds export PATH=<pinned>; <argv>; exec $SHELL -i (lines 136-144) using the login shell spec -lc (commands.go:97). The trailing exec $SHELL -i re-sources the user's interactive/login rc files, which can re-prepend dirs (nvm, etc.) ahead of the pinned daemon dir. The pin is therefore correct in the parent env but is clobbered inside the interactive shell the agent and its hook child-shells actually run in.
• Hook command is bare ao: backend/internal/adapters/agent/claudecode/hooks.go:24 (claudeHookCommandPrefix = "ao hooks claude-code ") and hooks.go:62-68 (managed hooks incl. user-prompt-submit). Other adapters do the same (cline, kilocode, droid, devin all install bare ao hooks <agent> <event>). None use an absolute path to the daemon binary.

Net: there is no namespaced ReverbCode ao on PATH, and the one PATH-pin that would compensate is undone by the orchestrator's interactive-shell relaunch, so bare ao falls back to a foreign daemon's CLI.

Fix

Pick one or combine (defense in depth):

1. Install/namespace ReverbCode's ao on PATH so a bare ao is unambiguous regardless of shell rc order — e.g. symlink the bundled daemon binary to ~/.ao/bin/ao and have the app ensure ~/.ao/bin is on PATH, or ship a reverbcode-ao and have adapters install hooks/prompts that call it. This removes the reliance on PATH ordering entirely.
2. Make the hook commands not depend on PATH ordering. Install hook command strings using the absolute path to the daemon binary (the value already resolved in HookPATH/os.Executable) instead of bare ao. The adapters already have a stable prefix; inject the resolved binary path at install time.
3. Stop the interactive-shell relaunch from clobbering the pin. Re-assert the pinned PATH after exec $SHELL -i re-sources rc files (e.g. append a one-shot that re-prepends the daemon dir, or launch the agent without an rc-sourcing login shell), so the daemon dir stays first in the shell the agent and its hook child-shells run in. See backend/internal/adapters/runtime/zellij/commands.go:119-145.

Option 1 or 2 is the durable fix; option 3 alone is fragile across shells.

Impact

• Lifecycle hooks dead in the orchestrator session — SessionStart/UserPromptSubmit/Stop/Notification/SessionEnd callbacks fail (unknown command 'hooks'), so activity-state tracking and session metadata reporting stall for orchestrators on any machine with a foreign ao first in interactive PATH.
• Orchestration can hit the wrong daemon — any bare ao spawn/ao send the orchestrator agent runs (its prompt instructs it to) resolves to the legacy CLI on :3000, not ReverbCode's :3001.
• Affects the common dev setup (nvm/Homebrew + a previously-installed agent-orchestrator). manager.go does log a warning only when the pin cannot be applied — here the pin is applied yet still defeated, so the failure is silent.

Related

• #89 — earlier smoke-test triage flagged the same symptom (spawned agents resolve a foreign ao 0.9.2) and recommended a PATH pin. That pin (HookPATH/runtimeEnv) has since shipped but is incomplete: this issue identifies why it is still defeated for the orchestrator shell (the exec $SHELL -i rc re-source) and the missing namespaced CLI.
• #78 — added the ao hooks <agent> <event> subcommand the foreign CLI lacks.