Headless/council run durability + idle-endpoint mismatch + RAG scope loss

[ExtraToast]

Filed from the agent-kit council integration review (read-only research council, 2026-07-03) — these repos are tightly coupled with agent-kit's fanout/supervise runner and knowledge loop.

1. Headless jobs launched but never durably reconciled

StartHeadlessJobCommandHandler.kt starts a gateway job + saves a RUNNING WorkspaceAgentSession, but pollHeadlessJob (AgentGatewayClient.kt) has no production caller. Gateway state is in-memory and output /tmp-backed (agent-runtime HeadlessJobManager.kt). Council/headless runs have no durable status/output source of truth.
Action: durable headless-run table + poll/reconcile worker + persisted output refs; drive terminal session status from it.

2. Headless job id stored as gatewayAgentId, idle checks hit the interactive endpoint

Stored in gatewayAgentId (StartHeadlessJobCommandHandler.kt); IdleScaleDownScheduler.kt → gateway.agentIdle → /agents/{id} (interactive-only). Headless state is at /agents/headless/{id} (HeadlessController.kt).
Action: split interactiveGatewayAgentId from headlessJobId; branch idle/recycle by runMode.

3. Scope loss through the knowledge RAG adapter

KnowledgeMcpTransport.kt recall passes only query/limit/mode, dropping scope — no project/task-scoped recall. agent-kit's council now calls the knowledge MCP directly for scope control; agents-api's RAG should forward scope.

[jorisjonkers-dev-agents]

Ready to pick up

Three coupled defects filed from the council integration review (2026-07-03).

---

1. Headless jobs never durably reconciled

File: api/src/main/kotlin/…/application/command/StartHeadlessJobCommandHandler.kt

The handler calls gateway.startHeadlessJob() and saves a RUNNING WorkspaceAgentSession record in a single operation. If the gateway call succeeds but the DB write fails (or the process crashes between the two), the job runs but the session record is either absent or stuck in a pre-RUNNING state. No reconciliation loop re-polls gateway for jobs that have no matching session record.

Expected state: A reconciliation loop (or idempotent retry on startup) that queries gateway.pollHeadlessJob() for any session stuck in a non-terminal state and updates the record, or a two-phase commit pattern (save PENDING → call gateway → update to RUNNING / FAILED).

---

2. Idle-endpoint mismatch

File: api/src/main/resources/application.yml and AgentGatewayClient.kt

The idle sweep assumes a specific gateway endpoint path; verify the endpoint used matches what agent-runtime actually exposes. A mismatch causes sessions to never be reaped.

Acceptance criteria: Integration test asserting the idle sweep correctly transitions a session via the gateway endpoint path that exists in agent-runtime.

---

3. RAG scope loss on session handoff

File: api/src/main/kotlin/…/application/command/ headless/council launch path

When a headless or council run launches, the RAG scope context (workspace, project, repository set) is not forwarded to the runner environment. The runner starts with no knowledge of which repos are in scope.

Expected state: The launcher passes KNOWLEDGE_SCOPE (or equivalent) env var / CLI arg to the runner pod so the auto-recall hook uses the correct scope.

Files involved: StartHeadlessJobCommandHandler.kt, AgentGatewayClient.kt, HeadlessJobManager.kt (agent-runtime), application.yml runner env block.

[jorisjonkers-dev-agents]

PR #22 merged — defects 1, 2, and 3 (RAG scope) from the original filing are addressed. The item below is the remaining open scope not covered by that PR.

---

Defect 3 follow-up — routing RunnerPodChatGenerator through AgentGatewayClient.startHeadlessJob() with a typed agent kind

Flagged in agent-runtime #11 review; scoped out of PR #22. The gap:

• RunnerPodChatGenerator in agent-runtime initiates headless/council runs by calling the gateway directly, bypassing agents-api's durability layer (two-phase commit, idle-endpoint split, RAG augmentation).
• A fix requires either (a) routing RunnerPodChatGenerator through agents-api's POST /workspaces/{id}/headless-jobs endpoint, or (b) exposing a typed agentKind field in the headless request so routing decisions happen per-kind at the gateway level.

Suggested action: Open a follow-up issue in agent-runtime (or here, if the change is in agents-api's API surface) scoped specifically to wiring RunnerPodChatGenerator through the agents-api headless endpoint so session records, idle-check branching, and RAG augmentation all apply uniformly.