Skip to content

fix: parked approval escalations that leave chats permanently stuck - #39

Open
shipdocs wants to merge 3 commits into
HKUDS:mainfrom
shipdocs:agent/fix-approval-escalation-hangs
Open

fix: parked approval escalations that leave chats permanently stuck#39
shipdocs wants to merge 3 commits into
HKUDS:mainfrom
shipdocs:agent/fix-approval-escalation-hangs

Conversation

@shipdocs

@shipdocs shipdocs commented Aug 14, 2026

Copy link
Copy Markdown

Summary

Chats that hit a tool-approval escalation could get permanently stuck with no way to unblock them — restarting the engine just re-parks on the same escalation forever. Two commits, addressing the same failure class from different angles:

1. fix(ui): keep approval escalations actionable

  • Recognize read-only sed -n expressions that use regex addresses, avoiding unnecessary approval escalation.
  • Keep the chat composer available during an active turn so the backend can queue or route follow-up messages.
  • Render a complete, clickable approval card in the active session when the canonical escalation card was mirrored elsewhere.

2. fix: recover parked approval escalations and outbound dispatch hangs

Root-caused via a live .opc/ui_state.db inspection of a stuck run: for the incident under investigation, zero human_escalation card messages existed anywhere (session or activity channel) even though the task had correctly parked.

  • EventBus.publish() awaits every escalation_created listener through asyncio.gather(..., return_exceptions=True). Any exception inside _mirror_escalation was silently swallowed — no log, no card in any channel, and critically no pending-escalation future ever got registered, so even a later correct reply had nothing live to resolve. _mirror_escalation now catches failures at each resolution step, logs them, and degrades to the project activity channel instead of losing the card outright.
  • Even when a card is missing, free-text replies were deliberately never treated as a decision (to avoid ambiguity) — and the synonym list didn't even recognize the word "proceed". As a safety net, an unambiguous decision word typed in chat (approve/deny/proceed/go ahead/continue/...) can now resume a parked approval checkpoint directly, without requiring a card click. Ambiguous text still falls through as ordinary conversation, unchanged.
  • The checkpoint-resume progress note was written directly onto task.metadata, but progress_log is WorkItem-owned metadata — this tripped the metadata-ownership invariant checker (metadata_ownership_conflict/violation warnings) on every resume of a work-item-linked task. Now routed through the WorkItem-owned path (append_work_item_progress) when the task is linked to a work item.
  • Separately: the channel outbound-dispatch loop is the sole consumer for every configured channel (Telegram, WhatsApp, Discord, ...); a single send that never returns used to stall delivery for all channels until restart. Outbound sends are now bounded by a configurable dispatch_timeout_seconds (default 30s).

User impact

Read-only inspection commands no longer request approval unnecessarily, users can send a recovery or follow-up message while work is active, pending approvals remain actionable from the chat where the user is working, a card failing to render can no longer strand a task forever, and one hung channel send can no longer stall message delivery for every other channel.

Test plan

  • tests/test_shell_safety.py, tests/test_ws_handler_escalations.py, tests/test_approval_escalation_reply.py, tests/test_approval_engine.py, tests/test_chat_store_progress_folding.py, tests/test_metadata_ownership.py — all green
  • tests/test_attempt_ledger.py, tests/test_checkpoint_answer_live_dispatcher.py, tests/test_company_recruiter.py, tests/test_company_runtime_suspend_resume.py, tests/test_request_user_input_collab.py, tests/test_run_failure_settlement.py, tests/test_runtime_config_enforcement.py, tests/test_runtime_v2_migration.py, tests/test_self_evolution_pipeline.py, tests/test_stale_checkpoint_selfheal.py — 199 passed
  • tests/test_session_integration.py (192 passed) — new regression: escalation card survives a routing-resolution exception
  • New regression: plain "proceed" resumes a parked approval checkpoint without a card click (test_stale_checkpoint_selfheal.py), "proceed"/"go ahead"/"continue" normalize to an approval decision (test_approval_escalation_reply.py)
  • npm run test:contracts (9 passed), npm run typecheck, npm run build with Node.js 22

@shipdocs
shipdocs marked this pull request as ready for review August 14, 2026 12:18
Approval cards could go missing without a trace: EventBus.publish()
awaits escalation_created listeners via asyncio.gather(return_exceptions=True),
so any exception inside _mirror_escalation was silently swallowed — no
log, no card in any channel, and no pending-escalation future, leaving
the parked task with no way to be answered. _mirror_escalation now
degrades to the activity channel and logs loudly instead of losing the
card. As a safety net for cards that still fail to render, an
unambiguous decision word typed in chat (now including "proceed") can
resume a parked approval checkpoint directly, without requiring a card
click. Also routes the checkpoint-resume progress note through the
WorkItem-owned metadata path instead of writing task.metadata directly,
which was tripping the metadata-ownership invariant checker on every
resume of a work-item-linked task.

Separately, the channel outbound-dispatch loop is the sole consumer for
every channel; a single send that never returns used to stall delivery
for all channels until restart. Outbound sends are now bounded by a
configurable dispatch_timeout_seconds.
@shipdocs shipdocs changed the title Fix approval escalations that can leave chats stuck fix: parked approval escalations that leave chats permanently stuck Aug 14, 2026
…on, team chat

Sequences the fixes into phases: proactive orphan-checkpoint sweep, giving
the Secretary a real delegate-to-team action, and a multi-party team chat
built on the existing activity channel and AgentMessage fan-out.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant