feat: human-in-the-loop ambient integration with smart escalation (closes #482)

[SakethSumanBathini]

Summary

Closes #482. AgentWatch can block dangerous actions, but has no proactive way to bring a human into the loop for the actions that fall short of an outright block but still warrant judgment — today, a user has to manually check a dashboard to find out anything needs attention. This PR adds an ambient human-in-the-loop (HITL) layer: agent actions are triaged by risk level into one of three interaction patterns — a fire-and-forget notify, an question where the agent asks a human for clarification, or a blocking review requiring approve/reject/edit — with every decision recorded in an audit trail and a bounded feedback loop that adjusts escalation thresholds based on observed human behavior.

Why this design

As with the circuit breaker in #483, this PR deliberately builds on infrastructure AgentWatch already has rather than introducing a parallel notification system:

• Delivery is delegated to the existing AlertingEngine (agentwatch/alerting/engine.py) — surfaced requests are routed through alert_event(), reusing its Slack/PagerDuty webhook delivery, retry logic, and signing. The HITL layer adds escalation logic and decision tracking on top; it doesn't reimplement webhook plumbing.
• Risk classification reuses the existing RiskLevel enum and AgentEvent.safety (agentwatch/core/schema.py) — the same risk tier the safety engine already assigns to every tool call.
• The alerting engine is an optional, dependency-injected parameter, not a hard import — an HITLOrchestrator works standalone (e.g. for tests, or a deployment without alerting configured yet), and a delivery failure is caught and logged rather than breaking request bookkeeping (mirroring the same graceful-degradation pattern from the [Feat] Active Circuit Breaker with Safe Pause & Resume #483 circuit breaker).

The three interaction patterns

Pattern	When	Behavior
NOTIFY	Low/medium risk (configurable)	Delivered through the alerting engine, then auto-approved — informational, doesn't block the agent
QUESTION	Medium risk, or explicitly requested by the agent	A human is asked for clarification; the agent waits for a text answer()
REVIEW	High/critical risk (configurable)	Blocking; a human must approve(), reject(), or edit() before the action proceeds

RiskLevel.SAFE actions (and events with no safety check at all) are auto-approved immediately and never surfaced — no notification noise for routine activity. An agent can also force the QUESTION pattern explicitly (by supplying a question= argument to submit()) regardless of the event's risk level, since "the agent is asking for clarification" is a QUESTION by definition, independent of risk.

Configurable thresholds

HITLThresholds maps each RiskLevel to a pattern (or None for auto-approve), validates its own configuration (a non-positive timeout raises), and backfills any risk levels the caller didn't specify from sensible defaults — so a partially-specified policy can never produce a KeyError on lookup. The default policy: SAFE auto-approves, LOW/MEDIUM notify, HIGH/CRITICAL require review.

Decision workflow and audit trail

• approve() / reject() / edit(payload) resolve a REVIEW request; answer(text) resolves a QUESTION; expire() marks a stale PENDING request as timed out.
• Every decision — including automatic ones (auto-approved SAFE events, notify-then-proceed) — produces an immutable HumanDecision in the audit trail, queryable globally (audit_trail()) or per-session (audit_for_session()).
• Attempting to decide an unknown request, a request that's already resolved, or a request via the wrong pattern (e.g. calling answer() on a REVIEW request) raises HITLError with a clear message rather than silently no-oping or corrupting state.

Learning from human feedback — a bounded, explainable heuristic

The issue's "learning from human feedback" criterion is intentionally the most open-ended one, so this PR implements a concrete, explainable version rather than an opaque model: FeedbackLearner tracks, per risk level, the approval-vs-rejection rate of genuine human REVIEW decisions (auto-approvals are explicitly excluded — they carry no signal about human judgment). Once a level accumulates enough samples (min_samples, default 10):

• A consistently high approval rate (>= relax_above, default 95%) suggests relaxing that level one step (e.g. REVIEW → QUESTION) — humans keep approving it, so blocking review may be unnecessary friction.
• A low approval rate (<= tighten_below, default 50%) suggests tightening it one step (e.g. NOTIFY → QUESTION) — frequent rejections mean the level deserves more scrutiny, not less.
• CRITICAL is never relaxed below REVIEW, regardless of approval rate — a hard floor, not a suggestion the learner can override.
• suggest() returns proposed changes with a human-readable reason (e.g. "approval rate 100% over 10 decisions; review may be unnecessary overhead"); apply() actually mutates the thresholds. Both are available — a deployment can choose to log suggestions for human review, or apply them automatically.

This is explicitly not a machine-learning system: it's a small, auditable rate-based heuristic with hard-coded safety bounds, so its behavior is always predictable and explainable — the same design principle behind the honest, scoped-down proposal from the #401 feasibility discussion, applied here proactively rather than after overreaching.

Testing

45 tests, 100% line coverage across all five new files. Coverage breakdown:

• Thresholds — default policy resolves every risk level; requires_human() correctly distinguishes NOTIFY from QUESTION/REVIEW; invalid timeout raises; a partially-specified policy is backfilled; set_pattern() mutates correctly.
• Submit / triage — SAFE and no-safety-data events auto-approve; NOTIFY events deliver through the alerting engine then auto-approve without blocking; REVIEW events stay PENDING; an explicit question= forces the QUESTION pattern regardless of risk; custom summaries are respected.
• Decisions — approve/reject/edit/answer/expire all tested on their happy paths; edit correctly replaces the payload and is reflected in the audit entry; deciding an unknown request, a resolved request, or via the wrong pattern all raise HITLError; expiring an already-resolved request raises.
• Queries and audit — pending() correctly excludes auto-approved/resolved requests; get() returns None for unknown ids; the audit trail records every decision (including auto-approvals) and can be filtered per-session; HumanDecision.to_dict() and HITLRequest.to_dict() round-trip correctly.
• Delivery — a delivery failure through a broken alerting engine is caught and logged, and does not prevent the request from being created and later decided; the orchestrator works with no alerting engine configured at all.
• Feedback learner — invalid configuration (tighten ≥ relax, non-positive min_samples) raises; approval-rate tracking (including that EDIT counts as approval); no suggestion below min_samples; correct relax/tighten suggestions at the configured boundaries; CRITICAL never relaxed below REVIEW even at 100% approval; a mid-range approval rate produces no suggestion; a level already at its floor (auto-approve) or ceiling (REVIEW) produces no further suggestion; apply() correctly mutates the thresholds in place.
• Orchestrator + feedback integration — genuine human decisions feed the learner; auto-approvals (SAFE, NOTIFY) explicitly do not pollute the learner's approval-rate signal; one full end-to-end lifecycle test (submit a HIGH-risk event → delivered through alerting → pending REVIEW → human approves → audited → fed to the learner).

Additionally verified (learned directly from the #483 review, applied proactively here):

• ruff check — clean.
• ruff format --check — clean, run explicitly on both source and test files from the start (this was the gate that initially failed on [Feat] Active Circuit Breaker with Safe Pause & Resume #483's first push; verified up front this time rather than after the fact).
• mypy — clean, no type errors across all 5 source files.
• bandit — clean.
• No regressions: tests/test_alerting.py (the module this integrates with) and a broader slice of the existing suite still pass; the full package still imports cleanly.
• Two internal defensive guards (if current not in order: return current in the relax/tighten logic) were initially unreachable dead code once test coverage revealed they could never be hit — rather than suppressing the coverage gap, they were removed and replaced with a single exhaustive _PATTERN_ORDER constant, since the order list is provably exhaustive over InteractionPattern | None.

Scope note

This PR delivers the core HITL orchestration: all three interaction patterns, configurable per-risk thresholds, the full approve/reject/edit/answer decision workflow, a complete audit trail, and the bounded feedback-learning heuristic — reusing the existing Slack/PagerDuty delivery from AlertingEngine. Discord as a third delivery channel is intentionally left as a focused follow-up: it's an additive channel-validator change to agentwatch/alerting/channels.py that doesn't touch the HITL orchestration logic itself, and keeping it separate lets this PR's review stay focused on the escalation/decision/audit/feedback logic, which is where the substantive new behavior lives.

Closes #482.

[vercel]

@SakethSumanBathini is attempting to deploy a commit to the sreerevanth's projects Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

Warning

Review limit reached

@SakethSumanBathini, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 18 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3362fede-a04c-4966-af45-fc64feff08b8

📥 Commits

Reviewing files that changed from the base of the PR and between 49932f6 and 8913ca9.

📒 Files selected for processing (6)

• agentwatch/hitl/__init__.py
• agentwatch/hitl/decisions.py
• agentwatch/hitl/feedback.py
• agentwatch/hitl/orchestrator.py
• agentwatch/hitl/thresholds.py
• tests/test_hitl.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🧪 PR Test Results

Check	Result
Tests (pytest tests/)	✅ success
Lint (ruff check .)	❌ failure
Coverage (agentwatch)	74.24%

_{Python 3.12 · commit 8913ca9}