Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/memory/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ Channel DB modules: `db/slack_channels.py` (workspace connections, channel-agent
- `stores/auth.js` - Email/admin authentication + JWT
- `stores/collaborations.js` - Collaboration graph state, WebSocket integration
- `stores/loops.js` - Sequential agent loops UI state, agent-scoped, WebSocket-driven (#1106)
- `stores/executions.js` - Fleet execution list/stats + agent Overview analytics (`fetchAgentAnalytics`, cached per `${name}:${window}`, never polled) (#1107)
- `stores/executions.js` - Fleet execution list/stats + agent Overview analytics (`fetchAgentAnalytics`, cached per `${name}:${window}`, never polled) (#1107) + per-schedule performance rollups (`fetchSchedulesSummary`, same `${name}:${window}` cache; one fetch shared by the Overview "Schedules performance" section and the Schedules-tab inline stats) (#1115)
- `stores/sessions.js` - Session tab state

**Real-time:** WebSocket client at `utils/websocket.js` with auto-reconnect; tracks `_eid` and replays via `last-event-id` — see [Real-time Delivery](#real-time-delivery-reliability-003-306).
Expand Down Expand Up @@ -558,6 +558,7 @@ Lookup keys: S-01/E-02/L-03 shipped via #653; S-02/E-01/E-05/B-01 (Phase 2) and
| GET/PUT/DELETE | `/api/agents/{name}/schedules/{id}` | Get / update (same 400 on timeout) / soft-delete |
| POST | `/api/agents/{name}/schedules/{id}/enable` · `/disable` · `/trigger` | Enable / disable / manual trigger |
| GET | `/api/agents/{name}/schedules/{id}/executions` | Execution history |
| GET | `/api/agents/{name}/schedules/analytics-summary` | **Per-schedule performance rollups for the whole agent** in one compact call (#1115). `?window=` ∈ {7d,14d,30d}→168/336/720h (422 otherwise). One row per **non-deleted** schedule (zero-run schedules included): terminal `success_rate` (`None`→`—` when zero terminal), `avg_duration_ms` (NULL-skip), `cost_total`, `context_avg`, `tool_call_total`, last-run outcome. Backs BOTH the Overview "Schedules performance" section AND the Schedules-tab inline stats from a single fetch (no N round-trips). **Declared before `/{id}` in `routers/schedules.py`** so the literal `analytics-summary` isn't captured as a `schedule_id` (Invariant #4). DB: `get_agent_schedules_summary` (generalises #868). Tool-call totals parsed over the newest 5,000 rows agent-wide (`tool_calls_sampled` flag) |
| GET | `/api/agents/{name}/schedules/{id}/analytics` | Per-schedule analytics: counts, success rate, duration p50/p95/p99, cost, tool-call top-5, daily timeline. `?window_hours=` ∈ {24,168,720}, default 168 (#868). Percentiles Python-side over the newest 5,000 success rows (`sampled:true` reported when capped); counts + timeline full-set; UTC day buckets gap-filled. Tenant boundary in the DB layer (`agent_name` passed through) — `AuthorizedAgent` validates only the path agent name, NOT that `schedule_id` belongs to it, so the DB-layer filter is the actual boundary. Soft-deleted schedules 404 |
| POST/GET/DELETE | `/api/agents/{name}/schedules/{id}/webhook` | Generate/rotate token · status + URL · revoke (WEBHOOK-001) |

Expand Down
18 changes: 18 additions & 0 deletions docs/memory/feature-flows/agent-overview-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,11 +153,29 @@ capabilities, platforms, tools) is tucked behind a native collapsible
- The three tab-validity sites were deduped into one `DEEP_LINK_TABS` constant
(includes `'overview'`); the invalid-tab fallback resets to `'overview'`.

## Schedules performance section (#1115)
Below the trend charts, a **Schedules performance** section lists the agent's
non-deleted schedules — one compact scorecard each (command/name + cron,
terminal success rate, avg duration, runs-in-window, tool calls), honoring the
same 7/14/30d window selector. Each row deep-links to the Schedules tab (the
#868 per-schedule deep view stays the drill-in target). Hidden when the agent
has no schedules. The Schedules tab itself (`SchedulesPanel.vue`) renders the
**same** rollups as inline mini-stats per row. Both consume one
`executions.js` `fetchSchedulesSummary(name, window)` call
(`GET /api/agents/{name}/schedules/analytics-summary`, DB
`get_agent_schedules_summary`) — cached per `${name}:${window}`, no
N per-schedule round-trips.

## Testing
- `tests/unit/test_agent_analytics.py` — bucketing (incl. `Other` fallback),
terminal-based success rate, **full-set avg vs sampled p95** correctness,
NULL-skipping context avg, window boundary, empty agent, gap-filled timeline.
12 tests; mirrors the #868 `test_schedule_analytics.py` fixture machinery.
- `tests/unit/test_1115_schedules_summary.py` — per-schedule rollups:
terminal success rate (incl. `error`, non-terminal excluded), NULL-skipping
avg duration, tool-call total (malformed JSON skipped), zero-run schedule
still appears (rate `None`), soft-deleted excluded, out-of-window excluded.
6 tests.

## Related Flows
- [executions-dashboard.md](executions-dashboard.md) — fleet-level sibling; shares
Expand Down
3 changes: 3 additions & 0 deletions src/backend/database.py
Original file line number Diff line number Diff line change
Expand Up @@ -1603,6 +1603,9 @@ def get_schedule_analytics(self, schedule_id: str, hours: int,
def get_agent_analytics(self, agent_name: str, hours: int):
return self._schedule_ops.get_agent_analytics(agent_name, hours)

def get_agent_schedules_summary(self, agent_name: str, hours: int):
return self._schedule_ops.get_agent_schedules_summary(agent_name, hours)

def get_agent_token_stats(self, agent_name: str):
return self._schedule_ops.get_agent_token_stats(agent_name)

Expand Down
179 changes: 179 additions & 0 deletions src/backend/db/schedules.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,31 @@ def _bucket_for_trigger(trigger: Optional[str]) -> str:
return _TRIGGER_BUCKETS.get(trigger or "", _OTHER_BUCKET)


# #1115: max chars for the per-schedule "command" label derived from a
# schedule's message (the Overview/Schedules-tab scorecard headline).
_SCHEDULE_LABEL_MAX = 80


def _schedule_command_label(message: Optional[str]) -> str:
"""Short headline for a schedule's scorecard, derived from its message.

Uses the first non-empty line (the command/intent, e.g. ``/do-something``),
collapsed and truncated. Empty when the message is blank — the frontend
falls back to the schedule name.
"""
if not message:
return ""
for line in message.splitlines():
stripped = line.strip()
if stripped:
return (
stripped[: _SCHEDULE_LABEL_MAX - 1] + "…"
if len(stripped) > _SCHEDULE_LABEL_MAX
else stripped
)
return ""


# #73: chunk size for scoped `IN (...)` lookups. SQLite caps host parameters at
# SQLITE_MAX_VARIABLE_NUMBER (999 before SQLite 3.32). Keep a safe margin below
# that so a large accessible-agent set can't blow the limit. Read as a module
Expand Down Expand Up @@ -1779,6 +1804,160 @@ def get_schedule_analytics(
"sample_size": sample_size,
}

def get_agent_schedules_summary(self, agent_name: str, hours: int) -> Dict:
"""Per-schedule performance rollups for an agent over a window (#1115).

ONE row per non-deleted schedule (zero-run schedules included, with
zeros), so both the Overview "Schedules performance" section and the
Schedules-tab inline stats render from a single call — no N per-schedule
round-trips. The #868 deep view stays the drill-in target.

Per schedule: terminal **success_rate** (success / (success + failed
[incl. ``error``]); ``None`` when zero terminal runs so the UI shows
``—`` not a false 0%), **avg_duration_ms** (NULL-skipping AVG),
**cost_total**, **context_avg** (NULL-skipping), **tool_call_total**,
and last-run outcome. Read-only / DB-sourced (renders when stopped).

Tool-call totals are parsed from the newest ``_PERCENTILE_ROWSET_CAP``
rows agent-wide (matches the #868 sampling discipline); ``tool_calls_
sampled`` flags when that cap was hit. Window uses ``iso_cutoff``
(Invariant #16).
"""
cutoff = iso_cutoff(hours)
cap = _PERCENTILE_ROWSET_CAP
FAILED_STATES = (TaskExecutionStatus.FAILED, "error")

with get_db_connection() as conn:
cursor = conn.cursor()

# Schedules (non-deleted) — the authoritative row set so a
# zero-run schedule still appears.
cursor.execute(
"""
SELECT id, name, message, cron_expression, enabled
FROM agent_schedules
WHERE agent_name = ? AND deleted_at IS NULL
ORDER BY created_at ASC
""",
(agent_name,),
)
schedule_rows = cursor.fetchall()

# One grouped aggregate for every schedule's executions in window.
cursor.execute(
"""
SELECT
schedule_id,
COUNT(*) AS total,
SUM(CASE WHEN status = 'success' THEN 1 ELSE 0 END) AS success_count,
SUM(CASE WHEN status IN ('failed', 'error') THEN 1 ELSE 0 END) AS failed_count,
SUM(CASE WHEN status = 'cancelled' THEN 1 ELSE 0 END) AS cancelled_count,
SUM(COALESCE(cost, 0)) AS cost_total,
AVG(CASE WHEN duration_ms IS NOT NULL THEN duration_ms END) AS avg_duration_ms,
AVG(CASE WHEN context_used IS NOT NULL THEN context_used END) AS context_avg
FROM schedule_executions
WHERE agent_name = ? AND started_at > ?
GROUP BY schedule_id
""",
(agent_name, cutoff),
)
agg_by_sched = {r["schedule_id"]: r for r in cursor.fetchall()}

# Last-run outcome per schedule. SQLite's bare-column-with-MAX
# rule returns the row holding the max started_at.
cursor.execute(
"""
SELECT schedule_id, MAX(started_at) AS last_run_at, status AS last_status
FROM schedule_executions
WHERE agent_name = ? AND started_at > ?
GROUP BY schedule_id
""",
(agent_name, cutoff),
)
last_by_sched = {r["schedule_id"]: r for r in cursor.fetchall()}

# Tool-call totals — bounded JSON parse over newest rows agent-wide
# (cap + 1 to detect sampling), attributed back per schedule.
cursor.execute(
"""
SELECT schedule_id, tool_calls
FROM schedule_executions
WHERE agent_name = ? AND started_at > ? AND tool_calls IS NOT NULL
ORDER BY started_at DESC
LIMIT ?
""",
(agent_name, cutoff, cap + 1),
)
tool_rows = cursor.fetchall()

tool_calls_sampled = len(tool_rows) > cap
tool_total_by_sched: Dict[str, int] = defaultdict(int)
for row in tool_rows[:cap]:
raw = row["tool_calls"]
if not raw:
continue
try:
parsed = json.loads(raw)
except (json.JSONDecodeError, TypeError):
continue
if not isinstance(parsed, list):
continue
for entry in parsed:
if isinstance(entry, dict) and (entry.get("name") or entry.get("tool")):
tool_total_by_sched[row["schedule_id"]] += 1

schedules: List[Dict] = []
for s in schedule_rows:
sid = s["id"]
agg = agg_by_sched.get(sid)
last = last_by_sched.get(sid)

if agg:
total = int(agg["total"] or 0)
success_count = int(agg["success_count"] or 0)
failed_count = int(agg["failed_count"] or 0)
cancelled_count = int(agg["cancelled_count"] or 0)
cost_total = round(float(agg["cost_total"] or 0.0), 4)
avg_duration_ms = (
int(agg["avg_duration_ms"]) if agg["avg_duration_ms"] is not None else None
)
context_avg = (
int(agg["context_avg"]) if agg["context_avg"] is not None else None
)
else:
total = success_count = failed_count = cancelled_count = 0
cost_total = 0.0
avg_duration_ms = context_avg = None

terminal = success_count + failed_count
success_rate = round(success_count / terminal, 4) if terminal else None

schedules.append({
"schedule_id": sid,
"name": s["name"],
"command": _schedule_command_label(s["message"]),
"cron_expression": s["cron_expression"],
"enabled": bool(s["enabled"]),
"total_executions": total,
"success_count": success_count,
"failed_count": failed_count,
"cancelled_count": cancelled_count,
"success_rate": success_rate,
"avg_duration_ms": avg_duration_ms,
"cost_total": cost_total,
"context_avg": context_avg,
"tool_call_total": tool_total_by_sched.get(sid, 0),
"last_run_at": last["last_run_at"] if last else None,
"last_run_status": last["last_status"] if last else None,
})

return {
"window_hours": hours,
"schedule_count": len(schedules),
"tool_calls_sampled": tool_calls_sampled,
"schedules": schedules,
}

def get_agent_analytics(self, agent_name: str, hours: int) -> Dict:
"""Compute agent-scoped execution analytics over a rolling window (#1107).

Expand Down
42 changes: 42 additions & 0 deletions src/backend/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -708,3 +708,45 @@ class AgentAnalyticsResponse(BaseModel):
timeline: List[AgentAnalyticsTimelinePoint] = []
sampled: bool = False
sample_size: int = 0


class ScheduleSummaryRow(BaseModel):
"""One per-schedule performance rollup (#1115).

`success_rate` is terminal-based (success / (success + failed [incl.
`error`])) and `None` when there were zero terminal runs in the window —
the UI renders `—`, not a false 0%. `avg_duration_ms` / `context_avg` are
`None` when nothing measurable ran. A zero-run schedule still appears
(all counts 0, rates `None`).
"""
schedule_id: str
name: str
command: str = ""
cron_expression: str
enabled: bool
total_executions: int
success_count: int
failed_count: int
cancelled_count: int
success_rate: Optional[float] = None
avg_duration_ms: Optional[int] = None
cost_total: float
context_avg: Optional[int] = None
tool_call_total: int
last_run_at: Optional[str] = None
last_run_status: Optional[str] = None


class AgentSchedulesSummaryResponse(BaseModel):
"""Response envelope for GET /api/agents/{name}/schedules/analytics-summary (#1115).

One compact rollup row per non-deleted schedule for the window — consumed
by BOTH the Overview "Schedules performance" section and the Schedules-tab
inline stats from a single call (no N per-schedule round-trips).
`tool_calls_sampled` flags when the agent-wide tool-call parse pool was
capped. UTC window via `iso_cutoff`.
"""
window_hours: int
schedule_count: int
tool_calls_sampled: bool = False
schedules: List[ScheduleSummaryRow] = []
35 changes: 33 additions & 2 deletions src/backend/routers/schedules.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"scheduler" as an agent name.
"""

from fastapi import APIRouter, Depends, HTTPException, Request, status
from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
from typing import List, Optional
from pydantic import BaseModel
from datetime import datetime
Expand All @@ -17,12 +17,15 @@
import logging
import httpx

from models import User, ScheduleAnalyticsResponse
from models import User, ScheduleAnalyticsResponse, AgentSchedulesSummaryResponse
from dependencies import get_current_user, get_authorized_agent, AuthorizedAgent, CurrentUser
from database import db, Schedule, ScheduleCreate, ScheduleExecution
from services.platform_audit_service import platform_audit_service, AuditEventType

_ANALYTICS_VALID_WINDOWS = frozenset({24, 168, 720}) # #868
# #1115: Overview/Schedules-tab scorecard windows → hours (matches the #1107
# Overview selector: 7 / 14 / 30 days).
_SUMMARY_WINDOW_HOURS = {"7d": 168, "14d": 336, "30d": 720}

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -341,6 +344,34 @@ async def get_schedule_analytics(
return ScheduleAnalyticsResponse(**analytics)


@router.get(
"/{name}/schedules/analytics-summary",
response_model=AgentSchedulesSummaryResponse,
)
async def get_agent_schedules_summary(
name: AuthorizedAgent,
window: str = Query("7d", description="One of 7d, 14d, 30d"),
):
"""Per-schedule performance rollups for the agent over a window (#1115).

ONE compact row per non-deleted schedule — consumed by BOTH the Overview
"Schedules performance" section and the Schedules-tab inline stats from a
single call (no N per-schedule round-trips). Zero-run schedules are
included. Read-only / DB-sourced (renders when the agent is stopped). The
#868 per-schedule deep view stays the drill-in target.

NOTE: declared BEFORE `/{name}/schedules/{schedule_id}` so the literal
`analytics-summary` segment isn't captured as a schedule_id (Invariant #4).
"""
hours = _SUMMARY_WINDOW_HOURS.get(window)
if hours is None:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=f"window must be one of {sorted(_SUMMARY_WINDOW_HOURS)}",
)
return AgentSchedulesSummaryResponse(**db.get_agent_schedules_summary(name, hours))


@router.get("/{name}/schedules/{schedule_id}", response_model=ScheduleResponse)
async def get_schedule(
name: AuthorizedAgent,
Expand Down
Loading
Loading