-
Notifications
You must be signed in to change notification settings - Fork 5
Persist task description in tip entity metadata #61
Description
Problem
When tips are generated from agent trajectories, the task description (extracted from the first user message via parse_openai_agents_trajectory()) is available during tip generation but is discarded before the tip entities are stored. This means there's no way to know what task a given guideline tip originated from after it's persisted.
This blocks downstream work like clustering tips by task similarity — without the task description in metadata, there's nothing to embed and compare.
Proposed Solution
Persist task_description as a metadata field on every guideline entity at storage time. The task description comes from the trajectory (not the LLM), so metadata is the right place for it — no changes to the Tip model or LLM prompt schema are needed.
Changes
- kaizen/llm/tips/tips.py — Change generate_tips() to return a TipGenerationResult dataclass that pairs the list of Tip objects with the source task_description string
- kaizen/schema/tips.py — Add TipGenerationResult dataclass
- kaizen/sync/phoenix_sync.py — Include task_description in each tip entity's metadata when calling update_entities()
- kaizen/frontend/mcp/mcp_server.py — Same change for the MCP save_trajectory path; also guard against empty tips list before calling update_entities()
Example metadata after this change
metadata={
"category": "strategy",
"rationale": "...",
"trigger": "...",
"source_trace_id": "...",
"source_span_id": "...",
"task_description": "Fix the login bug", # NEW
}
Context
This is Phase 1 of the tip clustering effort. Phase 2 uses the persisted task descriptions to cluster related tips by cosine similarity.