Skip to content

Persist task description in tip entity metadata #61

@jayaramkr

Description

@jayaramkr

Problem

When tips are generated from agent trajectories, the task description (extracted from the first user message via parse_openai_agents_trajectory()) is available during tip generation but is discarded before the tip entities are stored. This means there's no way to know what task a given guideline tip originated from after it's persisted.

This blocks downstream work like clustering tips by task similarity — without the task description in metadata, there's nothing to embed and compare.

Proposed Solution

Persist task_description as a metadata field on every guideline entity at storage time. The task description comes from the trajectory (not the LLM), so metadata is the right place for it — no changes to the Tip model or LLM prompt schema are needed.

Changes

  • kaizen/llm/tips/tips.py — Change generate_tips() to return a TipGenerationResult dataclass that pairs the list of Tip objects with the source task_description string
  • kaizen/schema/tips.py — Add TipGenerationResult dataclass
  • kaizen/sync/phoenix_sync.py — Include task_description in each tip entity's metadata when calling update_entities()
  • kaizen/frontend/mcp/mcp_server.py — Same change for the MCP save_trajectory path; also guard against empty tips list before calling update_entities()

Example metadata after this change

metadata={
     "category": "strategy",
     "rationale": "...",
     "trigger": "...",
     "source_trace_id": "...",
     "source_span_id": "...",
     "task_description": "Fix the login bug",  # NEW
 }

Context

This is Phase 1 of the tip clustering effort. Phase 2 uses the persisted task descriptions to cluster related tips by cosine similarity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions