Skip to content

new skill: monitor-metrics#32

Closed
naitik-mixpanel wants to merge 2 commits into
mixpanel:mainfrom
naitik-mixpanel:monitor-metrics
Closed

new skill: monitor-metrics#32
naitik-mixpanel wants to merge 2 commits into
mixpanel:mainfrom
naitik-mixpanel:monitor-metrics

Conversation

@naitik-mixpanel

Copy link
Copy Markdown
Contributor

Add monitor-metrics skill

What

Adds a new skill, monitor-metrics, for single-metric diagnosis against the mixpanel-mcp connector (Mixpanel US). It is deliberately scoped to one metric, one diagnosis, one attribution pass — not a portfolio sweep.

The skill answers three distinct questions, each as its own command:

  • metric-anomaly — point-in-time anomaly detection (Z-score + IQR, time-bucketed) over 7-day hourly and 30-day daily series. Flags spikes, drops, and clusters; does not test for trend drift. (2 queries)

  • metric-drift — trend-level drift detection (mean-shift + variance-ratio) over 60-day daily and 16-week weekly windows, with a built-in outlier-contamination check so it can run standalone. Owns shape classification (step / slope / oscillating). (2 queries)

  • metric-rca — root-cause attribution that runs on top of an existing anomaly/drift diagnosis. Fans out across five segmentation branches (component decomposition, default-property breakdowns, distinct-id outliers, cohort comparison, calendar/market context), ranks findings by concentration × deviation, and appends them to the diagnosis board. Does not run cold.
    Shared skill-level scaffolding handles the parts every command depends on:

  • Step 0 — input validation (project + metric), with Get-Business-Context called once per session to resolve project nicknames/acronyms before falling back to Get-Projects.

  • Step 1 — metric ingestion into a normalized "metric series" object. Path A (saved Mixpanel Metric, preferred) lifts the full definition via Get-Metric; Path B rebuilds the query body from Get-Query-Schema for reports / dashboard tiles / prose.

  • Step 1.5 — project profile resolution: cheap metadata-only filter validation (List-Properties, Get-Property-Values) and an instrumentation health check (Get-Issues) that surfaces schema/null/type-drift issues into the verdict without aborting.

  • Step 2 / Step 3 — diagnosis payload handoff (held in conversation memory, not disk), an opt-in board prompt, and automatic RCA append to the existing board.
    Output contract is enforced across commands: compact verdict-first cards (headline → confidence → next step), always-on trend charts (annotations only when something is flagged), and explicit scope limits on every card.

Files added:

  • SKILL.md — routing, shared Steps 0–3, output contract
  • commands/metric-anomaly.md
  • commands/metric-drift.md
  • commands/metric-rca.md

Why

CSAs and PMs repeatedly hit the same three questions when a metric moves — is this point weird, has the baseline shifted, and where is the movement coming from — but those are statistically different tests with different customer conversations attached (an anomaly is an incident, drift is a trend, RCA is the segmentation story). Conflating them produces muddy verdicts. Separating detection from attribution, and forcing RCA to consume a prior diagnosis rather than run cold, keeps each answer clean and the cost predictable (anomaly is the cheap default at 2 queries).

It also fills a clear gap between existing skills: analyze-report reads what a chart shows but doesn't chase causes; weekly-pulse and gtm-customer-intelligence operate at portfolio/adoption scope. monitor-metrics is the single-metric statistical-detection + RCA workflow that sits between them, with explicit handoffs noted in "When not to use this skill."

Type

  • New skill
  • Update existing skill
  • Bug fix
  • Documentation

Skill review

  • /review-skill passed (no blockers or majors)

Testing

Sample Test case: https://claude.ai/share/f01d3424-437f-4e49-89b9-58b1accc6abd

@naitik-mixpanel naitik-mixpanel deleted the monitor-metrics branch June 10, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant