feat: Model Performance Registry — benchmark-driven model selection for Fusion

[Delqhi]

Motivation

Fusion v1 runs tournaments with a fixed provider list (fusion.providers). We don't know which model is best at which task type — we're guessing. This issue adds a persistent per-model-per-category performance database that:

1. Records benchmark results across all fusion providers
2. Recommends the best model for a given task category
3. Feeds back into Fusion provider selection — tournaments use top-N models for the detected task, not a fixed list

Architecture

sin-code fusion benchmark --dataset evals/skill-code.json
  │
  ├── Run dataset against ALL fusion providers (parallel)
  ├── Record per-model: pass_rate, avg_latency, avg_cost, avg_tokens
  └── Store in modelperf.db (SQLite, modernc, M2)
  
sin-code fusion recommend --task code-generation
  │
  └── Return top-3 models from modelperf.db sorted by pass_rate

sin-code fusion rank
  │
  └── Show leaderboard table: model × category × pass_rate × cost

Task categories (auto-detected from eval dataset metadata)

Category	Example datasets
code-generation	skill-code-*.json
debugging	skill-debug-*.json
planning	skill-planning-*.json
test-generation	test-generation.json
refactoring	skill-code-create.json
review	skill-code-audit.json

Schema (modelperf.db)

CREATE TABLE model_perf (
  id INTEGER PRIMARY KEY,
  model TEXT NOT NULL,
  category TEXT NOT NULL,
  dataset TEXT NOT NULL,
  pass_rate REAL NOT NULL,
  avg_latency_ms INTEGER,
  avg_cost_usd REAL,
  avg_tokens INTEGER,
  sample_count INTEGER DEFAULT 1,
  recorded_at TEXT NOT NULL,
  UNIQUE(model, category, dataset)
);

Upsert on (model, category, dataset) — latest result wins, sample_count increments.

Integration into Fusion

// Before: fixed provider list
providers := pool.Get(cfg.FusionProviders)

// After: task-aware selection
category := detectTaskCategory(prompt)
recommended := modelperf.Recommend(category, 3)
providers := pool.Get(recommended) // fall back to full pool if cold start

CLI surface

# Benchmark all fusion providers against a dataset
sin-code fusion benchmark --dataset evals/skill-code.json

# Show leaderboard
sin-code fusion rank

# Get recommendation for a task
sin-code fusion recommend --task code-generation

# Export for analysis
sin-code fusion rank --json > /tmp/modelperf.json

Config

Key	Default	Purpose
fusion.benchmark_on_verify_fail	true	Auto-run benchmark after tournament failures
fusion.recommendation_weight	0.7	Blend factor: 0=fixed list, 1=fully recommendation-driven

Non-breaking

• Additive: new internal/modelperf/ package, new CLI subcommands
• Cold-start: if modelperf.db is empty, Fusion falls back to current fixed-list behavior
• No change to existing tournament logic — only provider selection changes

Files to create

• cmd/sin-code/internal/modelperf/store.go — SQLite store
• cmd/sin-code/internal/modelperf/recommend.go — recommendation engine
• cmd/sin-code/internal/modelperf/benchmark.go — benchmark runner
• cmd/sin-code/internal/modelperf/*_test.go — tests
• cmd/sin-code/fusion_cmd.go — add benchmark/rank/recommend subcommands
• cmd/sin-code/internal/loopbuilder/builder.go — wire recommendation into provider selection

[Delqhi]

Shipped in PR #397 (commit efe6121). Model Performance Registry: SQLite-backed per-model-per-category benchmark DB. sin-code fusion benchmark/rank/recommend CLI. Auto-wired into loopbuilder provider selection. 15 tests race-clean.