Add roborev insights command for review pattern analysis

[darrenhaas]

Summary

• Adds roborev insights command that queries failing reviews from the database, builds a structured analysis prompt including current review guidelines, and sends it to an LLM agent for pattern identification (issue Add roborev insights command for LLM-powered review pattern analysis #359)
• Agent analyzes recurring finding patterns, hotspot areas, noise candidates, guideline gaps, and produces concrete guideline text suggestions for .roborev.toml
• Supports --repo, --branch, --since (default 30d), --agent, --model, --reasoning, --wait (default true), and --json flags
• Prioritizes unaddressed (open) findings over addressed (closed) ones when truncating to fit prompt size limits
• New files: cmd/roborev/insights.go, internal/prompt/insights.go, plus tests for both

🤖 Generated with Claude Code

[roborev-ci]

roborev: Combined Review (173295e)

Verdict: The PR implements the insights command for analyzing failing reviews, but requires fixes for repository path resolution, prompt size enforcement, and API pagination.

🟡 Medium Severity

**1. Worktree and non-root --repo paths will miss existing reviews
**
Locations: cmd/roborev/insights.go:103, [cmd/roborev/insights.go:138](/home/robore
v/repos/roborev/cmd/roborev/insights.go#L138), internal/daemon/server.go:657
run Insights uses git.GetRepoRoot or filepath.Abs as the /api/jobs?repo= filter, but the daemon stores jobs under git.GetMainRepoRoot(...). In a worktree, or if --repo points at a subdirectory, the filter path will not match the
stored repo root, causing roborev insights to incorrectly report no failing reviews.
Recommendation: Canonicalize the query path with git.GetMainRepoRoot before calling fetchFailingReviews and before loading repo config. Add tests for worktrees and subdirectory --repo inputs.

**
2. Prompt-size enforcement is inconsistent and can exceed the daemon’s enqueue limit**
Locations: cmd/roborev/insights.go:159, internal
/prompt/insights.go:105, [internal/prompt/insights.go:151](/home/roborev/repos/roborev/internal/prompt
/insights.go#L151), internal/daemon/server.go:605
The new builder ignores repo/global prompt-size config and uses the hard
-coded prompt.MaxPromptSize. It checks sb.Len() only after appending a review, so the final prompt can overshoot the intended cap by up to a full review chunk. On larger histories, this can make /api/enqueue fail with 413 request body too large.
*
Recommendation:* Pass the resolved prompt budget into InsightsData, trim based on remaining space before appending, and add a test ensuring the final prompt never exceeds the configured limit.

3. API limit silently omits valid reviews within the requested time window
Locations: cmd/roborev/insights.
go:224, [cmd/roborev/insights.go:227](/home/roborev/repos/roborev/cmd/roborev/
insights.go#L227), internal/storage/jobs.go:1065
The code hardcodes an API query limit of 200 (
params.Set("limit", "200")), then filters by FinishedAt, fix-job status, and verdict on the client side. In an active repo, recent passing jobs can easily push relevant failures beyond that first page, silently omitting valid reviews within the requested --since window.
*
Recommendation:* Handle pagination, pass the since parameter directly to the daemon API to filter before the limit is applied, or add a server-side "failing verdict" filter. A regression test with >200 done jobs is recommended.

---

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

[darrenhaas]

Fixing the above, had worktree as part of test case. I should have asked what of edge cases.

[darrenhaas]

I am that many multi repo... :)

[roborev-ci]

roborev: Combined Review (9fe0549)

Verdict: The PR introduces a new insights command but contains hard compile failures and logical bugs in pagination and prompt size enforcement that need to be addressed.

High

Severity

• cmd/roborev/insights.go:~205: Calls waitForPromptJob(cmd, serverAddr, job.ID, false, promptPollInterval),
  but cmd/roborev/run.go defines waitForPromptJob with only four parameters. This is a hard compile failure. Remove the extra argument or change the
  shared helper signature consistently across callers.
• cmd/roborev/insights.go:~273 and :~287: References job.IsFixJob() and
  review.Closed, but internal/storage/models.go only provides IsTaskJob(), and [internal/storage/models.go](/home/roborev/repos/
  roborev/internal/storage/models.go#L99) uses Addressed, not Closed. This is another compile failure in the new command. Use an existing discriminator or add a real IsFixJob helper, and pass review.Addressed into InsightsReviewFromJob .

Medium Severity

• cmd/roborev/insights.go:~255: Stops paginating as soon as it sees one FinishedAt < since, but
  internal/storage/jobs.go returns /api/jobs ordered by j.id DESC, not finished_at DESC. A slower older job can finish inside the time
  window and still land on a later page, so this can silently miss eligible failing reviews. Keep paginating until exhaustion/max results, or add server-side sort/filter by finish time. There’s also no test covering out-of-order finish times across pages.
• internal/prompt/
  insights.go:~83 and :~157: Only applies the size cap when appending review entries. A large review_guidelines block or the fixed prompt header can already exceed MaxPrompt Size, so the new prompt-size enforcement is incomplete and can still produce oversize prompts. Enforce the budget against the full prompt, including guidelines/front matter, and add a test with oversized guidelines.

---

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

[roborev-ci]

roborev: Combined Review (8761b0d)

Verdict: The PR introduces a new roborev insights command but contains several compile-blocking errors and an edge-case regression that must be fixed.

High

• cmd/roborev/insights.go:214
  Calls waitForPromptJob with five arguments, but the existing helper takes four. This does not compile
  . Drop the extra promptPollInterval argument or change the helper signature and all callers consistently.

• cmd/roborev/insights.go:284
  Calls job.IsFixJob(), but storage.ReviewJob does not define that method. This is
  another compile failure. Either implement a real fix-job predicate on ReviewJob or remove this check.

• internal/prompt/insights.go:138
  Declares included and only increments it. Go rejects unused locals, so this file also fails to build. Remove
  the variable or use it in the final output.

Medium

• cmd/roborev/insights.go:100
  Falls back to cwd when git.GetRepoRoot fails. That differs from roborev list, which leaves the repo filter empty
  outside a checkout. As written, running roborev insights from a non-repo directory queries a nonexistent repo path and misleadingly returns “No failing reviews found.” Leave repoRoot empty when the cwd is not a repo, and add a CLI test for that path.

---

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

[darrenhaas]

Addressed the two real findings from the latest review:

• Removed unused included variable
• Non-repo cwd now errors clearly instead of silently querying a nonexistent path

The two "compile failure" findings are false positives — waitForPromptJob takes 5 parameters (run.go:196) and IsFixJob() exists (models.go:132). Code compiles and all tests pass.

[roborev-ci]

roborev: Combined Review (2433ddd)

Verdict: The new insights command contains a medium severity issue regarding incorrect configuration resolution when executed from a git worktree.

Medium

• cmd/roborev/insights.
  go:123, [cmd/roborev/insights.go:159](/home/roborev/repos/roborev/cmd/roborev
  /insights.go:159), cmd/roborev/insights.go:175
  runInsights rewrites the caller’s path
  to the main repo root and then reuses that rewritten path for config lookup, max-prompt resolution, and GetCurrentBranch. In a worktree, that points at the main checkout’s .roborev.toml and branch, not the active worktree the user invoked from. The result is that
  insights can analyze the right review history but enqueue the job with the wrong guidelines, prompt budget, and branch metadata. Keep two paths here: a canonical/main repo path for daemon/database filtering, and the original repo/worktree path for config and branch resolution.

---

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

[roborev-ci]

roborev: Combined Review (d49ee72)

Verdict: The proposed insights command introduces hard compile failures and missing validation that must be addressed before merging.

High

• cmd/roborev/insights.go:222: Calls waitForPromptJob(cmd, serverAddr, job .ID, false, promptPollInterval), but the existing signature in run.go only accepts 4 arguments. This is a hard compile failure. Drop the extra argument or extend the helper consistently across call sites.
• cmd/roborev/insights.go:282-2 84: Calls job.IsFixJob(), but ReviewJob only defines IsTaskJob(). There is no IsFixJob method in the repo, causing another hard compile failure. Either add a real fix-job discriminator in storage or remove this call and filter with fields that
  actually exist.

Medium

• cmd/roborev/insights.go:101-120: Never validates an explicit --repo path. If the user passes a non-repo directory, GetMainRepoRoot errors are ignored and the command falls through to querying
  the daemon, which can end with a misleading success path saying "No failing reviews found". This should fail fast with the same "not a git repository" error used for cwd detection.

---

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

[roborev-ci]

roborev: Combined Review (b5b58a2)

Verdict: The pull request introduces the new insights command
but requires fixes for JSON output consistency, correct dataset sizing, branch scoping, and job type configuration.

High

• [cmd/roborev/insights.go:151](/home/roborev/repos/roborev/cmd/roborev/insights.go#L1
  51)
  When the --json flag is provided and no failing reviews are found, the command prints plain text to stdout, breaking JSON output expectations for programmatic consumers. Output a valid empty JSON structure (e.g., {"reviews_analyzed": 0}) when opts.jsonOutput is
  true.

Medium

• cmd/roborev/insights.go:170, [internal/prompt/insights.go:117](/
  home/roborev/repos/roborev/internal/prompt/insights.go#L117)
  runInsights fetches up to 100 failing reviews, but it never sets MaxReviews when calling BuildInsightsPrompt. The prompt builder falls back to 5
  0, so up to half of the fetched reviews are silently dropped before the model sees them, while the CLI/JSON still reports len(reviews) as “analyzing” or reviews_analyzed. Align the limits by passing MaxReviews: maxInsightsReviews or reporting the actual included count, and
  add a test with >50 reviews.

• cmd/roborev/insights.go:179
  The command filters source reviews by
  opts.branch, but the enqueued insights job stores git.GetCurrentBranch(repoRoot) instead of the requested branch. That makes the saved job metadata incorrect for --branch, and with current daemon behavior it also means branch-based handling is tied to the checkout branch rather than the branch being analyzed. Use
  opts.branch when provided, falling back to the current branch only when the flag is unset.

• [cmd/roborev/insights.go:180](/home/roborev/repos/roborev/cmd/roborev/insights.go#L18
  0)
  The comment says // Enqueue as a task job, but JobType: "task" (or equivalent) is missing from the daemon.EnqueueRequest struct. The daemon may process and store this as a standard code review, polluting review history. Add JobType: "task " to the daemon.EnqueueRequest literal.

• internal/prompt/insights.go:92, [internal/prompt/insights.go:175](/
  home/roborev/repos/roborev/internal/prompt/insights.go#L175)
  Prompt-size enforcement is not actually hard-bounded for small max_prompt_size values. guidelineCap := max(promptBudget/10, 10 24) can consume more than the entire budget, and the omission text appended later is not budget-checked either. A repo with a small configured prompt limit can still generate an oversized prompt. Cap each section against remaining budget, not just a fixed minimum, and add a test with a sub-2KB budget.

---

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

[wesm]

I'll take a closer look at this. I'm actually interested in putting more of the review and session intelligence in agentsview (and building a web UI for roborev there, too), but having some simple insights in roborev in the cli seems useful. I'll experiment and tweak things a bit, so just bear with me so I can wrap my head around how this should work

[wesm]

@mariusvniekerk if you also want to take a look at this and weight in I would appreciate it

[mariusvniekerk]

i think this is a reasonable initial approach.

We will likely want to deal with the many reviews problem by materializing the review json outputs to some temporary path and pointing the agent at that path and have them use their natural explore tools to assess the problem rather than just making a very large prompt. I would view that as out of scope of this intial approach.

[wesm]

superseded by #554