Harden profiling detection and agent reports

.changeset/detection-report-hardening.md

@@ -0,0 +1,11 @@
+---
+"@lanterna-profiler/core": minor
+"@lanterna-profiler/detectors": minor
+"@lanterna-profiler/cli": minor
+---
+
+Harden profiling reports and detector attribution.
+
+- Add richer CPU, memory, and async report signals for agent-oriented review.
+- Add generic CPU hotspot detection and improve source-aware attribution for built-in detectors.
+- Improve agent/text/markdown report output with user stacks, read-first targets, and clearer quality caveats.

docs/architecture.md

@@ -157,8 +157,8 @@ The pipeline transforms `CaptureBundle` into `LanternaReport` in four phases:
 
 1. **Kind contributors** — each `ProfileKind` writes its report section into `profiles.<reportSectionKey>` and publishes a typed view consumable via `context.forKind(id)`.
 2. **Section analyzers** — optional extensions write under `extensions.<namespace>` (not kind-specific).
-3. **Finding analyzers** — cross-cutting rules emit `Finding`s, each tagged with a `profileKind` string.
-4. **Finalize** — each kind's optional `finalize` hook mutates its own section based on the final findings (e.g. CPU sets `profiles.cpu.summary.dominantBlockingKind` and `topUserHotspot`).
+3. **Finding analyzers** — cross-cutting rules emit `Finding`s, each tagged with a `profileKind` string. After every analyzer, the in-progress `snapshot.findings` is updated so later detectors can suppress duplicates or build on earlier evidence.
+4. **Finalize** — each kind's optional `finalize` hook mutates its own section based on the final findings (e.g. CPU sets `profiles.cpu.summary.dominantBlockingKind`, `topCpuCulprit`, `topRequestEntry`, and `topUserHotspot`).
 
 ### Frame classification
 
@@ -200,7 +200,7 @@ Correlation is conservative: if no single user frame dominates, Lanterna reports
 
 Findings are detectors running on the enriched snapshot, not on the raw bundle. Each finding carries a required `profileKind: string` tag so consumers can filter by kind. The full catalog of built-in findings, grouped by kind, lives in [extending/detectors.md](./extending/detectors.md#built-in-findings).
 
-Findings are sorted by `priority.score` first, then by severity and attributed weight. Dominant user-code CPU is exposed as `profiles.cpu.summary.topUserHotspot` for context instead of as an actionable finding.
+Findings are sorted by `priority.score` first, then by severity and attributed weight. Known anti-pattern detectors own specific categories such as blocking I/O or sync crypto; the generic `cpu-hotspot:*` detector is the fallback for plain user-code CPU that still needs a file/line or caller lead. CPU summaries also expose `profiles.cpu.summary.topCpuCulprit` for the self-heavy culprit and `topRequestEntry` / `topUserHotspot` for caller context.
 
 Built-in findings may also expose:
 

docs/examples/report.agent.md

@@ -7,6 +7,7 @@ cwd: /repo
 kinds: [cpu, memory, async]
 lanterna_version: "1.5.1"
 cpu_quality: high
+memory_quality: high
 memory_signal: present
 async_quality: high
 integrity: ok
@@ -50,6 +51,7 @@ degrading_caveats: []
 
 ## Kind Review — memory
 
+- quality: high
 - memory_usage: 10 samples every 250ms
 - top_allocator: Buffer.alloc at node:buffer:10 — user_caller loadCache at src/cache.ts:18 (high, heap-sample-path, support 88.0%)
 

docs/extending/detectors.md

@@ -144,8 +144,9 @@ The default pack lives in `@lanterna-profiler/detectors` and pre-wires detectors
 | `blocking-io:<api>` | Sampled sync `fs` / `child_process` / `zlib` frame on the hot path. |
 | `json-on-hot-path:<api>` | `JSON.parse` / `JSON.stringify` consuming meaningful CPU. |
 | `node-modules-hotspot:<package>` | A dependency frame dominates CPU time. |
+| `cpu-hotspot:<frame>` | User-code self CPU ≥ 10%, or inclusive CPU ≥ 25% as a lower-confidence caller lead, when no more specific CPU detector already explains the frame. |
 | `excessive-gc` | `gcRatio > 10%` or `longestPauseMs > 100ms`. |
-| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`. |
+| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`; anchors to strong stall correlation when available, otherwise to the hottest user CPU fallback. |
 | `deopt-loop:<function>` | Same deoptimised function seen ≥ 5 times (`--deep`) and hot in the profile. |
 | `require-in-hot-path` | Module loading functions sampled on the hot path. |
 
@@ -191,6 +192,8 @@ import {
 - `buildAttributedFinding(...)` — one-shot helper that returns a fully-shaped `Finding`.
 - `CpuHotspotContext` — the attribution view (`fullHotspots`, `hotspotById`, `userCallerById`, `candidateCallersById`) reachable from a kind-scoped detector via `kinds.cpu.view.hotspotAnalysis`.
 
+Finding analyzers run incrementally: after each detector, `snapshot.findings` includes findings emitted so far. Use that shared state to avoid duplicate generic findings when a specific rule has already explained the same frame. The built-in `cpu-hotspot` detector uses this to defer to `sync-crypto`, `blocking-io`, `json-on-hot-path`, `node-modules-hotspot`, and `require-in-hot-path`.
+
 ## Thresholds
 
 `DETECTOR_THRESHOLDS` from `@lanterna-profiler/detectors` is the source of truth for tunable values:
@@ -203,6 +206,8 @@ console.log(DETECTOR_THRESHOLDS.eventLoopStall.p99);
 
 You can read these values in your own detectors so users get consistent thresholds across the pack.
 
+The generic CPU fallback is configured under `DETECTOR_THRESHOLDS.cpuHotspot`: `minSelfPct` gates self-heavy user functions (`evidence.extra.mode: "self"`), `minTotalPct` catches inclusive-only user callers when no self-heavy candidate exists (`mode: "inclusive-entry"`), `criticalPct` escalates severity, and `maxFindings` caps noise. Inclusive-entry findings use top-level `proofLevel: "heuristic"` so agents treat them as source-inspection leads rather than direct proof that the wrapper body is expensive.
+
 ## Where to next
 
 - [plugin-loading.md](./plugin-loading.md) — how Lanterna discovers your plugin (CLI flag, `.lanterna.json`, packaging).

docs/kinds/async.md

@@ -77,3 +77,4 @@ Async-specific options:
 - **Microtasks default to off.** Enabling `--async-include-microtasks` produces very noisy reports. Use it only for the `microtask-flood` finding.
 - **Dropped records are sampled, not lost forever.** `quality.recordsDropped > 0` means raise `--async-max-events` for the next run if completeness matters.
 - **User callers are anchors, not proof.** Async `userCaller` is derived from already captured user frames. Prefer high-confidence CPU-window attribution when present; stack-only callers should guide inspection rather than be treated as the definitive line to edit.
+- **Public async file paths are normalized.** When V8/CDP reports `file://` URLs, Lanterna converts them to normal filesystem paths before grouping hot files, chains, and finding evidence. Virtual bundler URLs are kept as-is.

docs/kinds/cpu.md

@@ -41,7 +41,9 @@ CPU-specific options:
 | `idleRatio` | Fraction of samples spent idle. |
 | `topCategory` | Dominant non-idle category. |
 | `dominantBlockingKind` | Coarse summary derived from emitted findings (e.g. `sync-crypto`, `blocking-io`). |
-| `topUserHotspot` | Set when a single user function dominates user CPU. |
+| `topCpuCulprit` | Strongest self-heavy user-code CPU frame. This is the first place to inspect when the question is "which line is burning CPU?". |
+| `topRequestEntry` | Dominant user-code request/caller entry after accounting for existing findings and stall correlation. Useful when the hottest work happens below a wrapper or builtin. |
+| `topUserHotspot` | Backward-compatible contextual hotspot. It currently mirrors `topRequestEntry` and remains useful for consumers that have not adopted the newer split fields. |
 
 ### `quality`
 
@@ -83,8 +85,9 @@ V8 deoptimisation clusters with `function`, `file`, `line`, `reason`, `bailoutTy
 | `blocking-io:<api>` | Sampled sync `fs` / `child_process` / `zlib` frame with meaningful CPU. |
 | `json-on-hot-path:<api>` | `JSON.parse` / `JSON.stringify` consuming meaningful CPU. |
 | `node-modules-hotspot:<package>` | A dependency frame dominates meaningful CPU time. |
+| `cpu-hotspot:<frame>` | Generic fallback for user-code CPU not explained by a more specific CPU detector. Self-heavy frames are direct hotspots; inclusive-only frames are lower-confidence caller leads. |
 | `excessive-gc` | `gcRatio > 10%` or `longestPauseMs > 100ms`. |
-| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`. |
+| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`; evidence points at a strongly correlated frame when possible, otherwise the hottest user CPU fallback. |
 | `deopt-loop:<function>` | Same deoptimised function seen ≥ 5 times (`--deep`) and hot in the CPU profile. |
 | `require-in-hot-path` | Module loading functions sampled on the hot path. |
 
@@ -93,7 +96,7 @@ Each finding ships with `confidence`, `proofLevel`, `evidence.file/line/function
 ## Reading order
 
 1. `quality.confidence` and `quality.reasons[]` — is the profile worth acting on?
-2. `summary.topCategory` and `summary.dominantBlockingKind` — where is CPU concentrated?
+2. `summary.topCpuCulprit`, `summary.topRequestEntry`, `summary.topCategory`, and `summary.dominantBlockingKind` — where CPU is concentrated and which line to inspect first.
 3. `findings[]` filtered to `profileKind === "cpu"` — prioritized hypotheses.
 4. Top 5 `hotspots` even when no finding fired — direct evidence.
 5. `eventLoop` — translates CPU pressure into latency.
@@ -124,5 +127,6 @@ The classification feeds `summary` ratios and several finding heuristics. Self-n
 ## Caveats
 
 - A hotspot in `node_modules` or `node:builtin` is often a **symptom**. Inspect the user caller before blaming the dependency.
+- A `cpu-hotspot:*` finding means no specialized detector explained the user-code CPU lead. If `evidence.extra.mode === "self"`, inspect the function body for tight loops, repeated transformations, cache misses, or CPU-bound algorithms. If `mode === "inclusive-entry"`, inspect the callees and hot stacks before blaming the wrapper body.
 - High `nativeRatio` is normal for CPU work that lives in C++ (crypto, compression, JSON). Look at user callers, not just the leaf.
 - `cwd` mismatch between Lanterna and the target can mis-classify your code as `node_modules`. Check `meta.cwd`.

docs/reading-a-report.md

@@ -18,7 +18,7 @@ Use `--format agent` when an AI agent or automation will consume the report. It
 | --- | --- | --- | --- |
 | 1 | `meta` | What was captured (mode, duration, `profileKinds`, integrity flags). | `durationMs` very short, or `captureIntegrity.*` flags `false`. |
 | 2 | `profiles.cpu.quality` | Whether CPU evidence is strong enough to trust. | `confidence = low`, high idle, low samples, untimed samples. |
-| 3 | `profiles.cpu.summary` | Where CPU time went (ratios, top category). | `idleRatio` > 0.8 — the profile is mostly idle. |
+| 3 | `profiles.cpu.summary` | Where CPU time went and the best first line to inspect (`topCpuCulprit`, `topRequestEntry`). | `idleRatio` > 0.8 — the profile is mostly idle. |
 | 4 | `findings` | Prioritized hypotheses backed by the capture (tagged `profileKind`). | Empty `findings[]` does not prove a healthy profile. |
 | 5 | `profiles.cpu.hotspots` | Where CPU is actually spent (self + inclusive). | A hot leaf in `node_modules` is usually a symptom, not a cause. |
 | 6 | `profiles.cpu.eventLoop` | Latency signal + stall windows. | `confidence = low` or `measurementBasis = histogram` alone. |
@@ -34,7 +34,7 @@ Use `--format agent` when an AI agent or automation will consume the report. It
 
 1. **Read `meta` and `captureIntegrity`** — sanity-check what you're about to interpret. Short `durationMs`, `controlChannel: false` in spawn mode, or missing `gcTimed` change how strongly you should weight downstream sections. See [signal-quality.md](./signal-quality.md).
 2. **Read `profiles.cpu.quality`** before `findings[]` — a low-confidence profile can identify leads but not prove root causes.
-3. **Filter `findings[]` by severity and `profileKind`** — start with `severity != "info"`; group by kind to know which specialist page to consult.
+3. **Filter `findings[]` by severity and `profileKind`** — start with `severity != "info"`; group by kind to know which specialist page to consult. A `cpu-hotspot:*` finding is the generic fallback when no more specific CPU detector explains a user-code CPU lead.
 4. **Open the implicated source file** — `evidence.file` and `evidence.line` are where the action should happen. For some detectors that points at the user caller rather than a builtin callee.
 5. **Cross-reference kinds when both were captured** — `alloc-in-hot-path` is the canonical example: a frame hot on CPU **and** in top allocators is the highest-leverage fix you can make.
 6. **Use `hotStacks` only when a hotspot is ambiguous** — it surfaces the surrounding call path without manual reconstruction.
@@ -55,8 +55,9 @@ The full catalog (with triggers and remediations) lives in [extending/detectors.
 | `blocking-io:<api>` | cpu | Sync `fs` / `child_process` / `zlib` on the hot path. Use the async equivalent. |
 | `json-on-hot-path:<api>` | cpu | `JSON.parse` / `JSON.stringify` is a meaningful share of CPU. Cache, stream, or reduce. |
 | `node-modules-hotspot:<package>` | cpu | A dependency dominates CPU. **Inspect the user caller path first.** |
+| `cpu-hotspot:<frame>` | cpu | Plain user code dominates CPU without matching a known anti-pattern. `mode: "self"` is a direct body hotspot; `mode: "inclusive-entry"` is a caller/context lead. |
 | `excessive-gc` | cpu | GC ratio or longest pause is too high. Hunt allocations in top user hotspots. |
-| `event-loop-stall` | cpu | The main thread stopped servicing tasks. Check `correlatedHotspots`. |
+| `event-loop-stall` | cpu | The main thread stopped servicing tasks. Prefer strong `correlatedHotspots`; otherwise treat the fallback hotspot as the best CPU lead, not proven stall causality. |
 | `deopt-loop:<function>` | cpu | A hot function keeps deoptimising under `--deep`. Stabilise shapes/types. |
 | `require-in-hot-path` | cpu | Module loading on the hot path. Hoist or memoize the lazy load. |
 | `memory-growth:rss` / `memory-growth:heapUsed` | memory | Sustained linear growth ≥ 1 MB/s. Inspect top allocators and lifetimes. |
@@ -90,6 +91,12 @@ For some detectors (e.g. `sync-crypto-on-hot-path`, `blocking-io:<api>`), `evide
 
 When `measurementBasis === "histogram"`, `correlatedHotspots[]` is based on overall CPU overlap — not temporal overlap with stall windows. Read [signal-quality.md](./signal-quality.md#profilescpueventloop) before claiming causality.
 
+`event-loop-stall` can also expose `evidence.extra.proofLevel: "hotspot-fallback"` plus `fallbackHotspots[]`. That means the event-loop lag is real, but no measured stall window dominated enough to blame one frame directly; Lanterna anchored the finding to the hottest user CPU frame so the report still points at a useful source line.
+
+#### `topCpuCulprit` vs `topRequestEntry`
+
+`profiles.cpu.summary.topCpuCulprit` is self-CPU first: it tries to answer "which function body burned CPU?". `topRequestEntry` is finding-aware and caller-oriented: it keeps the request/user entry that explains why the hot work happened. If they differ, inspect `topCpuCulprit` for the local algorithmic problem and `topRequestEntry` for call frequency, payload size, or routing context.
+
 #### `deopt-loop:<function>`
 
 Fires only when a function is **both** hot in the CPU profile **and** repeatedly deoptimised under `--deep`. Focus on stabilising shapes and types, then reprofile. One-off deopt entries are noise.
@@ -104,6 +111,8 @@ A dependency hotspot is often a symptom — your code controls when and how ofte
 
 > **Assuming no findings means no problem.** Lanterna's detectors are heuristic. A clean `findings[]` lowers the odds of the usual issues; it does not prove the profile is healthy.
 
+> **Ignoring `cpu-hotspot:*` because it is generic.** This finding intentionally covers custom CPU-heavy code. `mode: "self"` is often the most direct file/line when the workload is a loop, scoring function, transformation, or algorithm instead of a known Node API misuse. `mode: "inclusive-entry"` should be treated as a lead to inspect callees/hot stacks.
+
 > **Blaming `node_modules` immediately.** A dependency hotspot is often just where the CPU landed. The caller path is usually your code.
 
 > **Ignoring `idleRatio`.** A profile captured without real load can be technically valid but operationally misleading.