diff --git a/.changeset/detection-report-hardening.md b/.changeset/detection-report-hardening.md new file mode 100644 index 0000000..156b805 --- /dev/null +++ b/.changeset/detection-report-hardening.md @@ -0,0 +1,11 @@ +--- +"@lanterna-profiler/core": minor +"@lanterna-profiler/detectors": minor +"@lanterna-profiler/cli": minor +--- + +Harden profiling reports and detector attribution. + +- Add richer CPU, memory, and async report signals for agent-oriented review. +- Add generic CPU hotspot detection and improve source-aware attribution for built-in detectors. +- Improve agent/text/markdown report output with user stacks, read-first targets, and clearer quality caveats. diff --git a/docs/architecture.md b/docs/architecture.md index 9118523..6ecc6f9 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -157,8 +157,8 @@ The pipeline transforms `CaptureBundle` into `LanternaReport` in four phases: 1. **Kind contributors** — each `ProfileKind` writes its report section into `profiles.` and publishes a typed view consumable via `context.forKind(id)`. 2. **Section analyzers** — optional extensions write under `extensions.` (not kind-specific). -3. **Finding analyzers** — cross-cutting rules emit `Finding`s, each tagged with a `profileKind` string. -4. **Finalize** — each kind's optional `finalize` hook mutates its own section based on the final findings (e.g. CPU sets `profiles.cpu.summary.dominantBlockingKind` and `topUserHotspot`). +3. **Finding analyzers** — cross-cutting rules emit `Finding`s, each tagged with a `profileKind` string. After every analyzer, the in-progress `snapshot.findings` is updated so later detectors can suppress duplicates or build on earlier evidence. +4. **Finalize** — each kind's optional `finalize` hook mutates its own section based on the final findings (e.g. CPU sets `profiles.cpu.summary.dominantBlockingKind`, `topCpuCulprit`, `topRequestEntry`, and `topUserHotspot`). ### Frame classification @@ -200,7 +200,7 @@ Correlation is conservative: if no single user frame dominates, Lanterna reports Findings are detectors running on the enriched snapshot, not on the raw bundle. Each finding carries a required `profileKind: string` tag so consumers can filter by kind. The full catalog of built-in findings, grouped by kind, lives in [extending/detectors.md](./extending/detectors.md#built-in-findings). -Findings are sorted by `priority.score` first, then by severity and attributed weight. Dominant user-code CPU is exposed as `profiles.cpu.summary.topUserHotspot` for context instead of as an actionable finding. +Findings are sorted by `priority.score` first, then by severity and attributed weight. Known anti-pattern detectors own specific categories such as blocking I/O or sync crypto; the generic `cpu-hotspot:*` detector is the fallback for plain user-code CPU that still needs a file/line or caller lead. CPU summaries also expose `profiles.cpu.summary.topCpuCulprit` for the self-heavy culprit and `topRequestEntry` / `topUserHotspot` for caller context. Built-in findings may also expose: diff --git a/docs/examples/report.agent.md b/docs/examples/report.agent.md index 2c54c13..85d0ec2 100644 --- a/docs/examples/report.agent.md +++ b/docs/examples/report.agent.md @@ -7,6 +7,7 @@ cwd: /repo kinds: [cpu, memory, async] lanterna_version: "1.5.1" cpu_quality: high +memory_quality: high memory_signal: present async_quality: high integrity: ok @@ -50,6 +51,7 @@ degrading_caveats: [] ## Kind Review — memory +- quality: high - memory_usage: 10 samples every 250ms - top_allocator: Buffer.alloc at node:buffer:10 — user_caller loadCache at src/cache.ts:18 (high, heap-sample-path, support 88.0%) diff --git a/docs/extending/detectors.md b/docs/extending/detectors.md index d65c0df..cdb558e 100644 --- a/docs/extending/detectors.md +++ b/docs/extending/detectors.md @@ -144,8 +144,9 @@ The default pack lives in `@lanterna-profiler/detectors` and pre-wires detectors | `blocking-io:` | Sampled sync `fs` / `child_process` / `zlib` frame on the hot path. | | `json-on-hot-path:` | `JSON.parse` / `JSON.stringify` consuming meaningful CPU. | | `node-modules-hotspot:` | A dependency frame dominates CPU time. | +| `cpu-hotspot:` | User-code self CPU ≥ 10%, or inclusive CPU ≥ 25% as a lower-confidence caller lead, when no more specific CPU detector already explains the frame. | | `excessive-gc` | `gcRatio > 10%` or `longestPauseMs > 100ms`. | -| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`. | +| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`; anchors to strong stall correlation when available, otherwise to the hottest user CPU fallback. | | `deopt-loop:` | Same deoptimised function seen ≥ 5 times (`--deep`) and hot in the profile. | | `require-in-hot-path` | Module loading functions sampled on the hot path. | @@ -191,6 +192,8 @@ import { - `buildAttributedFinding(...)` — one-shot helper that returns a fully-shaped `Finding`. - `CpuHotspotContext` — the attribution view (`fullHotspots`, `hotspotById`, `userCallerById`, `candidateCallersById`) reachable from a kind-scoped detector via `kinds.cpu.view.hotspotAnalysis`. +Finding analyzers run incrementally: after each detector, `snapshot.findings` includes findings emitted so far. Use that shared state to avoid duplicate generic findings when a specific rule has already explained the same frame. The built-in `cpu-hotspot` detector uses this to defer to `sync-crypto`, `blocking-io`, `json-on-hot-path`, `node-modules-hotspot`, and `require-in-hot-path`. + ## Thresholds `DETECTOR_THRESHOLDS` from `@lanterna-profiler/detectors` is the source of truth for tunable values: @@ -203,6 +206,8 @@ console.log(DETECTOR_THRESHOLDS.eventLoopStall.p99); You can read these values in your own detectors so users get consistent thresholds across the pack. +The generic CPU fallback is configured under `DETECTOR_THRESHOLDS.cpuHotspot`: `minSelfPct` gates self-heavy user functions (`evidence.extra.mode: "self"`), `minTotalPct` catches inclusive-only user callers when no self-heavy candidate exists (`mode: "inclusive-entry"`), `criticalPct` escalates severity, and `maxFindings` caps noise. Inclusive-entry findings use top-level `proofLevel: "heuristic"` so agents treat them as source-inspection leads rather than direct proof that the wrapper body is expensive. + ## Where to next - [plugin-loading.md](./plugin-loading.md) — how Lanterna discovers your plugin (CLI flag, `.lanterna.json`, packaging). diff --git a/docs/kinds/async.md b/docs/kinds/async.md index 33d0a50..8779476 100644 --- a/docs/kinds/async.md +++ b/docs/kinds/async.md @@ -77,3 +77,4 @@ Async-specific options: - **Microtasks default to off.** Enabling `--async-include-microtasks` produces very noisy reports. Use it only for the `microtask-flood` finding. - **Dropped records are sampled, not lost forever.** `quality.recordsDropped > 0` means raise `--async-max-events` for the next run if completeness matters. - **User callers are anchors, not proof.** Async `userCaller` is derived from already captured user frames. Prefer high-confidence CPU-window attribution when present; stack-only callers should guide inspection rather than be treated as the definitive line to edit. +- **Public async file paths are normalized.** When V8/CDP reports `file://` URLs, Lanterna converts them to normal filesystem paths before grouping hot files, chains, and finding evidence. Virtual bundler URLs are kept as-is. diff --git a/docs/kinds/cpu.md b/docs/kinds/cpu.md index e9f1505..8345b55 100644 --- a/docs/kinds/cpu.md +++ b/docs/kinds/cpu.md @@ -41,7 +41,9 @@ CPU-specific options: | `idleRatio` | Fraction of samples spent idle. | | `topCategory` | Dominant non-idle category. | | `dominantBlockingKind` | Coarse summary derived from emitted findings (e.g. `sync-crypto`, `blocking-io`). | -| `topUserHotspot` | Set when a single user function dominates user CPU. | +| `topCpuCulprit` | Strongest self-heavy user-code CPU frame. This is the first place to inspect when the question is "which line is burning CPU?". | +| `topRequestEntry` | Dominant user-code request/caller entry after accounting for existing findings and stall correlation. Useful when the hottest work happens below a wrapper or builtin. | +| `topUserHotspot` | Backward-compatible contextual hotspot. It currently mirrors `topRequestEntry` and remains useful for consumers that have not adopted the newer split fields. | ### `quality` @@ -83,8 +85,9 @@ V8 deoptimisation clusters with `function`, `file`, `line`, `reason`, `bailoutTy | `blocking-io:` | Sampled sync `fs` / `child_process` / `zlib` frame with meaningful CPU. | | `json-on-hot-path:` | `JSON.parse` / `JSON.stringify` consuming meaningful CPU. | | `node-modules-hotspot:` | A dependency frame dominates meaningful CPU time. | +| `cpu-hotspot:` | Generic fallback for user-code CPU not explained by a more specific CPU detector. Self-heavy frames are direct hotspots; inclusive-only frames are lower-confidence caller leads. | | `excessive-gc` | `gcRatio > 10%` or `longestPauseMs > 100ms`. | -| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`. | +| `event-loop-stall` | `p99LagMs >= 100` or `maxLagMs >= 200`; evidence points at a strongly correlated frame when possible, otherwise the hottest user CPU fallback. | | `deopt-loop:` | Same deoptimised function seen ≥ 5 times (`--deep`) and hot in the CPU profile. | | `require-in-hot-path` | Module loading functions sampled on the hot path. | @@ -93,7 +96,7 @@ Each finding ships with `confidence`, `proofLevel`, `evidence.file/line/function ## Reading order 1. `quality.confidence` and `quality.reasons[]` — is the profile worth acting on? -2. `summary.topCategory` and `summary.dominantBlockingKind` — where is CPU concentrated? +2. `summary.topCpuCulprit`, `summary.topRequestEntry`, `summary.topCategory`, and `summary.dominantBlockingKind` — where CPU is concentrated and which line to inspect first. 3. `findings[]` filtered to `profileKind === "cpu"` — prioritized hypotheses. 4. Top 5 `hotspots` even when no finding fired — direct evidence. 5. `eventLoop` — translates CPU pressure into latency. @@ -124,5 +127,6 @@ The classification feeds `summary` ratios and several finding heuristics. Self-n ## Caveats - A hotspot in `node_modules` or `node:builtin` is often a **symptom**. Inspect the user caller before blaming the dependency. +- A `cpu-hotspot:*` finding means no specialized detector explained the user-code CPU lead. If `evidence.extra.mode === "self"`, inspect the function body for tight loops, repeated transformations, cache misses, or CPU-bound algorithms. If `mode === "inclusive-entry"`, inspect the callees and hot stacks before blaming the wrapper body. - High `nativeRatio` is normal for CPU work that lives in C++ (crypto, compression, JSON). Look at user callers, not just the leaf. - `cwd` mismatch between Lanterna and the target can mis-classify your code as `node_modules`. Check `meta.cwd`. diff --git a/docs/reading-a-report.md b/docs/reading-a-report.md index 3884a27..ebc37c6 100644 --- a/docs/reading-a-report.md +++ b/docs/reading-a-report.md @@ -18,7 +18,7 @@ Use `--format agent` when an AI agent or automation will consume the report. It | --- | --- | --- | --- | | 1 | `meta` | What was captured (mode, duration, `profileKinds`, integrity flags). | `durationMs` very short, or `captureIntegrity.*` flags `false`. | | 2 | `profiles.cpu.quality` | Whether CPU evidence is strong enough to trust. | `confidence = low`, high idle, low samples, untimed samples. | -| 3 | `profiles.cpu.summary` | Where CPU time went (ratios, top category). | `idleRatio` > 0.8 — the profile is mostly idle. | +| 3 | `profiles.cpu.summary` | Where CPU time went and the best first line to inspect (`topCpuCulprit`, `topRequestEntry`). | `idleRatio` > 0.8 — the profile is mostly idle. | | 4 | `findings` | Prioritized hypotheses backed by the capture (tagged `profileKind`). | Empty `findings[]` does not prove a healthy profile. | | 5 | `profiles.cpu.hotspots` | Where CPU is actually spent (self + inclusive). | A hot leaf in `node_modules` is usually a symptom, not a cause. | | 6 | `profiles.cpu.eventLoop` | Latency signal + stall windows. | `confidence = low` or `measurementBasis = histogram` alone. | @@ -34,7 +34,7 @@ Use `--format agent` when an AI agent or automation will consume the report. It 1. **Read `meta` and `captureIntegrity`** — sanity-check what you're about to interpret. Short `durationMs`, `controlChannel: false` in spawn mode, or missing `gcTimed` change how strongly you should weight downstream sections. See [signal-quality.md](./signal-quality.md). 2. **Read `profiles.cpu.quality`** before `findings[]` — a low-confidence profile can identify leads but not prove root causes. -3. **Filter `findings[]` by severity and `profileKind`** — start with `severity != "info"`; group by kind to know which specialist page to consult. +3. **Filter `findings[]` by severity and `profileKind`** — start with `severity != "info"`; group by kind to know which specialist page to consult. A `cpu-hotspot:*` finding is the generic fallback when no more specific CPU detector explains a user-code CPU lead. 4. **Open the implicated source file** — `evidence.file` and `evidence.line` are where the action should happen. For some detectors that points at the user caller rather than a builtin callee. 5. **Cross-reference kinds when both were captured** — `alloc-in-hot-path` is the canonical example: a frame hot on CPU **and** in top allocators is the highest-leverage fix you can make. 6. **Use `hotStacks` only when a hotspot is ambiguous** — it surfaces the surrounding call path without manual reconstruction. @@ -55,8 +55,9 @@ The full catalog (with triggers and remediations) lives in [extending/detectors. | `blocking-io:` | cpu | Sync `fs` / `child_process` / `zlib` on the hot path. Use the async equivalent. | | `json-on-hot-path:` | cpu | `JSON.parse` / `JSON.stringify` is a meaningful share of CPU. Cache, stream, or reduce. | | `node-modules-hotspot:` | cpu | A dependency dominates CPU. **Inspect the user caller path first.** | +| `cpu-hotspot:` | cpu | Plain user code dominates CPU without matching a known anti-pattern. `mode: "self"` is a direct body hotspot; `mode: "inclusive-entry"` is a caller/context lead. | | `excessive-gc` | cpu | GC ratio or longest pause is too high. Hunt allocations in top user hotspots. | -| `event-loop-stall` | cpu | The main thread stopped servicing tasks. Check `correlatedHotspots`. | +| `event-loop-stall` | cpu | The main thread stopped servicing tasks. Prefer strong `correlatedHotspots`; otherwise treat the fallback hotspot as the best CPU lead, not proven stall causality. | | `deopt-loop:` | cpu | A hot function keeps deoptimising under `--deep`. Stabilise shapes/types. | | `require-in-hot-path` | cpu | Module loading on the hot path. Hoist or memoize the lazy load. | | `memory-growth:rss` / `memory-growth:heapUsed` | memory | Sustained linear growth ≥ 1 MB/s. Inspect top allocators and lifetimes. | @@ -90,6 +91,12 @@ For some detectors (e.g. `sync-crypto-on-hot-path`, `blocking-io:`), `evide When `measurementBasis === "histogram"`, `correlatedHotspots[]` is based on overall CPU overlap — not temporal overlap with stall windows. Read [signal-quality.md](./signal-quality.md#profilescpueventloop) before claiming causality. +`event-loop-stall` can also expose `evidence.extra.proofLevel: "hotspot-fallback"` plus `fallbackHotspots[]`. That means the event-loop lag is real, but no measured stall window dominated enough to blame one frame directly; Lanterna anchored the finding to the hottest user CPU frame so the report still points at a useful source line. + +#### `topCpuCulprit` vs `topRequestEntry` + +`profiles.cpu.summary.topCpuCulprit` is self-CPU first: it tries to answer "which function body burned CPU?". `topRequestEntry` is finding-aware and caller-oriented: it keeps the request/user entry that explains why the hot work happened. If they differ, inspect `topCpuCulprit` for the local algorithmic problem and `topRequestEntry` for call frequency, payload size, or routing context. + #### `deopt-loop:` Fires only when a function is **both** hot in the CPU profile **and** repeatedly deoptimised under `--deep`. Focus on stabilising shapes and types, then reprofile. One-off deopt entries are noise. @@ -104,6 +111,8 @@ A dependency hotspot is often a symptom — your code controls when and how ofte > **Assuming no findings means no problem.** Lanterna's detectors are heuristic. A clean `findings[]` lowers the odds of the usual issues; it does not prove the profile is healthy. +> **Ignoring `cpu-hotspot:*` because it is generic.** This finding intentionally covers custom CPU-heavy code. `mode: "self"` is often the most direct file/line when the workload is a loop, scoring function, transformation, or algorithm instead of a known Node API misuse. `mode: "inclusive-entry"` should be treated as a lead to inspect callees/hot stacks. + > **Blaming `node_modules` immediately.** A dependency hotspot is often just where the CPU landed. The caller path is usually your code. > **Ignoring `idleRatio`.** A profile captured without real load can be technically valid but operationally misleading. diff --git a/docs/report-schema.md b/docs/report-schema.md index bd4238f..ef4789f 100644 --- a/docs/report-schema.md +++ b/docs/report-schema.md @@ -1,4 +1,4 @@ -# Report Schema (v2) +# Report Schema (v2.0.0) A `LanternaReport` is the structured JSON Lanterna emits after every capture. This page describes its shape. For interpretation rules, see [reading-a-report.md](./reading-a-report.md). @@ -99,7 +99,7 @@ interface SourceLocation { } ``` -When present, prefer `source.file:source.line` for human diagnosis and patching, but keep the generated `file:line` as fallback context. On-disk sources are relative to `meta.cwd` when possible. Bundler virtual sources such as `webpack://app/src/server.ts` or `vite:/src/server.ts` are kept verbatim and may not exist on disk. +When present, prefer `source.file:source.line` for human diagnosis and patching, but keep the generated `file:line` as fallback context. On-disk sources are relative to `meta.cwd` when possible. `file://` URLs observed from V8/CDP are normalized back to normal filesystem paths in public report entries when possible. Bundler virtual sources such as `webpack://app/src/server.ts` or `vite:/src/server.ts` are kept verbatim and may not exist on disk. `source?: SourceLocation` can appear on CPU hotspots, hot-stack frames and anchors, memory allocators and memory summaries, async frame-bearing entries, deopts, and `findings[].evidence`. @@ -109,7 +109,7 @@ When present, prefer `source.file:source.line` for human diagnosis and patching, | Section | Purpose | | --- | --- | -| `summary` | High-level CPU ratios (user / node_modules / builtin / native / GC / idle), `topCategory`, `dominantBlockingKind`, `topUserHotspot`. | +| `summary` | High-level CPU ratios (user / node_modules / builtin / native / GC / idle), `topCategory`, `dominantBlockingKind`, `topCpuCulprit`, `topRequestEntry`, `topUserHotspot`. | | `quality` | Confidence gate for CPU evidence — `confidence`, `sampleCount`, `durationMs`, `idleRatio`, `samplesTimed`, `durationBasis`, `reasons[]`, `recommendations[]`. | | `hotspots` | Aggregated functions with `selfMs`/`selfPct` and `totalMs`/`totalPct`, `callers[]`/`callees[]`, `category`, `optimizationState`, and optional `userCaller` for non-user frames. | | `hotStacks` | Most frequent complete sampled stacks with `weightPct` and `frames[]`. | @@ -125,6 +125,7 @@ Detail: [kinds/cpu.md](./kinds/cpu.md). | Section | Purpose | | --- | --- | | `summary` | Total sampled bytes, top allocator, RSS / heapUsed / external / arrayBuffers stats (start/end/min/max/mean/p95) plus linear `slopeBytesPerSec`. | +| `quality` | Memory confidence gate — `confidence`, `reasons[]`, `recommendations[]`. | | `hotAllocators` | Frames ranked by `selfBytes` / `totalBytes`, with file/line, frame category, and optional `userCaller` for external allocators. | | `memoryUsage` | Compact `process.memoryUsage()` metadata (`sampleCount`, first/last sample). Raw samples present only with `--include-memory-samples`. | | `heapSnapshotAnalysis` | Optional start/end retained-growth summary when `--heap-snapshot-analysis` is enabled. Very large snapshots return `available: false` with a warning instead of being parsed unbounded. | @@ -143,7 +144,7 @@ Each finding has the same shape regardless of which kind produced it: | Field | Purpose | | --- | --- | -| `id` | Detector-specific identifier (e.g. `blocking-io:fs.readFileSync`). | +| `id` | Detector-specific identifier (e.g. `blocking-io:fs.readFileSync`, `cpu-hotspot:`). | | `profileKind` | Source kind (`"cpu"`, `"memory"`, `"async"`, …). | | `severity` | `critical`, `warning`, or `info`. | | `category` | Grouping for filtering. | @@ -163,7 +164,7 @@ The full catalog of built-in findings, grouped by kind, is in [extending/detecto ## Schema versioning -This is **schema v2**. The defining trait of v2 is per-kind nesting under `profiles..*` and `meta.kinds..*`. Future schema changes will bump the version; consumers should branch on the discriminator they need rather than on the version itself. +This is **schema v2.0.0**. The defining trait of v2 is per-kind nesting under `profiles..*` and `meta.kinds..*`. Additive optional fields can appear within the same major schema; breaking changes should bump the version. Consumers should branch on the fields they need rather than on the version alone. ## See also diff --git a/docs/signal-quality.md b/docs/signal-quality.md index f77dbad..ec34fd6 100644 --- a/docs/signal-quality.md +++ b/docs/signal-quality.md @@ -66,9 +66,13 @@ The event-loop section has its own confidence pair so consumers can judge stall When `measurementBasis === "histogram"`, `correlatedHotspots[]` is based on overall CPU overlap, not temporal overlap with stall windows — interpret accordingly. -## `profiles.memory.*` quality +The `event-loop-stall` finding mirrors this distinction in `evidence.extra.proofLevel`. `aggregate-correlation` means a measured stall window had a dominant user hotspot. `hotspot-fallback` means lag crossed the threshold but correlation was not strong enough, so the finding is anchored to the hottest user CPU frame as an inspection lead. -The memory kind exposes statistical signals rather than a discrete confidence enum: +## `profiles.memory.quality` + +The memory kind uses the same top-level quality shape as other kinds: `confidence`, `reasons[]`, and `recommendations[]`. Its confidence is derived from the availability and volume of memory-usage samples, heap-sampling data, and heap-snapshot warnings when snapshot analysis is enabled. + +Memory-specific signals to inspect alongside `profiles.memory.quality`: - `summary.rss.slopeBytesPerSec` — linear growth slope. Sustained slopes ≥ 1 MB/s trigger a `memory-growth` finding (warning); ≥ 5 MB/s upgrades to critical. Short captures with warm-up phases can produce artificially steep slopes. - `memoryUsage.sampleCount` — how many `process.memoryUsage()` samples landed. Below ~10 samples, the slope is unreliable. diff --git a/docs/source-maps.md b/docs/source-maps.md index be9ec3d..451de5c 100644 --- a/docs/source-maps.md +++ b/docs/source-maps.md @@ -52,6 +52,8 @@ Whenever a frame is mapped successfully, a `source` object is attached: `source` appears on: - `profiles.cpu.hotspots[].source` +- `profiles.cpu.summary.topCpuCulprit.source` +- `profiles.cpu.summary.topRequestEntry.source` - `profiles.cpu.summary.topUserHotspot.source` - `profiles.cpu.hotStacks[].frames[].source` - `profiles.cpu.hotStackClusters[].anchor.source` diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 3dab587..6c20cd8 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -117,6 +117,8 @@ Common fixes: 3. **Deopts not detected — missing `--deep`.** The `deopt-loop` detector only fires when `--deep` is passed and only for functions also hot in the CPU profile. Without `--deep`, `deopts[]` is empty by design. 4. **GC findings suppressed on very short captures.** If `durationMs < 250` and no timed GC events were captured, the `excessive-gc` detector suppresses findings to avoid false positives. Run for longer. +If `findings[]` is empty but `profiles.cpu.summary.topCpuCulprit` or `profiles.cpu.hotspots[0]` is strong, inspect that frame anyway. A missing specialized finding only means Lanterna did not match a known pattern; custom CPU-heavy code can still be the bottleneck. + --- ## Low-confidence CPU profile @@ -182,6 +184,7 @@ Low confidence does not make the report useless — use it to choose what to ins 1. **Low-confidence histogram measurement.** If `eventLoop.measurementBasis === "histogram"` and `confidence === "low"`, thresholds are already raised (p99 ≥ 200 ms, max ≥ 400 ms). Check `eventLoop.histogram` directly. 2. **One-off startup cost inflated the max.** The very first event-loop tick after module loading may be long. If `stallIntervals` shows a single stall near `atMs: 0`, it may be startup, not steady-state behavior. 3. **Heartbeats not available.** When `measurementBasis === "histogram"`, Lanterna cannot reconstruct which user-code frames ran during the stall window. `correlatedHotspots` is then based on overall CPU overlap, not temporal overlap. +4. **Fallback attribution is not direct causality.** If `findings[].evidence.extra.proofLevel === "hotspot-fallback"`, the lag crossed the threshold but no stall window had strong attribution. Treat the reported file/line as the best CPU lead and confirm with source or a longer rerun. --- diff --git a/packages/cli/src/renderers/agent-renderer.ts b/packages/cli/src/renderers/agent-renderer.ts index 4aa2b5d..3f8c17f 100644 --- a/packages/cli/src/renderers/agent-renderer.ts +++ b/packages/cli/src/renderers/agent-renderer.ts @@ -16,6 +16,8 @@ type Frame = { source?: { file: string; line: number }; }; +type CpuStackFrame = Frame & { category?: string }; + type ReadTargetSource = 'finding' | 'cpu' | 'memory' | 'async'; type ReadTargetDecision = 'read-first' | 'inspect-lead' | 'supporting-context'; type ReadTargetReason = @@ -24,7 +26,11 @@ type ReadTargetReason = | 'user-caller' | 'dependency-hotspot-caller' | 'runtime-hotspot-caller' + | 'cpu-user-stack' + | 'top-cpu-culprit' | 'top-cpu-hotspot' + | 'top-request-entry' + | 'top-user-hotspot' | 'hot-stack-cluster' | 'memory-allocator' | 'top-async-hot-file' @@ -55,11 +61,10 @@ export class AgentReportRenderer implements ReportRenderer { readonly format: RenderableFormat = 'agent'; render(report: LanternaReport): string { - const findings = report.findings ?? []; const lines: string[] = []; appendFrontmatter(lines, report); lines.push(''); - appendFindings(lines, findings); + appendFindings(lines, report); lines.push(''); appendKindReview(lines, report); lines.push(''); @@ -88,6 +93,9 @@ function appendFrontmatter(lines: string[], report: LanternaReport): void { lines.push(`kinds: ${yamlInlineList(meta?.profileKinds ?? [])}`); lines.push(`lanterna_version: ${yamlScalar(meta?.lanternaVersion ?? 'unknown')}`); lines.push(`cpu_quality: ${yamlScalar(report.profiles?.cpu?.quality?.confidence ?? 'absent')}`); + lines.push( + `memory_quality: ${yamlScalar(report.profiles?.memory?.quality?.confidence ?? 'absent')}`, + ); lines.push(`memory_signal: ${yamlScalar(memorySignalLabel(report.profiles?.memory))}`); lines.push( `async_quality: ${yamlScalar(report.profiles?.async?.quality?.confidence ?? 'absent')}`, @@ -128,7 +136,8 @@ function integrityLabel( // Findings — table summary + per-finding detail block. // --------------------------------------------------------------------------- -function appendFindings(lines: string[], findings: Finding[]): void { +function appendFindings(lines: string[], report: LanternaReport): void { + const findings = report.findings ?? []; lines.push('## Findings'); lines.push(''); if (findings.length === 0) { @@ -162,12 +171,17 @@ function appendFindings(lines: string[], findings: Finding[]): void { appendTable(lines, headers, rows); lines.push(''); findings.forEach((finding, index) => { - appendFindingDetail(lines, finding, index + 1); + appendFindingDetail(lines, finding, index + 1, report); if (index < findings.length - 1) lines.push(''); }); } -function appendFindingDetail(lines: string[], finding: Finding, position: number): void { +function appendFindingDetail( + lines: string[], + finding: Finding, + position: number, + report: LanternaReport, +): void { lines.push(`## Finding ${position} — ${finding.id}`); lines.push(''); lines.push(`- title: ${finding.title}`); @@ -178,6 +192,8 @@ function appendFindingDetail(lines: string[], finding: Finding, position: number if (candidateCallers.length > 0) { lines.push(`- candidate_callers: ${candidateCallers.map(formatUserCallerCompact).join('; ')}`); } + const userStack = cpuUserStackForFinding(finding, report); + if (userStack) lines.push(`- user_stack: ${userStack}`); lines.push(`- observed: ${formatMeasurements(finding.measurements?.observed)}`); lines.push(`- thresholds: ${formatMeasurements(finding.measurements?.thresholds)}`); lines.push(`- impact: ${formatImpact(finding)}`); @@ -227,7 +243,24 @@ function appendCpuKindReview(lines: string[], report: LanternaReport): void { return; } lines.push(`- quality: ${cpu.quality?.confidence ?? 'unknown'}`); - if (isRenderableReviewFrame(cpu.summary?.topUserHotspot)) { + if (isRenderableReviewFrame(cpu.summary?.topCpuCulprit)) { + lines.push( + `- top_cpu_culprit: ${frameLabel(cpu.summary.topCpuCulprit)} at ${frameLocation(cpu.summary.topCpuCulprit)} (${formatPct(cpu.summary.topCpuCulprit.selfPct)} self, ${formatPct(cpu.summary.topCpuCulprit.totalPct)} total)`, + ); + } + if ( + isRenderableReviewFrame(cpu.summary?.topRequestEntry) && + !sameFrameLocation(cpu.summary.topRequestEntry, cpu.summary?.topCpuCulprit) + ) { + lines.push( + `- top_request_entry: ${frameLabel(cpu.summary.topRequestEntry)} at ${frameLocation(cpu.summary.topRequestEntry)} (${formatPct(cpu.summary.topRequestEntry.totalPct)} total)`, + ); + } + if ( + isRenderableReviewFrame(cpu.summary?.topUserHotspot) && + !sameFrameLocation(cpu.summary.topUserHotspot, cpu.summary?.topCpuCulprit) && + !sameFrameLocation(cpu.summary.topUserHotspot, cpu.summary?.topRequestEntry) + ) { lines.push( `- top_user_hotspot: ${frameLabel(cpu.summary.topUserHotspot)} at ${frameLocation(cpu.summary.topUserHotspot)}`, ); @@ -287,6 +320,7 @@ function appendMemoryKindReview(lines: string[], report: LanternaReport): void { return; } const usage = memory.memoryUsage; + lines.push(`- quality: ${memory.quality?.confidence ?? 'unknown'}`); lines.push( `- memory_usage: ${ usage?.available @@ -460,6 +494,9 @@ function degradingSignalCaveats(report: LanternaReport): string[] { if (report.profiles?.memory?.memoryUsage?.available === false) { caveats.push('memory usage series unavailable'); } + if (report.profiles?.memory?.quality?.confidence === 'low') { + caveats.push('memory confidence low'); + } const heapSnapshotWarnings = report.profiles?.memory?.heapSnapshotAnalysis?.warnings ?? []; if (heapSnapshotWarnings.length > 0) { caveats.push(`heap snapshot warnings: ${heapSnapshotWarnings.join('; ')}`); @@ -484,11 +521,17 @@ function degradingSignalCaveats(report: LanternaReport): string[] { function hasInsufficientSignal(report: LanternaReport): boolean { return ( blockingIntegrityCaveats(report).length > 0 || - degradingSignalCaveats(report).length > 0 || + rerunRequiredSignalCaveats(report).length > 0 || (report.findings ?? []).some((finding) => decisionForFinding(finding) === 'rerun') ); } +function rerunRequiredSignalCaveats(report: LanternaReport): string[] { + return degradingSignalCaveats(report).filter( + (caveat) => caveat !== 'event-loop timing unavailable' && caveat !== 'GC timing unavailable', + ); +} + function decisionForFinding(finding: Finding): 'actionable' | 'hypothesis' | 'rerun' { if (finding.confidence === 'low') return 'hypothesis'; if (finding.priority?.actionConfidence === 'low') return 'hypothesis'; @@ -520,6 +563,135 @@ function candidateCallersFromEvidenceExtra(extra: unknown): UserCallerAttributio return Array.isArray(value) ? (value as UserCallerAttribution[]) : []; } +function cpuUserStackForFinding(finding: Finding, report: LanternaReport): string | undefined { + const stack = matchedCpuUserStackForFinding(finding, report); + if (!stack) return undefined; + const leafSuffix = + stack.leaf && !isUserStackFrame(stack.leaf) + ? `, leaf ${frameLabel(stack.leaf)} at ${frameLocation(stack.leaf)}` + : ''; + return `${formatCpuStackFrames(stack.userFrames)} (${formatPct(stack.weightPct)} stack${leafSuffix})`; +} + +function matchedCpuUserStackForFinding( + finding: Finding, + report: LanternaReport, +): { userFrames: CpuStackFrame[]; leaf?: CpuStackFrame; weightPct: number } | undefined { + if (finding.profileKind !== 'cpu') return undefined; + const hotStacks = report.profiles?.cpu?.hotStacks ?? []; + const matches = hotStacks + .map((stack) => ({ + stack, + score: scoreCpuStackForFinding(stack.frames, finding), + })) + .filter((match) => match.score > 0) + .sort( + (left, right) => right.score - left.score || right.stack.weightPct - left.stack.weightPct, + ); + const best = matches[0]; + if (!best) return undefined; + const stack = trimCpuUserStackForFinding(best.stack.frames, finding); + if (stack.userFrames.length === 0) return undefined; + return { ...stack, weightPct: best.stack.weightPct }; +} + +function scoreCpuStackForFinding(frames: readonly CpuStackFrame[], finding: Finding): number { + let score = frames.some((frame) => frameMatchesTarget(frame, finding.evidence)) ? 100 : 0; + const callee = calleeNameFromExtra(finding.evidence.extra); + if (callee && frames.some((frame) => frameMatchesFunction(frame, callee))) score += 50; + for (const caller of candidateCallersFromEvidenceExtra(finding.evidence.extra)) { + if (frames.some((frame) => frameMatchesTarget(frame, caller))) { + score += caller.stackDistance === 1 ? 12 : 8; + } + } + return score; +} + +function trimCpuUserStackForFinding( + frames: readonly CpuStackFrame[], + finding: Finding, +): { userFrames: CpuStackFrame[]; leaf?: CpuStackFrame } { + const causalFrames = [...frames].reverse(); + const endIndex = cpuStackEndIndex(causalFrames, finding); + const prefix = causalFrames.slice(0, endIndex + 1); + const firstUserIndex = prefix.findIndex(isUserStackFrame); + if (firstUserIndex < 0) return { userFrames: [], leaf: prefix[prefix.length - 1] }; + const userFrames = prefix.slice(firstUserIndex).filter(isUserStackFrame); + return { userFrames, leaf: prefix[prefix.length - 1] }; +} + +function cpuStackEndIndex(frames: readonly CpuStackFrame[], finding: Finding): number { + const callee = calleeNameFromExtra(finding.evidence.extra); + if (callee) { + const calleeIndex = frames.findIndex((frame) => frameMatchesFunction(frame, callee)); + if (calleeIndex >= 0) return calleeIndex; + } + const evidenceIndex = frames.findIndex((frame) => frameMatchesTarget(frame, finding.evidence)); + return evidenceIndex >= 0 ? evidenceIndex : frames.length - 1; +} + +function formatCpuStackFrames(frames: readonly CpuStackFrame[]): string { + const labels = frames.map((frame) => `${frameLabel(frame)} at ${frameLocation(frame)}`); + if (labels.length <= 12) return labels.join(' -> '); + return [...labels.slice(0, 5), '...', ...labels.slice(-6)].join(' -> '); +} + +function calleeNameFromExtra(extra: unknown): string | undefined { + if (!extra || typeof extra !== 'object') return undefined; + const value = Reflect.get(extra, 'callee'); + return typeof value === 'string' && value.length > 0 ? value : undefined; +} + +function frameMatchesTarget( + frame: CpuStackFrame, + target: { + function?: string; + file: string; + line: number; + source?: { file: string; line: number }; + }, +): boolean { + const locationMatches = + sameFrameFileLine(frame, target.file, target.line) || + (target.source ? sameFrameFileLine(frame, target.source.file, target.source.line) : false); + if (!locationMatches) return false; + if (!target.function || !frame.function) return true; + return frameMatchesFunction(frame, target.function); +} + +function sameFrameFileLine(frame: CpuStackFrame, file: string, line: number): boolean { + return ( + (frame.file === file && frame.line === line) || + (frame.source?.file === file && frame.source.line === line) + ); +} + +function frameMatchesFunction( + frame: Pick, + functionName: string, +): boolean { + const normalizedFrameName = stripV8OptimizationPrefix(frame.function ?? ''); + const normalizedTargetName = stripV8OptimizationPrefix(functionName); + return ( + normalizedFrameName === normalizedTargetName || + normalizedFrameName.endsWith(`.${normalizedTargetName}`) + ); +} + +function stripV8OptimizationPrefix(functionName: string): string { + return functionName.replace(/^[*~]/, ''); +} + +function isUserStackFrame(frame: CpuStackFrame): boolean { + return frame.category === 'user'; +} + +function isAnonymousUserStackFrame(frame: CpuStackFrame): boolean { + return ( + isUserStackFrame(frame) && (frame.function === '(anonymous)' || frame.function?.trim() === '') + ); +} + // --------------------------------------------------------------------------- // Location, files, frames helpers. // --------------------------------------------------------------------------- @@ -552,6 +724,11 @@ function userCallerCell(caller: UserCallerAttribution | undefined): string { return `${frameLocation(caller)} (${caller.confidence})`; } +function sameFrameLocation(left: Frame | undefined, right: Frame | undefined): boolean { + if (!left || !right) return false; + return frameLocation(left) === frameLocation(right) && left.function === right.function; +} + function userCallerSuffix(caller: UserCallerAttribution | undefined): string { if (!caller) return ''; return ` — user_caller ${formatUserCallerCompact(caller)}`; @@ -565,14 +742,15 @@ function formatUserCallerCompact(caller: UserCallerAttribution): string { function collectReadTargets(report: LanternaReport): ReadTarget[] { const targets: ReadTarget[] = []; - collectFindingReadTargets(targets, report.findings ?? []); + collectFindingReadTargets(targets, report); collectAggregateReadTargets(targets, report); return dedupeReadTargets(targets) .sort((a, b) => a.rank - b.rank || a.location.localeCompare(b.location)) .slice(0, 10); } -function collectFindingReadTargets(targets: ReadTarget[], findings: Finding[]): void { +function collectFindingReadTargets(targets: ReadTarget[], report: LanternaReport): void { + const findings = report.findings ?? []; findings.forEach((finding, index) => { const signal = formatImpact(finding); const findingDecision = decisionForFinding(finding); @@ -588,6 +766,7 @@ function collectFindingReadTargets(targets: ReadTarget[], findings: Finding[]): decision: findingIsActionable ? 'read-first' : 'inspect-lead', rank: findingIsActionable ? index : 100 + index, }); + collectCpuUserStackReadTargets(targets, finding, report, index); return; } const userCaller = userCallerFromEvidenceExtra(finding.evidence.extra); @@ -625,59 +804,101 @@ function collectFindingReadTargets(targets: ReadTarget[], findings: Finding[]): rank: 150 + index * 10 + candidateIndex, }); }); + collectCpuUserStackReadTargets(targets, finding, report, index); + }); +} + +function collectCpuUserStackReadTargets( + targets: ReadTarget[], + finding: Finding, + report: LanternaReport, + findingIndex: number, +): void { + const stack = matchedCpuUserStackForFinding(finding, report); + if (!stack) return; + const hasNamedUserFrame = stack.userFrames.some((frame) => !isAnonymousUserStackFrame(frame)); + stack.userFrames.forEach((frame, frameIndex) => { + if (hasNamedUserFrame && isAnonymousUserStackFrame(frame)) return; + const target = readTargetFrame(frame); + if (!target) return; + targets.push({ + ...target, + reason: 'cpu-user-stack', + source: 'finding', + signal: `${formatPct(stack.weightPct)} stack`, + decision: frameMatchesTarget(frame, finding.evidence) ? 'inspect-lead' : 'supporting-context', + rank: 80 + findingIndex * 20 + frameIndex, + }); }); } function collectAggregateReadTargets(targets: ReadTarget[], report: LanternaReport): void { const cpu = report.profiles?.cpu; if (cpu) { - pushReadTarget(targets, cpu.summary?.topUserHotspot, { - reason: 'top-cpu-hotspot', + const hasCpuFinding = (report.findings ?? []).some((finding) => finding.profileKind === 'cpu'); + pushReadTarget(targets, cpu.summary?.topCpuCulprit, { + reason: 'top-cpu-culprit', source: 'cpu', - signal: signalFromPctFrame(cpu.summary?.topUserHotspot), - decision: 'inspect-lead', - rank: 200, + signal: signalFromPctFrame(cpu.summary?.topCpuCulprit), + decision: 'read-first', + rank: 180, }); - for (const hotspot of cpu.hotspots ?? []) { - const userCaller = readTargetFrame(hotspot.userCaller); - if (userCaller && isExternalOrRuntimeFrame(hotspot)) { - targets.push({ - ...userCaller, - reason: reasonForExternalUserCaller(hotspot), + if (!hasCpuFinding) { + pushReadTarget(targets, cpu.summary?.topRequestEntry, { + reason: 'top-request-entry', + source: 'cpu', + signal: signalFromTotalPctFrame(cpu.summary?.topRequestEntry), + decision: 'inspect-lead', + rank: 195, + }); + pushReadTarget(targets, cpu.summary?.topUserHotspot, { + reason: 'top-user-hotspot', + source: 'cpu', + signal: signalFromTotalPctFrame(cpu.summary?.topUserHotspot), + decision: 'inspect-lead', + rank: 200, + }); + for (const hotspot of cpu.hotspots ?? []) { + const userCaller = readTargetFrame(hotspot.userCaller); + if (userCaller && isExternalOrRuntimeFrame(hotspot)) { + targets.push({ + ...userCaller, + reason: reasonForExternalUserCaller(hotspot), + source: 'cpu', + signal: signalFromPctFrame(hotspot), + decision: hotspot.userCaller?.confidence === 'high' ? 'read-first' : 'inspect-lead', + rank: hotspot.userCaller?.confidence === 'high' ? 210 : 230, + }); + } else { + pushReadTarget(targets, hotspot, { + reason: 'top-cpu-hotspot', + source: 'cpu', + signal: signalFromPctFrame(hotspot), + decision: 'inspect-lead', + rank: 220, + }); + } + } + for (const stack of cpu.hotStacks ?? []) { + const frame = stack.frames.find((candidate) => Boolean(readTargetFrame(candidate))); + pushReadTarget(targets, frame, { + reason: 'hot-stack-cluster', source: 'cpu', - signal: signalFromPctFrame(hotspot), - decision: hotspot.userCaller?.confidence === 'high' ? 'read-first' : 'inspect-lead', - rank: hotspot.userCaller?.confidence === 'high' ? 210 : 230, + signal: signalFromWeight(stack.weightPct), + decision: 'supporting-context', + rank: 240, }); - } else { - pushReadTarget(targets, hotspot, { - reason: 'top-cpu-hotspot', + } + for (const cluster of cpu.hotStackClusters ?? []) { + pushReadTarget(targets, cluster.anchor, { + reason: 'hot-stack-cluster', source: 'cpu', - signal: signalFromPctFrame(hotspot), - decision: 'inspect-lead', - rank: 220, + signal: signalFromWeight(cluster.weightPct), + decision: 'supporting-context', + rank: 250, }); } } - for (const stack of cpu.hotStacks ?? []) { - const frame = stack.frames.find((candidate) => Boolean(readTargetFrame(candidate))); - pushReadTarget(targets, frame, { - reason: 'hot-stack-cluster', - source: 'cpu', - signal: signalFromWeight(stack.weightPct), - decision: 'supporting-context', - rank: 240, - }); - } - for (const cluster of cpu.hotStackClusters ?? []) { - pushReadTarget(targets, cluster.anchor, { - reason: 'hot-stack-cluster', - source: 'cpu', - signal: signalFromWeight(cluster.weightPct), - decision: 'supporting-context', - rank: 250, - }); - } } const memory = report.profiles?.memory; if (memory) { @@ -813,7 +1034,7 @@ function pushReadTarget( function readTargetFrame( frame: Frame | undefined, ): Pick | undefined { - if (!frame || isPseudoFrameFunction(frame.function)) return undefined; + if (!frame) return undefined; if (frame.source && isEditableUserFile(frame.source.file)) { return { file: frame.source.file, @@ -869,8 +1090,16 @@ function formatReadTargetReason(reason: ReadTargetReason): string { return 'user caller for dependency hotspot'; case 'runtime-hotspot-caller': return 'user caller for runtime hotspot'; + case 'cpu-user-stack': + return 'CPU user stack'; + case 'top-cpu-culprit': + return 'top CPU culprit'; case 'top-cpu-hotspot': return 'top CPU hotspot'; + case 'top-request-entry': + return 'top request entry'; + case 'top-user-hotspot': + return 'top user hotspot'; case 'hot-stack-cluster': return 'hot stack cluster'; case 'memory-allocator': @@ -923,6 +1152,13 @@ function signalFromPctFrame(frame: (Frame & { selfPct?: number }) | undefined): return '—'; } +function signalFromTotalPctFrame(frame: (Frame & { totalPct?: number }) | undefined): string { + if (typeof frame?.totalPct === 'number' && Number.isFinite(frame.totalPct)) { + return `${formatPct(frame.totalPct)} total`; + } + return signalFromPctFrame(frame); +} + function signalFromWeight(weightPct: number | undefined): string { if (typeof weightPct === 'number' && Number.isFinite(weightPct)) { return `${formatPct(weightPct)} stack weight`; @@ -1037,7 +1273,23 @@ function isNonEmpty(value: string | undefined): value is string { function isEditableUserFile(value: string | undefined): value is string { if (!isNonEmpty(value)) return false; - return !isPseudoFile(value) && !isDependencyOrRuntimePath(value) && !isVirtualSourcePath(value); + return ( + looksLikeFilePath(value) && + !isPseudoFile(value) && + !isDependencyOrRuntimePath(value) && + !isVirtualSourcePath(value) + ); +} + +function looksLikeFilePath(value: string): boolean { + return ( + value.startsWith('/') || + value.startsWith('./') || + value.startsWith('../') || + value.includes('/') || + value.includes('\\') || + /\.[A-Za-z0-9]+$/.test(value) + ); } function isGeneratedOutputPath(file: string): boolean { diff --git a/packages/cli/src/renderers/markdown-renderer.ts b/packages/cli/src/renderers/markdown-renderer.ts index 6181c3e..f20be8d 100644 --- a/packages/cli/src/renderers/markdown-renderer.ts +++ b/packages/cli/src/renderers/markdown-renderer.ts @@ -52,6 +52,19 @@ export class MarkdownReportRenderer implements ReportRenderer { lines.push( `- GC: ${formatMs(cpu.gc?.totalPauseMs)} total pause, ${formatMs(cpu.gc?.longestPauseMs)} longest`, ); + if (cpu.summary?.topCpuCulprit) { + lines.push( + `- Top CPU culprit: ${escapePipe(cpu.summary.topCpuCulprit.function)} at \`${escapeBackticks(formatFrameLocation(cpu.summary.topCpuCulprit))}\` (${formatPct(cpu.summary.topCpuCulprit.selfPct)} self, ${formatPct(cpu.summary.topCpuCulprit.totalPct)} total)`, + ); + } + if ( + cpu.summary?.topRequestEntry && + !sameFrameLocation(cpu.summary.topRequestEntry, cpu.summary.topCpuCulprit) + ) { + lines.push( + `- Top request entry: ${escapePipe(cpu.summary.topRequestEntry.function)} at \`${escapeBackticks(formatFrameLocation(cpu.summary.topRequestEntry))}\` (${formatPct(cpu.summary.topRequestEntry.totalPct)} total)`, + ); + } lines.push(''); lines.push('### Top CPU Hotspots'); this.renderHotspots(lines, cpu.hotspots ?? []); @@ -62,6 +75,7 @@ export class MarkdownReportRenderer implements ReportRenderer { if (memory) { lines.push('## Memory'); lines.push(''); + lines.push(`- Quality: ${memory.quality?.confidence ?? 'unknown'}`); lines.push(`- Total sampled: ${formatBytes(memory.summary?.totalSampledBytes)}`); if (memory.summary?.topAllocator?.userCaller) { lines.push( @@ -214,6 +228,18 @@ function escapeBackticks(value: string): string { return value.replaceAll('`', '\\`'); } +function sameFrameLocation( + left: { function?: string; file: string; line: number; source?: { file: string; line: number } }, + right: + | { function?: string; file: string; line: number; source?: { file: string; line: number } } + | undefined, +): boolean { + if (!right) return false; + return ( + formatFrameLocation(left) === formatFrameLocation(right) && left.function === right.function + ); +} + function userCallerFromEvidenceExtra(extra: unknown): UserCallerAttribution | undefined { if (!extra || typeof extra !== 'object') return undefined; return (extra as { userCaller?: UserCallerAttribution }).userCaller; diff --git a/packages/cli/src/renderers/text-renderer.ts b/packages/cli/src/renderers/text-renderer.ts index 29b6e40..61fda32 100644 --- a/packages/cli/src/renderers/text-renderer.ts +++ b/packages/cli/src/renderers/text-renderer.ts @@ -49,6 +49,19 @@ export class TextReportRenderer implements ReportRenderer { lines.push( ` GC: ${formatMs(cpu.gc?.totalPauseMs)} total pause, ${formatMs(cpu.gc?.longestPauseMs)} longest`, ); + if (cpu.summary?.topCpuCulprit) { + lines.push( + ` Top CPU culprit: ${cpu.summary.topCpuCulprit.function} (${formatFrameLocation(cpu.summary.topCpuCulprit)}): self ${formatPct(cpu.summary.topCpuCulprit.selfPct)}, total ${formatPct(cpu.summary.topCpuCulprit.totalPct)}`, + ); + } + if ( + cpu.summary?.topRequestEntry && + !sameFrameLocation(cpu.summary.topRequestEntry, cpu.summary.topCpuCulprit) + ) { + lines.push( + ` Top request entry: ${cpu.summary.topRequestEntry.function} (${formatFrameLocation(cpu.summary.topRequestEntry)}): total ${formatPct(cpu.summary.topRequestEntry.totalPct)}`, + ); + } lines.push(' Top hotspots:'); this.renderHotspots(lines, cpu.hotspots ?? [], ' '); lines.push(''); @@ -57,6 +70,7 @@ export class TextReportRenderer implements ReportRenderer { const memory = report.profiles?.memory; if (memory) { lines.push('Memory'); + lines.push(` Quality: ${memory.quality?.confidence ?? 'unknown'}`); lines.push(` Total sampled: ${formatBytes(memory.summary?.totalSampledBytes)}`); if (memory.summary?.topAllocator?.userCaller) { lines.push( @@ -215,6 +229,18 @@ export class TextReportRenderer implements ReportRenderer { } } +function sameFrameLocation( + left: { function?: string; file: string; line: number; source?: { file: string; line: number } }, + right: + | { function?: string; file: string; line: number; source?: { file: string; line: number } } + | undefined, +): boolean { + if (!right) return false; + return ( + formatFrameLocation(left) === formatFrameLocation(right) && left.function === right.function + ); +} + function userCallerFromEvidenceExtra(extra: unknown): UserCallerAttribution | undefined { if (!extra || typeof extra !== 'object') return undefined; return (extra as { userCaller?: UserCallerAttribution }).userCaller; diff --git a/packages/cli/test/fixtures/report.agent.md b/packages/cli/test/fixtures/report.agent.md index 2c54c13..85d0ec2 100644 --- a/packages/cli/test/fixtures/report.agent.md +++ b/packages/cli/test/fixtures/report.agent.md @@ -7,6 +7,7 @@ cwd: /repo kinds: [cpu, memory, async] lanterna_version: "1.5.1" cpu_quality: high +memory_quality: high memory_signal: present async_quality: high integrity: ok @@ -50,6 +51,7 @@ degrading_caveats: [] ## Kind Review — memory +- quality: high - memory_usage: 10 samples every 250ms - top_allocator: Buffer.alloc at node:buffer:10 — user_caller loadCache at src/cache.ts:18 (high, heap-sample-path, support 88.0%) diff --git a/packages/cli/test/report-renderer.test.ts b/packages/cli/test/report-renderer.test.ts index c18ba0a..bb0bcfe 100644 --- a/packages/cli/test/report-renderer.test.ts +++ b/packages/cli/test/report-renderer.test.ts @@ -516,6 +516,11 @@ describe('renderReport', () => { }, }, memory: { + quality: { + confidence: 'medium', + reasons: ['heap snapshots unavailable'], + recommendations: ['Use the allocation samples as the primary evidence.'], + }, memoryUsage: { available: true, sampleIntervalMs: 250, @@ -610,6 +615,7 @@ describe('renderReport', () => { expect(output).toContain('command: "node server.js"'); expect(output).toContain('kinds: [cpu, memory]'); expect(output).toContain('cpu_quality: low'); + expect(output).toContain('memory_quality: medium'); expect(output).toContain('integrity: degraded'); expect(output).toContain('rerun_required: true'); expect(output).toContain('sourcemap_coverage: 0.9'); @@ -632,6 +638,7 @@ describe('renderReport', () => { expect(output).toContain('## Kind Review — cpu'); expect(output).toContain('- quality: low'); expect(output).toContain('## Kind Review — memory'); + expect(output).toContain('- quality: medium'); expect(output).toContain('- memory_usage: 8 samples every 250ms'); expect(output).toContain('## Files To Read First'); const filesSection = sectionText(output, 'Files To Read First'); @@ -682,6 +689,299 @@ describe('renderReport', () => { expect(output).not.toContain('## Kind Review — async'); }); + it('does not force rerun for CPU findings when only event-loop and GC timing are unavailable', () => { + const output = renderReport( + { + meta: { + ...baseMeta, + captureIntegrity: { + ...baseMeta.captureIntegrity, + eventLoopTimed: false, + gcTimed: false, + }, + }, + profiles: { + cpu: { + quality: { + confidence: 'high', + sampleCount: 250, + durationMs: 5000, + idleRatio: 0.2, + samplesTimed: true, + durationBasis: 'timeDeltas', + reasons: [], + recommendations: [], + }, + }, + }, + findings: [ + { + id: 'sync-crypto-on-hot-path', + profileKind: 'cpu', + severity: 'critical', + category: 'sync-crypto', + title: 'Synchronous crypto on hot path', + evidence: { + file: '/repo/src/auth.js', + line: 3, + function: 'hashPassword', + selfPct: 81, + }, + priority: { score: 90, actionConfidence: 'high' }, + confidence: 'high', + proofLevel: 'direct-sample', + why: 'Synchronous crypto was sampled in user code.', + suggestion: 'Move the crypto work off the request path.', + references: [], + }, + ], + }, + { format: 'agent' }, + ); + + expect(output).toContain('rerun_required: false'); + expect(output).toContain( + 'degrading_caveats: ["event-loop timing unavailable", "GC timing unavailable"]', + ); + }); + + it('labels inclusive CPU aggregate read targets with total signal when CPU has no findings', () => { + const output = renderReport( + { + meta: baseMeta, + profiles: { + cpu: { + summary: { + topRequestEntry: { + function: 'processBatch', + file: '/repo/src/app.js', + line: 7, + selfPct: 0.01, + totalPct: 62, + }, + topUserHotspot: { + function: 'processBatch', + file: '/repo/src/app.js', + line: 7, + selfPct: 0.01, + totalPct: 62, + }, + }, + quality: { + confidence: 'high', + sampleCount: 250, + durationMs: 5000, + idleRatio: 0.2, + samplesTimed: true, + durationBasis: 'timeDeltas', + reasons: [], + recommendations: [], + }, + }, + }, + findings: [], + }, + { format: 'agent' }, + ); + + const filesSection = sectionText(output, 'Files To Read First'); + expect(filesSection).toMatch( + /\| \/repo\/src\/app\.js:7 +\| top request entry +\| cpu +\| 62\.0% total +\| inspect-lead \|/, + ); + expect(filesSection).not.toContain('0.01% self'); + }); + + it('does not duplicate inclusive CPU aggregate read targets when CPU findings already point at code', () => { + const output = renderReport( + { + meta: baseMeta, + profiles: { + cpu: { + summary: { + topRequestEntry: { + function: 'processBatch', + file: '/repo/src/app.js', + line: 7, + selfPct: 0.01, + totalPct: 62, + }, + topUserHotspot: { + function: 'processBatch', + file: '/repo/src/app.js', + line: 7, + selfPct: 0.01, + totalPct: 62, + }, + }, + quality: { + confidence: 'high', + sampleCount: 250, + durationMs: 5000, + idleRatio: 0.2, + samplesTimed: true, + durationBasis: 'timeDeltas', + reasons: [], + recommendations: [], + }, + }, + }, + findings: [ + { + id: 'sync-crypto-on-hot-path:hashPassword', + profileKind: 'cpu', + severity: 'warning', + category: 'sync-crypto', + title: 'Synchronous crypto on hot path', + evidence: { + file: '/repo/src/app.js', + line: 3, + function: 'hashPassword', + selfPct: 28, + }, + priority: { score: 90, actionConfidence: 'high' }, + confidence: 'high', + proofLevel: 'direct-sample', + why: 'Synchronous crypto was sampled in user code.', + suggestion: 'Move the crypto work off the request path.', + references: [], + }, + ], + }, + { format: 'agent' }, + ); + + const kindReviewSection = sectionText(output, 'Kind Review — cpu'); + expect(kindReviewSection).toContain( + '- top_request_entry: processBatch at /repo/src/app.js:7 (62.0% total)', + ); + + const filesSection = sectionText(output, 'Files To Read First'); + expect(filesSection).toMatch( + /\| \/repo\/src\/app\.js:3 +\| finding location +\| finding \| 28\.0% self +\| read-first \|/, + ); + expect(filesSection).not.toContain('/repo/src/app.js:7'); + }); + + it('renders the matched CPU user stack on finding details', () => { + const output = renderReport( + { + meta: baseMeta, + profiles: { + cpu: { + summary: {}, + hotStacks: [ + { + weightPct: 81, + frames: [ + { + function: 'pbkdf2Sync', + file: 'node:internal/crypto/pbkdf2', + line: 62, + category: 'node:builtin', + }, + { + function: 'hashPassword', + file: '/repo/src/auth.js', + line: 3, + category: 'user', + }, + { + function: 'route', + file: '/repo/src/server.js', + line: 12, + category: 'user', + }, + ], + }, + ], + quality: { + confidence: 'high', + sampleCount: 250, + durationMs: 5000, + idleRatio: 0.2, + samplesTimed: true, + durationBasis: 'timeDeltas', + reasons: [], + recommendations: [], + }, + }, + }, + findings: [ + { + id: 'sync-crypto-on-hot-path', + profileKind: 'cpu', + severity: 'critical', + category: 'sync-crypto', + title: 'Synchronous crypto on hot path', + evidence: { + file: '/repo/src/auth.js', + line: 3, + function: 'hashPassword', + selfPct: 81, + extra: { + proofLevel: 'attributed-caller', + attributionBasis: 'sample-path', + attributionConfidence: 'high', + callee: 'pbkdf2Sync', + calleeTotalPct: 81, + userCaller: { + function: 'hashPassword', + file: '/repo/src/auth.js', + line: 3, + profilePct: 81, + supportPct: 100, + confidence: 'high', + basis: 'cpu-sample-path', + stackDistance: 1, + }, + candidateCallers: [ + { + function: 'hashPassword', + file: '/repo/src/auth.js', + line: 3, + profilePct: 81, + supportPct: 100, + confidence: 'high', + basis: 'cpu-sample-path', + stackDistance: 1, + }, + { + function: 'route', + file: '/repo/src/server.js', + line: 12, + profilePct: 81, + supportPct: 100, + confidence: 'high', + basis: 'cpu-sample-path', + stackDistance: 2, + }, + ], + }, + }, + priority: { score: 90, actionConfidence: 'high' }, + confidence: 'high', + proofLevel: 'direct-sample', + why: 'Synchronous crypto was sampled in user code.', + suggestion: 'Move the crypto work off the request path.', + references: [], + }, + ], + }, + { format: 'agent' }, + ); + + expect(output).toContain( + '- user_stack: route at /repo/src/server.js:12 -> hashPassword at /repo/src/auth.js:3 (81.0% stack, leaf pbkdf2Sync at node:internal/crypto/pbkdf2:62)', + ); + const filesSection = sectionText(output, 'Files To Read First'); + expect(filesSection).toMatch( + /\| \/repo\/src\/server\.js:12 +\| CPU user stack +\| finding \| 81\.0% stack +\| supporting-context \|/, + ); + expect(filesSection).toMatch( + /\| \/repo\/src\/auth\.js:3 +\| finding location +\| finding \| 81\.0% self +\| read-first +\|/, + ); + }); + it('does not require rerun for non-applicable source maps', () => { const output = renderReport( { @@ -1508,6 +1808,21 @@ describe('renderReport', () => { callees: [], optimizationState: 'unknown', }, + { + id: 'native-writev', + function: 'writev', + file: 'writev', + line: 0, + column: 0, + category: 'native', + selfMs: 12, + selfPct: 12, + totalMs: 12, + totalPct: 12, + callers: [], + callees: [], + optimizationState: 'unknown', + }, ], hotStacks: [ { @@ -1552,6 +1867,7 @@ describe('renderReport', () => { expect(filesSection).not.toContain('(program):0'); expect(filesSection).not.toContain('(garbage collector):0'); expect(filesSection).not.toContain('node:internal/streams/writable:300'); + expect(filesSection).not.toContain('writev:0'); expect(kindReviewSection).not.toContain('(idle):0'); expect(kindReviewSection).not.toContain('(program):0'); expect(kindReviewSection).not.toContain('(garbage collector):0'); @@ -1802,6 +2118,44 @@ describe('renderReport', () => { expect(filesSection).not.toContain('/repo/src/file-11.js:21'); }); + it('keeps anonymous frames when they point at editable user source', () => { + const output = renderReport( + { + meta: baseMeta, + profiles: {}, + findings: [ + { + id: 'cpu-hotspot:/repo/src/app.js:1:(anonymous)', + profileKind: 'cpu', + severity: 'critical', + category: 'cpu-hotspot', + title: 'Top-level module is hot', + evidence: { + file: '/repo/src/app.js', + line: 1, + function: '(anonymous)', + selfPct: 98, + }, + priority: { + score: 98, + actionConfidence: 'high', + }, + confidence: 'high', + proofLevel: 'direct-sample', + why: 'Top-level CPU work was sampled here.', + suggestion: 'Inspect the top-level module body.', + references: [], + }, + ], + }, + { format: 'agent' }, + ); + + expect(sectionText(output, 'Files To Read First')).toContain( + '| /repo/src/app.js:1 | finding location | finding | 98.0% self | read-first |', + ); + }); + it('keeps generated output fallbacks as inspection leads instead of read-first targets', () => { const output = renderReport( { @@ -2132,6 +2486,11 @@ function agentFixtureReport() { deopts: [], }, memory: { + quality: { + confidence: 'high', + reasons: [], + recommendations: [], + }, summary: { totalSampledBytes: 4096, samplingIntervalBytes: 524288, diff --git a/packages/core/src/analysis/core/pipeline.ts b/packages/core/src/analysis/core/pipeline.ts index 6fb047d..456082f 100644 --- a/packages/core/src/analysis/core/pipeline.ts +++ b/packages/core/src/analysis/core/pipeline.ts @@ -140,6 +140,7 @@ export class AnalysisPipeline { for (const analyzer of sortAnalyzers(this.findingAnalyzers)) { try { findings.push(...analyzer.run(context, snapshot)); + snapshot.findings = findings; } catch (error) { logger.warn({ analyzerId: analyzer.id, err: error }, 'analysis finding analyzer failed'); recordCaptureDiagnostic(bundle.captureIntegrity, { diff --git a/packages/core/src/analysis/model/summary.ts b/packages/core/src/analysis/model/summary.ts index b20ada2..2275e7c 100644 --- a/packages/core/src/analysis/model/summary.ts +++ b/packages/core/src/analysis/model/summary.ts @@ -112,6 +112,33 @@ export function deriveTopUserHotspot( return summary; } +export function deriveTopCpuCulprit( + hotspots: readonly Hotspot[], + correlatedHotspots: readonly CorrelatedHotspot[] = [], +): SummaryUserHotspot | undefined { + const candidates = hotspots + .filter( + (hotspot) => hotspot.category === 'user' && hotspot.selfPct >= TOP_USER_HOTSPOT_MIN_SELF_PCT, + ) + .map((hotspot) => ({ + hotspot, + correlated: findCorrelatedHotspot(hotspot, correlatedHotspots), + })); + const namedPool = candidates.some((candidate) => !isAnonymousWrapper(candidate.hotspot)) + ? candidates.filter((candidate) => !isAnonymousWrapper(candidate.hotspot)) + : candidates; + const matches = namedPool.sort(compareCpuCulpritCandidates); + const topCandidate = matches[0]; + const top = topCandidate?.hotspot; + if (!top) return undefined; + + return toSummaryUserHotspot( + top, + topCandidate.correlated, + matches.slice(1, 3).map(({ hotspot }) => hotspot), + ); +} + function compareTopHotspotCandidates( left: { hotspot: Hotspot; correlated?: CorrelatedHotspot }, right: { hotspot: Hotspot; correlated?: CorrelatedHotspot }, @@ -123,6 +150,17 @@ function compareTopHotspotCandidates( return right.hotspot.selfPct - left.hotspot.selfPct; } +function compareCpuCulpritCandidates( + left: { hotspot: Hotspot; correlated?: CorrelatedHotspot }, + right: { hotspot: Hotspot; correlated?: CorrelatedHotspot }, +): number { + const selfDelta = right.hotspot.selfPct - left.hotspot.selfPct; + if (selfDelta !== 0) return selfDelta; + const correlationDelta = correlationScore(right.correlated) - correlationScore(left.correlated); + if (correlationDelta !== 0) return correlationDelta; + return right.hotspot.totalPct - left.hotspot.totalPct; +} + function correlationScore(correlated: CorrelatedHotspot | undefined): number { if (!correlated) return 0; const confidenceWeight = @@ -142,6 +180,40 @@ function findCorrelatedHotspot( ); } +function toSummaryUserHotspot( + hotspot: Hotspot, + correlated: CorrelatedHotspot | undefined, + alternativeHotspots: readonly Hotspot[], +): SummaryUserHotspot { + const alternatives = alternativeHotspots.map((alternative) => { + const alt: SummaryUserHotspot['alternativeHotspots'] extends (infer T)[] | undefined + ? T + : never = { + id: alternative.id, + function: alternative.function, + file: alternative.file, + line: alternative.line, + selfPct: alternative.selfPct, + totalPct: alternative.totalPct, + }; + if (alternative.source) alt.source = alternative.source; + return alt; + }); + const summary: SummaryUserHotspot = { + function: hotspot.function, + file: hotspot.file, + line: hotspot.line, + selfPct: hotspot.selfPct, + totalPct: hotspot.totalPct, + eventLoopCorrelation: correlated + ? { overlapPct: correlated.overlapPct, samplePct: correlated.samplePct } + : undefined, + alternativeHotspots: alternatives.length > 0 ? alternatives : undefined, + }; + if (hotspot.source) summary.source = hotspot.source; + return summary; +} + function isAnonymousWrapper(hotspot: Hotspot): boolean { return hotspot.function === '(anonymous)' || hotspot.function.trim() === ''; } diff --git a/packages/core/src/capture/coordinator.ts b/packages/core/src/capture/coordinator.ts index 0dcbfa1..d55612f 100644 --- a/packages/core/src/capture/coordinator.ts +++ b/packages/core/src/capture/coordinator.ts @@ -123,6 +123,7 @@ export async function runCapture( const kindsData = await stopProbes( probeInstances, + connected, cdp, mode, options, @@ -240,6 +241,7 @@ function emitProbeStartProgress( async function stopProbes( probeInstances: readonly ProbeInstance[], + connected: RunCaptureConnectedSession, cdp: RunCaptureConnectedSession['cdp'], mode: ProbeLifecycleContext['mode'], options: RunCaptureOptions, @@ -248,7 +250,15 @@ async function stopProbes( ): Promise> { const kindsData: Record = {}; for (const probeInstance of probeInstances) { - const result = await stopProbe(probeInstance, cdp, mode, options, captureIntegrity, stopReason); + const result = await stopProbe( + probeInstance, + connected, + cdp, + mode, + options, + captureIntegrity, + stopReason, + ); if (!result.ok) continue; kindsData[probeInstance.kind.id] = result.value; } @@ -257,6 +267,7 @@ async function stopProbes( async function stopProbe( { kind, probe }: ProbeInstance, + connected: RunCaptureConnectedSession, cdp: RunCaptureConnectedSession['cdp'], mode: ProbeLifecycleContext['mode'], options: RunCaptureOptions, @@ -270,6 +281,9 @@ async function stopProbe( const ctx = createProbeLifecycleContext(cdp, mode, kind.id, { abortSignal: options.abortSignal, stopReason, + ...(connected.drainLiveSignals + ? { liveSourceSignals: connected.drainLiveSignals.bind(connected) } + : {}), }); const result = stopTimeoutMs === false diff --git a/packages/core/src/capture/core/deopts.ts b/packages/core/src/capture/core/deopts.ts index 6588907..71ba24a 100644 --- a/packages/core/src/capture/core/deopts.ts +++ b/packages/core/src/capture/core/deopts.ts @@ -8,6 +8,8 @@ export function parseDeoptsFromStderr(stderr: string): RawDeopt[] { const lines = stderr.split('\n'); const bailoutPattern = /bailout .*?kind:\s*([^,]+),\s*reason:\s*([^)]+)\).*?<[^>]*>\s+(\S+)\s+at\s+(\S+):(\d+)/i; + const nodeBailoutPattern = + /bailout\s+\(kind:\s*([^,]+),\s*reason:\s*(.*?)\):\s*(?:begin|end)\.\s+deoptimizing\s+.*?]*>\s+(\S+).*?reason:\s*([^,;]+)/i; @@ -35,6 +37,27 @@ export function parseDeoptsFromStderr(stderr: string): RawDeopt[] { continue; } + match = nodeBailoutPattern.exec(line); + if (match) { + const reason = (match[2] ?? '').trim(); + const functionName = (match[3] ?? '').trim(); + const key = `${functionName}|${reason}`; + const existing = deoptCountsByKey.get(key); + if (existing) { + existing.count += 1; + continue; + } + deoptCountsByKey.set(key, { + function: functionName, + file: '', + line: 0, + reason, + bailoutType: (match[1] ?? '').trim(), + count: 1, + }); + continue; + } + match = genericBailoutPattern.exec(line); if (match) { const reason = (match[2] ?? '').trim(); diff --git a/packages/core/src/capture/core/types.ts b/packages/core/src/capture/core/types.ts index 3d7ffe6..fd14082 100644 --- a/packages/core/src/capture/core/types.ts +++ b/packages/core/src/capture/core/types.ts @@ -1,4 +1,5 @@ import type { CaptureKindDataMap } from '../../kinds/core/types.js'; +import type { MemoryUsageSample } from '../../report/types.js'; import type { EventLoopSampleData, ParsedTargetInfo, @@ -147,6 +148,8 @@ export interface LiveSourceSignals { eventLoopSamplesAbs: EventLoopSample[]; eventLoopAvailable: boolean; eventLoopResolutionMs?: number; + memoryUsageSamples?: MemoryUsageSample[]; + memoryUsageSampleIntervalMs?: number; integrityCounters?: { controlChannelWriteErrors: number; gcObserverSetupFailed: number; diff --git a/packages/core/src/capture/spawn/index.ts b/packages/core/src/capture/spawn/index.ts index a6f6480..43b5076 100644 --- a/packages/core/src/capture/spawn/index.ts +++ b/packages/core/src/capture/spawn/index.ts @@ -4,6 +4,7 @@ import { rm, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { basename, join } from 'node:path'; import { connectCdp } from '../../inspector/client.js'; +import type { MemoryUsageSample } from '../../report/types.js'; import { attachControlChannel } from '../../runtime-signals/control-channel.js'; import { createCaptureIntegrity, mergeCaptureIntegrityCounters } from '../core/session.js'; import type { @@ -78,8 +79,10 @@ export class SpawnSource implements ProfileSource { const captureIntegrity = createCaptureIntegrity({ controlChannelExpected: true }); const gcEventsAbs: RawGcEvent[] = []; const eventLoopSamplesAbs: EventLoopSample[] = []; + const memoryUsageSamples: MemoryUsageSample[] = []; let eventLoopAvailable = false; let eventLoopResolutionMs: number | undefined; + let memoryUsageSampleIntervalMs: number | undefined; let appCompleted = false; let resolveAppCompletion = () => {}; const appCompletionPromise = new Promise((resolve) => { @@ -117,6 +120,19 @@ export class SpawnSource implements ProfileSource { }); return; } + if (event.type === 'memory-usage') { + if (!event.captureStarted) return; + memoryUsageSampleIntervalMs = event.sampleIntervalMs; + memoryUsageSamples.push({ + atMs: event.atMs, + rss: event.rss, + heapTotal: event.heapTotal, + heapUsed: event.heapUsed, + external: event.external, + arrayBuffers: event.arrayBuffers, + }); + return; + } if (event.type === 'app-complete') { mergeCaptureIntegrityCounters(captureIntegrity, event.integrity); appCompleted = true; @@ -166,6 +182,8 @@ export class SpawnSource implements ProfileSource { eventLoopSamplesAbs: [...eventLoopSamplesAbs], eventLoopAvailable, eventLoopResolutionMs, + memoryUsageSamples: [...memoryUsageSamples], + memoryUsageSampleIntervalMs, integrityCounters: { controlChannelWriteErrors: captureIntegrity.controlChannelWriteErrors, gcObserverSetupFailed: captureIntegrity.gcObserverSetupFailed, diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index cb34d85..bbfacca 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -181,6 +181,7 @@ export type { BlockingIoEvidenceExtra, BuiltinFinding, BuiltinFindingCategory, + CpuHotspotEvidenceExtra, CpuProfileReport, CpuSummary, DeoptEntry, diff --git a/packages/core/src/kinds/async/analysis.ts b/packages/core/src/kinds/async/analysis.ts index e80efd5..1adb109 100644 --- a/packages/core/src/kinds/async/analysis.ts +++ b/packages/core/src/kinds/async/analysis.ts @@ -1,3 +1,4 @@ +import { fileURLToPath } from 'node:url'; import type { SourceMapResolver } from '../../analysis/sourcemap/resolver.js'; import type { CaptureBundle, RawCpuProfile } from '../../capture/core/types.js'; import type { @@ -176,19 +177,26 @@ function effectiveDuration(rec: AsyncOperationRecord, captureDurationMs: number) } function toReportFrame(frame: AsyncStackFrame): AsyncStackFrameReport { + const source = activeResolver?.resolve(frame.file, frame.line, frame.column); const reportFrame: AsyncStackFrameReport = { function: frame.function, - file: frame.file, + file: normalizeFrameFile(frame.file), line: frame.line, column: frame.column, }; - if (activeResolver && frame.file && frame.line > 0) { - const source = activeResolver.resolve(frame.file, frame.line, frame.column); - if (source) reportFrame.source = source; - } + if (source) reportFrame.source = source; return reportFrame; } +function normalizeFrameFile(file: string): string { + if (!file.startsWith('file://')) return file; + try { + return fileURLToPath(file); + } catch { + return file; + } +} + function userCallerFromAsyncFrame( frame: AsyncStackFrameReport | undefined, options: Pick, @@ -445,7 +453,10 @@ function scoreCdpMatch( } function sameFrameFile(left: AsyncStackFrame, right: AsyncStackFrame): boolean { - return left.file === right.file && Math.abs(left.line - right.line) <= 2; + return ( + normalizeFrameFile(left.file) === normalizeFrameFile(right.file) && + Math.abs(left.line - right.line) <= 2 + ); } function buildChainTree( @@ -546,7 +557,10 @@ function buildChains( totalOps += 1; totalDuration += node.durationMs; const frame = recordById.get(node.asyncId)?.initStack[0]; - if (frame) fileCounts.set(frame.file, (fileCounts.get(frame.file) ?? 0) + 1); + if (frame) { + const file = normalizeFrameFile(frame.file); + fileCounts.set(file, (fileCounts.get(file) ?? 0) + 1); + } if (depthFromRoot > maxDepth) { maxDepth = depthFromRoot; deepestLeaf = node; @@ -645,9 +659,7 @@ function buildQuality( reasons.push( `only ${(sampledStackRatio * 100).toFixed(0)}% of async operations include init stacks`, ); - recommendations.add( - 'Increase async stack depth or capture from process start for better file attribution.', - ); + recommendations.add('Increase async stack depth for better file attribution.'); } if (data.integrity.recordsDropped > 0) { reasons.push( @@ -775,16 +787,14 @@ function buildHotFiles(args: { const cpuPctByFile = new Map(); for (const entry of cpuAttribution.topChains) { if (!entry.rootFrame) continue; - cpuPctByFile.set( - entry.rootFrame.file, - (cpuPctByFile.get(entry.rootFrame.file) ?? 0) + entry.cpuPct, - ); + const file = normalizeFrameFile(entry.rootFrame.file); + cpuPctByFile.set(file, (cpuPctByFile.get(file) ?? 0) + entry.cpuPct); } for (const rec of records) { const frame = rec.initStack[0]; if (!frame) continue; - const file = frame.file; + const file = normalizeFrameFile(frame.file); const durationMs = effectiveDuration(rec, captureDurationMs); const ageMs = rec.orphan ? Math.max(0, captureDurationMs - rec.initAtMs) : 0; const aggregate = byFile.get(file) ?? { @@ -831,7 +841,7 @@ function buildHotFiles(args: { const rootFrame = rootId ? records.find((candidate) => candidate.asyncId === rootId)?.initStack[0] : undefined; - if (rootFrame && rootFrame.file !== file) { + if (rootFrame && normalizeFrameFile(rootFrame.file) !== file) { cpuPctByFile.set(file, cpuPctByFile.get(file) ?? 0); } } @@ -1004,7 +1014,7 @@ function buildCpuAttribution(args: BuildAttributionArgs): AsyncCpuAttribution { ); const frame = cpuFrameForNode(nodeById.get(samples[i] ?? -1)); if (frame) { - const key = `${frame.file}:${frame.line}:${frame.column}:${frame.function}`; + const key = `${normalizeFrameFile(frame.file)}:${frame.line}:${frame.column}:${frame.function}`; const current = bucket.frameCounts.get(key); if (current) current.count += 1; else bucket.frameCounts.set(key, { frame, count: 1 }); diff --git a/packages/core/src/kinds/core/types.ts b/packages/core/src/kinds/core/types.ts index a138026..ac849d6 100644 --- a/packages/core/src/kinds/core/types.ts +++ b/packages/core/src/kinds/core/types.ts @@ -1,5 +1,6 @@ import type { ZodType } from 'zod'; import type { FindingAnalyzer, SectionAnalyzer } from '../../analysis/core/types.js'; +import type { LiveSourceSignals } from '../../capture/core/types.js'; import type { CdpClient } from '../../inspector/client.js'; import type { HookInstaller } from '../../runtime-signals/hooks/framework.js'; @@ -40,6 +41,7 @@ export interface ProbeLifecycleContext { cdp: CdpClient; mode: 'spawn' | 'attach'; kindId: string; + liveSourceSignals?: () => LiveSourceSignals; } export type ProbeStopReason = 'exit' | 'timeout' | 'signal'; diff --git a/packages/core/src/kinds/cpu/analysis.ts b/packages/core/src/kinds/cpu/analysis.ts index 5625e47..263c740 100644 --- a/packages/core/src/kinds/cpu/analysis.ts +++ b/packages/core/src/kinds/cpu/analysis.ts @@ -18,6 +18,7 @@ import { buildCpuProfileQuality } from '../../analysis/model/profile-quality.js' import { buildCpuSummary, deriveDominantBlockingKind, + deriveTopCpuCulprit, deriveTopUserHotspot, } from '../../analysis/model/summary.js'; import type { CaptureBundle } from '../../capture/core/types.js'; @@ -136,6 +137,11 @@ export const cpuFinalize: KindFinalizeHook = ({ snapshot }) => { cpu.eventLoop.correlatedHotspots ?? [], snapshot.findings, ); + cpu.summary.topRequestEntry = cpu.summary.topUserHotspot; + cpu.summary.topCpuCulprit = deriveTopCpuCulprit( + cpu.hotspots, + cpu.eventLoop.correlatedHotspots ?? [], + ); }; export type { KindViews }; diff --git a/packages/core/src/kinds/memory/analysis.ts b/packages/core/src/kinds/memory/analysis.ts index 6bea3f2..5fbce75 100644 --- a/packages/core/src/kinds/memory/analysis.ts +++ b/packages/core/src/kinds/memory/analysis.ts @@ -9,6 +9,7 @@ import type { CaptureBundle } from '../../capture/core/types.js'; import type { FrameCategory, MemoryHotAllocator, + MemoryProfileQuality, MemoryProfileReport, MemorySeriesStats, MemorySummary, @@ -104,10 +105,12 @@ export function createMemoryAnalysisContributor( data.heapSnapshotAnalysis, options.heapSnapshotAnalysis, ); + const quality = buildMemoryQuality(data, totalSampledBytes, heapSnapshotAnalysis); const report: MemoryProfileReport = { summary, hotAllocators: hotAllocators.slice(0, MAX_PUBLIC_HOT_ALLOCATORS), + quality, memoryUsage: { available: data.memoryUsage.available, sampleIntervalMs: data.memoryUsage.sampleIntervalMs || 0, @@ -135,6 +138,57 @@ export function createMemoryAnalysisContributor( }; } +function buildMemoryQuality( + data: MemoryKindData, + totalSampledBytes: number, + heapSnapshotAnalysis: HeapSnapshotAnalysisReport | undefined, +): MemoryProfileQuality { + const reasons: string[] = []; + const recommendations = new Set(); + const memorySampleCount = data.memoryUsage.samples.length; + + if (!data.memoryUsage.available || memorySampleCount === 0) { + reasons.push('process.memoryUsage() samples were unavailable'); + recommendations.add( + 'Keep the target alive until finalization or use spawn mode so live memory samples can be preserved.', + ); + } + + if (data.heapSamplingAvailable === false) { + reasons.push('V8 heap sampling profile was unavailable'); + recommendations.add('Rerun the capture while the target process remains reachable over CDP.'); + } + + for (const warning of data.warnings ?? []) { + reasons.push(warning); + } + + if (heapSnapshotAnalysis && !heapSnapshotAnalysis.available) { + reasons.push('heap snapshot analysis was unavailable'); + if (heapSnapshotAnalysis.warnings.length > 0) { + reasons.push(...heapSnapshotAnalysis.warnings); + } + recommendations.add( + 'Use profiles.memory.hotAllocators and memoryUsage as the primary evidence when heap snapshots are unavailable.', + ); + } + + const hasHeapSampling = data.heapSamplingAvailable !== false && totalSampledBytes > 0; + const hasMemoryUsage = data.memoryUsage.available && memorySampleCount > 0; + const confidence: MemoryProfileQuality['confidence'] = + hasHeapSampling && hasMemoryUsage + ? 'high' + : hasHeapSampling || hasMemoryUsage + ? 'medium' + : 'low'; + + return { + confidence, + reasons, + recommendations: Array.from(recommendations), + }; +} + function resolveHeapSnapshotAnalysisReport( capturedOrReport: MemoryKindData['heapSnapshotAnalysis'] | HeapSnapshotAnalysisReport | undefined, options: NormalizedHeapSnapshotAnalysisOptions | undefined, diff --git a/packages/core/src/kinds/memory/heap-snapshot-analysis.ts b/packages/core/src/kinds/memory/heap-snapshot-analysis.ts index 0e77de7..9e68b00 100644 --- a/packages/core/src/kinds/memory/heap-snapshot-analysis.ts +++ b/packages/core/src/kinds/memory/heap-snapshot-analysis.ts @@ -1,4 +1,4 @@ -import { createWriteStream, mkdirSync, readFileSync, statSync } from 'node:fs'; +import { createWriteStream, mkdirSync, readFileSync, rmSync, statSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { isNoiseRetainerPath, shouldKeepNoiseFrames } from '../../analysis/noise-filters.js'; import type { CdpClient } from '../../inspector/client.js'; @@ -107,6 +107,7 @@ const DEFAULT_MAX_RETAINER_DEPTH = 8; const DEFAULT_MAX_GROUPS = 20; const DEFAULT_MAX_PATHS_PER_GROUP = 3; const DEFAULT_MAX_SNAPSHOT_BYTES = 512 * 1024 * 1024; +export const DEFAULT_HEAP_SNAPSHOT_CAPTURE_TIMEOUT_MS = 30_000; const WEAK_EDGE_TYPE = 'weak'; export function normalizeHeapSnapshotAnalysisOptions( @@ -329,8 +330,8 @@ export function buildHeapSnapshotAnalysisReport( ); } try { - assertSnapshotWithinLimit(captured.start.path, options.maxSnapshotBytes); - assertSnapshotWithinLimit(captured.end.path, options.maxSnapshotBytes); + assertSnapshotUsable(captured.start.path, options.maxSnapshotBytes); + assertSnapshotUsable(captured.end.path, options.maxSnapshotBytes); const start = parseHeapSnapshot(JSON.parse(readFileSync(captured.start.path, 'utf8'))); const end = parseHeapSnapshot(JSON.parse(readFileSync(captured.end.path, 'utf8'))); const analysis = analyzeHeapSnapshotGrowth(start, end, options); @@ -352,8 +353,12 @@ export function buildHeapSnapshotAnalysisReport( } } -function assertSnapshotWithinLimit(path: string, maxSnapshotBytes: number): void { +function assertSnapshotUsable(path: string, maxSnapshotBytes: number): void { const size = statSync(path).size; + if (size === 0) { + rmSync(path, { force: true }); + throw new Error(`heap snapshot ${path} is 0 bytes; ignoring it`); + } if (size > maxSnapshotBytes) { throw new Error( `heap snapshot ${path} is ${size} bytes, above the ${maxSnapshotBytes} byte analysis limit`, @@ -405,13 +410,21 @@ export function resolveHeapSnapshotPath(outputDir: string, label: 'start' | 'end export async function takeHeapSnapshotToFile( cdp: CdpClient, path: string, - options: { abortSignal?: AbortSignal } = {}, + options: { abortSignal?: AbortSignal; timeoutMs?: number } = {}, ): Promise { mkdirSync(dirname(path), { recursive: true }); await cdp.send('HeapProfiler.enable'); await new Promise((resolve, reject) => { const stream = createWriteStream(path, { encoding: 'utf8' }); let settled = false; + const timeoutMs = options.timeoutMs ?? DEFAULT_HEAP_SNAPSHOT_CAPTURE_TIMEOUT_MS; + const timeout = + timeoutMs > 0 + ? setTimeout(() => { + finish(new Error(`heap snapshot capture timed out after ${timeoutMs}ms`)); + }, timeoutMs) + : undefined; + if (typeof timeout?.unref === 'function') timeout.unref(); const offChunk = cdp.on('HeapProfiler.addHeapSnapshotChunk', (params) => { const chunk = (params as { chunk?: unknown }).chunk; if (typeof chunk === 'string') stream.write(chunk); @@ -427,9 +440,11 @@ export async function takeHeapSnapshotToFile( settled = true; offChunk(); offClose(); + if (timeout) clearTimeout(timeout); options.abortSignal?.removeEventListener('abort', onAbort); if (error) { stream.destroy(); + rmSync(path, { force: true }); reject(error); } else { stream.end(() => resolve()); diff --git a/packages/core/src/kinds/memory/index.ts b/packages/core/src/kinds/memory/index.ts index cba51a0..83fda5b 100644 --- a/packages/core/src/kinds/memory/index.ts +++ b/packages/core/src/kinds/memory/index.ts @@ -69,6 +69,7 @@ export function createMemoryProfileKind( samplingIntervalBytes: data.samplingIntervalBytes, memoryUsageIntervalMs, memoryUsageSampleCount: data.memoryUsage.samples.length, + heapSamplingAvailable: data.heapSamplingAvailable ?? true, heapSnapshotAnalysisEnabled: heapSnapshotAnalysis.enabled, ...(data.heapSnapshotAnalysis ? { heapSnapshotAnalysisAvailable: data.heapSnapshotAnalysis.available } @@ -77,6 +78,7 @@ export function createMemoryProfileKind( contributeIntegrity: (data) => ({ memoryUsageAvailable: data.memoryUsage.available, memoryUsageSampleCount: data.memoryUsage.samples.length, + heapSamplingAvailable: data.heapSamplingAvailable ?? true, ...(data.heapSnapshotAnalysis ? { heapSnapshotAnalysisAvailable: data.heapSnapshotAnalysis.available } : {}), diff --git a/packages/core/src/kinds/memory/probe.ts b/packages/core/src/kinds/memory/probe.ts index bd9b165..233ab65 100644 --- a/packages/core/src/kinds/memory/probe.ts +++ b/packages/core/src/kinds/memory/probe.ts @@ -11,6 +11,7 @@ import { import type { CaptureProbe, ProbeLifecycleContext, ProbeStopReason } from '../core/types.js'; import { type CapturedHeapSnapshots, + DEFAULT_HEAP_SNAPSHOT_CAPTURE_TIMEOUT_MS, type HeapSnapshotAnalysisReport, type NormalizedHeapSnapshotAnalysisOptions, resolveHeapSnapshotPath, @@ -26,6 +27,8 @@ export interface MemoryKindData { sampleIntervalMs: number; }; heapSnapshotAnalysis?: CapturedHeapSnapshots | HeapSnapshotAnalysisReport; + heapSamplingAvailable?: boolean; + warnings?: string[]; } export interface MemoryProbeOptions { @@ -42,6 +45,7 @@ export interface MemoryProbeOptions { */ export function createMemoryProbe(options: MemoryProbeOptions): CaptureProbe { let capturedHeapSnapshots: CapturedHeapSnapshots | undefined; + const warnings: string[] = []; return { ...(options.heapSnapshotAnalysis?.enabled ? { stopTimeoutMs: false as const } : {}), ...(options.heapSnapshotAnalysis?.enabled @@ -67,6 +71,7 @@ export function createMemoryProbe(options: MemoryProbeOptions): CaptureProbe { - const samplingProfile = await stopHeapSampling(ctx.cdp); + let samplingProfile: RawSamplingHeapProfile; + let heapSamplingAvailable = true; + try { + if (ctx.cdp.closed) throw new Error('CDP connection closed before heap sampling stopped'); + samplingProfile = await stopHeapSampling(ctx.cdp); + } catch (error) { + heapSamplingAvailable = false; + warnings.push( + `failed to stop heap sampling: ${error instanceof Error ? error.message : String(error)}`, + ); + samplingProfile = emptySamplingProfile(); + } if (options.heapSnapshotAnalysis?.enabled && capturedHeapSnapshots) { if (ctx.stopReason === 'signal') { capturedHeapSnapshots.available = false; @@ -93,6 +109,7 @@ export function createMemoryProbe(options: MemoryProbeOptions): CaptureProbe 0 ? { warnings: [...warnings] } : {}), ...(capturedHeapSnapshots ? { heapSnapshotAnalysis: capturedHeapSnapshots } : {}), }; }, @@ -124,3 +141,42 @@ export function createMemoryProbe(options: MemoryProbeOptions): CaptureProbe import('../../capture/core/types.js').LiveSourceSignals; + }, + fallbackIntervalMs: number, +) { + const live = ctx.liveSourceSignals?.(); + if (!ctx.cdp.closed) { + const memoryUsage = await readMemoryUsageSeries(ctx.cdp); + if (memoryUsage.available || memoryUsage.samples.length > 0) return memoryUsage; + } + if (live?.memoryUsageSamples && live.memoryUsageSamples.length > 0) { + return { + samples: live.memoryUsageSamples, + available: true, + sampleIntervalMs: live.memoryUsageSampleIntervalMs ?? fallbackIntervalMs, + }; + } + return { samples: [], available: false, sampleIntervalMs: fallbackIntervalMs }; +} + +function emptySamplingProfile(): RawSamplingHeapProfile { + return { + head: { + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: 0, + columnNumber: 0, + }, + selfSize: 0, + id: 1, + children: [], + }, + samples: [], + }; +} diff --git a/packages/core/src/report/schema/cpu-profile.ts b/packages/core/src/report/schema/cpu-profile.ts index 5a15027..e49b9ee 100644 --- a/packages/core/src/report/schema/cpu-profile.ts +++ b/packages/core/src/report/schema/cpu-profile.ts @@ -27,6 +27,30 @@ const cpuSummarySchema = z.object({ idleRatio: z.number(), topCategory: frameCategorySchema, dominantBlockingKind: z.union([z.literal('sync-crypto'), z.literal('blocking-io'), z.null()]), + topCpuCulprit: z + .object({ + function: z.string(), + file: z.string(), + line: z.number().int(), + selfPct: z.number(), + totalPct: z.number(), + eventLoopCorrelation: stallCorrelationSchema.optional(), + alternativeHotspots: z.array(alternativeHotspotEvidenceSchema).optional(), + source: sourceLocationSchema.optional(), + }) + .optional(), + topRequestEntry: z + .object({ + function: z.string(), + file: z.string(), + line: z.number().int(), + selfPct: z.number(), + totalPct: z.number(), + eventLoopCorrelation: stallCorrelationSchema.optional(), + alternativeHotspots: z.array(alternativeHotspotEvidenceSchema).optional(), + source: sourceLocationSchema.optional(), + }) + .optional(), topUserHotspot: z .object({ function: z.string(), diff --git a/packages/core/src/report/schema/findings.ts b/packages/core/src/report/schema/findings.ts index 5a8f30d..055b21c 100644 --- a/packages/core/src/report/schema/findings.ts +++ b/packages/core/src/report/schema/findings.ts @@ -8,6 +8,7 @@ import { findingConfidenceSchema, findingReportProofLevelSchema, findingSeveritySchema, + frameCategorySchema, gcCountSchema, measurementBasisSchema, measurementConfidenceSchema, @@ -53,7 +54,7 @@ export const excessiveGcExtraSchema = z.object({ }); export const eventLoopStallExtraSchema = z.object({ - proofLevel: z.literal('aggregate-correlation'), + proofLevel: z.union([z.literal('aggregate-correlation'), z.literal('hotspot-fallback')]), p99LagMs: z.number(), maxLagMs: z.number(), sampleCount: z.number().int().nonnegative(), @@ -62,6 +63,7 @@ export const eventLoopStallExtraSchema = z.object({ histogram: eventLoopHistogramSchema.optional(), stallIntervals: z.array(stallIntervalSchema), candidateHotspots: z.array(correlatedHotspotSchema), + fallbackHotspots: z.array(alternativeHotspotEvidenceSchema).optional(), correlationCoverage: correlationCoverageSchema.optional(), }); @@ -82,6 +84,16 @@ export const nodeModulesHotspotExtraSchema = attributionEvidenceSchema.extend({ alternativeHotspots: z.array(alternativeHotspotEvidenceSchema).optional(), }); +export const cpuHotspotExtraSchema = z.object({ + proofLevel: z.union([z.literal('direct-user-hotspot'), z.literal('inclusive-user-entry')]), + mode: z.enum(['self', 'inclusive-entry']), + category: frameCategorySchema, + selfPct: z.number(), + totalPct: z.number(), + eventLoopCorrelation: stallCorrelationSchema.optional(), + alternativeHotspots: z.array(alternativeHotspotEvidenceSchema), +}); + const builtinFindingExtraSchema = z.union([ blockingIoExtraSchema, syncCryptoExtraSchema, @@ -91,6 +103,7 @@ const builtinFindingExtraSchema = z.union([ eventLoopStallExtraSchema, jsonHotPathExtraSchema, nodeModulesHotspotExtraSchema, + cpuHotspotExtraSchema, ]); const genericFindingExtraSchema = z.record(z.string(), z.unknown()); @@ -161,6 +174,7 @@ export const findingSchema = z 'event-loop-stall': eventLoopStallExtraSchema, 'json-on-hot-path': jsonHotPathExtraSchema, 'node-modules-hotspot': nodeModulesHotspotExtraSchema, + 'cpu-hotspot': cpuHotspotExtraSchema, } as const; const extraSchema = schemaByCategory[category as keyof typeof schemaByCategory]; diff --git a/packages/core/src/report/schema/memory-profile.ts b/packages/core/src/report/schema/memory-profile.ts index 95f6afa..eec5b38 100644 --- a/packages/core/src/report/schema/memory-profile.ts +++ b/packages/core/src/report/schema/memory-profile.ts @@ -1,6 +1,7 @@ import { z } from 'zod'; import { frameCategorySchema, + profileConfidenceSchema, sourceLocationSchema, userCallerAttributionSchema, } from './primitives.js'; @@ -91,9 +92,16 @@ const heapSnapshotAnalysisSchema = z.object({ warnings: z.array(z.string()), }); +const memoryProfileQualitySchema = z.object({ + confidence: profileConfidenceSchema, + reasons: z.array(z.string()), + recommendations: z.array(z.string()), +}); + export const memoryProfileReportSchema = z.object({ summary: memorySummarySchema, hotAllocators: z.array(memoryHotAllocatorSchema), + quality: memoryProfileQualitySchema, memoryUsage: z.object({ available: z.boolean(), sampleIntervalMs: z.number().positive(), diff --git a/packages/core/src/report/types.ts b/packages/core/src/report/types.ts index f6aed81..1e0d467 100644 --- a/packages/core/src/report/types.ts +++ b/packages/core/src/report/types.ts @@ -33,7 +33,8 @@ export type BuiltinFindingCategory = | 'excessive-gc' | 'event-loop-stall' | 'json-on-hot-path' - | 'node-modules-hotspot'; + | 'node-modules-hotspot' + | 'cpu-hotspot'; export type FindingCategory = BuiltinFindingCategory | (string & {}); @@ -125,6 +126,8 @@ export interface CpuSummary { idleRatio: number; topCategory: FrameCategory; dominantBlockingKind: 'sync-crypto' | 'blocking-io' | null; + topCpuCulprit?: SummaryUserHotspot; + topRequestEntry?: SummaryUserHotspot; topUserHotspot?: SummaryUserHotspot; } @@ -328,7 +331,7 @@ export interface ExcessiveGcEvidenceExtra { } export interface EventLoopStallEvidenceExtra { - proofLevel: 'aggregate-correlation'; + proofLevel: 'aggregate-correlation' | 'hotspot-fallback'; p99LagMs: number; maxLagMs: number; sampleCount: number; @@ -337,6 +340,7 @@ export interface EventLoopStallEvidenceExtra { histogram?: EventLoopReport['histogram']; stallIntervals: EventLoopReport['stallIntervals']; candidateHotspots: CorrelatedHotspot[]; + fallbackHotspots?: AlternativeHotspotEvidence[]; correlationCoverage?: CorrelationCoverage; } @@ -357,6 +361,16 @@ export interface NodeModulesHotspotEvidenceExtra extends AttributionEvidence { alternativeHotspots?: AlternativeHotspotEvidence[]; } +export interface CpuHotspotEvidenceExtra { + proofLevel: 'direct-user-hotspot' | 'inclusive-user-entry'; + mode: 'self' | 'inclusive-entry'; + category: FrameCategory; + selfPct: number; + totalPct: number; + eventLoopCorrelation?: StallCorrelation; + alternativeHotspots: AlternativeHotspotEvidence[]; +} + export interface BuiltinFindingEvidenceExtraMap { 'blocking-io': BlockingIoEvidenceExtra; 'sync-crypto': SyncCryptoEvidenceExtra; @@ -366,6 +380,7 @@ export interface BuiltinFindingEvidenceExtraMap { 'event-loop-stall': EventLoopStallEvidenceExtra; 'json-on-hot-path': JsonHotPathEvidenceExtra; 'node-modules-hotspot': NodeModulesHotspotEvidenceExtra; + 'cpu-hotspot': CpuHotspotEvidenceExtra; } export type BuiltinFindingEvidenceExtra = Exclude< @@ -552,6 +567,12 @@ export interface MemorySummary { externalRatio?: number; } +export interface MemoryProfileQuality { + confidence: ProfileConfidence; + reasons: string[]; + recommendations: string[]; +} + /** * Memory profile report section — lives under `report.profiles.memory`. Built * from V8 sampling heap profiler output plus a `process.memoryUsage()` time @@ -560,6 +581,7 @@ export interface MemorySummary { export interface MemoryProfileReport { summary: MemorySummary; hotAllocators: MemoryHotAllocator[]; + quality: MemoryProfileQuality; memoryUsage: { available: boolean; sampleIntervalMs: number; diff --git a/packages/core/src/runtime-signals/hooks/installers/memory-usage.ts b/packages/core/src/runtime-signals/hooks/installers/memory-usage.ts index 4f7b721..0e4796f 100644 --- a/packages/core/src/runtime-signals/hooks/installers/memory-usage.ts +++ b/packages/core/src/runtime-signals/hooks/installers/memory-usage.ts @@ -22,6 +22,7 @@ export function createMemoryUsageInstaller( interface MemoryUsageInstallerApi { performance: typeof globalThis.performance; + controlChannel: { emit(event: object): boolean }; registerGlobal(name: string, value: unknown): void; addResetHook(fn: () => void): void; addDisposeHook?(fn: () => void): void; @@ -39,18 +40,26 @@ function installMemoryUsage(api: MemoryUsageInstallerApi, sampleIntervalMs: numb }> = []; let intervalMs = sampleIntervalMs; let captureStartMs = api.performance.now(); + let captureStarted = false; const sample = () => { try { const now = api.performance.now(); const usage = process.memoryUsage(); - samples.push({ + const entry = { atMs: Math.max(0, now - captureStartMs), rss: usage.rss, heapTotal: usage.heapTotal, heapUsed: usage.heapUsed, external: usage.external, arrayBuffers: usage.arrayBuffers ?? 0, + }; + samples.push(entry); + api.controlChannel.emit({ + type: 'memory-usage', + ...entry, + sampleIntervalMs: intervalMs, + captureStarted, }); } catch { // process.memoryUsage() can throw in unusual hosts (e.g. workers without @@ -66,6 +75,7 @@ function installMemoryUsage(api: MemoryUsageInstallerApi, sampleIntervalMs: numb api.addResetHook(() => { captureStartMs = api.performance.now(); + captureStarted = true; samples.length = 0; sample(); }); diff --git a/packages/core/src/runtime-signals/schemas.ts b/packages/core/src/runtime-signals/schemas.ts index 206d2d9..0287d37 100644 --- a/packages/core/src/runtime-signals/schemas.ts +++ b/packages/core/src/runtime-signals/schemas.ts @@ -11,6 +11,15 @@ export const eventLoopSampleSchema = z.object({ lagMs: z.number(), }); +export const memoryUsageSampleSchema = z.object({ + atMs: z.number(), + rss: z.number().nonnegative(), + heapTotal: z.number().nonnegative(), + heapUsed: z.number().nonnegative(), + external: z.number().nonnegative(), + arrayBuffers: z.number().nonnegative(), +}); + const eventLoopSummarySchema = z.object({ max: z.number(), mean: z.number(), @@ -72,6 +81,12 @@ export const controlGcSchema = z.object({ durationMs: z.number(), }); +export const controlMemoryUsageSchema = memoryUsageSampleSchema.extend({ + type: z.literal('memory-usage'), + sampleIntervalMs: z.number().positive(), + captureStarted: z.boolean().optional(), +}); + export const controlAppCompleteSchema = z.object({ type: z.literal('app-complete'), atMs: z.number().optional(), @@ -83,6 +98,7 @@ export const controlEventSchema = z.discriminatedUnion('type', [ controlCaptureStartSchema, controlHeartbeatSchema, controlGcSchema, + controlMemoryUsageSchema, controlAppCompleteSchema, ]); diff --git a/packages/core/test/async-kind.test.ts b/packages/core/test/async-kind.test.ts index f16a77a..ade6fbc 100644 --- a/packages/core/test/async-kind.test.ts +++ b/packages/core/test/async-kind.test.ts @@ -254,7 +254,7 @@ describe('async kind round-trip', () => { expect(chain?.executionFrame?.function).toBe('executeWork'); expect(chain?.userCaller).toMatchObject({ function: 'executeWork', - file: 'file:///app/src/worker.js', + file: '/app/src/worker.js', line: 32, profilePct: 100, supportPct: 100, @@ -346,12 +346,12 @@ describe('async kind round-trip', () => { cpuAttributionCoveragePct: section?.cpuAttribution.attributedCpuPct, }); expect(section?.hotFiles[0]).toMatchObject({ - file: 'file:///app/src/users.js', + file: '/app/src/users.js', operationCount: 2, totalDurationMs: 180, primaryFrame: { function: 'loadUser', - file: 'file:///app/src/users.js', + file: '/app/src/users.js', line: 12, }, kindBreakdown: { promise: 2 }, @@ -359,7 +359,7 @@ describe('async kind round-trip', () => { }); expect(section?.summary.topAsyncHotFile).toEqual({ function: 'loadUser', - file: 'file:///app/src/users.js', + file: '/app/src/users.js', line: 12, score: section?.hotFiles[0]?.score, confidence: section?.hotFiles[0]?.confidence, @@ -404,6 +404,25 @@ describe('async kind round-trip', () => { ); }); + it('does not recommend process-start capture for spawn captures with partial stacks', () => { + const records = [ + record(1, 0, 100, 0), + withFrame(record(2, 0, 50, 5), { + function: 'partlySampled', + file: 'file:///app/src/partial.js', + line: 2, + }), + ]; + const pipeline = createAnalysisPipeline({ kinds: [createAsyncProfileKind()] }); + const result = pipeline.run(makeBundle(records), { + command: ['node', 'app.js'], + mode: 'spawn', + }); + + expect(result.profiles.async?.quality.recommendations.join('\n')).not.toMatch(/process start/); + expect(result.profiles.async?.quality.recommendations.join('\n')).toMatch(/async stack depth/); + }); + it('adds root, deepest, and dominant file frames to async chains', () => { const root = withFrame(record(1, 0, 10, 0), { function: 'routeHandler', @@ -428,7 +447,7 @@ describe('async kind round-trip', () => { expect(chain?.rootFrame?.function).toBe('routeHandler'); expect(chain?.deepestFrame?.function).toBe('repoCall'); - expect(chain?.dominantFile).toBe('file:///app/src/service.js'); + expect(chain?.dominantFile).toBe('/app/src/service.js'); }); it('passes asyncStackDepth through to the async hook installer', () => { diff --git a/packages/core/test/deopts.test.ts b/packages/core/test/deopts.test.ts index c2ef6a0..4b26de8 100644 --- a/packages/core/test/deopts.test.ts +++ b/packages/core/test/deopts.test.ts @@ -2,6 +2,33 @@ import { describe, expect, it } from 'vitest'; import { parseDeoptsFromStderr } from '../src/capture/core/deopts.js'; describe('parseDeoptsFromStderr', () => { + it('parses Node 24 bailout lines with JSFunction names and no source location', () => { + const trace = [ + '[bailout (kind: deopt-eager, reason: prepare for on stack replacement (OSR)): begin. deoptimizing 0x1a9113053409 , 0x03b4315437a9 , opt id 3, bytecode offset 247, deopt exit 27, FP to SP delta 120, caller SP 0x7fffeb7988f0, pc 0x7c0b96807624]', + '[bailout (kind: deopt-eager, reason: not a String): begin. deoptimizing 0x1a9113053409 , 0x236a86703151 , opt id 4, bytecode offset 86, deopt exit 14, FP to SP delta 160, caller SP 0x7fffeb7988f0, pc 0x7c0b96808331]', + '[bailout (kind: deopt-eager, reason: prepare for on stack replacement (OSR)): begin. deoptimizing 0x1a9113053409 , 0x236a867042c9 , opt id 5, bytecode offset 247, deopt exit 23, FP to SP delta 120, caller SP 0x7fffeb7988f0, pc 0x7c0b968099a0]', + ].join('\n'); + + expect(parseDeoptsFromStderr(trace)).toEqual([ + { + function: 'deoptLoop', + file: '', + line: 0, + reason: 'prepare for on stack replacement (OSR)', + bailoutType: 'deopt-eager', + count: 2, + }, + { + function: 'deoptLoop', + file: '', + line: 0, + reason: 'not a String', + bailoutType: 'deopt-eager', + count: 1, + }, + ]); + }); + it('parses dependent-code deoptimization lines emitted by V8', () => { const trace = '[marking dependent code 0x391aa99446d1 (0x0ddd9864dfe1 ) (opt id 0) for deoptimization, reason: dependent field representation changed]'; diff --git a/packages/core/test/heap-snapshot-analysis.test.ts b/packages/core/test/heap-snapshot-analysis.test.ts index 5c2c33a..091c8ca 100644 --- a/packages/core/test/heap-snapshot-analysis.test.ts +++ b/packages/core/test/heap-snapshot-analysis.test.ts @@ -224,6 +224,32 @@ describe('heap snapshot parsing and analysis', () => { } }); + it('treats zero-byte snapshots as unavailable instead of parsing them', async () => { + const dir = await mkdtemp(join(tmpdir(), 'lanterna-empty-heap-test-')); + try { + const startPath = join(dir, 'start.heapsnapshot'); + const endPath = join(dir, 'end.heapsnapshot'); + await writeFile(startPath, ''); + await writeFile(endPath, '{}'); + + const report = buildHeapSnapshotAnalysisReport( + { + available: true, + mode: 'start-end', + start: { path: startPath }, + end: { path: endPath }, + warnings: [], + }, + normalizeHeapSnapshotAnalysisOptions({ enabled: true }), + ); + + expect(report.available).toBe(false); + expect(report.warnings.join('\n')).toMatch(/0 bytes/); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + it('classifies listener, timer, cache and closure retainer paths', () => { expect( classifyRetainerPath([ diff --git a/packages/core/test/memory-kind.test.ts b/packages/core/test/memory-kind.test.ts index 6ae9e72..7c773bd 100644 --- a/packages/core/test/memory-kind.test.ts +++ b/packages/core/test/memory-kind.test.ts @@ -1,5 +1,8 @@ +import { mkdtemp, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; import vm from 'node:vm'; -import { describe, expect, it } from 'vitest'; +import { describe, expect, it, vi } from 'vitest'; import { createAnalysisPipeline } from '../src/analysis/core/pipeline.js'; import type { RawSamplingHeapProfile } from '../src/capture/core/heap.js'; import type { CaptureBundle } from '../src/capture/core/types.js'; @@ -222,6 +225,25 @@ describe('memory kind analysis', () => { expect(memoryProfileReportSchema.safeParse(report).success).toBe(true); }); + it('adds memory quality using the shared confidence/reasons/recommendations contract', () => { + const data: MemoryKindData = { + samplingProfile: profile(), + samplingIntervalBytes: 512 * 1024, + memoryUsage: { samples: series(0), available: true, sampleIntervalMs: 200 }, + }; + const pipeline = createAnalysisPipeline({ kinds: [createMemoryProfileKind()] }); + const result = pipeline.run(bundle(data), { command: ['node', 'app.js'], mode: 'spawn' }); + + const report = result.profiles.memory as MemoryProfileReport; + + expect(report.quality).toEqual({ + confidence: 'high', + reasons: [], + recommendations: [], + }); + expect(memoryProfileReportSchema.safeParse(report).success).toBe(true); + }); + it('attributes external allocators to the nearest user caller', () => { const data: MemoryKindData = { samplingProfile: externalAllocatorProfile(), @@ -338,6 +360,8 @@ describe('memory kind analysis', () => { expect(report.hotAllocators).toEqual([]); expect(report.summary.totalSampledBytes).toBe(0); expect(report.summary.rss).toBeUndefined(); + expect(report.quality.confidence).toBe('low'); + expect(report.quality.reasons).toContain('process.memoryUsage() samples were unavailable'); const lanternaReport = buildLanternaReport(bundle(data), result, [memoryKind], { command: ['node', 'app.js'], @@ -498,6 +522,99 @@ describe('memory kind analysis', () => { expect(memory.read().samples[0]?.atMs).toBe(0); }); + it('preserves live memory usage samples when the target exits before CDP final reads', async () => { + const liveSamples = series(0, 3); + const cdp = { + closed: true, + send: async () => { + throw new Error('CDP connection closed'); + }, + evaluate: async () => null, + on: () => () => {}, + onClose: () => () => {}, + close: async () => {}, + }; + const probe = createMemoryProbe({ + samplingIntervalBytes: 512 * 1024, + memoryUsageIntervalMs: 250, + }); + + const data = await probe.stop({ + cdp, + mode: 'spawn', + kindId: 'memory', + stopReason: 'exit', + liveSourceSignals: () => ({ + gcEventsAbs: [], + eventLoopSamplesAbs: [], + eventLoopAvailable: false, + memoryUsageSamples: liveSamples, + memoryUsageSampleIntervalMs: 250, + }), + } as never); + + expect(data.memoryUsage).toEqual({ + samples: liveSamples, + available: true, + sampleIntervalMs: 250, + }); + }); + + it('bounds heap snapshot capture and still returns standard memory data', async () => { + vi.useFakeTimers(); + const outputDir = await mkdtemp(join(tmpdir(), 'lanterna-heap-timeout-')); + let rejectSnapshot!: (error: Error) => void; + const hangingSnapshot = new Promise((_resolve, reject) => { + rejectSnapshot = reject; + }); + const sent: string[] = []; + const cdp = { + closed: false, + send: async (method: string) => { + sent.push(method); + if (method === 'HeapProfiler.takeHeapSnapshot') return hangingSnapshot; + if (method === 'HeapProfiler.stopSampling') return { profile: profile() }; + return {}; + }, + evaluate: async () => ({ samples: series(0, 2), sampleIntervalMs: 250 }), + on: () => () => {}, + onClose: () => () => {}, + close: async () => {}, + }; + const probe = createMemoryProfileKind({ + heapSnapshotAnalysis: { enabled: true, outputDir }, + }).createProbe(); + let startState: 'pending' | 'resolved' | 'rejected' = 'pending'; + const startPromise = probe.start({ cdp, mode: 'spawn', kindId: 'memory' }).then( + () => { + startState = 'resolved'; + }, + () => { + startState = 'rejected'; + }, + ); + + try { + await vi.advanceTimersByTimeAsync(31_000); + await Promise.resolve(); + expect(startState).toBe('resolved'); + expect(sent).toContain('HeapProfiler.startSampling'); + + const stopPromise = probe.stop({ cdp, mode: 'spawn', kindId: 'memory' }); + await vi.advanceTimersByTimeAsync(31_000); + const data = await stopPromise; + const snapshot = data.heapSnapshotAnalysis; + expect(snapshot?.available).toBe(false); + expect(snapshot?.warnings.join('\n')).toMatch(/timed out/i); + expect(data.memoryUsage.available).toBe(true); + } finally { + rejectSnapshot(new Error('cleanup')); + await startPromise.catch(() => {}); + vi.useRealTimers(); + await rm(outputDir, { recursive: true, force: true }); + } + }); + it('installs memory hooks on a later attach when the framework already exists', () => { const context = { process, diff --git a/packages/core/test/summary.test.ts b/packages/core/test/summary.test.ts index c045b7a..06f1e03 100644 --- a/packages/core/test/summary.test.ts +++ b/packages/core/test/summary.test.ts @@ -3,6 +3,7 @@ import type { EnrichedTree, NodeEnriched } from '../src/analysis/model/hotspots. import { buildCpuSummary, deriveDominantBlockingKind, + deriveTopCpuCulprit, deriveTopUserHotspot, } from '../src/analysis/model/summary.js'; import type { @@ -375,3 +376,47 @@ describe('deriveTopUserHotspot', () => { }); }); }); + +describe('deriveTopCpuCulprit', () => { + it('returns the self-heavy user frame', () => { + const wrapper = makeHotspot({ + category: 'user', + function: 'processBatch', + file: 'src/batch.ts', + line: 7, + selfPct: 0.04, + totalPct: 99, + }); + const compute = makeHotspot({ + category: 'user', + function: 'scoreRecommendations', + file: 'src/search.ts', + line: 13, + selfPct: 70, + totalPct: 75, + }); + + expect(deriveTopCpuCulprit([wrapper, compute])?.function).toBe('scoreRecommendations'); + }); + + it('does not report an inclusive-only wrapper as the CPU culprit', () => { + const wrapper = makeHotspot({ + category: 'user', + function: 'processBatch', + file: 'src/batch.ts', + line: 7, + selfPct: 0.04, + totalPct: 99, + }); + const caller = makeHotspot({ + category: 'user', + function: 'hashPassword', + file: 'src/auth.ts', + line: 3, + selfPct: 0.01, + totalPct: 98, + }); + + expect(deriveTopCpuCulprit([wrapper, caller])).toBeUndefined(); + }); +}); diff --git a/packages/detectors/README.md b/packages/detectors/README.md index 0b9a44d..2d8967f 100644 --- a/packages/detectors/README.md +++ b/packages/detectors/README.md @@ -2,7 +2,7 @@ Default detector pack for [Lanterna](https://github.com/arkerone/lanterna) — agent-first Node.js CPU, memory & experimental async profiler. -This package contains the built-in CPU, memory, and async detectors, thresholds, attribution helpers, and kind factories that pre-wire those detectors. Capture orchestration and the `KindScopedDetector` seam itself live in [`@lanterna-profiler/core`](https://www.npmjs.com/package/@lanterna-profiler/core). +This package contains the built-in CPU, memory, and async detectors, thresholds, attribution helpers, and kind factories that pre-wire those detectors. The CPU pack includes both pattern-specific rules (`sync-crypto`, `blocking-io`, `json-on-hot-path`, dependency hotspots, require-on-hot-path) and the generic `cpu-hotspot:*` fallback for plain user-code CPU that needs a concrete file/line or caller lead. Capture orchestration and `KindScopedDetector` itself live in [`@lanterna-profiler/core`](https://www.npmjs.com/package/@lanterna-profiler/core). ## Install diff --git a/packages/detectors/src/config.ts b/packages/detectors/src/config.ts index 1d2f5f0..7e68fea 100644 --- a/packages/detectors/src/config.ts +++ b/packages/detectors/src/config.ts @@ -98,6 +98,7 @@ export interface DetectorThresholds { readonly minTotalPct: number; readonly warningSelfPct: number; }; + readonly cpuHotspot: CpuHotspotThresholds; readonly deoptLoop: { readonly minCount: number; readonly criticalCount: number; @@ -126,6 +127,17 @@ export interface HotAsyncContextThresholds { readonly maxFindings: number; } +export interface CpuHotspotThresholds { + /** User-code self CPU that is actionable without a pattern-specific detector. */ + readonly minSelfPct: number; + /** User-code inclusive CPU that becomes a lower-confidence caller lead when no self hotspot exists. */ + readonly minTotalPct: number; + /** Above either value the generic hotspot is `critical`. */ + readonly criticalPct: number; + /** Emit at most this many generic hotspots. */ + readonly maxFindings: number; +} + export interface LongAwaitThresholds { /** An async operation lasting longer than this (ms) is reported. */ readonly minDurationMs: number; @@ -261,6 +273,10 @@ export const DETECTOR_THRESHOLDS: DetectorThresholds = { // module resolution is happening per request. warningSelfPct escalates // when the resolver itself is churning (3% self = lots of cache misses). requireInHotPath: { minSelfPct: 0.5, minTotalPct: 1, warningSelfPct: 3 }, + // Generic user-code CPU hotspot fallback. Pattern detectors should own + // known anti-patterns; self-heavy user frames are actionable, inclusive-only + // user frames are emitted as lower-confidence caller/context leads. + cpuHotspot: { minSelfPct: 10, minTotalPct: 25, criticalPct: 40, maxFindings: 3 }, // 5 deopts for the same function is meaningful (V8 stops optimising // after a few tries); 20+ is pathological. deoptLoop: { minCount: 5, criticalCount: 20 }, diff --git a/packages/detectors/src/detectors/alloc-in-hot-path.ts b/packages/detectors/src/detectors/alloc-in-hot-path.ts index 98e1f44..8321b26 100644 --- a/packages/detectors/src/detectors/alloc-in-hot-path.ts +++ b/packages/detectors/src/detectors/alloc-in-hot-path.ts @@ -19,8 +19,8 @@ export const allocInHotPathDetector: KindScopedDetector<'cpu' | 'memory'> = { kindIds: ['cpu', 'memory'], detect({ cpu, memory }): Finding[] { const thresholds = DETECTOR_THRESHOLDS.allocInHotPath; - const cpuHotspots = cpu.report.hotspots; - const memAllocators = memory.report.hotAllocators; + const cpuHotspots = cpu.report.hotspots.filter(isActionableFrame); + const memAllocators = memory.report.hotAllocators.filter(isActionableFrame); if (cpuHotspots.length === 0 || memAllocators.length === 0) return []; const memByKey = new Map(); @@ -104,3 +104,15 @@ function buildFinding( function frameKey(fn: string, file: string, line: number): string { return `${file}::${fn}::${line}`; } + +function isActionableFrame( + frame: Pick, +) { + if (frame.category !== 'user' && frame.category !== 'node_modules') return false; + if (frame.file.startsWith('node:')) return false; + if (/^(?:native |extensions::|evalmachine\.|node:internal\/)/.test(frame.file)) return false; + if (/^\((?:root|idle|program|garbage collector|anonymous)\)$/.test(frame.function)) { + return false; + } + return true; +} diff --git a/packages/detectors/src/detectors/blocking-io.ts b/packages/detectors/src/detectors/blocking-io.ts index b469323..5415ac3 100644 --- a/packages/detectors/src/detectors/blocking-io.ts +++ b/packages/detectors/src/detectors/blocking-io.ts @@ -125,10 +125,14 @@ import { findStallCorrelation, isBuiltinRuntimeHotspot, maxHotspotPct, + readFrameSourceText, resolveAttribution, severityForPct, } from './shared.js'; +const ZLIB_PROCESS_CHUNK_SYNC = 'processChunkSync'; +const ZLIB_SYNC_APIS = ['gzipSync', 'gunzipSync', 'deflateSync', 'inflateSync'] as const; + export const blockingIoDetector: KindScopedDetector<'cpu'> = { id: 'blocking-io', kindIds: ['cpu'], @@ -136,38 +140,80 @@ export const blockingIoDetector: KindScopedDetector<'cpu'> = { const report = cpu.report; const context: CpuHotspotContext = cpu.view.hotspotAnalysis; const thresholds = DETECTOR_THRESHOLDS.blockingIo; - const { categoryTotalPct } = aggregateByPatterns(context.fullHotspots, BLOCKING_IO_PATTERNS, { + const aggregate = aggregateByPatterns(context.fullHotspots, BLOCKING_IO_PATTERNS, { normalize: stripOptPrefix, }); + const zlibProcessChunkTotalPct = context.fullHotspots + .filter(isZlibProcessChunkSyncHotspot) + .reduce((sum, hotspot) => sum + hotspot.totalPct, 0); + const categoryTotalPct = aggregate.categoryTotalPct + zlibProcessChunkTotalPct; const familyExceeded = exceedsCategoryThreshold(categoryTotalPct, thresholds.categoryTotalPct); const findings: Finding[] = []; for (const hotspot of context.fullHotspots) { if (!isBuiltinRuntimeHotspot(hotspot)) continue; const normalizedFunctionName = stripOptPrefix(hotspot.function); - const patternMatch = BLOCKING_IO_PATTERNS.find((pattern) => - pattern.re.test(normalizedFunctionName), - ); + const patternMatch = + BLOCKING_IO_PATTERNS.find((pattern) => pattern.re.test(normalizedFunctionName)) ?? + zlibProcessChunkPattern(hotspot, context, cpu.view.bundle.target.cwd); if (!patternMatch) continue; const perFrameHit = exceedsAnyHotspotThreshold(hotspot, thresholds); if (!perFrameHit && !familyExceeded) continue; - findings.push(buildFinding(hotspot, patternMatch.api, categoryTotalPct, report, context)); + const callee = 'callee' in patternMatch ? patternMatch.callee : undefined; + findings.push( + buildFinding(hotspot, patternMatch.api, categoryTotalPct, report, context, { callee }), + ); } return findings; }, }; +function isZlibProcessChunkSyncHotspot(hotspot: Hotspot): boolean { + return ( + stripOptPrefix(hotspot.function) === ZLIB_PROCESS_CHUNK_SYNC && + (hotspot.file === 'node:zlib' || hotspot.file.endsWith('/zlib.js')) + ); +} + +function zlibProcessChunkPattern( + hotspot: Hotspot, + context: CpuHotspotContext, + cwd: string, +): { api: string; callee?: string } | undefined { + if (!isZlibProcessChunkSyncHotspot(hotspot)) return undefined; + return { + api: inferZlibSyncApi(hotspot, context, cwd), + callee: 'node:zlib processChunkSync', + }; +} + +function inferZlibSyncApi(hotspot: Hotspot, context: CpuHotspotContext, cwd: string): string { + const attribution = context.userCallerById.get(hotspot.id); + const candidates = [attribution, ...(context.candidateCallersById?.get(hotspot.id) ?? [])].filter( + (candidate): candidate is NonNullable => Boolean(candidate), + ); + for (const candidate of candidates) { + const source = readFrameSourceText(candidate, cwd); + const match = source?.match(/\b(gzipSync|gunzipSync|deflateSync|inflateSync)\s*\(/); + if (match?.[1] && ZLIB_SYNC_APIS.includes(match[1] as (typeof ZLIB_SYNC_APIS)[number])) { + return `zlib.${match[1]}`; + } + } + return 'zlib.gzipSync'; +} + function buildFinding( hotspot: Hotspot, api: string, categoryTotalPct: number, report: { eventLoop: EventLoopReport }, context: CpuHotspotContext, + options: { callee?: string } = {}, ): BuiltinFinding<'blocking-io'> { const asyncApi = api.replace(/Sync$/, ''); const { attribution, caller, candidateCallers } = resolveAttribution(hotspot, context); const evidenceExtra: BlockingIoEvidenceExtra = { api, - callee: hotspot.function, + callee: options.callee ?? hotspot.function, ...buildAttributionEvidence(attribution, caller, candidateCallers), eventLoopCorrelation: findStallCorrelation(caller, report), categoryTotalPct: categoryTotalPct > 0 ? categoryTotalPct : undefined, diff --git a/packages/detectors/src/detectors/cpu-hotspot.ts b/packages/detectors/src/detectors/cpu-hotspot.ts new file mode 100644 index 0000000..ce32e96 --- /dev/null +++ b/packages/detectors/src/detectors/cpu-hotspot.ts @@ -0,0 +1,221 @@ +import type { + AlternativeHotspotEvidence, + BuiltinFinding, + CpuHotspotEvidenceExtra, + Finding, + Hotspot, + KindScopedDetector, + LanternaReport, + StallCorrelation, +} from '@lanterna-profiler/core'; +import { defineBuiltinFinding } from '@lanterna-profiler/core'; +import { DETECTOR_THRESHOLDS } from '../config.js'; + +const SPECIFIC_CPU_FINDING_CATEGORIES = new Set([ + 'blocking-io', + 'sync-crypto', + 'json-on-hot-path', + 'node-modules-hotspot', + 'require-in-hot-path', +]); + +type CpuHotspotMode = CpuHotspotEvidenceExtra['mode']; + +/** + * Generic fallback for user-code CPU bottlenecks. Pattern detectors explain + * known anti-patterns; this explains "plain code is just hot" and gives agents + * a concrete file/line or caller lead even when no specialized category applies. + */ +export const cpuHotspotDetector: KindScopedDetector<'cpu'> = { + id: 'cpu-hotspot', + kindIds: ['cpu'], + order: 90, + detect({ cpu }, shared): Finding[] { + const thresholds = DETECTOR_THRESHOLDS.cpuHotspot; + const candidates = cpu.report.hotspots + .filter((hotspot) => hotspot.category === 'user') + .filter( + (hotspot) => + hotspot.selfPct >= thresholds.minSelfPct || hotspot.totalPct >= thresholds.minTotalPct, + ) + .filter( + (hotspot) => + !( + hasSpecificCpuFindings(shared.findings) && + isAnonymousWrapper(hotspot) && + hotspot.selfPct < thresholds.minSelfPct + ), + ) + .filter((hotspot) => !isExplainedBySpecificFinding(hotspot, shared.findings)) + .sort(compareHotspots); + const selfHotspots = candidates.filter((hotspot) => hotspot.selfPct >= thresholds.minSelfPct); + const mode: CpuHotspotMode = selfHotspots.length > 0 ? 'self' : 'inclusive-entry'; + const explanationPool = selfHotspots.length > 0 ? selfHotspots : candidates; + const namedPool = explanationPool.some((hotspot) => !isAnonymousWrapper(hotspot)) + ? explanationPool.filter((hotspot) => !isAnonymousWrapper(hotspot)) + : explanationPool; + + return namedPool.slice(0, thresholds.maxFindings).map((hotspot, index) => + buildFinding( + hotspot, + candidates.filter((candidate) => candidate.id !== hotspot.id).slice(0, 2), + cpu.report.eventLoop.correlatedHotspots?.find((candidate) => sameFrame(candidate, hotspot)), + mode, + index, + ), + ); + }, +}; + +function compareHotspots(left: Hotspot, right: Hotspot): number { + const selfDelta = right.selfPct - left.selfPct; + if (selfDelta !== 0) return selfDelta; + return right.totalPct - left.totalPct; +} + +function isAnonymousWrapper(hotspot: Hotspot): boolean { + return hotspot.function === '(anonymous)' || hotspot.function.trim() === ''; +} + +function hasSpecificCpuFindings(findings: readonly LanternaReport['findings'][number][]): boolean { + return findings.some((finding) => SPECIFIC_CPU_FINDING_CATEGORIES.has(finding.category)); +} + +function buildFinding( + hotspot: Hotspot, + alternatives: Hotspot[], + eventLoopCorrelation: StallCorrelation | undefined, + mode: CpuHotspotMode, + index: number, +): BuiltinFinding<'cpu-hotspot'> { + const thresholds = DETECTOR_THRESHOLDS.cpuHotspot; + const score = Math.max(hotspot.selfPct, hotspot.totalPct); + const severity: BuiltinFinding<'cpu-hotspot'>['severity'] = + score >= thresholds.criticalPct ? 'critical' : 'warning'; + const evidenceExtra: CpuHotspotEvidenceExtra = { + proofLevel: mode === 'self' ? 'direct-user-hotspot' : 'inclusive-user-entry', + mode, + category: hotspot.category, + selfPct: hotspot.selfPct, + totalPct: hotspot.totalPct, + ...(eventLoopCorrelation + ? { + eventLoopCorrelation: { + overlapPct: eventLoopCorrelation.overlapPct, + samplePct: eventLoopCorrelation.samplePct, + }, + } + : {}), + alternativeHotspots: alternatives.map(toAlternativeHotspotEvidence), + }; + + return defineBuiltinFinding({ + id: `cpu-hotspot:${hotspot.id}`, + profileKind: 'cpu', + severity, + category: 'cpu-hotspot', + title: + mode === 'self' + ? `${hotspot.function} is a CPU hotspot` + : `${hotspot.function} leads to unexplained CPU work`, + confidence: mode === 'self' ? (index === 0 ? 'high' : 'medium') : 'medium', + proofLevel: mode === 'self' ? 'direct-sample' : 'heuristic', + evidence: { + file: hotspot.source?.file ?? hotspot.file, + line: hotspot.source?.line ?? hotspot.line, + function: hotspot.source?.name ?? hotspot.function, + selfPct: hotspot.selfPct, + ...(hotspot.source ? { source: hotspot.source } : {}), + extra: evidenceExtra, + }, + measurements: { + observed: { + selfPct: hotspot.selfPct, + totalPct: hotspot.totalPct, + }, + thresholds: { + minSelfPct: thresholds.minSelfPct, + minTotalPct: thresholds.minTotalPct, + criticalPct: thresholds.criticalPct, + }, + }, + remediation: { + kind: mode === 'self' ? 'offload-worker' : 'other', + notes: + mode === 'self' + ? 'This is not a known blocking API pattern; inspect the sampled function body directly. Reduce algorithmic cost, cache stable results, or move CPU-bound work to worker_threads/piscina.' + : 'This user frame is inclusive-heavy but not self-heavy. Inspect the callees and hot stacks first; the caller may only be the entry point to missing detector coverage or external CPU work.', + }, + why: + mode === 'self' + ? `\`${hotspot.function}\` accounts for ${hotspot.selfPct.toFixed(1)}% self CPU and ${hotspot.totalPct.toFixed(1)}% inclusive CPU. No more specific built-in detector explained this frame, so the function body itself is the bottleneck.` + : `\`${hotspot.function}\` accounts for only ${hotspot.selfPct.toFixed(1)}% self CPU but ${hotspot.totalPct.toFixed(1)}% inclusive CPU. No more specific built-in detector explained the downstream work, so this is a caller/context lead rather than proof that the function body is expensive.`, + suggestion: + mode === 'self' + ? 'Open this function first. Look for tight loops, repeated transformations, excessive object work, or synchronous CPU-heavy algorithms. If the work is inherently expensive, move it off the main event loop.' + : 'Open this caller and inspect the top callees/hot stacks. If the downstream work is a known API pattern, add or tune a specialized detector; otherwise reduce call frequency, input size, or move the downstream CPU off the main event loop.', + references: [ + 'https://nodejs.org/en/docs/guides/dont-block-the-event-loop', + 'https://nodejs.org/api/worker_threads.html', + ], + }); +} + +function toAlternativeHotspotEvidence(hotspot: Hotspot): AlternativeHotspotEvidence { + return { + id: hotspot.id, + function: hotspot.function, + file: hotspot.source?.file ?? hotspot.file, + line: hotspot.source?.line ?? hotspot.line, + selfPct: hotspot.selfPct, + totalPct: hotspot.totalPct, + ...(hotspot.source ? { source: hotspot.source } : {}), + }; +} + +function isExplainedBySpecificFinding( + hotspot: Hotspot, + findings: readonly LanternaReport['findings'][number][], +): boolean { + return findings.some((finding) => { + if (!SPECIFIC_CPU_FINDING_CATEGORIES.has(finding.category)) return false; + if (sameFrame(finding.evidence, hotspot)) return true; + if (sameSourceFrame(finding.evidence.source, hotspot)) return true; + const extra = finding.evidence.extra as + | { userCaller?: unknown; candidateCallers?: unknown } + | undefined; + if (sameUnknownFrame(extra?.userCaller, hotspot)) return true; + if (!Array.isArray(extra?.candidateCallers)) return false; + return extra.candidateCallers.some((candidate) => sameUnknownFrame(candidate, hotspot)); + }); +} + +function sameUnknownFrame(candidate: unknown, hotspot: Hotspot): boolean { + if (!candidate || typeof candidate !== 'object') return false; + return sameFrame(candidate as { file?: string; line?: number; function?: string }, hotspot); +} + +function sameFrame( + candidate: { file?: string; line?: number; function?: string }, + hotspot: Hotspot, +): boolean { + return ( + candidate.file === hotspot.file && + candidate.line === hotspot.line && + candidate.function === hotspot.function + ); +} + +function sameSourceFrame( + source: { file?: string; line?: number; name?: string } | undefined, + hotspot: Hotspot, +): boolean { + if (!source || !hotspot.source) return false; + return ( + source.file === hotspot.source.file && + source.line === hotspot.source.line && + (source.name === undefined || + hotspot.source.name === undefined || + source.name === hotspot.source.name) + ); +} diff --git a/packages/detectors/src/detectors/deep-async-chain.ts b/packages/detectors/src/detectors/deep-async-chain.ts index 4020eca..00b3575 100644 --- a/packages/detectors/src/detectors/deep-async-chain.ts +++ b/packages/detectors/src/detectors/deep-async-chain.ts @@ -21,6 +21,7 @@ export const deepAsyncChainDetector: KindScopedDetector<'async'> = { for (const chain of report.chains) { if (findings.length >= thresholds.maxFindings) break; if (chain.depth < thresholds.minDepth) continue; + if (isTimerOnlySyntheticChain(chain)) continue; const severity: BaseFinding['severity'] = chain.depth >= thresholds.criticalDepth ? 'critical' : 'warning'; const rootFrame = chain.rootFrame; @@ -73,3 +74,16 @@ export const deepAsyncChainDetector: KindScopedDetector<'async'> = { return findings; }, }; + +function isTimerOnlySyntheticChain(chain: { + deepestPath: readonly string[]; + rootFrame?: unknown; + deepestFrame?: unknown; + dominantFile?: string; +}): boolean { + if (chain.rootFrame || chain.deepestFrame || chain.dominantFile) return false; + return ( + chain.deepestPath.length > 0 && + chain.deepestPath.every((kind) => kind === 'timer' || kind === 'immediate') + ); +} diff --git a/packages/detectors/src/detectors/deopt-loop.ts b/packages/detectors/src/detectors/deopt-loop.ts index 45ec0f6..a896fda 100644 --- a/packages/detectors/src/detectors/deopt-loop.ts +++ b/packages/detectors/src/detectors/deopt-loop.ts @@ -1,7 +1,9 @@ import type { BuiltinFinding, + DeoptEntry, DeoptLoopEvidenceExtra, Finding, + Hotspot, KindScopedDetector, } from '@lanterna-profiler/core'; import { defineBuiltinFinding } from '@lanterna-profiler/core'; @@ -18,7 +20,7 @@ export const deoptLoopDetector: KindScopedDetector<'cpu'> = { const context: CpuHotspotContext = cpu.view.hotspotAnalysis; const thresholds = DETECTOR_THRESHOLDS.deoptLoop; const findings: Finding[] = []; - for (const deopt of report.deopts) { + for (const deopt of aggregateDeopts(report.deopts)) { if (deopt.count < thresholds.minCount) continue; const matchingHotspot = findHotDeoptHotspot(deopt.function, deopt.file, deopt.line, context); if (!matchingHotspot) continue; @@ -38,8 +40,8 @@ export const deoptLoopDetector: KindScopedDetector<'cpu'> = { confidence: 'medium', proofLevel: 'trace-only', evidence: { - file: deopt.file, - line: deopt.line, + file: deopt.file || matchingHotspot.file, + line: deopt.line || matchingHotspot.line, function: deopt.function, selfPct: 0, ...(matchingHotspot.source ? { source: matchingHotspot.source } : {}), @@ -62,12 +64,49 @@ export const deoptLoopDetector: KindScopedDetector<'cpu'> = { }, }; +function aggregateDeopts(deopts: readonly DeoptEntry[]): DeoptEntry[] { + const grouped = new Map(); + const output: DeoptEntry[] = []; + for (const deopt of deopts) { + if (deopt.file && deopt.line > 0) { + output.push(deopt); + continue; + } + const key = deopt.function; + const existing = grouped.get(key); + if (!existing) { + grouped.set(key, { ...deopt }); + continue; + } + existing.count += deopt.count; + existing.reason = mergeLabel(existing.reason, deopt.reason); + existing.bailoutType = mergeLabel(existing.bailoutType, deopt.bailoutType); + } + output.push(...grouped.values()); + return output.sort((a, b) => b.count - a.count); +} + +function mergeLabel(left: string, right: string): string { + if (left === right) return left; + const parts = new Set( + [...left.split(';'), ...right.split(';')].map((part) => part.trim()).filter(Boolean), + ); + return Array.from(parts).join('; '); +} + function findHotDeoptHotspot( functionName: string, file: string, line: number, context: CpuHotspotContext, -) { +): Hotspot | undefined { + if (!file || line <= 0) { + const matches = context.fullHotspots.filter( + (hotspot) => + hotspot.function === functionName && hotspot.category === 'user' && hotspot.totalPct > 1, + ); + return matches.length === 1 ? matches[0] : undefined; + } return context.fullHotspots.find( (hotspot) => hotspot.function === functionName && diff --git a/packages/detectors/src/detectors/event-loop-stall.ts b/packages/detectors/src/detectors/event-loop-stall.ts index b58f21a..213d847 100644 --- a/packages/detectors/src/detectors/event-loop-stall.ts +++ b/packages/detectors/src/detectors/event-loop-stall.ts @@ -1,7 +1,9 @@ import type { + AlternativeHotspotEvidence, BuiltinFinding, EventLoopStallEvidenceExtra, Finding, + Hotspot, KindScopedDetector, } from '@lanterna-profiler/core'; import { defineBuiltinFinding } from '@lanterna-profiler/core'; @@ -30,9 +32,13 @@ export const eventLoopStallDetector: KindScopedDetector<'cpu'> = { eventLoop.confidence === 'high' && topCandidate !== undefined && topCandidate.overlapPct >= thresholds.strongCorrelationOverlapPct; + const fallbackHotspots = fallbackUserHotspots(report.hotspots); + const fallbackCandidate = strongCorrelation ? undefined : fallbackHotspots[0]; + const anchor = strongCorrelation ? topCandidate : fallbackCandidate; + const proofLevel = strongCorrelation ? 'aggregate-correlation' : 'hotspot-fallback'; const severity: Finding['severity'] = maxLagMs > thresholds.critical ? 'critical' : 'warning'; const evidenceExtra: EventLoopStallEvidenceExtra = { - proofLevel: 'aggregate-correlation', + proofLevel, p99LagMs, maxLagMs, sampleCount: eventLoop.sampleCount, @@ -41,6 +47,7 @@ export const eventLoopStallDetector: KindScopedDetector<'cpu'> = { histogram: eventLoop.histogram, stallIntervals: eventLoop.stallIntervals, candidateHotspots: eventLoop.correlatedHotspots ?? [], + fallbackHotspots: fallbackHotspots.length > 0 ? fallbackHotspots : undefined, correlationCoverage: eventLoop.correlationCoverage, }; @@ -51,14 +58,14 @@ export const eventLoopStallDetector: KindScopedDetector<'cpu'> = { severity, category: 'event-loop-stall', title: `Event loop stalled (max ${maxLagMs.toFixed(0)}ms)`, - confidence: strongCorrelation ? 'high' : 'medium', - proofLevel: 'correlated-window', + confidence: strongCorrelation ? 'high' : fallbackCandidate ? 'medium' : 'medium', + proofLevel: strongCorrelation ? 'correlated-window' : 'heuristic', evidence: { - file: strongCorrelation ? topCandidate.file : '(process)', - line: strongCorrelation ? topCandidate.line : 0, - function: strongCorrelation ? topCandidate.function : '(aggregate)', - selfPct: strongCorrelation ? topCandidate.samplePct : 0, - ...(strongCorrelation && topCandidate.source ? { source: topCandidate.source } : {}), + file: anchor?.file ?? '(process)', + line: anchor?.line ?? 0, + function: anchor?.function ?? '(aggregate)', + selfPct: strongCorrelation ? topCandidate.samplePct : (fallbackCandidate?.selfPct ?? 0), + ...(anchor?.source ? { source: anchor.source } : {}), extra: evidenceExtra, }, measurements: { @@ -72,7 +79,9 @@ export const eventLoopStallDetector: KindScopedDetector<'cpu'> = { }, why: strongCorrelation ? `The event loop spent up to ${maxLagMs.toFixed(0)}ms (p99 ${p99LagMs.toFixed(0)}ms) without being able to pick up tasks. During those measured stall windows, \`${topCandidate.function}\` accounted for ${topCandidate.overlapPct.toFixed(1)}% of the user-code CPU samples.` - : `The event loop spent up to ${maxLagMs.toFixed(0)}ms (p99 ${p99LagMs.toFixed(0)}ms) without being able to pick up tasks. The report includes ranked correlated hotspots, but no single user frame dominated the measured stall windows strongly enough to blame it on its own.`, + : fallbackCandidate + ? `The event loop spent up to ${maxLagMs.toFixed(0)}ms (p99 ${p99LagMs.toFixed(0)}ms) without being able to pick up tasks. No measured stall window had enough attribution to blame a single frame, but \`${fallbackCandidate.function}\` is the hottest user-code CPU frame in the same capture (${fallbackCandidate.selfPct.toFixed(1)}% self, ${fallbackCandidate.totalPct.toFixed(1)}% total).` + : `The event loop spent up to ${maxLagMs.toFixed(0)}ms (p99 ${p99LagMs.toFixed(0)}ms) without being able to pick up tasks. The report includes ranked correlated hotspots, but no single user frame dominated the measured stall windows strongly enough to blame it on its own.`, suggestion: `Identify the hottest user-code function in this report and move its work off the main thread. Use \`worker_threads\` or \`piscina\` for CPU-bound work; chunk long loops with \`setImmediate\` or a queue; prefer streaming JSON for large payloads.`, references: [ 'https://nodejs.org/en/docs/guides/dont-block-the-event-loop', @@ -82,3 +91,24 @@ export const eventLoopStallDetector: KindScopedDetector<'cpu'> = { ]; }, }; + +function fallbackUserHotspots(hotspots: readonly Hotspot[]): AlternativeHotspotEvidence[] { + return hotspots + .filter((hotspot) => hotspot.category === 'user') + .filter((hotspot) => hotspot.selfPct >= 10 || hotspot.totalPct >= 25) + .sort((left, right) => { + const selfDelta = right.selfPct - left.selfPct; + if (selfDelta !== 0) return selfDelta; + return right.totalPct - left.totalPct; + }) + .slice(0, 3) + .map((hotspot) => ({ + id: hotspot.id, + function: hotspot.function, + file: hotspot.source?.file ?? hotspot.file, + line: hotspot.source?.line ?? hotspot.line, + selfPct: hotspot.selfPct, + totalPct: hotspot.totalPct, + ...(hotspot.source ? { source: hotspot.source } : {}), + })); +} diff --git a/packages/detectors/src/detectors/index.ts b/packages/detectors/src/detectors/index.ts index 39c98f5..b41dbde 100644 --- a/packages/detectors/src/detectors/index.ts +++ b/packages/detectors/src/detectors/index.ts @@ -4,6 +4,7 @@ import { type KindScopedDetector, } from '@lanterna-profiler/core'; import { blockingIoDetector } from './blocking-io.js'; +import { cpuHotspotDetector } from './cpu-hotspot.js'; import { deoptLoopDetector } from './deopt-loop.js'; import { eventLoopStallDetector } from './event-loop-stall.js'; import { excessiveGcDetector } from './excessive-gc.js'; @@ -21,6 +22,7 @@ export const DETECTORS: KindScopedDetector<'cpu'>[] = [ deoptLoopDetector, requireInHotPathDetector, nodeModulesHotspotDetector, + cpuHotspotDetector, ]; export function createBuiltInFindingAnalyzers(): FindingAnalyzer[] { diff --git a/packages/detectors/src/detectors/json-on-hot-path.ts b/packages/detectors/src/detectors/json-on-hot-path.ts index ad909a3..a56c12b 100644 --- a/packages/detectors/src/detectors/json-on-hot-path.ts +++ b/packages/detectors/src/detectors/json-on-hot-path.ts @@ -14,6 +14,7 @@ import { buildAttributionEvidence, type CpuHotspotContext, findStallCorrelation, + readFrameSourceText, resolveAttribution, } from './shared.js'; @@ -25,11 +26,13 @@ export const jsonOnHotPathDetector: KindScopedDetector<'cpu'> = { const report = cpu.report; const context: CpuHotspotContext = cpu.view.hotspotAnalysis; const thresholds = DETECTOR_THRESHOLDS.jsonHotPath; + const cwd = cpu.view.bundle.target.cwd; const { categoryTotalPct } = aggregateByPatterns(context.fullHotspots, JSON_FUNCTION_PATTERNS, { normalize: stripOptPrefix, }); const familyExceeded = categoryTotalPct >= thresholds.categoryTotalPct; const findings: Finding[] = []; + const seen = new Set(); for (const hotspot of context.fullHotspots) { const normalizedFunctionName = stripOptPrefix(hotspot.function); const patternMatch = JSON_FUNCTION_PATTERNS.find((pattern) => @@ -38,12 +41,54 @@ export const jsonOnHotPathDetector: KindScopedDetector<'cpu'> = { if (!patternMatch) continue; if (hotspot.category !== 'node:builtin' && hotspot.category !== 'native') continue; if (hotspot.totalPct < thresholds.minTotalPct && !familyExceeded) continue; - findings.push(buildFinding(hotspot, patternMatch.api, categoryTotalPct, report, context)); + const finding = buildFinding(hotspot, patternMatch.api, categoryTotalPct, report, context); + if (seen.has(finding.id)) continue; + seen.add(finding.id); + findings.push(finding); + } + for (const hotspot of context.fullHotspots) { + if (hotspot.category !== 'user') continue; + if (hotspot.totalPct < thresholds.minTotalPct && !familyExceeded) continue; + if (!hasDominantSelfCost(hotspot, thresholds.minTotalPct)) continue; + const api = inlinedJsonApi(readFrameSourceText(hotspot, cwd), hotspot.line); + if (!api) continue; + const finding = buildFinding(hotspot, api, categoryTotalPct, report, context); + if (seen.has(finding.id)) continue; + seen.add(finding.id); + findings.push(finding); } return findings; }, }; +function hasDominantSelfCost(hotspot: Hotspot, minSelfPct: number): boolean { + if (hotspot.selfPct < minSelfPct) return false; + if (hotspot.totalPct <= 0) return true; + return hotspot.selfPct / hotspot.totalPct >= 0.5; +} + +function inlinedJsonApi(sourceText: string | undefined, line: number): string | undefined { + const sourceWindow = sourceTextAroundLine(sourceText, line); + if (!sourceWindow) return undefined; + if (/\bJSON\.stringify\s*\(/.test(sourceWindow)) return 'JSON.stringify'; + if (/\bJSON\.parse\s*\(/.test(sourceWindow)) return 'JSON.parse'; + return undefined; +} + +function sourceTextAroundLine( + sourceText: string | undefined, + line: number, + radius = 3, +): string | undefined { + if (!sourceText || line <= 0) return undefined; + const lines = sourceText.split(/\r?\n/); + const index = line - 1; + if (index < 0 || index >= lines.length) return undefined; + const start = Math.max(0, index - radius); + const end = Math.min(lines.length, index + radius + 1); + return lines.slice(start, end).join('\n'); +} + function buildFinding( hotspot: Hotspot, api: string, @@ -53,7 +98,7 @@ function buildFinding( ): BuiltinFinding<'json-on-hot-path'> { const { attribution, caller, candidateCallers } = resolveAttribution(hotspot, context); const evidenceExtra: JsonHotPathEvidenceExtra = { - callee: hotspot.function, + callee: hotspot.category === 'user' ? api : hotspot.function, calleeTotalPct: hotspot.totalPct, ...buildAttributionEvidence(attribution, caller, candidateCallers), eventLoopCorrelation: findStallCorrelation(caller, report), diff --git a/packages/detectors/src/detectors/memory-growth.ts b/packages/detectors/src/detectors/memory-growth.ts index 1ef7360..1f6ee90 100644 --- a/packages/detectors/src/detectors/memory-growth.ts +++ b/packages/detectors/src/detectors/memory-growth.ts @@ -28,7 +28,9 @@ export const memoryGrowthDetector: KindScopedDetector<'memory'> = { const heapUsed = memory.view.series.heapUsed; if (rss) { - const finding = buildGrowthFinding('rss', rss, durationMs, sampleCount); + const finding = hasRssRetentionCorroboration(memory.view.series) + ? buildGrowthFinding('rss', rss, durationMs, sampleCount) + : null; if (finding) findings.push(finding); } if (heapUsed) { @@ -39,13 +41,27 @@ export const memoryGrowthDetector: KindScopedDetector<'memory'> = { }, }; +function hasRssRetentionCorroboration(series: { + rss?: MemorySeriesStats; + heapUsed?: MemorySeriesStats; + external?: MemorySeriesStats; + arrayBuffers?: MemorySeriesStats; +}): boolean { + const thresholds = DETECTOR_THRESHOLDS.memoryGrowth; + return ( + toMBPerSec(series.heapUsed) >= thresholds.heapGrowthWarnMBPerSec || + toMBPerSec(series.external) >= thresholds.rssGrowthWarnMBPerSec || + toMBPerSec(series.arrayBuffers) >= thresholds.rssGrowthWarnMBPerSec + ); +} + function buildGrowthFinding( metric: 'rss' | 'heapUsed', stats: MemorySeriesStats, durationMs: number, sampleCount: number, ): BaseFinding> | null { - const slopeMBPerSec = stats.slopeBytesPerSec / BYTES_PER_MB; + const slopeMBPerSec = toMBPerSec(stats); const thresholds = DETECTOR_THRESHOLDS.memoryGrowth; const warn = metric === 'rss' ? thresholds.rssGrowthWarnMBPerSec : thresholds.heapGrowthWarnMBPerSec; @@ -106,6 +122,10 @@ function buildGrowthFinding( }; } +function toMBPerSec(stats: MemorySeriesStats | undefined): number { + return (stats?.slopeBytesPerSec ?? 0) / BYTES_PER_MB; +} + function formatRate(mbPerSec: number): string { if (mbPerSec >= 1) return `${mbPerSec.toFixed(2)} MB/s`; return `${(mbPerSec * 1024).toFixed(1)} KB/s`; diff --git a/packages/detectors/src/detectors/shared.ts b/packages/detectors/src/detectors/shared.ts index 6d0e3ea..2c309ec 100644 --- a/packages/detectors/src/detectors/shared.ts +++ b/packages/detectors/src/detectors/shared.ts @@ -1,3 +1,5 @@ +import { existsSync, readFileSync } from 'node:fs'; +import { isAbsolute, join } from 'node:path'; import type { AlternativeHotspotEvidence, AttributionEvidence, @@ -282,3 +284,24 @@ export function buildAttributedFinding< references, }; } + +export function readFrameSourceText( + frame: { file?: string; source?: { file: string } } | undefined, + cwd: string, +): string | undefined { + if (!frame) return undefined; + const candidates = [frame.source?.file, frame.file].filter((file): file is string => + Boolean(file), + ); + for (const candidate of candidates) { + if (candidate.startsWith('node:') || candidate.startsWith('native ')) continue; + const path = isAbsolute(candidate) ? candidate : join(cwd, candidate); + if (!existsSync(path)) continue; + try { + return readFileSync(path, 'utf8'); + } catch { + // Source inspection is best-effort; detectors can still use sample evidence. + } + } + return undefined; +} diff --git a/packages/detectors/src/detectors/sync-crypto.ts b/packages/detectors/src/detectors/sync-crypto.ts index ec83ae6..bc5c915 100644 --- a/packages/detectors/src/detectors/sync-crypto.ts +++ b/packages/detectors/src/detectors/sync-crypto.ts @@ -5,6 +5,7 @@ import type { FindingRemediation, Hotspot, SyncCryptoEvidenceExtra, + UserCallerAttribution, } from '@lanterna-profiler/core'; import { defineBuiltinFinding, stripOptPrefix } from '@lanterna-profiler/core'; import { DETECTOR_THRESHOLDS, SYNC_CRYPTO_FNS, SYNC_CRYPTO_PATTERNS } from '../config.js'; @@ -62,6 +63,7 @@ import { exceedsCategoryThreshold, findStallCorrelation, isBuiltinRuntimeHotspot, + readFrameSourceText, resolveAttribution, severityForPct, } from './shared.js'; @@ -91,7 +93,9 @@ export const syncCryptoDetector: KindScopedDetector<'cpu'> = { continue; } if (hotspot.totalPct < thresholds.minTotalPct && !familyExceeded) continue; - findings.push(buildFinding(hotspot, categoryTotalPct, report, context)); + findings.push( + buildFinding(hotspot, categoryTotalPct, report, context, cpu.view.bundle.target.cwd), + ); } return findings; }, @@ -102,13 +106,16 @@ function buildFinding( categoryTotalPct: number, report: { eventLoop: EventLoopReport }, context: CpuHotspotContext, + cwd: string, ): BuiltinFinding<'sync-crypto'> { const { attribution, caller, candidateCallers } = resolveAttribution(hotspot, context); + const sourceCallsiteCaller = + findSourceCallsiteCaller(candidateCallers, hotspot.function, cwd) ?? caller; const evidenceExtra: SyncCryptoEvidenceExtra = { callee: hotspot.function, calleeTotalPct: hotspot.totalPct, ...buildAttributionEvidence(attribution, caller, candidateCallers), - eventLoopCorrelation: findStallCorrelation(attribution, report), + eventLoopCorrelation: findStallCorrelation(caller ?? attribution, report), categoryTotalPct: categoryTotalPct > 0 ? categoryTotalPct : undefined, }; const thresholds = DETECTOR_THRESHOLDS.syncCrypto; @@ -119,7 +126,7 @@ function buildFinding( severity: severityForPct(hotspot.totalPct, thresholds.criticalPct), title: `Synchronous crypto on hot path (${hotspot.function})`, hotspot, - caller, + caller: sourceCallsiteCaller, selfPct: hotspot.totalPct, extra: evidenceExtra, measurements: { @@ -144,3 +151,78 @@ function buildFinding( }), ); } + +function findSourceCallsiteCaller( + candidates: readonly UserCallerAttribution[], + callee: string, + cwd: string, +): UserCallerAttribution | undefined { + const pattern = new RegExp(`\\b${escapeRegExp(callExpressionName(callee))}\\s*\\(`); + for (const candidate of candidates) { + const source = readFrameSourceText(candidate, cwd); + const anchorLine = candidate.source?.line ?? candidate.line; + const line = + findPatternLineNearAnchor(source, anchorLine, pattern) ?? + findPatternLineInFunctionBlock(source, anchorLine, pattern); + if (line !== undefined) { + return { + ...candidate, + line, + ...(candidate.source ? { source: { ...candidate.source, line } } : {}), + }; + } + } + return undefined; +} + +function callExpressionName(callee: string): string { + const normalized = stripOptPrefix(callee); + return normalized.split('.').at(-1) ?? normalized; +} + +function findPatternLineNearAnchor( + sourceText: string | undefined, + line: number, + pattern: RegExp, + radius = 2, +): number | undefined { + if (!sourceText || line <= 0) return undefined; + const lines = sourceText.split(/\r?\n/); + const index = line - 1; + if (index < 0 || index >= lines.length) return undefined; + const start = Math.max(0, index - radius); + const end = Math.min(lines.length, index + radius + 1); + for (let current = start; current < end; current += 1) { + if (pattern.test(lines[current] ?? '')) return current + 1; + } + return undefined; +} + +function findPatternLineInFunctionBlock( + sourceText: string | undefined, + line: number, + pattern: RegExp, +): number | undefined { + if (!sourceText || line <= 0) return undefined; + const lines = sourceText.split(/\r?\n/); + let depth = 0; + let enteredBlock = false; + for (let current = line - 1; current < lines.length; current += 1) { + const text = lines[current] ?? ''; + if (enteredBlock && pattern.test(text)) return current + 1; + for (const char of text) { + if (char === '{') { + depth += 1; + enteredBlock = true; + } else if (char === '}') { + depth -= 1; + if (enteredBlock && depth <= 0) return undefined; + } + } + } + return undefined; +} + +function escapeRegExp(value: string): string { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} diff --git a/packages/detectors/src/index.ts b/packages/detectors/src/index.ts index a2a1616..1f0309b 100644 --- a/packages/detectors/src/index.ts +++ b/packages/detectors/src/index.ts @@ -15,6 +15,7 @@ export { ASYNC_DETECTORS as defaultAsyncDetectors, createBuiltInAsyncFindingAnalyzers, } from './detectors/async-index.js'; +export { cpuHotspotDetector } from './detectors/cpu-hotspot.js'; export { deepAsyncChainDetector } from './detectors/deep-async-chain.js'; export { externalBufferPressureDetector } from './detectors/external-buffer-pressure.js'; export { hotAsyncContextDetector } from './detectors/hot-async-context.js'; diff --git a/packages/detectors/test/async-detectors.test.ts b/packages/detectors/test/async-detectors.test.ts index 557ae3f..ee594c2 100644 --- a/packages/detectors/test/async-detectors.test.ts +++ b/packages/detectors/test/async-detectors.test.ts @@ -211,6 +211,22 @@ describe('deep-async-chain detector', () => { const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); expect(result.findings).toEqual([]); }); + + it('does not fire on timer-only chains without user or promise evidence', () => { + const records: AsyncOperationRecord[] = []; + for (let i = 0; i < 35; i++) { + const trigger = i === 0 ? 0 : i; + records.push(makeRecord(i + 1, trigger, 'timer', 5, i)); + } + const bundle = makeBundle({ records }); + const pipeline = createAnalysisPipeline({ + kinds: [createAsyncProfileKind()], + findingAnalyzers: [createFindingAnalyzerFromKindScopedDetector(deepAsyncChainDetector)], + }); + const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); + + expect(result.findings).toEqual([]); + }); }); describe('microtask-flood detector', () => { @@ -253,7 +269,7 @@ describe('microtask-flood detector', () => { const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); const finding = result.findings.find((f) => f.id === 'microtask-flood'); - expect(finding?.evidence.file).toBe('file:///app/src/fanout.js'); + expect(finding?.evidence.file).toBe('/app/src/fanout.js'); expect(finding?.evidence.function).toBe('scheduleFanout'); expect(finding?.evidence.extra).toMatchObject({ asyncQuality: 'high', @@ -279,7 +295,7 @@ describe('microtask-flood detector', () => { const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); const finding = result.findings.find((f) => f.id === 'long-await:999'); expect(finding?.evidence.function).toBe('fetchUser'); - expect(finding?.evidence.file).toBe('file:///app/src/users.js'); + expect(finding?.evidence.file).toBe('/app/src/users.js'); expect(finding?.evidence.line).toBe(42); }); @@ -334,7 +350,7 @@ describe('microtask-flood detector', () => { const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); const finding = result.findings.find((f) => f.id === 'long-await:999'); - expect(finding?.evidence.file).toBe('file:///app/src/queue.js'); + expect(finding?.evidence.file).toBe('/app/src/queue.js'); expect(finding?.evidence.function).toBe('queueWork'); expect(finding?.confidence).toBe('medium'); expect(finding?.evidence.extra).toMatchObject({ @@ -404,7 +420,7 @@ describe('hot-async-context detector', () => { const finding = result.findings.find((f) => f.id.startsWith('hot-async-context:')); expect(finding).toBeDefined(); expect(finding?.evidence.function).toBe('requestHandler'); - expect(finding?.evidence.file).toBe('file:///app/src/server.js'); + expect(finding?.evidence.file).toBe('/app/src/server.js'); expect(finding?.severity).toBe('critical'); }); @@ -448,11 +464,11 @@ describe('deep async chain anchoring', () => { const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); const finding = result.findings.find((f) => f.id.startsWith('deep-async-chain:')); - expect(finding?.evidence.file).toBe('file:///app/src/deep.js'); + expect(finding?.evidence.file).toBe('/app/src/deep.js'); expect(finding?.evidence.extra).toMatchObject({ - dominantFile: 'file:///app/src/deep.js', - rootFrame: expect.objectContaining({ file: 'file:///app/src/root.js' }), - deepestFrame: expect.objectContaining({ file: 'file:///app/src/deep.js' }), + dominantFile: '/app/src/deep.js', + rootFrame: expect.objectContaining({ file: '/app/src/root.js' }), + deepestFrame: expect.objectContaining({ file: '/app/src/deep.js' }), asyncQuality: 'high', }); }); diff --git a/packages/detectors/test/findings.test.ts b/packages/detectors/test/findings.test.ts index fb35c8d..fb884bf 100644 --- a/packages/detectors/test/findings.test.ts +++ b/packages/detectors/test/findings.test.ts @@ -1,4 +1,8 @@ import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { pathToFileURL } from 'node:url'; import { buildLanternaReport, type CaptureBundle, @@ -358,6 +362,139 @@ describe('findings – sync-crypto split caller attribution', () => { }); }); +describe('findings – sync-crypto exact source callsite', () => { + const dir = mkdtempSync(join(tmpdir(), 'lanterna-sync-source-')); + const sourcePath = join(dir, 'app.mjs'); + writeFileSync( + sourcePath, + [ + "import { pbkdf2Sync } from 'node:crypto';", + '', + 'function hashPassword(password, salt) {', + " return pbkdf2Sync(password, salt, 100_000, 32, 'sha256').toString('hex');", + '}', + '', + 'function processBatch(size) {', + ' const out = [];', + ' for (let i = 0; i < size; i++) {', + ' out.push(hashPassword(`user-${i}`, `salt-${i}`));', + ' }', + ' return out;', + '}', + '', + 'processBatch(20);', + ].join('\n'), + ); + + const profile: RawCpuProfile = { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: '(anonymous)', + scriptId: '1', + url: pathToFileURL(sourcePath).href, + lineNumber: 0, + columnNumber: 0, + }, + hitCount: 0, + children: [3], + }, + { + id: 3, + callFrame: { + functionName: 'processBatch', + scriptId: '1', + url: pathToFileURL(sourcePath).href, + lineNumber: 6, + columnNumber: 0, + }, + hitCount: 2, + children: [4, 5], + }, + { + id: 4, + callFrame: { + functionName: 'pbkdf2Sync', + scriptId: '2', + url: 'node:crypto', + lineNumber: 100, + columnNumber: 0, + }, + hitCount: 60, + children: [], + }, + { + id: 5, + callFrame: { + functionName: 'hashPassword', + scriptId: '1', + url: pathToFileURL(sourcePath).href, + lineNumber: 2, + columnNumber: 0, + }, + hitCount: 0, + children: [6], + }, + { + id: 6, + callFrame: { + functionName: 'pbkdf2Sync', + scriptId: '2', + url: 'node:crypto', + lineNumber: 100, + columnNumber: 0, + }, + hitCount: 40, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(60).fill(4).concat(Array(40).fill(6)).concat(Array(2).fill(3)), + timeDeltas: [], + }; + + const report = createReport(makeRaw(profile, { target: { cwd: dir } }), { + sampleIntervalMicros: 1000, + deep: false, + command: ['node', sourcePath], + }); + + it('uses the source line that directly calls the sync API as finding location', () => { + const finding = findFindingOrFail( + report, + (f) => f.id === 'sync-crypto-on-hot-path', + 'sync-crypto finding', + ); + + assert.equal(finding.evidence.function, 'hashPassword'); + assert.equal(finding.evidence.file, 'app.mjs'); + assert.equal(finding.evidence.line, 4); + }); + + it('does not add an anonymous inclusive cpu-hotspot when sync-crypto explains the work', () => { + assert.equal( + report.findings.some((finding) => finding.category === 'cpu-hotspot'), + false, + ); + }); + + rmSync(dir, { recursive: true, force: true }); +}); + describe('capture integrity – attach mode clean regression', () => { const report = makeReport('sync-crypto', { captureIntegrity: { @@ -627,6 +764,67 @@ describe('findings – blocking-io', () => { ); assert.equal((f.evidence.extra as Record)?.proofLevel, 'attributed-caller'); }); + + it('detects Node 24 zlib sync work reported as processChunkSync', () => { + const zlibProfile: RawCpuProfile = { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: 'compressPayload', + scriptId: '1', + url: `file://${CWD}/src/zlib.js`, + lineNumber: 10, + columnNumber: 0, + }, + hitCount: 5, + children: [3], + }, + { + id: 3, + callFrame: { + functionName: 'processChunkSync', + scriptId: '0', + url: 'node:zlib', + lineNumber: 0, + columnNumber: 0, + }, + hitCount: 95, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(95).fill(3).concat(Array(5).fill(2)), + timeDeltas: [], + }; + const zlibReport = createReport(makeRaw(zlibProfile), { + sampleIntervalMicros: 1000, + deep: false, + command: ['node', 'app.js'], + }); + + const finding = findFindingOrFail( + zlibReport, + (f) => f.id.startsWith('blocking-io:zlib.'), + 'zlib blocking-io finding', + ); + + assert.match(String((finding.evidence.extra as Record)?.callee), /zlib/); + assert.equal(finding.evidence.function, 'compressPayload'); + }); }); describe('findings – blocking-io false positive suppression', () => { @@ -799,6 +997,77 @@ describe('findings – deopt-loop', () => { assert.equal(f.severity, 'critical'); }); + it('aggregates deopt traces without file and line by function name', () => { + const raw = makeRaw( + { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: 'shapeShift', + scriptId: '1', + url: `file://${CWD}/src/shapes.js`, + lineNumber: 22, + columnNumber: 0, + }, + hitCount: 100, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(100).fill(2), + timeDeltas: [], + }, + { + deopts: [ + { + function: 'shapeShift', + file: '', + line: 0, + reason: 'wrong map', + bailoutType: 'soft', + count: 3, + }, + { + function: 'shapeShift', + file: '', + line: 0, + reason: 'not a Smi', + bailoutType: 'soft', + count: 3, + }, + ], + }, + ); + const deoptReport = createReport(raw, { + sampleIntervalMicros: 1000, + deep: true, + command: ['node', 'app.js'], + }); + + const finding = findFindingOrFail( + deoptReport, + (f) => f.id.startsWith('deopt-loop:shapeShift'), + 'deopt-loop finding without file/line', + ); + + assert.equal((finding.evidence.extra as Record).count, 6); + assert.equal(finding.evidence.file, 'src/shapes.js'); + }); + it('does not emit deopt-loop without --deep mode', () => { const noDeepReport = createReport( makeRaw( @@ -1069,6 +1338,247 @@ describe('findings – json-on-hot-path', () => { assert.match(String((f.evidence.extra as Record)?.callee), /JSON\.stringify/); assert.equal((f.evidence.extra as Record)?.proofLevel, 'attributed-caller'); }); + + it('detects JSON work inlined into a hot user frame when source contains JSON calls', () => { + const dir = mkdtempSync(join(tmpdir(), 'lanterna-json-inline-')); + try { + const sourcePath = join(dir, 'http.js'); + writeFileSync( + sourcePath, + [ + 'export function serializeResponse(payload) {', + ' return JSON.stringify(payload);', + '}', + ].join('\n'), + ); + const inlineProfile: RawCpuProfile = { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: 'serializeResponse', + scriptId: '1', + url: pathToFileURL(sourcePath).href, + lineNumber: 1, + columnNumber: 0, + }, + hitCount: 100, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(100).fill(2), + timeDeltas: [], + }; + const inlineReport = createReport( + makeRaw(inlineProfile, { + target: { + pid: 99999, + nodeVersion: 'v24.0.0', + v8Version: '12.0.0', + platform: 'linux', + arch: 'x64', + cwd: dir, + }, + }), + { + sampleIntervalMicros: 1000, + deep: false, + command: ['node', 'http.js'], + }, + ); + + const finding = findFindingOrFail( + inlineReport, + (f) => f.id === 'json-on-hot-path:JSON.stringify', + 'inlined json-on-hot-path finding', + ); + + assert.equal(finding.evidence.function, 'serializeResponse'); + assert.equal((finding.evidence.extra as Record).callee, 'JSON.stringify'); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('does not report JSON for a hot user frame when JSON appears only elsewhere in the file', () => { + const dir = mkdtempSync(join(tmpdir(), 'lanterna-json-nearby-')); + try { + const sourcePath = join(dir, 'worker.js'); + writeFileSync( + sourcePath, + [ + 'export function compute(payload) {', + ' let total = 0;', + ' for (let i = 0; i < payload.length; i += 1) total += payload[i];', + ' return total;', + '}', + '', + 'export function logPayload(payload) {', + ' return JSON.stringify(payload);', + '}', + ].join('\n'), + ); + const profileWithUnrelatedJson: RawCpuProfile = { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: 'compute', + scriptId: '1', + url: pathToFileURL(sourcePath).href, + lineNumber: 0, + columnNumber: 0, + }, + hitCount: 100, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(100).fill(2), + timeDeltas: [], + }; + const unrelatedReport = createReport( + makeRaw(profileWithUnrelatedJson, { + target: { + pid: 99999, + nodeVersion: 'v24.0.0', + v8Version: '12.0.0', + platform: 'linux', + arch: 'x64', + cwd: dir, + }, + }), + { + sampleIntervalMicros: 1000, + deep: false, + command: ['node', 'worker.js'], + }, + ); + + assert.equal( + unrelatedReport.findings.some((f) => f.id.startsWith('json-on-hot-path:')), + false, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('does not report source-only JSON for a wrapper frame dominated by child work', () => { + const dir = mkdtempSync(join(tmpdir(), 'lanterna-json-wrapper-')); + try { + const sourcePath = join(dir, 'worker.js'); + writeFileSync( + sourcePath, + [ + 'export function expensiveWork() {', + ' return 42;', + '}', + '', + 'export function wrapper(payload) {', + ' const value = expensiveWork();', + ' console.error(JSON.stringify({ value, payload }));', + ' return value;', + '}', + ].join('\n'), + ); + const wrapperProfile: RawCpuProfile = { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: 'wrapper', + scriptId: '1', + url: pathToFileURL(sourcePath).href, + lineNumber: 4, + columnNumber: 0, + }, + hitCount: 3, + children: [3], + }, + { + id: 3, + callFrame: { + functionName: 'expensiveWork', + scriptId: '1', + url: pathToFileURL(sourcePath).href, + lineNumber: 0, + columnNumber: 0, + }, + hitCount: 97, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(97).fill(3).concat(Array(3).fill(2)), + timeDeltas: [], + }; + const wrapperReport = createReport( + makeRaw(wrapperProfile, { + target: { + pid: 99999, + nodeVersion: 'v24.0.0', + v8Version: '12.0.0', + platform: 'linux', + arch: 'x64', + cwd: dir, + }, + }), + { + sampleIntervalMicros: 1000, + deep: false, + command: ['node', 'worker.js'], + }, + ); + + assert.equal( + wrapperReport.findings.some((f) => f.id.startsWith('json-on-hot-path:')), + false, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); }); describe('findings – node-modules-hotspot', () => { @@ -1257,7 +1767,7 @@ describe('findings – node-modules-hotspot selection uses inclusive cost', () = }); }); -describe('summary – topUserHotspot', () => { +describe('findings – cpu-hotspot direct user code', () => { const profile: RawCpuProfile = { nodes: [ { @@ -1297,17 +1807,96 @@ describe('summary – topUserHotspot', () => { command: ['node', 'app.js'], }); - it('exposes a dominant user-code hotspot in summary instead of findings', () => { + it('emits an actionable cpu-hotspot for self-heavy user code', () => { const cpuProfile = getCpuProfile(report); - assert.equal( - report.findings.some((f) => f.id.startsWith('cpu-bound-user-hotspot:')), - false, + const finding = findFindingOrFail( + report, + (f) => f.category === 'cpu-hotspot', + 'cpu-hotspot finding', ); + assert.equal(finding.proofLevel, 'direct-sample'); + assert.equal(finding.confidence, 'high'); + assert.equal((finding.evidence.extra as Record)?.mode, 'self'); + assert.match(finding.evidence.function, /computeRanking/); + assert.match(cpuProfile.summary.topCpuCulprit?.function ?? '', /computeRanking/); assert.match(cpuProfile.summary.topUserHotspot?.function ?? '', /computeRanking/); assert.equal(cpuProfile.summary.topUserHotspot?.totalPct, 100); }); }); +describe('findings – cpu-hotspot inclusive fallback', () => { + const profile: RawCpuProfile = { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: 'handleRequest', + scriptId: '1', + url: `file://${CWD}/src/handler.js`, + lineNumber: 12, + columnNumber: 0, + }, + hitCount: 0, + children: [3], + }, + { + id: 3, + callFrame: { + functionName: 'unknownBuiltinWork', + scriptId: '0', + url: 'node:internal/custom', + lineNumber: 99, + columnNumber: 0, + }, + hitCount: 100, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(100).fill(3), + timeDeltas: [], + }; + + const report = createReport(makeRaw(profile), { + sampleIntervalMicros: 1000, + deep: false, + command: ['node', 'app.js'], + }); + + it('emits inclusive-only callers as hypotheses, not direct body hotspots', () => { + const finding = findFindingOrFail( + report, + (f) => f.category === 'cpu-hotspot', + 'cpu-hotspot finding', + ); + + assert.equal(finding.evidence.function, 'handleRequest'); + assert.equal(finding.proofLevel, 'heuristic'); + assert.equal(finding.confidence, 'medium'); + assert.equal((finding.evidence.extra as Record)?.mode, 'inclusive-entry'); + assert.equal( + (finding.evidence.extra as Record)?.proofLevel, + 'inclusive-user-entry', + ); + assert.match(finding.why, /caller\/context lead/); + assert.equal(getCpuProfile(report).summary.topCpuCulprit, undefined); + assert.match(getCpuProfile(report).summary.topRequestEntry?.function ?? '', /handleRequest/); + }); +}); + describe('summary – topUserHotspot selection uses inclusive cost', () => { const profile: RawCpuProfile = { nodes: [ @@ -1380,9 +1969,9 @@ describe('summary – topUserHotspot selection uses inclusive cost', () => { describe('findings – cpu-bound-user-hotspot suppression', () => { const report = makeReport('sync-crypto'); - it('does not emit cpu-bound-user-hotspot when a more specific detector explains the work', () => { + it('does not emit cpu-hotspot when a more specific detector explains the work', () => { assert.equal( - report.findings.some((f) => f.id.startsWith('cpu-bound-user-hotspot:')), + report.findings.some((f) => f.category === 'cpu-hotspot'), false, ); assert.match(getCpuProfile(report).summary.topUserHotspot?.function ?? '', /hashPassword/); diff --git a/packages/detectors/test/memory-detectors.test.ts b/packages/detectors/test/memory-detectors.test.ts index b627eea..2a5408f 100644 --- a/packages/detectors/test/memory-detectors.test.ts +++ b/packages/detectors/test/memory-detectors.test.ts @@ -118,12 +118,45 @@ function growingSeries(slopeBytesPerMs: number, externalMB = 1, count = 25): Mem return out; } +function rssOnlyGrowthSeries(slopeBytesPerMs: number, count = 25): MemoryUsageSample[] { + const out: MemoryUsageSample[] = []; + for (let i = 0; i < count; i++) { + const atMs = i * 200; + out.push({ + atMs, + rss: 100 * MB + atMs * slopeBytesPerMs, + heapTotal: 128 * MB, + heapUsed: 48 * MB, + external: 1 * MB, + arrayBuffers: 0.5 * MB, + }); + } + return out; +} + +function externalGrowthSeries(slopeBytesPerMs: number, count = 25): MemoryUsageSample[] { + const out: MemoryUsageSample[] = []; + for (let i = 0; i < count; i++) { + const atMs = i * 200; + const external = 1 * MB + atMs * slopeBytesPerMs; + out.push({ + atMs, + rss: 100 * MB + atMs * slopeBytesPerMs, + heapTotal: 64 * MB, + heapUsed: 48 * MB, + external, + arrayBuffers: external / 2, + }); + } + return out; +} + describe('memory-growth detector', () => { it('fires `warning` for ~1 MB/s RSS growth and `critical` for ~5 MB/s', () => { const bundle = makeBundle({ samplingProfile: singleAllocatorProfile('alloc', 'file:///app/src/a.js', 1, 1024), // ~6 MB/s RSS growth, comfortably above the critical threshold (5 MB/s). - memoryUsageSamples: growingSeries(6 * 1024), + memoryUsageSamples: externalGrowthSeries(6 * 1024), }); const pipeline = createAnalysisPipeline({ @@ -141,7 +174,7 @@ describe('memory-growth detector', () => { it('recommends Lanterna heap snapshot analysis before external heap tooling', () => { const bundle = makeBundle({ samplingProfile: singleAllocatorProfile('alloc', 'file:///app/src/a.js', 1, 1024), - memoryUsageSamples: growingSeries(6 * 1024), + memoryUsageSamples: externalGrowthSeries(6 * 1024), }); const pipeline = createAnalysisPipeline({ kinds: [createMemoryProfileKind()], @@ -167,6 +200,20 @@ describe('memory-growth detector', () => { const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); expect(result.findings).toEqual([]); }); + + it('does not report RSS-only growth when heap and external memory stay flat', () => { + const bundle = makeBundle({ + samplingProfile: singleAllocatorProfile('churn', 'file:///app/src/churn.js', 1, 1024), + memoryUsageSamples: rssOnlyGrowthSeries(12 * 1024), + }); + const pipeline = createAnalysisPipeline({ + kinds: [createMemoryProfileKind()], + findingAnalyzers: [createFindingAnalyzerFromKindScopedDetector(memoryGrowthDetector)], + }); + const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); + + expect(result.findings.some((finding) => finding.id === 'memory-growth:rss')).toBe(false); + }); }); describe('large-allocator detector', () => { @@ -553,4 +600,54 @@ describe('alloc-in-hot-path detector', () => { const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); expect(result.findings).toEqual([]); }); + + it('ignores Node internal frames even when CPU and allocation keys match', () => { + const cpuProfile: RawCpuProfile = { + nodes: [ + { + id: 1, + callFrame: { + functionName: '(root)', + scriptId: '0', + url: '', + lineNumber: -1, + columnNumber: -1, + }, + hitCount: 0, + children: [2], + }, + { + id: 2, + callFrame: { + functionName: 'processChunkSync', + scriptId: '0', + url: 'node:zlib', + lineNumber: 0, + columnNumber: 0, + }, + hitCount: 100, + children: [], + }, + ], + startTime: 1000000, + endTime: 2000000, + samples: Array(100).fill(2), + timeDeltas: [], + }; + const bundle = makeBundle({ + samplingProfile: singleAllocatorProfile('processChunkSync', 'node:zlib', 0, 9000), + memoryUsageSamples: growingSeries(0), + cpuProfile, + }); + const pipeline = createAnalysisPipeline({ + kinds: [ + createCpuProfileKind({ readStderrSoFar: () => '', sampleIntervalMicros: 1000 }), + createMemoryProfileKind(), + ], + findingAnalyzers: [createFindingAnalyzerFromKindScopedDetector(allocInHotPathDetector)], + }); + const result = pipeline.run(bundle, { command: ['node', 'app.js'], mode: 'spawn' }); + + expect(result.findings).toEqual([]); + }); }); diff --git a/skills/lanterna-profiler/SKILL.md b/skills/lanterna-profiler/SKILL.md index 1f6eb52..6179290 100644 --- a/skills/lanterna-profiler/SKILL.md +++ b/skills/lanterna-profiler/SKILL.md @@ -40,8 +40,8 @@ Lanterna produces agent-facing Node.js profiling reports. Your job is not to sum - Classify each lead as `proven/actionable`, `hypothesis needing source confirmation`, `hypothesis needing another measurement`, or `non-representative signal requiring rerun`. Treat `rerun_required: true` as the report-level signal for that last class, then use `blocking_caveats`, `degrading_caveats`, and any `decision = rerun` finding to explain why. 4. **Diagnose by subsystem** (see per-kind references for interpretation rules: [cpu-profiling.md](references/cpu-profiling.md), [memory-profiling.md](references/memory-profiling.md), [async-profiling.md](references/async-profiling.md)) - - CPU: check top user hotspot, dependency/runtime hotspot with user caller, sync crypto, blocking I/O, JSON/serialization, require/import in hot path, deopt loops, and GC-correlated CPU. - - Event loop: only claim causality when event-loop timing is available and hotspot correlation is strong. + - CPU: check `top_cpu_culprit` first for the self-heavy line, then `top_request_entry` / `top_user_hotspot` for caller context, dependency/runtime hotspot with user caller, sync crypto, blocking I/O, JSON/serialization, require/import in hot path, generic `cpu-hotspot:*`, deopt loops, and GC-correlated CPU. + - Event loop: only claim causality when event-loop timing is available and hotspot correlation is strong. If `event-loop-stall` is rendered with `hotspot-fallback`, treat the file/line as the best CPU lead, not proof that it caused every stall. - Memory: distinguish allocation churn, JS heap growth, RSS/off-heap growth, external Buffer pressure, snapshot-retained growth, and weak short-window slopes. - Async/I/O: distinguish CPU work from long awaits, deep chains, orphan resources, low concurrency, external service waits, and attach-mode partial capture. - Architecture/dependency/environment: separate app code defects, dependency hotspots, architectural bottlenecks, insufficient load, machine/container limits, and capture artifacts. @@ -50,6 +50,7 @@ Lanterna produces agent-facing Node.js profiling reports. Your job is not to sum - Open `read-first` files before proposing changes. - Treat `inspect-lead` as confirmation targets, not patch targets. - For `node_modules`, `node:`, native, generated output, or virtual source-map frames, follow the rendered `user_caller` to editable user code. + - For `cpu-hotspot:*`, inspect `evidence.extra.mode`: `self` means inspect the reported function body directly; `inclusive-entry` means inspect callees and hot stacks before blaming the wrapper. - Confirm whether hot code is on the critical request/job path, repeated per request, unbounded, synchronous, allocation-heavy, or missing backpressure/concurrency control. 6. **Iterate** diff --git a/skills/lanterna-profiler/references/analysis-output.md b/skills/lanterna-profiler/references/analysis-output.md index 9945810..73ef464 100644 --- a/skills/lanterna-profiler/references/analysis-output.md +++ b/skills/lanterna-profiler/references/analysis-output.md @@ -52,6 +52,9 @@ Start from `$LANTERNA report report.json --format agent --output report.agent.md - Lead with quality when confidence is not high. - Include the specific report observation: finding id, decision, proof, metric, threshold, hotspot, allocator, async operation, caveat, or kind review line. +- For CPU reports, separate the self-heavy culprit from caller context when both are present: `top_cpu_culprit` answers which function body burned CPU; `top_request_entry` / `top_user_hotspot` explains the request or caller path. +- Treat `cpu-hotspot:*` according to `evidence.extra.mode`: `self` can be actionable direct CPU evidence when quality and source inspection support it; `inclusive-entry` is a caller/context hypothesis until callees or hot stacks confirm the expensive body. +- For `event-loop-stall` with `hotspot-fallback`, say event-loop lag was observed but causality is weaker; use the fallback frame as the next source lead or rerun target. - Include code observations only after reading the relevant files. Name the file/function and why it confirms or weakens the lead. - Keep `user_caller` confidence, support percentage, and generated/source-map fallback visible when those details affect actionability. - Treat `decision = actionable` as eligible for a recommendation only after source inspection. diff --git a/skills/lanterna-profiler/references/async-profiling.md b/skills/lanterna-profiler/references/async-profiling.md index 2da6486..a7a7cb3 100644 --- a/skills/lanterna-profiler/references/async-profiling.md +++ b/skills/lanterna-profiler/references/async-profiling.md @@ -89,6 +89,8 @@ Prefer findings that the `Findings.decision` column marks actionable, with high Await sites, resource origins, and async findings may carry a resolved `source` object. Prefer `source.file:source.line` over the raw `file:line` — raw coordinates point at compiled JS, `source.*` at the original TypeScript or bundled source. Fall back when `source` is missing. Use `source.name` for anonymous frames. Treat virtual paths (`webpack://`, `vite:/`) as bundler labels, not editable files, unless they resolve on disk. Quality gate: `meta.captureIntegrity.sourceMaps`; when `applicable !== false` and `coverage` is low, treat mapped positions as hints. `applicable: false` means plain JS without source maps, not degraded mapping. +When V8/CDP supplies `file://` URLs for async frames, Lanterna normalizes public report paths to normal filesystem paths before grouping hot files, chains, and finding evidence. Virtual bundler URLs remain virtual. + For analysis, use the rendered agent location first. Consult raw async frames only as a targeted JSON lookup when `Kind Review` does not render the specific frame or `userCaller` you need. In `## Files To Read First`, async rows use specific reasons such as `top async hot file`, `long async operation`, `long async operation caller`, `async hot file`, and `async CPU attribution`. Prefer `read-first` user callers for external async work; treat `inspect-lead` rows as places to confirm the async chain before editing. Pseudo/runtime async frames are filtered out of Kind Review tables unless an editable user caller can anchor the operation. diff --git a/skills/lanterna-profiler/references/common-pitfalls.md b/skills/lanterna-profiler/references/common-pitfalls.md index f8be461..0e1b752 100644 --- a/skills/lanterna-profiler/references/common-pitfalls.md +++ b/skills/lanterna-profiler/references/common-pitfalls.md @@ -31,6 +31,37 @@ const hash = await pool.run({ pw, salt }); --- +## Plain user-code CPU hotspots + +`cpu-hotspot:*` means Lanterna did not match a known API anti-pattern. When `evidence.extra.mode === "self"`, the reported user function itself is burning CPU. When `mode === "inclusive-entry"`, the reported function is the caller/context for downstream CPU and its callees need inspection first. + +**Common causes:** +- nested loops over request-size data; +- repeated sorting, filtering, regex, or scoring per request; +- recomputing stable values instead of caching; +- parsing or transforming large payloads in one synchronous block; +- doing CPU-bound work that belongs in a worker pool. + +**Fix pattern:** +```js +// BAD — recomputes for every request +function score(items, query) { + return items + .map((item) => expensiveScore(item, query)) + .sort((a, b) => b.score - a.score) + .slice(0, 20); +} + +// GOOD — reduce input, cache stable pieces, or offload the expensive part +const normalized = new Map(); +function normalizedItem(item) { + if (!normalized.has(item.id)) normalized.set(item.id, precompute(item)); + return normalized.get(item.id); +} +``` + +--- + ## Garbage collection pressure Too many short-lived allocations trigger frequent minor GCs (scavenge). Too many surviving objects → major GC (mark-compact) pauses. diff --git a/skills/lanterna-profiler/references/cpu-profiling.md b/skills/lanterna-profiler/references/cpu-profiling.md index 4d90d28..af3da80 100644 --- a/skills/lanterna-profiler/references/cpu-profiling.md +++ b/skills/lanterna-profiler/references/cpu-profiling.md @@ -18,7 +18,7 @@ The current built-in kind id and report section key are both `cpu`, so CPU analy These are targeted JSON lookup paths. For analysis, read the agent report first and use its frontmatter, `## Findings` table, `## Finding N` blocks, `Findings.decision` column, `Kind Review`, and `Files To Read First` sections as the contract. -- `profiles.cpu.summary`: on-CPU ratio, idle ratio, category ratios, top user hotspot. +- `profiles.cpu.summary`: on-CPU ratio, idle ratio, category ratios, `topCpuCulprit`, `topRequestEntry`, and backward-compatible `topUserHotspot`. - `profiles.cpu.hotspots[]`: aggregated frames by `(file, function, line)`. - `profiles.cpu.hotStacks[]`: frequent full sampled stacks, leaf-to-root. - `profiles.cpu.hotStackClusters[]`: hot stacks grouped by user-code anchor. @@ -54,6 +54,7 @@ When the needed signal is degraded, say so explicitly and avoid strong causal la - `measurementBasis === "heartbeats" | "both"`: timed heartbeats are available. - `confidence === "low" | "none"`: name suspects, but do not assert root cause. - Treat a specific hotspot as causal only when its correlation `confidence === "high"` and the report has timed stall intervals. +- For `event-loop-stall`, check `evidence.extra.proofLevel`: `aggregate-correlation` is stronger; `hotspot-fallback` means lag crossed the threshold but Lanterna anchored the finding to the hottest user CPU frame because no stall window had strong attribution. Prefer `eventLoop.correlatedHotspots[]` over generic hotspot guesses. If `correlatedHotspots[].overlapPct` is absent or weak, frame the result as a hypothesis. @@ -67,6 +68,7 @@ Start with the `## Findings` table, which renders findings in priority order. Va - Compare `measurements.observed` to `measurements.thresholds`; a large threshold ratio is stronger than severity alone. - Patch mechanically only when attribution is high-confidence and `remediation` is populated. - For attributed findings (`blocking-io`, `sync-crypto`, `require-in-hot-path`, `node-modules-hotspot`, `json-on-hot-path`), do not patch the user caller when `evidence.extra.attributionConfidence === "low"`. +- For `cpu-hotspot:*`, check `evidence.extra.mode`. `self` is direct user-code CPU evidence: inspect the reported function body for loops, repeated transformations, CPU-bound scoring/parsing, cache misses, or work that belongs in `worker_threads` / Piscina. `inclusive-entry` is lower-confidence caller evidence: inspect callees and hot stacks first. - For legacy reports without top-level `finding.proofLevel`, fall back to `evidence.extra.proofLevel`. - If `categoryTotalPct` is much larger than `calleeTotalPct`, prefer a structural fix for the family of calls over replacing one call site. @@ -78,7 +80,7 @@ Strongest actionable lead: ## Source Positions -Every CPU frame may carry an optional `source` object resolved from a source map: `hotspots[].source`, `summary.topUserHotspot.source`, `hotStacks[].frames[].source`, `hotStackClusters[].anchor.source`, and `findings[].evidence.source`. +Every CPU frame may carry an optional `source` object resolved from a source map: `hotspots[].source`, `summary.topCpuCulprit.source`, `summary.topRequestEntry.source`, `summary.topUserHotspot.source`, `hotStacks[].frames[].source`, `hotStackClusters[].anchor.source`, and `findings[].evidence.source`. - When `source` is present, cite `source.file:source.line` (the original TypeScript / bundled source) — do not cite `file:line` (the compiled `dist/` output). - When `source` is absent, fall back to `file:line` (no map was found for that frame — common for `node:` builtins or stripped bundles). @@ -90,7 +92,7 @@ Every CPU frame may carry an optional `source` object resolved from a source map 1. Read agent frontmatter. 2. Summarize actionable findings from `## Findings` table, `## Finding N` blocks, and `Findings.decision` column. -3. Use `## Kind Review` for top user-relevant hotspots, even when no detector fired. +3. Use `## Kind Review` for top user-relevant hotspots, even when no detector fired. Prefer `top_cpu_culprit` for "which line burns CPU?" and `top_request_entry` / `top_user_hotspot` for the caller or request context. 4. Use `## Files To Read First` as a table, not a plain list: `read-first` rows are the source-reading queue, `inspect-lead` rows need confirmation, and `supporting-context` rows explain the sampled stack. Generated output fallbacks (`dist/`, `build/`, `.next/`, etc.) are `inspect-lead` rows until resolved back to editable source. 5. If frontmatter has `rerun_required: true`, explain the caveat or `decision = rerun` finding and request a better capture before patching. 6. Summarize GC only when pauses or ratios are materially high and supported by the agent report or a targeted JSON lookup. diff --git a/skills/lanterna-profiler/references/detectors-and-plugins.md b/skills/lanterna-profiler/references/detectors-and-plugins.md index d502e7b..5228afc 100644 --- a/skills/lanterna-profiler/references/detectors-and-plugins.md +++ b/skills/lanterna-profiler/references/detectors-and-plugins.md @@ -87,6 +87,21 @@ Use top-level `confidence` and `proofLevel` when the detector can characterize i Set `confidence` to `high`, `medium`, or `low` based on sample volume, attribution strength, and whether the detector points to a direct edit location. +Finding analyzers run incrementally: after each analyzer completes, the in-progress `snapshot.findings` contains findings emitted so far. Later detectors can use that shared state to avoid duplicates or defer to stronger evidence. The built-in `cpu-hotspot` detector relies on this to suppress generic CPU findings when `sync-crypto`, `blocking-io`, `json-on-hot-path`, `node-modules-hotspot`, or `require-in-hot-path` already explains the frame. + +## Built-In CPU Fallback + +`cpu-hotspot:` is the generic CPU detector for plain user-code hotspots. It emits when a user frame crosses the configured self/inclusive CPU gates and no more specific CPU detector has already claimed it. Use `evidence.extra.mode` to interpret it: `self` is a direct body hotspot for custom loops, scoring functions, transformations, parsers, and other CPU-bound code; `inclusive-entry` is a caller/context lead for downstream CPU that still needs callees or hot stacks. + +The threshold block is `DETECTOR_THRESHOLDS.cpuHotspot`: + +- `minSelfPct`: self-heavy user-code gate. +- `minTotalPct`: inclusive fallback when no self-heavy candidate exists; emits `mode: "inclusive-entry"` and top-level `proofLevel: "heuristic"`. +- `criticalPct`: severity escalation. +- `maxFindings`: cap to avoid noisy generic output. + +`event-loop-stall` also has two evidence modes in `evidence.extra.proofLevel`: `aggregate-correlation` for strong stall-window attribution, and `hotspot-fallback` when the event-loop lag is real but the source location is only the hottest user CPU lead. + ## Multi-Kind Contract - `ProfileKind.id`: CLI/runtime identity and `meta.kinds.`; it appears in `meta.profileKinds[]` only when capture data was produced. diff --git a/skills/lanterna-profiler/references/report-schema.md b/skills/lanterna-profiler/references/report-schema.md index ebf0016..43a8c78 100644 --- a/skills/lanterna-profiler/references/report-schema.md +++ b/skills/lanterna-profiler/references/report-schema.md @@ -2,6 +2,8 @@ Use this only for targeted JSON field lookup after reading the agent report. The agent report drives the interactive investigation; this schema is a fallback when a specific field is needed and not rendered. For CPU-specific interpretation, see [cpu-profiling.md](cpu-profiling.md). +Current built-in report schema version is `2.0.0`. Schema v2 stores built-in data under `profiles..*` and per-kind metadata under `meta.kinds..*`. Additive optional fields can appear without changing the major shape. + For agent analysis, capture in JSON (`--format json --output report.json`), then render the agent contract with `$LANTERNA report report.json --format agent --output report.agent.md` (set `$LANTERNA` per the SKILL prefix block), and read that output in skill order. Do not start with `--format text`, `--format markdown`, or raw JSON. The JSON paths below are a schema dictionary for targeted clarification only when the agent report omits a field you need. The agent format renders the contract sections: frontmatter with `rerun_required`, `## Findings` table, `## Finding N` blocks, `Findings.decision` column, `Kind Review`, and `Files To Read First`. ## Top-Level Shape @@ -51,7 +53,7 @@ Common fields: | Field | Meaning | |---|---| -| `schemaVersion` | Report schema version | +| `schemaVersion` | Report schema version, currently `2.0.0` | | `nodeVersion`, `v8Version`, `platform`, `arch` | Target runtime metadata | | `pid`, `cwd`, `startedAt`, `durationMs` | Capture context | | `command` | Spawned command, or `[]` in attach mode | @@ -75,13 +77,13 @@ Important global integrity flags: ### `SourceLocation` -Optional field on every frame-bearing object (`hotspots[]`, `summary.topUserHotspot`, `hotStacks[].frames[]`, `hotStackClusters[].anchor`, `hotAllocators[]`, async frame-bearing entries such as `topOperations[]` / `chains[]` / `orphans[]`, `findings[].evidence`): +Optional field on every frame-bearing object (`hotspots[]`, `summary.topCpuCulprit`, `summary.topRequestEntry`, `summary.topUserHotspot`, `hotStacks[].frames[]`, `hotStackClusters[].anchor`, `hotAllocators[]`, async frame-bearing entries such as `topOperations[]` / `chains[]` / `orphans[]`, `findings[].evidence`): ```json { "file": "src/server.ts", "line": 42, "column": 18, "name": "handleRequest" } ``` -- `file` — relative to capture cwd when on disk; otherwise the raw map source URL (`webpack://app/src/...`, `vite:/src/...`). +- `file` — relative to capture cwd when on disk; otherwise the raw map source URL (`webpack://app/src/...`, `vite:/src/...`). Public async frame paths from V8/CDP `file://` URLs are normalized to normal filesystem paths when possible. - `line` — 1-based. - `column` — 1-based, optional. - `name` — original symbol name from the map's `names` array, useful when the generated `function` is `(anonymous)`. @@ -127,7 +129,11 @@ Location rule: use the agent report's rendered `User caller` first. If targeted { "profiles": { "cpu": { - "summary": {}, + "summary": { + "topCpuCulprit": {}, + "topRequestEntry": {}, + "topUserHotspot": {} + }, "hotspots": [], "hotStacks": [], "gc": {}, @@ -212,6 +218,8 @@ Common fields: Rules: - Read the agent report's `Source` and `Generated fallback` before proposing code changes. +- `cpu-hotspot:` is the built-in generic CPU fallback for hot user code not explained by a more specific detector. `evidence.extra.mode: "self"` is a direct source-inspection lead. `mode: "inclusive-entry"` is a lower-confidence caller/context lead and should be confirmed through callees or hot stacks. +- `event-loop-stall` can include `evidence.extra.proofLevel: "aggregate-correlation"` or `"hotspot-fallback"`. The fallback form anchors lag to the hottest user CPU frame when direct stall-window attribution is not strong enough. - In agent reports, `## Findings` table may include `User caller: () [confidence, support X%]`. Use that location before dependency/runtime frames, but only treat high-confidence user callers as potentially actionable. - `Files To Read First` is a table of `location`, `reason`, `source`, `signal`, and `decision`. It excludes `node_modules`, `node:`, pnpm store, virtual source-map paths, pseudo-files, and runtime locations unless an editable user-code `userCaller` location is available. Generated output folders such as `dist/`, `build/`, `out/`, `.next/`, `.nuxt/`, `.svelte-kit/`, `.vite/`, and `coverage/` are rendered as `generated output fallback` with `decision = inspect-lead`, not `read-first`. Treat `read-first` as the source-reading queue, `inspect-lead` as a confirmation lead, and `supporting-context` as surrounding evidence. Reasons distinguish finding locations, dependency callers, runtime callers, CPU hotspots/stacks, memory allocators, and async leads such as `top async hot file`, `long async operation`, and `async CPU attribution`. - Use `confidence`, `proofLevel`, `measurements`, and `priority`, not severity alone.