Context
NetworkLogSource.parse currently caps rendered events at MAX_DISPLAY_LINES = 5_000, keeping only the most recent N via takeLast. For typical web sessions (a few hundred to low thousands of events) this is invisible; for a 30-minute test on a chatty SPA hitting tens of thousands of requests, the panel only shows the last 5_000 and the rest are silently dropped from the view.
The cap was kept deliberately for the initial wiring; this issue tracks lifting it.
Why a hard cap exists
Two reasons:
LazyColumn stays responsive only up to a point; rendering 50k JSON-derived rows starts to stutter on scrub.parse() materializes the full filtered list before slicing (lineSequence().filter { … }.toList().takeLast(N)) — the upper bound on memory cost is set by the cap, not by file size.
Strategies to evaluate
A) Streaming parse with a ring buffer.
Replace lineSequence().filter{}.toList().takeLast(N) with a streaming pass that maintains an ArrayDeque<ParsedLogLine> of size N. O(1) memory per line during parse instead of O(file size). Doesn't change the cap, just stops doubling memory transiently. Cheapest win; ~5 LOC.
B) Range-based windowed parse.
Parse only events whose timestampMs falls within the user's currently scrubbed window (plus a buffer on either side). Re-parse on scrub. Requires keeping the file content in memory but materializing only the visible slice. Cap effectively becomes "N events around the cursor" rather than "last N events ever." Best UX for long sessions.
C) Pre-built index file.
At capture-write time (or first-load time), build a sidecar index of (byteOffset, timestampMs) per event. Range queries become disk-seek + parse-only-visible-bytes. Rules out re-parsing the whole file on every scrub. More machinery; pairs naturally with future response-body capture work.
D) Two-pane: counter header + paginated detail.
The header always shows the true total ("12,432 events captured"). The list shows a window with paging controls. No data is hidden, but interaction shifts from "scroll forever" to "page through ranges." Low-effort, high-clarity.
Recommendation
Start with A as a free win (no API change, no behavior change for sessions under the cap), then evaluate B as a v2 that genuinely supports long sessions. C depends on broader on-disk schema changes. D is a UX call — nice to have alongside any of the above.
Estimated diff
A: ~10 LOC. B: ~80 LOC. C: ~200 LOC + on-disk schema. D: ~50 LOC of Compose.
Context
NetworkLogSource.parsecurrently caps rendered events atMAX_DISPLAY_LINES = 5_000, keeping only the most recent N viatakeLast. For typical web sessions (a few hundred to low thousands of events) this is invisible; for a 30-minute test on a chatty SPA hitting tens of thousands of requests, the panel only shows the last 5_000 and the rest are silently dropped from the view.The cap was kept deliberately for the initial wiring; this issue tracks lifting it.
Why a hard cap exists
Two reasons:
LazyColumnstays responsive only up to a point; rendering 50k JSON-derived rows starts to stutter on scrub.parse()materializes the full filtered list before slicing (lineSequence().filter { … }.toList().takeLast(N)) — the upper bound on memory cost is set by the cap, not by file size.Strategies to evaluate
A) Streaming parse with a ring buffer.
Replace
lineSequence().filter{}.toList().takeLast(N)with a streaming pass that maintains anArrayDeque<ParsedLogLine>of size N. O(1) memory per line during parse instead of O(file size). Doesn't change the cap, just stops doubling memory transiently. Cheapest win; ~5 LOC.B) Range-based windowed parse.
Parse only events whose
timestampMsfalls within the user's currently scrubbed window (plus a buffer on either side). Re-parse on scrub. Requires keeping the file content in memory but materializing only the visible slice. Cap effectively becomes "N events around the cursor" rather than "last N events ever." Best UX for long sessions.C) Pre-built index file.
At capture-write time (or first-load time), build a sidecar index of
(byteOffset, timestampMs)per event. Range queries become disk-seek + parse-only-visible-bytes. Rules out re-parsing the whole file on every scrub. More machinery; pairs naturally with future response-body capture work.D) Two-pane: counter header + paginated detail.
The header always shows the true total ("12,432 events captured"). The list shows a window with paging controls. No data is hidden, but interaction shifts from "scroll forever" to "page through ranges." Low-effort, high-clarity.
Recommendation
Start with A as a free win (no API change, no behavior change for sessions under the cap), then evaluate B as a v2 that genuinely supports long sessions. C depends on broader on-disk schema changes. D is a UX call — nice to have alongside any of the above.
Estimated diff
A: ~10 LOC. B: ~80 LOC. C: ~200 LOC + on-disk schema. D: ~50 LOC of Compose.