Skip to content

Stabilize streams memory and read performance#3

Merged
sorenbs merged 24 commits intomainfrom
codex/stream-profiles
Apr 6, 2026
Merged

Stabilize streams memory and read performance#3
sorenbs merged 24 commits intomainfrom
codex/stream-profiles

Conversation

@sorenbs
Copy link
Copy Markdown
Member

@sorenbs sorenbs commented Apr 2, 2026

Summary

  • stabilize ingest, upload, WAL GC, and delete-time cleanup so stale acceleration state and retained WAL do not poison long-running servers
  • improve cold and warm read performance with cached full-segment fetches, mmap-backed local reads, planned candidate scans, and faster JSON/raw response assembly
  • expand memory observability with /_mem, runtime/process breakdowns, SQLite/runtime counters, high-water tracking, top-stream contributors, and operational docs
  • harden GH Archive demo ingest and packaging/release workflows, including package smoke coverage and release documentation

Verification

  • bun run verify
  • bun run test:conformance
  • bun run test:conformance:local
  • bun run test:node-local-package
  • bun run test:bun-local-package
  • bun run test:bun-server-package
  • npm pack --dry-run ./dist/npm/streams-local
  • npm pack --dry-run ./dist/npm/streams-server

Notes

  • remote soak and live-server work was validated during development
  • PR remains draft and should not be merged yet

@sorenbs sorenbs changed the title [codex] Improve bundled indexing and ingest resilience Stabilize streams memory and read performance Apr 6, 2026
@sorenbs sorenbs marked this pull request as ready for review April 6, 2026 13:33
@sorenbs sorenbs merged commit 9251679 into main Apr 6, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant