feat(P3): WebGL provider, ExperienceReplay, ClusterStability, benchmark suite, hotpath guard, docs by Copilot · Pull Request #73 · devlux76/cortex

Copilot · 2026-03-13T22:16:15Z

Original prompt

This section details on the original issue you should resolve

<issue_title>P3</issue_title>
<issue_description>All P3 tasks, issues and sub issues belong here, either directly or by reference. Please subtask every issue with P3 in its name for ease of tracking.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes P3 #72

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

…tability, benchmarks, hotpath guard, docs Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>

Copilot

Pull request overview

This PR adds P3 “polish/ship” features across embeddings, Daydreamer background maintenance, benchmarking, policy-constant enforcement, and documentation to round out developer experience and operational safeguards.

Changes:

Adds a WebGL-based embedding backend (OrtWebglEmbeddingBackend) and a resolver candidate (createWebglProviderCandidate) as a hardware-accelerated fallback tier.
Introduces Daydreamer maintenance modules (ExperienceReplay, ClusterStability) plus new benchmark suites for query latency, storage overhead, and hotpath scaling.
Adds a CI guard script for preventing hardcoded hotpath policy constants outside core/HotpathPolicy.ts, plus documentation updates.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
embeddings/OrtWebglEmbeddingBackend.ts	New embedding backend using Transformers.js with `device: "webgl"`.
embeddings/ProviderResolver.ts	Adds `createWebglProviderCandidate()` for WebGL capability-gated selection.
daydreamer/ExperienceReplay.ts	Implements idle-time synthetic queries that apply Hebbian LTP edge reinforcement.
daydreamer/ClusterStability.ts	Implements volume split/merge maintenance and PageActivity community relabeling.
scripts/guard-hotpath-policy.mjs	New source guard to prevent hardcoded hotpath policy constants outside `core/HotpathPolicy.ts`.
package.json	Adds `guard:hotpath-policy` and benchmark scripts.
tests/embeddings/OrtWebglEmbeddingBackend.test.ts	Unit tests for WebGL embedding backend and provider candidate support gating.
tests/daydreamer/ExperienceReplay.test.ts	Unit tests for replay cycle behavior and edge weight clamping.
tests/daydreamer/ClusterStability.test.ts	Unit tests for split/merge behavior and community label updates.
tests/benchmarks/QueryLatency.bench.ts	New query latency benchmark vs corpus size.
tests/benchmarks/StorageOverhead.bench.ts	New storage overhead benchmark via ingest + vector reads.
tests/benchmarks/HotpathScaling.bench.ts	New hotpath promotion-sweep scaling benchmark + Williams Bound invariant checks.
benchmarks/BASELINES.md	Template baseline recording doc for benchmark output.
docs/api.md	Adds/updates API reference sections for ingest/query/Daydreamer modules.
docs/development.md	Adds guard usage, troubleshooting, performance tuning, and benchmark instructions.
docs/product-surfaces.md	Defines UX/privacy contracts for app-vs-library product surfaces.

scripts/guard-hotpath-policy.mjs

tests/benchmarks/StorageOverhead.bench.ts

tests/benchmarks/HotpathScaling.bench.ts

devlux76 · 2026-03-14T00:58:19Z

daydreamer/ClusterStability.ts

+    // --- Pass 1: split high-variance volumes ---
+    for (const volume of volumes) {
+      if (
+        volume.variance > this.varianceThreshold &&
+        volume.bookIds.length >= 2
+      ) {
+        const splits = await this.splitVolume(volume, metadataStore);
+        if (splits !== null) {
+          splitCount++;
+          communityUpdates += await this.updateCommunityLabels(
+            splits,
+            metadataStore,
+          );
+          // Remove the original volume from storage (replace with two new ones)
+          await this.replaceVolumeInShelves(
+            volume.volumeId,
+            splits,
+            metadataStore,
+          );
+        }


@copilot apply changes based on this feedback

Fixed in 0341e62. Three changes made:

deleteVolume(volumeId) added to MetadataStore interface — removes the volume record, cleans up bookToVolume reverse-index entries for each of the volume's books, and deletes the volumeToShelf index entry.

IndexedDbMetadataStore.deleteVolume — implemented as a single atomic IDB transaction spanning volumes, bookToVolume, and volumeToShelf stores.

ClusterStability.run() — now calls metadataStore.deleteVolume(oldVolumeId) after replaceVolumeInShelves for every split parent and every merge source; tests assert getVolume(oldId) === undefined after both operations.

Also took the opportunity to fix the O(n²) indexOf inside kmeansAssign — replaced with a precomputed Map<bookId, index> so each K-means iteration is O(n).

docs/api.md

package.json

devlux76 · 2026-03-14T01:01:37Z

docs/product-surfaces.md

+| Data                            | Stays local?   | Notes                                      |
+|---------------------------------|:--------------:|--------------------------------------------|
+| Page content                    | Yes            | Stored in OPFS; never leaves the device.   |
+| Embeddings                      | Yes            | Stored in OPFS; never leaves the device.   |
+| Hotpath index                   | Yes            | IndexedDB; never leaves the device.        |
+| Query text                      | Yes            | Never logged or transmitted.               |
+| P2P curiosity probe (opt-in)    | Partial        | Public-interest graph slice only; no PII.  |
+| Model weights                   | Fetched once   | Cached locally via `@huggingface/transformers`. |
+


@copilot apply changes based on this feedback

devlux76 · 2026-03-14T01:02:02Z

benchmarks/BASELINES.md

+## Williams Bound H(t) — Sublinear Growth Curve
+
+| Graph mass (t) | H(t) = ceil(0.5 * sqrt(t * log2(1+t))) | H(t)/t ratio |
+|---------------:|----------------------------------------:|-------------:|
+|          1 000 |                                      ~22 |        0.022 |
+|         10 000 |                                      ~99 |        0.010 |
+|        100 000 |                                     ~408 |        0.004 |
+|      1 000 000 |                                   ~1 576 |        0.002 |
+


@copilot apply changes based on this feedback

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…lse positives, fix CI typecheck Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>

Initial plan

48e9f29

Copilot AI assigned Copilot and devlux76 Mar 13, 2026

Copilot started work on behalf of devlux76 March 13, 2026 22:16 View session

feat: implement P3 tasks — WebGL provider, ExperienceReplay, ClusterS…

6321731

…tability, benchmarks, hotpath guard, docs Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] [P3] Create subtasks for all P3 issues~~ feat(P3): WebGL provider, ExperienceReplay, ClusterStability, benchmark suite, hotpath guard, docs Mar 13, 2026

Copilot AI requested a review from devlux76 March 13, 2026 22:37

Copilot finished work on behalf of devlux76 March 13, 2026 22:37

devlux76 mentioned this pull request Mar 14, 2026

P3 #72

Closed

github-actions bot added layer: embeddings Embedding providers and resolver layer: daydreamer Background consolidation (LTP/LTD, recalc) layer: testing Test coverage and integration tests layer: documentation API docs, developer guide, architecture diagrams labels Mar 14, 2026

devlux76 marked this pull request as ready for review March 14, 2026 00:42

Copilot AI review requested due to automatic review settings March 14, 2026 00:42

Copilot started reviewing on behalf of devlux76 March 14, 2026 00:42 View session

Copilot AI reviewed Mar 14, 2026

View reviewed changes

devlux76 and others added 3 commits March 13, 2026 18:56

Potential fix for pull request finding

a5e38e8

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

91a74d3

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

02d8968

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of devlux76 March 14, 2026 00:58 View session

devlux76 and others added 5 commits March 13, 2026 18:58

Potential fix for pull request finding

0a1dc36

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

4064892

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

2d5a9c8

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

dd56622

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

fix: align with main's P1 API renames, add deleteVolume, fix guard fa…

0341e62

…lse positives, fix CI typecheck Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>

Copilot finished work on behalf of devlux76 March 14, 2026 01:16

Copilot started work on behalf of devlux76 March 14, 2026 01:16 View session

github-actions bot added layer: foundation Core types, model profiles, crypto layer: storage OPFS vector store and IndexedDB metadata store labels Mar 14, 2026

github-actions bot added layer: hippocampus Ingest orchestration (chunk → embed → persist) layer: cortex Retrieval orchestration (rank → expand → order) labels Mar 14, 2026

devlux76 merged commit f4dc669 into main Mar 14, 2026
3 of 4 checks passed

devlux76 deleted the copilot/p3-subtask-creation branch March 14, 2026 01:18

Copilot stopped work on behalf of devlux76 due to an error March 14, 2026 01:18
The session was cancelled by the user.

devlux76 mentioned this pull request Mar 14, 2026

Priority List #74

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(P3): WebGL provider, ExperienceReplay, ClusterStability, benchmark suite, hotpath guard, docs#73

feat(P3): WebGL provider, ExperienceReplay, ClusterStability, benchmark suite, hotpath guard, docs#73
devlux76 merged 10 commits intomainfrom
copilot/p3-subtask-creation

Copilot AI commented Mar 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devlux76 Mar 14, 2026

Uh oh!

Copilot AI Mar 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devlux76 Mar 14, 2026

Uh oh!

devlux76 Mar 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comments on the Issue (you are @copilot in this section)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devlux76 Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devlux76 Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

devlux76 Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 13, 2026 •

edited

Loading