Select unbounded MemWAL retention and close fold boundary

.github/workflows/ci.yml

@@ -211,6 +211,52 @@ jobs:
         # omnigraph/failpoints, so each line rebuilds only its crate.
         run: |
           cargo test --locked -p omnigraph-engine --features failpoints --test failpoints
+          set +e
+          output=$(cargo test --locked -p omnigraph-engine --features failpoints --test memwal_stream_cost gate_r0_ -- --nocapture 2>&1); status=$?
+          set -e
+          echo "$output"
+          [ "$status" -eq 0 ] || exit "$status"
+          echo "$output" | grep -Eq "test result: ok\. [1-9][0-9]* passed" \
+            || { echo "::error::Gate R0 filter matched no tests — vacuous pass"; exit 1; }
+          echo "$output" | grep -Fq "Gate R0 widest local:" \
+            || { echo "::error::Gate R0 local closure cell did not produce evidence"; exit 1; }
+          for test_name in \
+            gate_r0_current_inventory_path_classifier_is_canonical_and_fail_closed \
+            gate_r0_source_audit_revision_tripwire_matches_surveyed_lance \
+            gate_r0_retain_all_current_object_growth_is_swept_explicitly \
+            gate_r0_referenced_cut_retry_reuses_the_same_generation_root_local \
+            gate_r0_widest_generation_closes_and_records_retain_all_growth_local
+          do
+            echo "$output" | grep -Eq "test ${test_name} \.\.\. ok" \
+              || { echo "::error::required positive Gate R0 cell '${test_name}' did not execute normally"; exit 1; }
+          done
+          for test_name in \
+            gate_r0_current_inventory_refuses_manifest_for_another_shard \
+            gate_r0_current_inventory_refuses_duplicate_generation_authority \
+            gate_r0_current_inventory_refuses_zero_current_generation \
+            gate_r0_current_inventory_refuses_non_advanced_current_generation \
+            gate_r0_current_inventory_refuses_manifest_filename_body_version_mismatch \
+            gate_r0_current_inventory_refuses_gapped_manifest_version_chain \
+            gate_r0_current_inventory_refuses_listed_shard_without_manifest
+          do
+            echo "$output" | grep -Eq "test ${test_name} - should panic \.\.\. ok" \
+              || { echo "::error::required negative Gate R0 cell '${test_name}' did not execute as should-panic"; exit 1; }
+          done
+
+          # This is a remeasurement tripwire for the current single-exclusive-
+          # fold shape, not a runtime allocator limit. Keep it outside the
+          # gate_r0_ filter because the subprocess owner has a stable older name.
+          set +e
+          output=$(cargo test --locked -p omnigraph-engine --features failpoints --test memwal_stream_cost widest_legal_generation_records_no_roll_estimates_and_peak_rss -- --exact --nocapture 2>&1); status=$?
+          set -e
+          echo "$output"
+          [ "$status" -eq 0 ] || exit "$status"
+          echo "$output" | grep -Eq "test result: ok\. 1 passed" \
+            || { echo "::error::Gate R0 RSS filter matched no tests — vacuous pass"; exit 1; }
+          echo "$output" | grep -Eq "test widest_legal_generation_records_no_roll_estimates_and_peak_rss \.\.\. ok" \
+            || { echo "::error::Gate R0 RSS remeasurement cell did not execute normally"; exit 1; }
+          echo "$output" | grep -Fq "fold_delta_remeasure_bytes=402653184" \
+            || { echo "::error::Gate R0 RSS evidence line or 384-MiB tripwire changed"; exit 1; }
           cargo test --locked -p omnigraph-cluster --features failpoints --test failpoints
 
       - name: Commit regenerated openapi.json to PR branch
@@ -472,6 +518,16 @@ jobs:
           echo "$output" | grep -Eq "test result: ok\. [1-9][0-9]* passed" \
             || { echo "::error::filter 's3_' matched no tests — vacuous pass"; exit 1; }
 
+          output=$(cargo test --locked -p omnigraph-engine --features failpoints --test memwal_stream_cost gate_r0_widest_generation_closes_and_records_retain_all_growth_on_configured_rustfs -- --exact --nocapture 2>&1); status=$?
+          echo "$output"
+          [ "$status" -eq 0 ] || exit "$status"
+          echo "$output" | grep -Eq "test result: ok\. 1 passed" \
+            || { echo "::error::RustFS Gate R0 filter matched no tests — vacuous pass"; exit 1; }
+          echo "$output" | grep -Eq "test gate_r0_widest_generation_closes_and_records_retain_all_growth_on_configured_rustfs \.\.\. ok" \
+            || { echo "::error::RustFS Gate R0 cell did not execute with positive-test polarity"; exit 1; }
+          echo "$output" | grep -Fq "Gate R0 widest rustfs:" \
+            || { echo "::error::RustFS Gate R0 test skipped instead of measuring configured storage"; exit 1; }
+
       - name: Dump RustFS logs on failure
         if: failure()
         run: docker logs rustfs

AGENTS.md

@@ -33,7 +33,7 @@ OmniGraph is a typed property-graph engine built as a coordination layer over ma
 - **Multi-modal querying**: vector ANN (`nearest`), full-text (`search`/`fuzzy`/`match_text`/`bm25`), Reciprocal Rank Fusion (`rrf`), and graph traversal (`Expand`, anti-join `not { … }`) in one runtime.
 - **Branches and commits across the whole graph**: Git-style — every successful publish appends to a commit DAG; merges are three-way at the row level.
 - **Atomic per-query writes**: `mutate_as` and `load` accumulate insert/update batches into an in-memory `MutationStaging.pending` per touched table. Strict insert and upsert both route through the sealed exact-`id`, filter-bearing adapter; bare Lance Append is test-only. Their RFC-022 adapter resolves or rejects relevant recovery intents before base capture, captures `(native branch id, exact graph_head, schema identity)`, then rechecks recovery and authority under schema → branch → table gates. Mutation/Load keeps one keyed transaction per table and rejects more than 8,192 rows or 32 MiB before recovery arm; mutation update scans stream into the remaining table budget after pending-key shadowing, with blob sizes checked before payload reads. It then arms an identity-bearing recovery-v9 sidecar containing exact Lance transaction identities + pre-minted lineage, commits each table with zero transparent conflict retries, confirms the achieved effects, and publishes under the same token. Unrelated retryable pre-effect authority movement may fully reprepare without changing logical mode. A proven effect-free strict conflict returns `KeyConflict` only after a fresh exact-ID probe; no exact match triggers bounded full strict-mode reprepare, while an effect-free upsert conflict also fully reprepares. Any earlier effect or ambiguity returns `RecoveryRequired` with the sidecar retained. Strict read-set conflicts return `ReadSetChanged`. Deletes stage through the same path. D₂ at parse time remains the constructive (insert/update) XOR destructive (delete) boundary.
-- **RFC-026 private streaming core**: Phase A's main-only adapter enrolls one exact-`id` table into one empty unsharded Lance MemWAL under recovery-v10, then publishes its physical binding and `OPEN` lifecycle. Phase B1 adds a root-singleflight worker for one no-roll generation (8,192 rows / 32 MiB), watcher-only durable acknowledgement, conservative replay/fold-only recovery, and one recovery-v11 strict exact-`id` fold whose sole `__manifest` CAS advances the table pointer, lifecycle witness/epoch floor, and graph lineage together. Cheap raw caller row/byte bounds reject obviously over-cap input before recovery I/O; raw-fit input then receives exact post-tombstone validation at that same pre-recovery boundary. After any recovery/authority prelude, every put follows exact charge → shared admission → same-key input queue → worker mode; the permit transfers into the resident generation. Cold replay installs exact accounting plus a fold-only marker before releasing the opener queue; previously charged callers may create a recorded transient overlap above the nominal 32-MiB root budget while they drain, but new charge is refused. A cold fold reserves the full generation/resident/pending envelope before its owned opener and retains it with exclusive authority across the original seal deadline. Shared admission is retained through put/watcher; exclusive admission spans seal, drain proof, table effect, and manifest publication. Post-invocation ambiguity is typed `AckUnknown`; unresolved recovery blocks progress. The topology remains main-only, unsharded, one resident writer and one live writer process, with no fresh reads or generation GC. The core remains crate-private; its only callable integration seam is a feature-gated, doc-hidden test method. B2-0 specifies explicit enrollment, compare-and-chain tokens, trusted attribution, a manifest-selected graph-global current-token participant, monotonic lifecycle revisions with bounded management receipts, strict correction, Lance-owned reclamation with an enforced retained-storage admission watermark, and a manifest-authoritative graph-global `GraphHistoryBudget` checked by every manifest publisher. That budget reserves each publication's source-bounded physical-growth envelope and preserves dynamic per-stream correction/quiesce/`SEALED` closure reserves; none of B2 is implemented. Two RC.1 guards prove generic cleanup does not reclaim `_mem_wal` and raw deletion of the successor's empty WAL fence sentinel is unsafe, so OmniGraph never deletes those paths. Public B2 additionally waits for durable reclaim attempt/receipt recovery, bounded history checkpointing, strong inventory or Lance-owned accounting, and source-derived enforced physical-growth reservations. No schema-v9, SDK, CLI, HTTP, Cedar, or OpenAPI streaming contract ships until the Lance patch and all B2 evidence gates pass.
+- **RFC-026 private streaming core**: Phase A's main-only adapter enrolls one exact-`id` table into one empty unsharded Lance MemWAL under recovery-v10, then publishes its physical binding and `OPEN` lifecycle. Phase B1 adds a root-singleflight worker for one no-roll generation (8,192 rows / 32 MiB), watcher success plus the same writer's post-durability `check_fenced()` before clean acknowledgement, conservative replay/fold-only recovery, and one recovery-v11 strict exact-`id` fold whose sole `__manifest` CAS advances the table pointer, lifecycle witness/epoch floor, and graph lineage together. Cheap raw bounds and exact post-tombstone validation run before recovery; put ownership follows charge → shared admission → same-key queue → worker mode; cold replay installs exact fold-only accounting; exclusive admission spans seal, drain proof, table effect, and publication. Fold now charges the scanner's logical slices against the same 32-MiB generation cap and copies each scanner emission into dense owned arrays before retaining it, so the 8,192-row high-entropy near-cap closure cell folds and publishes successfully; its isolated fold RSS delta was 284,934,144 bytes (about 272 MiB), below the 384-MiB remeasurement tripwire. Post-invocation ambiguity is `AckUnknown`; unresolved recovery blocks progress. The topology remains main-only, unsharded, one resident writer and one live writer process, with no fresh reads or generation GC. The selected next profile is **unbounded retain-all**: OmniGraph imposes no retained-byte, object-count, file-count, or history quota and never deletes raw `_mem_wal` paths. Provider exhaustion is a loud storage failure and may halt admission, fold, or recovery progress; it never permits silent loss or publication of partial state. Row count, Arrow memory, deadlines, retries, and ambiguous outcomes remain bounded. RC.1's missing cross-open materialization-attempt receipt and complete physical-output envelope are therefore not blockers for this profile; they remain relevant to the future B2b managed-reclamation design. Compare-and-chain tokens, trusted attribution, lifecycle receipts, correction, authorization, and product parity remain required and inactive. The core remains crate-private behind one feature-gated, doc-hidden test seam; no schema-v9, SDK, CLI, HTTP, Cedar, or OpenAPI streaming contract ships from this result.
 - **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Cluster-only boot** (RFC-011): the server always boots from a cluster directory (`--cluster <dir | s3://…>`, RFC-005) and serves N graphs (N ≥ 1) under multi-graph routes (`/graphs/{graph_id}/...` + read-only `GET /graphs` enumeration); there are no single-graph flat routes and no positional-URI boot. Per-graph + server-level Cedar policies. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators run `cluster apply` and restart.
 - **CLI** with two-surface config (RFC-007/008): the team-owned cluster directory (`cluster.yaml`) plus the per-operator `~/.omnigraph/config.yaml` (servers, clusters, credentials, actor, profiles, aliases, defaults). Graphs are addressed via `--store`/`--server`/`--cluster`/`--profile`/operator defaults (RFC-011). Multi-format output (json/jsonl/csv/kv/table).
 
@@ -280,7 +280,7 @@ omnigraph policy explain --cluster ./company-brain --graph knowledge --actor act
 | Cleanup (`cleanup_old_versions`) | ✅ | `omnigraph cleanup` derives requested `--keep` / `--older-than` cutoffs from each table's available versions; Lance refs plus OmniGraph's live-lazy-branch and recovery floors may retain additional versions. It fails closed on unopenable pins, recovery intent, or uncovered main-table HEAD drift |
 | BTREE / inverted (FTS) / vector indexes | ✅ | `@index`/`@key` declares intent; the physical index is derived state that never fails a logical op. Built per column through one chokepoint (`build_indices_on_dataset_for_catalog`, type-dispatched by `node_prop_index_kind`: enum + orderable scalar → BTREE, free-text String → FTS, Vector → vector); idempotent; lazy across branches. **Schema apply and mutation/load build no indexes inline**: they publish only logical data/schema effects, leaving physical intent pending. `ensure_indices` first runs the roll-forward-only recovery barrier before base capture or planning, then materializes every declared-but-missing artifact for one table through one staged mixed CreateIndex transaction under its identity-bearing recovery-v9 authority/lineage/delta envelope; it continues to report untrainable Vector columns as pending. `Armed` is rollback-only, `EffectsConfirmed` rolls forward only while captured authority holds, and first-touch refs carry exact identity. `optimize` separately restores fragment coverage through its bounded identity-bearing v9 maintenance envelope. |
 | Strict insert / upsert ingestion | ✅ transaction conflict filters + uncommitted fragment staging | Internal schema v6 introduced the explicit logical mode and v8 preserves it. General strict insert and upsert use the sealed exact-`id`, forced-v2 MergeInsert adapter; strict insert exact-probes its pinned parent before minting `omnigraph.insert_absence=v1`, while an all-new upsert may mint the same optional certificate only from its completed effect statistics. Mutation/Load remains one transaction per table, capped before arm at 8,192 rows / 32 MiB. BranchMerge's proven all-new route accepts only a complete certificate chain plus final source/target native-incarnation checks; it stages bounded fragments with `InsertBuilder`, commits them as exact-`id` filtered `Update` transactions, and performs zero target preflights, target merge joins, or committed Appends. Missing or malformed proof falls back to the general ordered diff. Raw Lance writers are outside the supported graph-writer topology, and the certificate is an internal, non-cryptographic capability. The final five-pair production gate passed at 10K (3.875× median; 24,297,472-byte max paired RSS overhead) and 100K (~3.886×; 32,604,160 bytes). |
-| MemWAL streaming foundation | ✅ MemWAL system index, durable WAL generations, epoch-fenced shard writer | Internal schema v8 carries exact identity-keyed physical binding, current-HEAD witness, `OPEN`/`DRAINING`/`SEALED` state, per-shard epoch floor, and stream-config v2. Phase A's recovery-v10 adapter enrolls one empty unsharded shard. Private B1 adds one root-singleflight, no-roll generation capped at 8,192 rows / 32 MiB; watcher-only durable ack; detached ownership across cancellation/timeouts; fold-only replay using the pinned BatchStore watermark bridge; explicit seal/drain proof; and recovery-v11 exact-transaction fold followed by one graph-visibility CAS. Root-shared admission and lifecycle guards exclude competing table effects; a claimed/warm ambiguity retires and aborts before reuse. B1 performs no generation GC and exposes no fresh reads. B2-0 specifies the inactive schema-v9/config-v3/state-v2/recovery-v12 explicit-enrollment, token, attribution, lifecycle-revision/management-receipt, correction, Lance-owned reclamation, per-binding admission-watermark, and graph-global `GraphHistoryBudget` contracts. The latter is checked by every manifest writer and keeps independent per-stream closure reserves. There is still no public stream schema, enrollment, put/fold/status API, Cedar action, CLI, HTTP route, or OpenAPI contract; those remain B2. |
+| MemWAL streaming foundation | ✅ MemWAL system index, durable WAL generations, epoch-fenced shard writer | Internal schema v8 carries exact identity-keyed physical binding, current-HEAD witness, `OPEN`/`DRAINING`/`SEALED` state, per-shard epoch floor, and stream-config v2. Phase A's recovery-v10 adapter enrolls one empty unsharded shard. Private B1 adds one root-singleflight, no-roll generation capped at 8,192 rows / 32 MiB; watcher success plus same-writer post-durability epoch check; detached ownership; fold-only replay; seal/drain proof; and recovery-v11 exact-transaction fold followed by one graph-visibility CAS. The fold charges logical slices and densifies each scanner emission, so the deterministic 8,192-row/near-32-MiB high-entropy generation now closes; the measured isolated fold RSS delta is 284,934,144 bytes (about 272 MiB), below the 384-MiB remeasurement tripwire. It performs no generation GC or fresh reads. The selected unbounded retain-all profile has no OmniGraph byte/object/file/history quota and no raw `_mem_wal` deletion; provider exhaustion fails loudly and may stop progress. Row/logical-Arrow admission, deadline, retry, and ambiguity bounds remain. B2b retains the future Lance-owned reclamation design. Common schema-v9/config-v3/state-v2/recovery-v12 enrollment/token/attribution/lifecycle/correction contracts remain inactive. There is no public stream schema, enrollment, put/fold/status API, Cedar action, CLI, HTTP route, or OpenAPI contract. |
 | Vector search | ✅ | `nearest()` query op; embedding pipeline (Gemini / OpenAI clients); `@embed` in schema |
 | Full-text search | ✅ | `search/fuzzy/match_text/bm25` query ops |
 | Hybrid ranking | — | `rrf(...)` Reciprocal Rank Fusion in one runtime |