perf(vector/search): no cross-segment HNSW graph — segments are separate islands; recall depends on per-segment ef

[mosuka]

Round-3 perf push sub-issue (tracked under umbrella #536).

[L] No cross-segment HNSW graph — segments are separate islands; recall depends on per-segment ef

• Where: SegmentedVectorField::search_managed_segments runs each segment as an
  independent HNSW; no cross-segment graph or global entry-point.
• Current behavior: ef_search × N_segments work, recall is union-of-locals. No segment
  can know it has the global winners.
• Why it might be a bottleneck / risk: At segment counts of 50-100 (typical for
  high-ingest indexes), per-segment work multiplies and global top-K accuracy drops.
• Reference precedent: Vamana's "FreshDiskANN" merges segments via mini-graph; FAISS
  IVF over flat is essentially segment-free.
• Suggested direction: Background merge to keep segment count ≤ ~10. Or maintain a
  "leader" graph over segment-level centroids for routing.
• Risk / scope: Large. Aligns with existing merge infrastructure.

---

Cross-cutting themes

• No filter-aware ANN traversal: vector + filter is always post-filter, collapsing
  recall at high filter selectivity and wasting work at low selectivity. Largest missing
  piece for production hybrid workloads.
• Per-query allocation of heaps / bitmaps: every search allocates fresh BinaryHeaps
  and zero-inits a BitVec of size N. A SearchContext reusing these via thread-local +
  generational visited markers would eliminate megabytes of churn at high QPS.
• No batched search API: B queries processed serially with full per-query setup. Core
  hot data (Arc<QuantizedVectorPool>, field_position_index, graph entry point) is
  shared but the API doesn't expose batched access.
• HashMap with default (SipHash) hasher on u64 keys in hot paths: cheap fix with
  ahash / nohash / dense Vec<u32>; affects field_position_index,
  prefetch_index, id_to_index.
• Multi-segment search is serial + does full reader load per query:
  SegmentedVectorField re-parses the on-disk index file per query, no segment-reader
  cache, no per-segment parallelism. Hidden show-stopper.
• No FastScan / 4-bit packed PQ: state-of-the-art PQ ADC kernels are 2-4× faster than
  scalar lut[m*K + code]; single largest pure-distance improvement available.
• Distance kernel SoA + alignment: current AoS (int8 || sum_q || norm_q per record)
  pays for misaligned reads and bounds-check overhead. SoA arrays unlock PMADDUBSW /
  VPMADDUBSW intrinsics.
• HNSW graph is Vec<Vec<Vec<u64>>> — triple pointer chase: CSR arena with u32 ids
  packs neighbour lists contiguously, major cache-locality win.
• Hybrid fusion lacks top-K bound + runs lex/vec serial: minor at low limits, blocks
  parallel scheduling and full-sort cost at high limit.
• No async / mmap-paging for cold-cache HNSW: Lazy mode is correct but slow at
  out-of-RAM scale; DiskANN-style beam search + madvise would unlock disk-resident HNSW.
• Score / candidate width: f32 + u64 packing in heaps could shrink ~50 % with f16 / u32
  ids; combined with total_cmp resolves a silent NaN reorder risk.
• Field name as String everywhere: a FieldId(u16) registry would shave both memory
  and hot-loop hashing cost.
• Per-stage rerank pipeline hardcoded: 3-stage PQ→SQ→f32 rerank not expressible;
  Flat/IVF can't use the rerank sidecar. A generic RerankScorer trait would unify this.

---

ID: VS-40 — see ~/.claude/tasks/laurus/20260523_perf_round3_audit/task_list.md for the full Round-3 issue list.