perf(vector/search): per-field `String` keys everywhere — wasted hashing & alloc when field count is tiny

[mosuka]

Round-3 perf push sub-issue (tracked under umbrella #536).

[M] Per-field String keys everywhere — wasted hashing & alloc when field count is tiny

• Where: QuantizedVectorPool.field_index: HashMap<String, ...>
  (quantized_storage.rs:45); HnswIndexReader.prefetch_index: HashMap<String, ...>
  (hnsw/reader.rs:64); vector_ids: Vec<(u64, String)> everywhere;
  FieldSearchInput.field: String.

• Current behavior: Every per-field lookup hashes a String (or clones from &str
  via to_string()). In practice indexes have 1-10 vector fields per collection.

• Why it might be a bottleneck / risk: Memory waste from dual storage (each
  (u64, String) tuple holds a copy of the field name). Hashing a 20-char field name is
  ~50 cycles per call.

• Reference precedent: Lucene interns field names to integer ords at segment open;
  Tantivy uses FieldId(u32); Qdrant uses payload schema IDs.

• Data structure (proposed):

  struct FieldId(u16);                       // 65K fields per index
  struct FieldRegistry {
      id_by_name: ahash::AHashMap<String, FieldId>,
      names: Vec<String>,
  }
  // Storage pools key by FieldId:
  pub field_index: SmallVec<[Arc<DocIdMap>; 4]>,   // indexed by FieldId.0
  pub vector_ids: Vec<(u64, FieldId)>,             // (u64, u16) = 16 bytes (12 packed)
  

• Suggested direction: Introduce per-index FieldRegistry. Storage pools indexed by
  FieldId. String only at public API boundary.

• Risk / scope: Medium-Large diff; mechanical. Doubles per-field hot-loop speed, cuts
  memory ~30 %.

---

ID: VS-31 — see ~/.claude/tasks/laurus/20260523_perf_round3_audit/task_list.md for the full Round-3 issue list.

[mosuka]

Substantially resolved by #862 (and #853 before it): per-record field-name Strings are gone from the retained reader structures — records intern to (u64, u16) against one shared per-segment Arc<[Arc]> dictionary (legacy v1/v2 files included), VectorStorage::get/contains take (doc_id, &str) with an allocation-free linear-scan resolve, the IVF probe filter is a u16 compare, and the HNSW hot loop uses ordinals with no field-keyed probe at all. What remains String-keyed are the per-FIELD maps (position indexes, per-field caches) probed once per search — with 1–3 fields per segment those are not a measurable cost.