Skip to content

HNSW push-down: indexed_path_finds_top_result_and_traverses missed the aligned winner on x86_64 CI (recall, not scoring) #781

Description

@ohdearquant

Summary

typed_graph::tests::indexed_path_finds_top_result_and_traverses failed on a GitHub-hosted
x86_64 Linux runner with res[0].seed_id = "d2" where the test expects "winner", while the
same test at the same commit passes on aarch64/macOS. The failure mode indicates the HNSW
candidate set missed the top three documents entirely, not that scoring or ordering drifted.

Observed

  • Failing run: Workspace CITests (vector-index), run 30745926341, on commit 738206ef
    (a PR head whose diff touches only ruvector-core; ruvector-graph is untouched there).
    Panic: assertion 'left == right' failed; left: "d2"; right: "winner" at
    crates/ruvector-graph/src/typed_graph.rs:809.
  • Same commit, aarch64/macOS: cargo test -p ruvector-graph --lib typed_graph::tests::indexed_path_finds_top_result_and_traverses1 passed.
  • Current main (74870553), aarch64/macOS: same command, 1 passed.
  • No completed CI baseline exists on main to compare against: the last eight Workspace CI
    runs on main all ended cancelled before this job completed.

Why this is a recall failure, not a scoring one

The test's geometry makes the margins large. winner is exactly aligned with the query
(score 1.0); the decoys start at 0.2 rad, so the best decoy d0 scores cos(0.2) ≈ 0.98007 and
d2 scores ≈ 0.97845. For d2 to be ranked first, winner (margin 0.022), d0 and d1
must all be absent from the candidate set that exact rescore ran over. A 0.022 gap cannot be
produced by f32 accumulation differences between platforms; it can be produced by the ANN
stage returning a candidate list that excludes the true top results.

Suggested angle

Whether HNSW construction or search in the push-down path has platform- or run-dependent
behaviour (e.g. RNG-driven level assignment interacting with insertion order) that can drop
recall@5 below 1.0 on this 301-vector index. If construction is randomized, the test may want
either a fixed seed or an over-fetch margin large enough to make recall deterministic, so that
it fails only when the index is actually broken.

One data point, not a trend: I have only observed this once, because Workspace CI rarely
runs to completion on this repository at the moment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions