[Feature](lance) Support additional vector index algorithms

[Gabriel39]

Parent

Tracked by #66340. IVF_PQ query support was introduced and verified by #65730.

Scope

• IVF_FLAT query support.
• IVF_SQ query support.
• IVF_HNSW_FLAT query support.
• IVF_HNSW_SQ query support.
• IVF_HNSW_PQ query support.
• Define compatible vector element types, distance metrics, and ANN tuning parameters for each algorithm.

Completion criteria

• Query execution works through Lance automatic index selection for every listed algorithm.
• Regression fixtures and tests cover every algorithm and compatible metric combination.
• EXPLAIN and error handling make index selection and incompatibilities diagnosable.
• User documentation contains a supported algorithm compatibility matrix.

[Gabriel39]

@FANNG1, would you be available to take ownership of this issue and implement it?

[FANNG1]

Thanks for the invitation! I’d be happy to take this issue and plan to start working on it soon. Could you please assign it to me?

[Gabriel39]

Thanks! Yes, please go ahead. The issue is assigned to you.