refactor: Integrate LanceDB for Fast Nearest-Neighbor Vector Storage

[scheuclu]

Description

Replace in-memory array search loops with a local, embedded vector database (LanceDB) to enable sub-millisecond querying as our sentence and contextual databases grow.

Implementation Checklist

• Database Setup:
  • Install and initialize lancedb in the Python pipeline.
  • Load precomputed vectors (multilingual and sentence bases) directly into local LanceDB tables.
• Query Refactor:
  • Rewrite query endpoints to execute vector search directly using LanceDB indices instead of raw matrix dot products.