feat: Add Valkey as a vector store backend

[daric93]

Is your feature request related to a problem? Please describe.

LocalAI's stores subsystem is already pluggable — the /stores/* API and the gRPC Stores* contract both carry a per-request backend field — but the project only ships the in-memory local-store backend. That has two consequences:

• No durability. Every vector is lost on restart. The face/voice biometric registries (core/services/facerecognition, core/services/voicerecognition) and the router embedding cache (core/backend/stores.go) built on top of the store must be fully repopulated after any redeploy or crash.
• No scaling path. local-store's Find is an O(N) linear scan, so it doesn't scale to large corpora.

This was discussed back in #1792 ("feat: Vector search"), which weighed (1) in-memory brute force, (2) adding a vector DB as a backend, and (3) an external DB. Option (1) shipped as local-store; options (2)/(3) were never realized, and the issue is now closed/stale. Users who already run Valkey have no way to consolidate LocalAI's vector store onto it.

Describe the solution you'd like

Add a new built-in Go gRPC store backend, valkey-store, that realizes option (2) from #1792 through the existing pluggable store-backend seam — no proto change and no HTTP API change. It is selected by passing backend: "valkey-store" (alias "valkey") on the existing /stores/* requests, mirroring how local-store is wired.

• Implements the four Stores RPCs (Set/Get/Delete/Find) against the Valkey Search module (FT.* vector similarity) running on the valkey/valkey-bundle image.
• Mirrors backend/go/local-store in structure and error semantics, so it's a drop-in alternative.
• Durability: vectors persist across restarts via Valkey's RDB/AOF — the capability local-store fundamentally can't offer. This is the headline reason to prefer Valkey.
• Exact-search parity by default: uses a FLAT index with COSINE distance so results match local-store exactly. Cosine similarity is derived as sim = 1 - distance (Valkey Search returns cosine distance ∈ [0,2]).
• Opt-in HNSW (VALKEY_INDEX_ALGO=HNSW, default FLAT) for approximate ANN on large corpora.
• Vector-as-key: since the vector is the key in the store contract, it's encoded losslessly as prefix + hex(float32 little-endian bytes), with the raw bytes stored in an indexed vec HASH field and the opaque payload in val. The FT.CREATE index is created lazily on first Set once the dimension is known.
• Client library: the official pure-Go client github.com/valkey-io/valkey-go (no CGO), so the backend keeps LocalAI's CGO_ENABLED=0 static build and the Linux + Darwin backend matrix intact.

Per CONTRIBUTING ("Before jumping into a PR for a massive feature or big change, it is preferred to discuss it first via an issue"), this issue is that pre-PR discussion. I intend to implement it and open a PR.

Describe alternatives you've considered

• Keep the in-memory store only — no durability and no scaling path; the problem above remains.
• Integrate an external vector DB (Qdrant, Pinecone, ...) — a heavier dependency and more operational surface than reusing the store seam that already exists, and less appealing for users who already operate Valkey.
• valkey-glide (Go client) — rejected: its Rust core is reached over FFI and requires CGO plus a per-arch native library, which breaks the CGO_ENABLED=0 static build and the Linux/Darwin backend matrix.
• Other Valkey-compatible Go clients (rueidis, go-redis) — both viable and pure-Go, but valkey-go was chosen as the official valkey-io client with first-class FT.*/vector helpers (VectorString32, typed FtSearch + AsFtSearch, and a gomock-based mock package for unit tests).

This realizes option (2) from #1792 without the "classic database issues" that discussion worried about, because Valkey (not LocalAI) owns persistence and indexing.

Additional context

• Valkey is the open-source, Linux Foundation-backed fork of Redis, and valkey/valkey-bundle ships the native Valkey Search module that provides the FT.* vector commands used here.
• Testing plan: unit tests with a mocked client (asserting the exact commands built for each RPC, the sim = 1 - distance conversion, and the vector-as-key round-trip incl. -0.0/NaN edges), plus env-gated integration tests against a live valkey/valkey-bundle instance with the Search module (index creation, ingestion, KNN retrieval, delete, cleanup, and a set → restart → still-present durability check).
• Deliberately out of scope for the first iteration (possible follow-ups): hybrid/metadata-filtered search (needs new API surface beyond the four Stores RPCs), cluster mode, and multi-namespace consolidation onto a single instance.
• References:
  • Valkey Search module: https://github.com/valkey-io/valkey-search
  • valkey-go client: https://github.com/valkey-io/valkey-go
  • Reference pattern in-tree: backend/go/local-store/ and tests/integration/stores_test.go
  • Prior art / related discussion: feat: Vector search #1792

[daric93]

I'd like to work on this if it aligns with the project roadmap.