feat(rag): remote TEI embedding and reranker providers (FEAT-024)

.env.example

@@ -64,6 +64,9 @@
 # Embedding provider and model.
 # VEKTRA_EMBEDDING_PROVIDER=sentence-transformers
 # VEKTRA_EMBEDDING_MODEL=paraphrase-multilingual-MiniLM-L12-v2
+# Remote embedding via Text Embeddings Inference (VEKTRA_EMBEDDING_PROVIDER=tei):
+# VEKTRA_TEI_URL=http://localhost:8080
+# VEKTRA_TEI_API_KEY=
 
 # --------------------------------------------------------------------------
 # Vector store
@@ -106,6 +109,11 @@
 # VEKTRA_RERANK_PROVIDER=cross-encoder
 # VEKTRA_RERANK_MODEL=BAAI/bge-reranker-v2-m3
 # VEKTRA_RERANK_TOP_K=5
+# API key for API-based providers (cohere):
+# VEKTRA_RERANK_API_KEY=
+# Remote reranking via TEI (VEKTRA_RERANK_PROVIDER=tei, one instance per model):
+# VEKTRA_RERANK_TEI_URL=http://localhost:8080
+# VEKTRA_RERANK_TEI_API_KEY=
 
 # --------------------------------------------------------------------------
 # Ingestion

.s2s/BACKLOG.md

@@ -393,8 +393,9 @@ The `title` field would contain `filename + page` (e.g., "Costituzione italiana.
 
 ### FEAT-024: Remote embedding and reranker providers (TEI)
 
-**Status**: planned | **Priority**: medium | **Created**: 2026-07-12
+**Status**: completed | **Priority**: medium | **Created**: 2026-07-12 | **Completed**: 2026-07-13 | **PR**: #93
 **Origin**: deployment modularity review 2026-07-12 - the host workstation already serves TEI instances (bge-m3, qwen3-embedding); Vektra cannot use them.
+**Evidence**: `vektra-internal/stack/20260713-feat024-tei-providers.md` (+ artifacts in `20260713-feat024-eval-artifacts/`)
 
 **Context**: `VEKTRA_EMBEDDING_PROVIDER` documents a `tei` option (config.py:74) but **no TEI provider exists**: `main.py:129-137` unconditionally instantiates in-process `SentenceTransformersProvider`; the compose even ships a `tei` profile service nobody can talk to. The reranker likewise runs in-process only (`rerankers` lib; the `cohere` path never passes an api_key, so it is dead as wired - reranker.py:122). Consequences: every Vektra instance duplicates embedding/reranker compute in-container (CPU), and shared GPU/CPU inference services on the host cannot be reused.
 
@@ -406,14 +407,16 @@ The `title` field would contain `filename + page` (e.g., "Costituzione italiana.
 
 **Why it matters beyond dedup**: the current embedding model (paraphrase-multilingual-MiniLM-L12-v2) has **max_seq_length 128 tokens** - our 500-token chunks are silently truncated at embedding time (dense sees only the chunk head; BM25 sees the full text). bge-m3 (8192-token window, MIRACL dense nDCG@10 69.2 vs mE5-large 66.6; MiniLM sits 16-22 nDCG points below even mE5 on European-language retrieval per PL-MTEB) is the natural upgrade candidate, testable via TEI without fattening the container.
 
+**Resolution (2026-07-13)**: implemented as designed with two deltas: (1) TEI 1.9.3 `/info` does not expose the embedding size, so `dimensions()` probes `/embed` as fallback (both paths unit-tested); (2) found and fixed in passing a startup blocker: `check_embedding_model` resolved the provider by the hardcoded `sentence-transformers` name, so startup failed with any other provider (now uses the `default` alias). Measured on eval-full questions (same dual chunks, reindexed via a fresh `eval-tei` namespace): **bge-m3 via TEI retrieval hit 93.5% / MRR 0.8478 vs MiniLM dual 82.6% / 0.7029 (+10.9pp)** - beats even the fixed-chunking MiniLM baseline (89.1%/0.8062), confirming the 128-token truncation hypothesis; e2e grounded 54/55 stable, MC answers improve in substance (MC-02 produces a real bi-document comparison; kw 7/25 vs 4/25), p50 +0.7s (TEI on CPU). TEI reranker smoke: factual query scores 0.75 (2 survive the threshold), comparative query all-below-threshold rescued by TECH-007 (`rescued=3`) - full funnel verified with both remote providers, authenticated. Discovery filed under BUG-021: `run_reindex` stores through hardcoded pgvector, so reindex-into-Qdrant silently writes nothing (worked around via fresh-namespace ingest). Switching the default embedding to bge-m3 is a separate decision (needs full corpus re-ingest and a TECH-005-grade bench).
+
 **Traceability**: ADR-0013 (EmbeddingProvider Protocol), ARCH-035, ARCH-036, ADR-0021
 
 **Acceptance criteria**:
-- [ ] `VEKTRA_EMBEDDING_PROVIDER=tei` works end-to-end (ingest + query) against a TEI instance with api key
-- [ ] Collection created with the provider's real dimensions; clear error on dimension mismatch with an existing collection
-- [ ] `VEKTRA_RERANK_PROVIDER=tei` reranks via TEI /rerank with scores compatible with the threshold filter
-- [ ] Embedding-model comparison (MiniLM in-process vs bge-m3 via TEI) run with the TECH-005/existing harness and recorded
-- [ ] Docs: configuration.md + .env.example cover the new provider options
+- [x] `VEKTRA_EMBEDDING_PROVIDER=tei` works end-to-end (ingest + query) against a TEI instance with api key
+- [x] Collection created with the provider's real dimensions; clear error on dimension mismatch with an existing collection (verified live: 384-vs-1024 startup warning with remediation)
+- [x] `VEKTRA_RERANK_PROVIDER=tei` reranks via TEI /rerank with scores compatible with the threshold filter
+- [x] Embedding-model comparison (MiniLM in-process vs bge-m3 via TEI) run with the TECH-005/existing harness and recorded
+- [x] Docs: configuration.md + .env.example cover the new provider options
 
 ---
 
@@ -1087,7 +1090,7 @@ Three near-duplicates is the threshold where extraction starts to pay off (a fou
 
 **Resolution**: the endpoint now resolves embedding, sparse embedding, and vector store from `request.app.state.registry` (same contract as the pipeline). The per-request `SentenceTransformersProvider` instantiation and the now-unused `session` dependency were removed. Unit tests added (`vektra-index/tests/test_api_search.py`): registry resolution, hybrid→dense fallback without sparse, hybrid with sparse.
 
-**Same family, not fixed here**: `POST /documents/{id}/chunks`, `DELETE /documents/{id}` and `GET /stats` still hardcode pgvector (the stats-vs-Qdrant mismatch was already a known issue). Track separately if needed.
+**Same family, not fixed here**: `POST /documents/{id}/chunks`, `DELETE /documents/{id}` and `GET /stats` still hardcode pgvector (the stats-vs-Qdrant mismatch was already a known issue). Also `run_reindex` (`vektra-index/reindex.py`): it re-embeds with the registry's active embedding provider but stores through a hardcoded `PgvectorProvider`, so in Qdrant mode a reindex reports "completed" while the Qdrant collection receives nothing (found during the FEAT-024 live smoke, 2026-07-13: reindexing eval-full to v2 wrote to Postgres only). Track separately if needed.
 
 **Traceability**: ARCH-039 (ProviderRegistry), ARCH-051 (full-store contract), TECH-002 (eval harness)
 

.s2s/plans/20260712-sprint3-rag-quality.md

@@ -323,3 +323,25 @@ the bundle and a manual smoke in the Moodle dev stack.
   `vektra-internal/stack/20260713-tech007-retrieval-rescue.md`; per-question
   artifacts in `20260713-tech007-eval-artifacts/`. Dev `.env` adds
   `VEKTRA_RETRIEVAL_RESCUE_TOP_K=3`, `VEKTRA_RETRIEVAL_RESCUE_FLOOR=0.005`.
+- 2026-07-13: **FEAT-024 implemented and measured (post-sprint, PR #93)** —
+  TEI remote providers, `feat/feat-024-tei-providers`. TEIEmbeddingProvider
+  (native /embed, /info dimensions with /embed probe fallback — TEI 1.9.3
+  does not expose the size in /info), TEIRerankerService (/rerank, same
+  RerankResult semantics), Qdrant collection sized from the active provider
+  (fixes the hardcoded-384 latent bug; live 384-vs-1024 mismatch produces a
+  clear startup warning), cohere api_key pass-through fixed, startup warmup
+  fixed to resolve the `default` embedding alias (was hardcoded to
+  'sentence-transformers' and broke any alternative provider).
+  **Comparison on eval-full questions (same dual chunks, ns eval-tei,
+  bge-m3 float32 CPU via TEI): retrieval hit 82.6% -> 93.5%, MRR 0.7029 ->
+  0.8478 — beats even the fixed-chunking MiniLM baseline (89.1%/0.8062),
+  confirming MiniLM's 128-token truncation as a real retrieval cap.** E2e
+  grounded 54/55 stable, multi-chunk substance improves (MC-02 real
+  bi-document comparison, kw 7/25 vs 4/25), p50 +0.7s (CPU TEI). Reranker
+  smoke: factual 0.75 survives threshold, comparative all-below rescued by
+  TECH-007 (rescued=3) — full funnel verified with both remote providers.
+  Found in passing (filed under BUG-021 family): run_reindex stores through
+  hardcoded pgvector — reindex-into-Qdrant silently no-ops. Dev stack
+  restored to reference config (MiniLM + rescue); eval-tei namespace and
+  vektra-tei collection (105 points, 1024-dim) left in place for follow-ups;
+  bge-reranker TEI cache kept at /mnt/scratch/tei-rerank-cache.

CHANGELOG.md

@@ -21,6 +21,7 @@ Convention (Keep a Changelog 1.1.0):
 
 ### Added
 
+- **rag**: remote embedding and reranking via HuggingFace Text Embeddings Inference (FEAT-024). `VEKTRA_EMBEDDING_PROVIDER=tei` embeds through a TEI instance (`VEKTRA_TEI_URL`/`VEKTRA_TEI_API_KEY`, native `/embed` API) instead of in-process sentence-transformers, enabling shared host inference and long-window models (bge-m3: 8192 tokens vs MiniLM's 128, which silently truncates 500-token chunks today). `VEKTRA_RERANK_PROVIDER=tei` reranks through TEI `/rerank` (`VEKTRA_RERANK_TEI_URL`/`VEKTRA_RERANK_TEI_API_KEY`). The Qdrant collection is now sized from the active embedding provider's dimensions instead of a hardcoded 384 (latent bug for any non-384 model), with a clear startup error on dimension mismatch against an existing collection. The `cohere` rerank option now actually passes its API key (`VEKTRA_RERANK_API_KEY`); it was dead as wired.
 - **rag**: optional retrieval-filter rescue for multi-part questions (TECH-007, `VEKTRA_RETRIEVAL_RESCUE_TOP_K` + `VEKTRA_RETRIEVAL_RESCUE_FLOOR`, default off). When `VEKTRA_MIN_RELEVANCE_SCORE` empties the candidate set, keep the top-N chunks above an absolute floor instead of refusing: the cross-encoder scores each partial-answer chunk of a comparative/multi-part question below the threshold (it answers only one part), so on the eval corpus 9/10 multi-chunk questions died at the filter with `before=5 after=0` despite 90% raw retrieval hit. With the rescue, borderline sets reach the LLM, which arbitrates via strict grounding. The `retrieval_filter` trace step now records a `rescued` count.
 - **rag**: optional per-namespace inline source citations (FEAT-021, `citations_enabled` in the namespace config JSONB via `PATCH /api/v1/admin/namespaces/{id}/config`, default off, advanced pipeline only). When enabled, the system prompt instructs the LLM to add inline `[n]` markers matching the `<source id>` elements, the context template carries a `title` attribute ("filename, p.N"), and each returned source includes a `title` field; the learn widget renders the markers as superscripts with a tooltip. Default-off renders byte-identical prompts; `prompt_version` changes anyway because the template files changed (trace comparability note).
 - **rag**: optional parent chunk expansion in the advanced query pipeline (FEAT-017, `VEKTRA_PARENT_EXPANSION_ENABLED`, default off). With `VEKTRA_CHUNKING_STRATEGY=dual`, retrieved child chunks are replaced with their parent chunk's text after the retrieval filter and before token budgeting; children of the same parent collapse into the highest-scored one. Parent-child linkage is now actually persisted (deterministic `uuid5(doc_id, position)` ids, `parent_id` in the Qdrant payload and in the pgvector column), a new `VectorStoreProvider.retrieve()` fetches chunks by id, and the trace records `children_expanded`/`siblings_merged`/`parents_fetched` in a `parent_expansion` step.

docs/reference/configuration.md

@@ -59,8 +59,10 @@ The model name must match the vLLM `--model` path exactly (e.g., `/models/qwen35
 
 | Variable | Type | Default | Description |
 |----------|------|---------|-------------|
-| `VEKTRA_EMBEDDING_PROVIDER` | str | `sentence-transformers` | Embedding provider implementation |
-| `VEKTRA_EMBEDDING_MODEL` | str | `paraphrase-multilingual-MiniLM-L12-v2` | Model name within the selected provider |
+| `VEKTRA_EMBEDDING_PROVIDER` | str | `sentence-transformers` | Embedding provider implementation: `sentence-transformers` (in-process), `tei` (remote) |
+| `VEKTRA_EMBEDDING_MODEL` | str | `paraphrase-multilingual-MiniLM-L12-v2` | Model name within the selected provider (`sentence-transformers` only; a TEI instance serves one fixed model) |
+| `VEKTRA_TEI_URL` | str | `http://localhost:8080` | TEI server base URL (native API, no `/v1` suffix). Used when provider is `tei`. The Qdrant collection is sized from the served model's dimensions at startup |
+| `VEKTRA_TEI_API_KEY` | str | - | Bearer token for the TEI embedding server (`--api-key`). Optional |
 | `VEKTRA_SPARSE_EMBEDDING_PROVIDER` | str | - | Sparse embedding provider: `fastembed-bm25`, `splade` |
 | `VEKTRA_SPARSE_EMBEDDING_MODEL` | str | - | Sparse embedding model name |
 
@@ -101,9 +103,12 @@ The model name must match the vLLM `--model` path exactly (e.g., `/models/qwen35
 | Variable | Type | Default | Description |
 |----------|------|---------|-------------|
 | `VEKTRA_RERANK_ENABLED` | bool | `true` | Enable cross-encoder reranking after retrieval |
-| `VEKTRA_RERANK_PROVIDER` | str | `cross-encoder` | Reranking provider: `flashrank`, `cross-encoder`, `cohere` |
+| `VEKTRA_RERANK_PROVIDER` | str | `cross-encoder` | Reranking provider: `flashrank`, `cross-encoder`, `cohere`, `tei` (remote) |
 | `VEKTRA_RERANK_MODEL` | str | `BAAI/bge-reranker-v2-m3` | Multilingual reranking model. For English-only lightweight deployments: provider=`flashrank`, model=`ms-marco-MiniLM-L-12-v2` |
 | `VEKTRA_RERANK_TOP_K` | int | `5` | Final top-k results after reranking |
+| `VEKTRA_RERANK_API_KEY` | str | - | API key for API-based providers (`cohere`) |
+| `VEKTRA_RERANK_TEI_URL` | str | `http://localhost:8080` | TEI reranker server base URL (one TEI instance per model, e.g. serving `BAAI/bge-reranker-v2-m3`). Used when provider is `tei`. Scores are sigmoid-normalized like the in-process path |
+| `VEKTRA_RERANK_TEI_API_KEY` | str | - | Bearer token for the TEI reranker server. Optional |
 
 ## Ingestion
 

vektra-app/src/vektra_app/main.py

@@ -31,6 +31,7 @@
 from vektra_shared.config import QueryPipelineConfig, VektraSettings
 from vektra_shared.db import init_db
 from vektra_shared.errors import ERR_CONFIG_001, ErrorCategory, ErrorResponse
+from vektra_shared.protocols import EmbeddingProvider
 from vektra_shared.registry import ProviderRegistry
 from vektra_shared.startup import (
     StartupValidationError,
@@ -126,15 +127,31 @@ async def _step_5_register_providers(
     registry.register("llm", "default", llm_provider)
 
     # --- Embedding ---
-    from vektra_index.providers.sentence_transformers import (
-        SentenceTransformersProvider,
-    )
+    embedding_provider: EmbeddingProvider
+    if settings.embedding_provider == "tei":
+        from vektra_index.providers.tei import TEIEmbeddingProvider
 
-    embedding_provider = SentenceTransformersProvider(
-        model_name=settings.embedding_model
-    )
-    registry.register("embedding", "default", embedding_provider)
-    registry.register("embedding", "sentence-transformers", embedding_provider)
+        embedding_provider = TEIEmbeddingProvider(
+            url=settings.tei_url,
+            api_key=settings.tei_api_key,
+        )
+        registry.register("embedding", "default", embedding_provider)
+        registry.register("embedding", "tei", embedding_provider)
+        log.info(
+            "embedding_registered",
+            provider="tei",
+            url=_redact_url(settings.tei_url),
+        )
+    else:
+        from vektra_index.providers.sentence_transformers import (
+            SentenceTransformersProvider,
+        )
+
+        embedding_provider = SentenceTransformersProvider(
+            model_name=settings.embedding_model
+        )
+        registry.register("embedding", "default", embedding_provider)
+        registry.register("embedding", "sentence-transformers", embedding_provider)
 
     # --- Vector store ---
     from vektra_index.adapters import VectorStoreServiceAdapter
@@ -168,6 +185,10 @@ async def _step_5_register_providers(
             api_key=settings.qdrant_api_key,
             collection_name=settings.qdrant_collection,
             active_index_version=settings.active_index_version,
+            # FEAT-024: size the collection from the active embedding model
+            # instead of the hardcoded 384 default (latent bug for any
+            # non-384 model; bge-m3 via TEI is 1024).
+            dense_dimensions=embedding_provider.dimensions(),
         )
         registry.register("vector_store", "default", qdrant_provider)
         registry.register("vector_store", "qdrant", qdrant_provider)
@@ -246,6 +267,9 @@ async def _step_5_register_providers(
 
     pipeline_config = QueryPipelineConfig()
     reranker = create_reranker(pipeline_config.rerank)
+    if reranker is not None:
+        # Registered so the lifespan teardown can close remote clients (FEAT-024)
+        registry.register("reranker", "default", reranker)
 
     # --- Query pipeline (Phase 2: select simple or advanced) ---
     templates_dir = (
@@ -538,6 +562,18 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
 
     # Shutdown
     log.info("shutdown_started")
+    # Close remote provider HTTP clients (TEI, FEAT-024) best-effort
+    for category in ("embedding", "reranker"):
+        try:
+            provider = registry.get(category, "default")
+        except Exception:
+            continue
+        aclose = getattr(provider, "aclose", None)
+        if aclose is not None:
+            try:
+                await aclose()
+            except Exception as exc:
+                log.warning("provider_close_failed", category=category, error=str(exc))
     from vektra_shared.db import get_engine
 
     engine = get_engine()

vektra-core/pyproject.toml

@@ -14,6 +14,7 @@ dependencies = [
     "presidio-analyzer>=2.2",
     "presidio-anonymizer>=2.2",
     "rerankers[flashrank]>=0.5",
+    "httpx>=0.27",
 ]
 
 [build-system]

vektra-core/src/vektra_core/advanced_pipeline.py

@@ -39,7 +39,7 @@
     _history_to_messages,
     _trace_to_dict,
 )
-from vektra_core.reranker import RerankerService
+from vektra_core.reranker import RerankerProtocol
 from vektra_core.templates import TemplateRenderer
 from vektra_shared.config import LLMConfig, QueryPipelineConfig
 from vektra_shared.protocols import (
@@ -87,7 +87,7 @@ def __init__(
         renderer: TemplateRenderer,
         pipeline_config: QueryPipelineConfig,
         sparse_embedding: SparseEmbeddingProvider | None = None,
-        reranker: RerankerService | None = None,
+        reranker: RerankerProtocol | None = None,
     ) -> None:
         self._embedding = embedding
         self._vector_store = vector_store