perf(faiss): batch Vector CSV embeddings

[greglum]

Summary

• embed every ordinary Vector CSV in one model call instead of one call per Source
• validate embedding row counts and dimensions before persisting per-Source artifacts
• slice the batched response back into the existing stable dataset and vector files
• retain the per-Source keyword-search fallback and one master-index rebuild per ingestion group

Why

A Vector CSV can contain many Sources. The prior implementation invoked the embedding engine once for every Source, which adds avoidable model overhead during grouped corpus ingestion. This keeps the existing artifact and removal contracts while changing only the embedding unit of work.

Behavior change

The previous code popped keywordSearch from the caller's parameter dict inside the per-Source loop, so keyword extraction only ever applied to the first Source of the first document and the caller's dict was mutated. This PR reads the flag once per document from a copy, so extraction now runs for every configured Source and the caller's dict is left intact. This is an intentional fix, covered by test_keyword_search_remains_a_per_source_fallback.

Verification

• focused FAISS batch tests: 5 passed
• Black formatting check passed
• Ruff passed for the new test and changed code; the file retains six pre-existing diagnostics outside this change

The older FAISS integration tests require a configured local model or a running Tomcat instance and fail during their environment setup before exercising this path.

Related work

• Add opt-in multipart file upload batching python-sdk#36 is the client-side counterpart: it batches the client-to-server upload transport, while this PR batches the embedding-engine calls during ingestion. The two are independent.
• feat(faiss): best-effort per-file embedding statuses #2762 builds on this branch and extends the best-effort per-file embedding statuses from Best-effort Vector Database Embedding #2205 to the FAISS engine.

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.