add client-side embeddings cached in IndexedDB

[Mbeaulne]

Description

Adds embedding-based semantic search as a pre-processing step before LLM reranking. When an AI provider API base is configured, the component search now fetches text embeddings for both the search query and all indexed components, ranks them by cosine similarity, and merges those results with the top lexical matches before passing candidates to the LLM reranker. This improves the quality of candidates surfaced to the reranker, particularly for queries that don't share keywords with component names or descriptions.

A new componentSearchEmbeddings service handles embedding generation, cosine similarity scoring, and persistent caching of embeddings in IndexedDB (keyed by a FNV-1a hash of the text and model name) to avoid redundant API calls across searches. A mergeUniqueMatches helper deduplicates and merges lexical and embedding results by digest before reranking.

The embedding fetch is tracked with an isEmbeddingSearchPending flag, which gates the rerank active state and disables the AI search buttons while in progress, keeping the spinner and disabled states consistent with the existing reranking UX.

Related Issue and Pull requests

Type of Change

• Bug fix
• New feature
• Improvement
• Cleanup/Refactor
• Breaking change
• Documentation update

Checklist

• I have tested this does not break current pipelines / runs functionality
• I have tested the changes on staging

Screenshots (if applicable)

Test Instructions

1. Configure an OpenAI-compatible API base (e.g. https://api.openai.com/v1) in AI provider settings.
2. Open the component search panel and enter a query that is semantically related to a component but does not share exact keywords.
3. Click the AI search (Sparkles) button and verify the spinner appears during both the embedding fetch and reranking phases.
4. Confirm that results are ranked by semantic relevance.
5. Repeat the same search and verify via network tooling that no additional embedding API calls are made (cache hit).
6. Verify that searches still work correctly when no API base is configured (embedding step is skipped gracefully).

Additional Comments

The embedding model is hardcoded to text-embedding-3-small. If no apiBase is set, the embedding step is skipped entirely and behaviour is identical to before this change.

[github-actions]

🎩 Preview

A preview build has been created at: 06-18-add_client-side_embeddings_cached_in_indexeddb/92a80e1

[Mbeaulne]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Debounce component search input #2433
• add client-side embeddings cached in IndexedDB #2432 👈 (View in Graphite)
• Add search quality expectation tests for lexical search #2431
• Add deep AI search to rerank all components in selected sources #2430
• Add source-diverse AI rerank candidate pool #2429
• Add negative constraint parsing to lexical search #2428
• Add fuzzy/typo-tolerant matching for component name and I/O fields #2427
• Improve component search scoring relevance #2426
• Add synonym expansion to component lexical search #2425
• Normalize component search tokens for better matching #2424
• Expand component search indexing fields #2423
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Mbeaulne]

🤖 This is an AI-generated code review comment.

[MEDIUM] The full unfiltered index is embedded on every search, and embedTextsWithCache sends all cache-missing texts in a single fetch body with no chunking. On a cold cache for a large library this exceeds OpenAI embeddings limits (2048 inputs / ~300k tokens) → 400, and the whole semantic layer silently returns [] (swallowed by the host catch). Consider batching missing texts (e.g. 256–512/request) and merging, embedding only the candidate set, or capping N.

[Mbeaulne]

🤖 This is an AI-generated code review comment.

[LOW/MEDIUM] The cache key and the textHash validation field both derive from the same 32-bit FNV-1a hash, so a key collision (~50% likelihood near ~77k distinct texts) returns a stale-but-"validated" vector with no detection. Consider storing the full text (or a wider hash, e.g. SHA-256, or including text length) as the validation field so a key collision is caught as a miss.

[Mbeaulne]

🤖 This is an AI-generated code review comment.

[LOW] Dexie schema is pinned at version(1); cache busting relies on CACHE_SCHEMA_VERSION embedded in the key, so bumping it orphans all prior rows permanently (no prune). Also, there's no QuotaExceededError handling on this bulkPut — the rejection is swallowed by the host catch, silently disabling caching. Consider pruning rows whose key lacks the current schemaVersion:model: prefix on a schema bump, and catching quota errors to trigger a prune.

[Mbeaulne]

🤖 This is an AI-generated code review comment.

[LOW] This mergeUniqueMatches (dedupe-by-digest) is duplicated verbatim in useComponentSearchV2State.ts (~line 53). Consider exporting it once (e.g. from componentSearchIndex.ts) and importing it in both hosts.

[Mbeaulne]

🤖 This is an AI-generated code review comment.

maxTypoDistance allows edit distance 1 on 4-char tokens, so short generic IO names collide: data<->date, path<->bath, list<->last. Fuzzy is name/io-only and scored at 0.75x so exact hits still win, but a typo-free data could pull in date-named IO. Consider raising the distance-1 floor to length >= 5.