Skip to content

fix(gateway): correct Codestral embedding dimension#3197

Open
marius-kilocode wants to merge 5 commits into
mainfrom
fix-embedding-encoding-response
Open

fix(gateway): correct Codestral embedding dimension#3197
marius-kilocode wants to merge 5 commits into
mainfrom
fix-embedding-encoding-response

Conversation

@marius-kilocode
Copy link
Copy Markdown
Contributor

@marius-kilocode marius-kilocode commented May 12, 2026

Kilo indexing uses /api/gateway/embedding-models as the source of truth for local vector-store dimensions. After #3196 stopped forwarding dimensions to upstream providers, mistralai/codestral-embed-2505 returns its native 1536-dimensional vectors.

The catalog still advertised Codestral as 256 dimensions, so the client created a 256-dimensional Qdrant collection and then upserted 1536-dimensional vectors. Qdrant rejects that with Wrong input: Vector dimension error: expected dim: 256, got 1536, which surfaced in the extension as a generic Bad Request.

This updates the Codestral catalog entry to 1536 so the vector store is created with the dimension the gateway actually returns.

const contentType = response.headers.get('content-type') ?? '';
if (!contentType.includes('application/json')) return response;

const body = await response.clone().json();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Unnecessary .clone() leaks the original response body stream

The function calls response.clone().json() to read the body, then constructs a brand-new Response from the re-serialized JSON. The original response body (uncloned) is never consumed or returned, so its readable stream is abandoned — a resource leak for every successful base64 conversion.

Since the original response is not returned or used after this point (a new Response is always returned on success), call response.json() directly to consume the stream:

Suggested change
const body = await response.clone().json();
const body = await response.json();

if (
!item ||
typeof item !== 'object' ||
!Array.isArray((item as { embedding?: unknown }).embedding)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: as casts here violate project coding standards

Both (item as { embedding?: unknown }) on this line and (item as { embedding: number[] }) on line 62 use the as operator, which the project rules flag as something to avoid in favor of flow-sensitive typing or satisfies.

Since item is already narrowed to object (non-null) by the typeof item !== 'object' guard above, you can use 'embedding' in item to narrow further without any cast:

if (!('embedding' in item) || !Array.isArray(item.embedding)) {
  return item;
}
return {
  ...item,
  embedding: floatEmbeddingToBase64(item.embedding as number[]),
};

This still needs one cast for item.embedding as number[] (since Array.isArray widens to unknown[]), but that's a much narrower, more defensible cast than the current two.

@kilo-code-bot
Copy link
Copy Markdown
Contributor

kilo-code-bot Bot commented May 12, 2026

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Changes Since Last Review

The new commit reverted buildDownstreamResponse entirely (resolving both previously flagged issues — the .clone() stream leak and the as cast violations), and instead fixes the root cause: the Codestral catalog dimension was wrong (256 → 1536). This is a cleaner fix.

  • RESOLVED — WARNING: .clone() stream leak (embedding-request.ts:49) — buildDownstreamResponse removed
  • RESOLVED — SUGGESTION: as casts (embedding-request.ts:56) — buildDownstreamResponse removed
Files Reviewed (5 files)
  • apps/web/src/lib/ai-gateway/embeddings/embedding-request.ts — no issues
  • apps/web/src/lib/ai-gateway/embeddings/embedding-request.test.ts — no issues
  • apps/web/src/lib/ai-gateway/embeddings/kilo-embedding-models.ts — no issues
  • apps/web/src/app/api/gateway/embedding-models/route.test.ts — no issues
  • apps/web/src/app/api/openrouter/embeddings/route.ts — no issues

Reviewed by claude-sonnet-4.6 · 245,185 tokens

@marius-kilocode marius-kilocode changed the title fix(gateway): support Kilo embedding request format fix(gateway): correct Codestral embedding dimension May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant