Embeddings generation support

[chuckbeasley]

Currently, without using Foundry Local, I can generate embeddings using the Phi series ONNX models. It would be nice for Foundry Local to add support for that so that I don't have external dependencies, like Ollama or loading the same Phi model in a Docker container.

[chuckbeasley]

Tagging @MaanavD for visibility.

[behroozbc]

I think this is a good thing to add to foundry local

[alxxdev]

Hi @chuckbeasley,

Embeddings are actually already supported in Foundry Local via the OpenAI-compatible /v1/embeddings endpoint. The catalog includes embedding models:

$ foundry model list --type embedding -v
+---------------------+-----------+--------+--------+-------+--------+----------+------------------------------------+------------------------------------+
| Model Name          | Type      | Size   | Device | Tools | Cached | License  | Display Name                       | Model ID                           |
+---------------------+-----------+--------+--------+-------+--------+----------+------------------------------------+------------------------------------+
| qwen3-embedding-0.6b| Embedding | 495 MB | CPU    | ○     | ●      | apache-2.0| qwen3-embedding-0.6b-generic-cpu  | qwen3-embedding-0.6b-generic-cpu:1 |
| qwen3-embedding-8b  | Embedding | 5.6 GB | CPU    | ○     | ○      | apache-2.0| qwen3-embedding-8b-generic-cpu    | qwen3-embedding-8b-generic-cpu:1   |
+---------------------+-----------+--------+--------+-------+--------+----------+------------------------------------+------------------------------------+

To use them:

foundry model download qwen3-embedding-0.6b
foundry model load qwen3-embedding-0.6b

Then call the API:

curl http://127.0.0.1:<port>/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-embedding-0.6b","input":"hello world"}'

Or from Python with the OpenAI SDK:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:<port>/v1", api_key="none")
resp = client.embeddings.create(model="qwen3-embedding-0.6b", input="hello world")
print(resp.data[0].embedding)

No CLI command for embeddings yet (only HTTP API), but the functionality is there. Hope this helps!