Skip to content

Disabling semantic search doesn't release the embedding model's memory #2

Description

@s3rverro0m

Disabling semantic search doesn't actually free the embedding model. unloadEmbeddingModel() in src/lib/server/embeddings/model.ts just nulls out pipelinePromise/loadedOnnxId:

export function unloadEmbeddingModel(): void {
	pipelinePromise = null;
	loadedOnnxId = null;
}

That drops the JS reference but never calls .dispose() on the transformers.js pipeline, so the onnxruntime session (model weights + its worker thread pool) stays resident in the process. If semantic search was ever turned on once, RAM never comes back down after disabling it — only a full process restart clears it.

Noticed this because my instance sits right at its container memory limit and gets OOM-killed every ~1h, even with semantic search (and everything else that touches embeddings) toggled off in settings. Confirmed via /proc/1/status the process RSS climbs to the limit, and thread count sits around 80 for a plain SvelteKit app, which lines up with an ORT thread pool never getting torn down.

Fix is small, keep a reference to the extractor returned by pipeline() and call extractor.dispose() in unloadEmbeddingModel() before dropping the refs. Happy to open a PR if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions