Skip to content

fix Qwen3Embed panic when concurrency#186

Merged
akshayballal95 merged 1 commit into
StarlightSearch:mainfrom
st0nie:main
Nov 7, 2025
Merged

fix Qwen3Embed panic when concurrency#186
akshayballal95 merged 1 commit into
StarlightSearch:mainfrom
st0nie:main

Conversation

@st0nie

@st0nie st0nie commented Nov 6, 2025

Copy link
Copy Markdown
Contributor

Original code get the rwlock only when embedding is done

When we have multiple concurrent embedding at the same time, kvcache from one request may pollute another embedding process

Like:

Failed to process embedding for "/zion0/smbtest/1111 (1).txt":
embed_file panicked: called `Result::unwrap()` on an `Err` value: shape
mismatch in broadcast_add, lhs: [10, 16, 335, 654], rhs: [10, 1, 335,
335]

Now we hold the write lock during the whole procedure, allow preprocessing to be performed concurrently; but vector generation will not be concurrent.

Original code get the rwlock only when embedding is done

When we have multiple concurrent embedding at the same time, kvcache
from one request may pollute another embedding process

Like:
```txt
Failed to process embedding for "/zion0/smbtest/1111 (1).txt":
embed_file panicked: called `Result::unwrap()` on an `Err` value: shape
mismatch in broadcast_add, lhs: [10, 16, 335, 654], rhs: [10, 1, 335,
335]
```

Now we hold the write lock during the whole procedure, allow
preprocessing to be performed concurrently; but vector generation will
not be concurrent.
@akshayballal95 akshayballal95 merged commit 290a1b8 into StarlightSearch:main Nov 7, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants