Skip to content

vector-store: BM25 full-text search HTTP endpoint#471

Merged
ewienik merged 1 commit into
scylladb:masterfrom
knowack1:vector-685-http-api-full-text-search
Jun 10, 2026
Merged

vector-store: BM25 full-text search HTTP endpoint#471
ewienik merged 1 commit into
scylladb:masterfrom
knowack1:vector-685-http-api-full-text-search

Conversation

@knowack1

@knowack1 knowack1 commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

Add the POST /api/v1/indexes/{keyspace}/{index}/bm25 endpoint for BM25 full-text search.

Changes

  • Add PostIndexBm25Request and PostIndexBm25Response types to httpapi crate
  • Add post_index_25 route handler with OpenAPI documentation
  • Endpoint currently returns 501 Not Implemented as a stub
  • Update openapi.json with the new endpoint spec

API

Request:

{
  "query": "search text",
  "limit": 10
}

Response (200):

{
  "primary_keys": { "column_name": [...] },
  "scores": [0.95, 0.87, ...]
}

Fixes: VECTOR-685

@knowack1 knowack1 requested a review from m-szymon June 2, 2026 09:53
@knowack1 knowack1 force-pushed the vector-685-http-api-full-text-search branch from 04197e9 to c5cc093 Compare June 3, 2026 15:52
@knowack1 knowack1 requested a review from Copilot June 3, 2026 15:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an HTTP API surface (types + routing + OpenAPI spec) for a planned full-text search (BM25) capability in the vector-store service.

Changes:

  • Registers a new POST endpoint: /api/v1/indexes/{keyspace}/{index}/bm25 (currently returns 501 Not Implemented).
  • Introduces request/response schemas for BM25 search in the httpapi crate.
  • Updates the checked-in OpenAPI spec (api/openapi.json) to include the new endpoint and schemas.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
crates/vector-store/src/httproutes.rs Adds router registration and the (stub) handler + OpenAPI annotations for the BM25 endpoint.
crates/httpapi/src/lib.rs Adds PostIndexBm25Request / PostIndexBm25Response types for the new endpoint.
api/openapi.json Adds the new /bm25 path and component schemas to the published OpenAPI document.

Comment thread crates/vector-store/src/httproutes.rs Outdated
Comment thread crates/vector-store/src/httproutes.rs Outdated
Comment thread crates/vector-store/src/httproutes.rs
@knowack1 knowack1 changed the title HTTP API full text search vector-store: BM25 full-text search HTTP endpoint Jun 3, 2026
@knowack1 knowack1 requested a review from ewienik June 3, 2026 15:59
m-szymon added a commit to m-szymon/scylladb that referenced this pull request Jun 5, 2026
ScyllaDB needs to support full-text search queries through the vector
store service. This adds the client-side BM25 endpoint so ScyllaDB can
request ranked full-text search results from the vector store, alongside
the existing ANN (approximate nearest neighbor) endpoint.

Tests are added mirroring the ann() test suite. The test infrastructure
is extended with a mock server mode and a lightweight schema builder
helper to avoid spinning up a full CQL environment where it is not needed.

Fixes SCYLLADB-1511
Ref scylladb/vector-store#471
@knowack1 knowack1 force-pushed the vector-685-http-api-full-text-search branch from c5cc093 to d935e3b Compare June 8, 2026 11:18
@knowack1 knowack1 requested a review from Copilot June 8, 2026 11:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

Comment thread crates/vector-store/src/httproutes.rs
Comment thread crates/vector-store/src/httproutes.rs
Comment thread crates/vector-store/src/httproutes.rs Outdated
Comment thread crates/httpapi/src/lib.rs
Comment thread api/openapi.json
Comment thread api/openapi.json
Comment thread api/openapi.json
Comment thread api/openapi.json Outdated
@knowack1 knowack1 force-pushed the vector-685-http-api-full-text-search branch from dd96def to 93a5325 Compare June 8, 2026 11:48
@knowack1 knowack1 requested a review from Copilot June 8, 2026 11:48
@knowack1 knowack1 force-pushed the vector-685-http-api-full-text-search branch from 93a5325 to ae19911 Compare June 8, 2026 11:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comment thread crates/vector-store/src/httproutes.rs
Comment thread api/openapi.json Outdated
@knowack1 knowack1 force-pushed the vector-685-http-api-full-text-search branch from ae19911 to d1c07cb Compare June 8, 2026 11:52
@knowack1 knowack1 requested a review from Copilot June 8, 2026 11:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread crates/vector-store/src/httproutes.rs
@knowack1 knowack1 marked this pull request as ready for review June 8, 2026 11:59
@knowack1

knowack1 commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author

@m-szymon @ewienik please review API for full-text search. I think that we can merge API stub and it's documentation before the desired implementation. The API agreement is also required to accomplish client changes on scylladb side.

Comment thread api/openapi.json
m-szymon added a commit to m-szymon/scylladb that referenced this pull request Jun 9, 2026
ScyllaDB needs to support full-text search queries through the vector
store service. This adds the client-side BM25 endpoint so ScyllaDB can
request ranked full-text search results from the vector store, alongside
the existing ANN (approximate nearest neighbor) endpoint.

Tests are added mirroring the ann() test suite. The test infrastructure
is extended with a mock server mode and a lightweight schema builder
helper to avoid spinning up a full CQL environment where it is not needed.

Fixes SCYLLADB-1511
Ref scylladb/vector-store#471
m-szymon added a commit to m-szymon/scylladb that referenced this pull request Jun 9, 2026
ScyllaDB needs to support full-text search queries through the vector
store service. This adds the client-side BM25 endpoint so ScyllaDB can
request ranked full-text search results from the vector store, alongside
the existing ANN (approximate nearest neighbor) endpoint.

Tests are added mirroring the ann() test suite. The test infrastructure
is extended with a mock server mode and a lightweight schema builder
helper to avoid spinning up a full CQL environment where it is not needed.

Fixes SCYLLADB-1511
Ref scylladb/vector-store#471
Add the /api/v1/indexes/{keyspace}/{index}/bm25 POST endpoint that
will perform BM25 full-text search. Currently returns 501 Not
Implemented.

Fixes: VECTOR-685
@knowack1 knowack1 force-pushed the vector-685-http-api-full-text-search branch from d1c07cb to dbd1c9d Compare June 10, 2026 13:39
@knowack1 knowack1 requested a review from ewienik June 10, 2026 13:40
@ewienik ewienik added this pull request to the merge queue Jun 10, 2026
Merged via the queue into scylladb:master with commit 8a058c1 Jun 10, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants