vector-store: add Tantivy dependency and runtime FTS configuration#439
vector-store: add Tantivy dependency and runtime FTS configuration#439knowack1 wants to merge 1 commit into
Conversation
As I recall we were talking about dynamic flag, not static one. Maybe better would be to add configuration env? |
Unfortunately we did not make notes in jira ticket afterward :) We can switch to dynamic config. |
knowack1
force-pushed
the
vector-620-add-tantivy
branch
2 times, most recently
from
0b5e910 to
3d5a009
Compare
QuerthDP
left a comment
QuerthDP
left a comment
There was a problem hiding this comment.
Looks good.
Although I think the cargo machete will fail since we're not using the tantivy anywhere.
| @@ -0,0 +1,4 @@ | |||
| /* | |||
| * Copyright 2025-present ScyllaDB | |||
There was a problem hiding this comment.
vector-store: add Tantivy dependency and fts feature flag
Add tantivy 0.22 as an optional dependency gated behind the 'fts'
cargo feature flag. When compiled without --features fts (the default),
the dependency is excluded from the build.
Update the commit message
knowack1
force-pushed
the
vector-620-add-tantivy
branch
2 times, most recently
from
3d5a009 to
3068b6d
Compare
Add Tantivy 0.22 as a workspace dependency for full-text search support. Introduce VECTOR_STORE_FTS_ENABLED env var (bool, default: false) to control FTS at runtime, parsed in config_manager alongside existing env vars. Add fts_enabled field to Config and a placeholder fts module. Fixes VECTOR-620
knowack1
force-pushed
the
vector-620-add-tantivy
branch
from
3068b6d to
3dfb661
Compare
knowack1
changed the title
Done |
Done. Added use in fts.rs. |
There was a problem hiding this comment.
Pull request overview
This PR lays groundwork for future full-text search (FTS) support by adding Tantivy to the workspace and introducing a runtime configuration flag (VECTOR_STORE_FTS_ENABLED, default false) that is parsed into the Vector Store Config.
Changes:
- Add Tantivy (
0.22) as a workspace dependency and enable it for thevector-storecrate. - Extend
Configwithfts_enabled: booland parseVECTOR_STORE_FTS_ENABLEDinload_config. - Add a placeholder
ftsmodule to ensure the Tantivy dependency is wired into the crate.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/vector-store/src/lib.rs | Adds fts module and introduces Config::fts_enabled with default false. |
| crates/vector-store/src/fts.rs | Placeholder module that references Tantivy. |
| crates/vector-store/src/config_manager.rs | Parses VECTOR_STORE_FTS_ENABLED into Config. |
| crates/vector-store/Cargo.toml | Adds tantivy as a crate dependency (via workspace). |
| Cargo.toml | Adds tantivy = "0.22" to workspace dependencies. |
| Cargo.lock | Locks Tantivy and its transitive dependencies. |
| if let Some(fts_enabled) = env("VECTOR_STORE_FTS_ENABLED") | ||
| .ok() | ||
| .map(|v| { | ||
| v.trim().parse().or(Err(anyhow!( | ||
| "Unable to parse VECTOR_STORE_FTS_ENABLED env (true/false)" | ||
| ))) | ||
| }) | ||
| .transpose()? | ||
| { | ||
| config.fts_enabled = fts_enabled; | ||
| } |
| * Copyright 2026-present ScyllaDB | ||
| * SPDX-License-Identifier: LicenseRef-ScyllaDB-Source-Available-1.1 | ||
| */ | ||
|
|
| #[derive(Clone, Debug)] | ||
| pub struct Config { | ||
| pub vector_store_addr: std::net::SocketAddr, | ||
| pub scylladb_uri: String, | ||
| pub threads: Option<usize>, | ||
| pub memory_limit: Option<u64>, | ||
| pub memory_usage_check_interval: Option<Duration>, | ||
| pub opensearch_addr: Option<String>, | ||
| pub credentials: Option<Credentials>, | ||
| pub usearch_simulator: Option<Vec<Duration>>, | ||
| pub alter_index_simulator: bool, | ||
| pub cql_connection_timeout: Option<Duration>, | ||
| pub cql_keepalive_interval: Option<Duration>, | ||
| pub cql_keepalive_timeout: Option<Duration>, | ||
| pub cql_tcp_keepalive_interval: Option<Duration>, | ||
| pub cql_uri_translation_map: Option<HashMap<SocketAddr, SocketAddr>>, | ||
| pub cdc_safety_interval: Option<Duration>, | ||
| pub cdc_sleep_interval: Option<Duration>, | ||
| pub cdc_fine_safety_interval: Option<Duration>, | ||
| pub cdc_fine_sleep_interval: Option<Duration>, | ||
| pub disable_colors: bool, | ||
| pub fts_enabled: bool, | ||
| pub tls_cert_path: Option<std::path::PathBuf>, |
Add Tantivy 0.22 as a workspace dependency for full-text search support.
Introduce
VECTOR_STORE_FTS_ENABLEDenv var (true/false, default:false) tocontrol FTS at runtime, parsed in
config_manageralongside existing env vars.Add
fts_enabledfield toConfigand a placeholderftsmodule.Fixes: VECTOR-620