Skip to content

validator: add Alternator FLOAT32VECTOR type end-to-end tests#437

Open
QuerthDP wants to merge 1 commit into
scylladb:masterfrom
QuerthDP:alternator-v-type-tests
Open

validator: add Alternator FLOAT32VECTOR type end-to-end tests#437
QuerthDP wants to merge 1 commit into
scylladb:masterfrom
QuerthDP:alternator-v-type-tests

Conversation

@QuerthDP
Copy link
Copy Markdown
Member

@QuerthDP QuerthDP commented May 6, 2026

Add end-to-end coverage for the Alternator optimized vector type using requests with the new encoding injected into the JSON body.

Verify that queries work for items stored with the optimized vector representation and remain compatible with the existing L-based vector representation in the same index.

Fixes: VECTOR-666

@QuerthDP QuerthDP force-pushed the alternator-v-type-tests branch from b9b620c to 7952f36 Compare May 27, 2026 10:51
@QuerthDP QuerthDP force-pushed the alternator-v-type-tests branch 3 times, most recently from fdb2975 to 4dae248 Compare May 28, 2026 09:36
@QuerthDP QuerthDP marked this pull request as ready for review May 28, 2026 09:38
@QuerthDP QuerthDP requested review from ewienik and m-szymon May 28, 2026 09:38
Copy link
Copy Markdown
Contributor

@m-szymon m-szymon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I was considering adding such tests to types.rs tests (added here #467). I deliberately named it types, not key_types.
  2. Adding optimized data to index and querying are slightly separate issues.
  3. I was considering defining test datasets using "binary" type and only extending the json injector to convert them to FLOAT32VECTOR when expected - it could better reuse existing utils.

But it works, so consider if you would like to improve anything and we can merge it. I can later refactor.

Comment thread crates/validator/src/alternator/mod.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
Comment thread crates/validator/src/alternator/query.rs Outdated
Add end-to-end coverage for the Alternator optimized vector type
using requests with the new encoding injected into the JSON body.

Verify that queries work for items stored with the optimized vector
representation and remain compatible with the existing L-based
vector representation in the same index.
@QuerthDP QuerthDP force-pushed the alternator-v-type-tests branch from 4dae248 to 97c9968 Compare May 28, 2026 14:11
@QuerthDP
Copy link
Copy Markdown
Member Author

Changelog:

  • rebased on master
  • moved tests from query.rs to types.rs
  • merged the tests into one, which checks both insert and query

@QuerthDP QuerthDP requested a review from m-szymon May 28, 2026 14:13
@m-szymon
Copy link
Copy Markdown
Contributor

It is easier to ask AI to do it, than explain it: 3459c12

The problem is the pk_v_scan is not inserted at scan phase.
I was thinking about adding same trick as @nyh used for Java driver: scylladb/alternator-client-java#87

I also noticed that other test in that file don't actually check returned values. With that fixed, we could extend and use the query_with_key_type function.

@ewienik
Copy link
Copy Markdown
Collaborator

ewienik commented Jun 3, 2026

It is easier to ask AI to do it, than explain it: 3459c12

The problem is the pk_v_scan is not inserted at scan phase. I was thinking about adding same trick as @nyh used for Java driver: scylladb/alternator-client-java#87

I also noticed that other test in that file don't actually check returned values. With that fixed, we could extend and use the query_with_key_type function.

@m-szymon Does it mean that something needs to be done in this PR?

@m-szymon
Copy link
Copy Markdown
Contributor

m-szymon commented Jun 3, 2026

It is easier to ask AI to do it, than explain it: 3459c12
The problem is the pk_v_scan is not inserted at scan phase. I was thinking about adding same trick as @nyh used for Java driver: scylladb/alternator-client-java#87
I also noticed that other test in that file don't actually check returned values. With that fixed, we could extend and use the query_with_key_type function.

@m-szymon Does it mean that something needs to be done in this PR?

Yes - one small inconsistency that pk_v_scan is not a scan test.
We can either remove/ignore it and enhance tests separately or fix from the beginning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants