Skip to content

Add VECTOR and BSON data types support#16

Open
okramarenko wants to merge 9 commits intoadd10supportfrom
add_vector
Open

Add VECTOR and BSON data types support#16
okramarenko wants to merge 9 commits intoadd10supportfrom
add_vector

Conversation

@okramarenko
Copy link
Copy Markdown
Collaborator

@okramarenko okramarenko commented Apr 28, 2026

Note

Medium Risk
Introduces new extended-type parsing and session configuration plus new read/write paths for BSON/VECTOR values; mistakes could cause incorrect binary encoding/decoding or compatibility issues on older SingleStore versions.

Overview
Adds BSON and VECTOR as provider-level types backed by SingleStore extended column metadata. This includes parsing new fixed-length field extensions in ColumnDefinitionPayload, mapping them through TypeMapper/SingleStoreDbType, and exposing VECTOR schema details (VectorDimensions, VectorElementTypeName) via SingleStoreDbColumn.

Implements end-to-end value handling: new Vector*ColumnReader implementations for result decoding, SingleStoreBinaryValueConverter for parameter inference and vector/bson byte conversion, and updates SingleStoreParameter/SingleStoreBulkCopy to serialize these values correctly (including bulk copy casting expressions).

Adds a connection-string flag EnableExtendedDataTypes (default true) and session initialization/reset logic to SET SESSION enable_extended_types_metadata = TRUE when supported (>= 8.5.28), with explicit opt-in producing an error on unsupported servers; updates tests accordingly.

Reviewed by Cursor Bugbot for commit 08a4139. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread src/SingleStoreConnector/Protocol/Payloads/ColumnDefinitionPayload.cs Outdated
Comment thread src/SingleStoreConnector/Core/TypeMapper.cs Outdated
Comment thread src/SingleStoreConnector/SingleStoreParameter.cs
valueBytes.CopyTo(bytes, i * sizeof(double));
}
return bytes;
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big-endian double conversion missing NET5 optimization path

Low Severity

ConvertDoublesToBytes uses the BitConverter.GetBytes/Array.Reverse fallback on big-endian platforms, but ConvertFloatsToBytes (and the corresponding VectorFloat64ColumnReader) both use BinaryPrimitives.WriteDoubleLittleEndian/ReadDoubleLittleEndian under #if NET5_0_OR_GREATER. The double converter is functionally correct but inconsistent with every other conversion method in the PR.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 64c0676. Configure here.

Comment thread src/SingleStoreConnector/Core/ServerSession.cs Outdated
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default mode and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 08a4139. Configure here.

((column.ColumnFlags & ColumnFlags.Blob) != 0 || column.ColumnType is ColumnType.TinyBlob or ColumnType.Blob or ColumnType.MediumBlob or ColumnType.LongBlob);
IsLong = mySqlDbType != SingleStoreDbType.Vector &&
column.ColumnLength > 255 &&
((column.ColumnFlags & ColumnFlags.Blob) != 0 || column.ColumnType is ColumnType.TinyBlob or ColumnType.Blob or ColumnType.MediumBlob or ColumnType.LongBlob);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BSON column not excluded from IsLong like Vector

Low Severity

The IsLong calculation explicitly excludes SingleStoreDbType.Vector but does not exclude SingleStoreDbType.Bson. Since BSON is transported as ColumnType.Blob with columnSize: uint.MaxValue, it will have column.ColumnLength > 255 and the Blob flag set, causing IsLong to be true. If the intent was to treat these new extended types consistently (similar to how Vector is excluded), BSON may also need exclusion here. If BSON being IsLong = true is intentional since BSON documents can genuinely be large, this can be dismissed.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 08a4139. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant