Support custom PostgreSQL column types: RawText binds + select_as_text reads#20
Merged
Conversation
…t reads Custom column types (pgvector's vector, inet, citext, ...) were unusable on PostgreSQL: a text rendering from to_db bound with a declared text type, which the server will not implicitly cast to the column's type (SQLSTATE 42804), and reads failed because the engine requests binary result values it cannot decode for such types — the custom-fields guide's own pgvector example hit both. - yara_orm.RawText (a thin str subclass, like Array) binds as an *untyped* parameter on PostgreSQL: the engine declares OID 0 so the server infers the type from context, and sends the value with the text format code so the target type's input function parses it. The Rust Value gets a RawText variant (pg_type None, encode_format Text); every other backend treats it exactly like Text. - Field.select_as_text (new class attribute) makes PostgreSQL SELECT and RETURNING projections read the column through CAST(col AS text) — a bare column cast keeps the column's result name, so decode plans are unaffected. Rendering goes through a new BaseDialect.select_column hook applied at every projection site: full-row selects, only()/defer(), select_related, values()/values_list(), m2m prefetch and RETURNING. Comparison and grouping references keep the column's real type. - The custom-fields guide documents both on a complete pgvector VectorField. - tests/test_raw_text_custom_types.py exercises the shape on every backend via a custom inet field kind (a built-in PostgreSQL type with no implicit text cast and no engine decoder — the exact pgvector failure mode, minus the extension dependency): create/get/only, filter/update/values_list, bulk_create/save, select_related and m2m prefetch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Custom column types (pgvector
vector,inet,citext, ...) were unusable on PostgreSQL, even though the custom-fields guide advertises a pgvectorVectorFieldas its flagship example:to_dbbinds with a declaredtextparameter type, and PostgreSQL has no implicittext→ custom-type assignment cast — every ORM INSERT/UPDATE fails with SQLSTATE 42804. Downstream apps had to install a globalCREATE CAST (text AS vector) WITH INOUT AS IMPLICITvia migration.unsupported PostgreSQL type OID, forcing.defer()workarounds on every read.Fix
yara_orm.RawText— a thinstrsubclass (same pattern asArray). On PostgreSQL it binds untyped: the engine declares OID 0 so the server infers the parameter type from context (the target column, a<=>operand), and sends the value with the text format code so the type's own input function parses it. Implemented as aValue::RawTextvariant in the Rust engine (pg_type→None,encode_format→Format::Text); every other backend treats it exactly like a plain string.Field.select_as_text— new class attribute; PostgreSQL SELECT/RETURNING projections read the column throughCAST(col AS text)(a bare column cast keeps the column's result name, so decode plans are unaffected). Rendering goes through a newBaseDialect.select_columnhook applied at every projection site: full-row selects,only()/defer(),select_related,values()/values_list(), m2m prefetch and RETURNING. Comparison and grouping references keep the column's real type.VectorField.Tests
tests/test_raw_text_custom_types.pyruns on every configured backend using a custominetfield kind — a built-in PostgreSQL type with no implicit text cast and no engine decoder, i.e. the exact pgvector failure shape without the extension dependency. Covers create/get/only(), filter/update/values_list(),bulk_create/save,select_relatedand m2m prefetch, plus unit tests for the dialect hook and the marker type.Verified end-to-end against a real pgvector database:
create, full-rowget,update,bulk_create,values_listand a raw-SQLORDER BY embedding <=> $1KNN query with aRawTextparameter (no::vectorcast, no CREATE CAST) all pass.Full suite green (1221 sqlite under xdist + serial postgres modules),
make lintclean, coverage unchanged vs main (identical missed-line set, all backend-specific code needing live mysql/mariadb/oracle/mssql servers).