Skip to content

Support custom PostgreSQL column types: RawText binds + select_as_text reads#20

Merged
vsdudakov merged 1 commit into
mainfrom
fix/pg-custom-column-types
Jul 16, 2026
Merged

Support custom PostgreSQL column types: RawText binds + select_as_text reads#20
vsdudakov merged 1 commit into
mainfrom
fix/pg-custom-column-types

Conversation

@vsdudakov

Copy link
Copy Markdown
Owner

Problem

Custom column types (pgvector vector, inet, citext, ...) were unusable on PostgreSQL, even though the custom-fields guide advertises a pgvector VectorField as its flagship example:

  • Writes: a text rendering returned from to_db binds with a declared text parameter type, and PostgreSQL has no implicit text → custom-type assignment cast — every ORM INSERT/UPDATE fails with SQLSTATE 42804. Downstream apps had to install a global CREATE CAST (text AS vector) WITH INOUT AS IMPLICIT via migration.
  • Reads: the engine requests binary result values, which it cannot decode for a custom type — selecting the column raised unsupported PostgreSQL type OID, forcing .defer() workarounds on every read.

Fix

  • yara_orm.RawText — a thin str subclass (same pattern as Array). On PostgreSQL it binds untyped: the engine declares OID 0 so the server infers the parameter type from context (the target column, a <=> operand), and sends the value with the text format code so the type's own input function parses it. Implemented as a Value::RawText variant in the Rust engine (pg_typeNone, encode_formatFormat::Text); every other backend treats it exactly like a plain string.
  • Field.select_as_text — new class attribute; PostgreSQL SELECT/RETURNING projections read the column through CAST(col AS text) (a bare column cast keeps the column's result name, so decode plans are unaffected). Rendering goes through a new BaseDialect.select_column hook applied at every projection site: full-row selects, only()/defer(), select_related, values()/values_list(), m2m prefetch and RETURNING. Comparison and grouping references keep the column's real type.
  • The custom-fields guide now documents both on a complete pgvector VectorField.

Tests

tests/test_raw_text_custom_types.py runs on every configured backend using a custom inet field kind — a built-in PostgreSQL type with no implicit text cast and no engine decoder, i.e. the exact pgvector failure shape without the extension dependency. Covers create/get/only(), filter/update/values_list(), bulk_create/save, select_related and m2m prefetch, plus unit tests for the dialect hook and the marker type.

Verified end-to-end against a real pgvector database: create, full-row get, update, bulk_create, values_list and a raw-SQL ORDER BY embedding <=> $1 KNN query with a RawText parameter (no ::vector cast, no CREATE CAST) all pass.

Full suite green (1221 sqlite under xdist + serial postgres modules), make lint clean, coverage unchanged vs main (identical missed-line set, all backend-specific code needing live mysql/mariadb/oracle/mssql servers).

…t reads

Custom column types (pgvector's vector, inet, citext, ...) were unusable on
PostgreSQL: a text rendering from to_db bound with a declared text type,
which the server will not implicitly cast to the column's type (SQLSTATE
42804), and reads failed because the engine requests binary result values it
cannot decode for such types — the custom-fields guide's own pgvector example
hit both.

- yara_orm.RawText (a thin str subclass, like Array) binds as an *untyped*
  parameter on PostgreSQL: the engine declares OID 0 so the server infers the
  type from context, and sends the value with the text format code so the
  target type's input function parses it. The Rust Value gets a RawText
  variant (pg_type None, encode_format Text); every other backend treats it
  exactly like Text.
- Field.select_as_text (new class attribute) makes PostgreSQL SELECT and
  RETURNING projections read the column through CAST(col AS text) — a bare
  column cast keeps the column's result name, so decode plans are unaffected.
  Rendering goes through a new BaseDialect.select_column hook applied at
  every projection site: full-row selects, only()/defer(), select_related,
  values()/values_list(), m2m prefetch and RETURNING. Comparison and grouping
  references keep the column's real type.
- The custom-fields guide documents both on a complete pgvector VectorField.
- tests/test_raw_text_custom_types.py exercises the shape on every backend
  via a custom inet field kind (a built-in PostgreSQL type with no implicit
  text cast and no engine decoder — the exact pgvector failure mode, minus
  the extension dependency): create/get/only, filter/update/values_list,
  bulk_create/save, select_related and m2m prefetch.
@vsdudakov
vsdudakov merged commit 7111f4d into main Jul 16, 2026
4 checks passed
@vsdudakov
vsdudakov deleted the fix/pg-custom-column-types branch July 16, 2026 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant