Skip to content

fix(sl): project dotted physical columns to contract-valid aliases#353

Open
sricursion wants to merge 1 commit into
Kaelio:mainfrom
sricursion:fix/337-dotted-manifest-columns
Open

fix(sl): project dotted physical columns to contract-valid aliases#353
sricursion wants to merge 1 commit into
Kaelio:mainfrom
sricursion:fix/337-dotted-manifest-columns

Conversation

@sricursion

@sricursion sricursion commented Jul 12, 2026

Copy link
Copy Markdown

Fixes #337

Makes manifest-backed sources with dotted physical column names queryable instead of hard-failing the TS/Python contract on every sl_query.

Warehouse column names may legally contain . (quoted identifiers on Postgres/Snowflake — the telemetry case had CJK names like 注文.金額 — and MongoDB field names). projectManifestEntry copied them verbatim into columns[].name and grain, and toResolvedWire then rejected the source with ComposeContractError: must be unqualified (no '.') — ktx generated a source it refused to query, on every request that touched it (root-cause possibility 1 from the issue).

Approach

Manifest projection now emits contract-valid output names and keeps the physical reference in expr:

  • A dotted physical name gets a deterministic alias (._, deduped against sibling columns with a numeric suffix), in columns[].name and grain both.
  • The physical reference survives as the column's expr, quoted with the connection dialect's identifier character (reusing each connector's quoteIdentifier). The Python planner parses column expr in the connection's native dialect (the convention fix(sl): valid reserved-word and week-grain SQL on BigQuery/MySQL #340 enforced), so a hardcoded postgres-style quote would break backtick dialects — this is why the quoter is threaded per connection rather than hardcoded.
  • The Python side needs no changes: an aliased column is exactly a computed column, an already-supported path.
  • Non-SQL connections (MongoDB) alias without expr: their sources are searchable, and they are never SQL-queried (assertSqlQueryableConnection blocks that earlier).
  • The authoring contract stays strict: a dot in an agent-written source is still rejected, because there it genuinely signals a mistaken table.column qualification. Only the manifest projection — where names are physical by definition — normalizes.

Plumbing: manifestIdentifierQuoterForConnectionType resolves the quoter from the catalog's connectionType (accepting both POSTGRESQL descriptor form and postgres driver form via a reverse map derived from the existing DRIVER_TO_CONNECTION_TYPE), cached per connection in SemanticLayerService; local-sl.ts resolves it from project config through the same localConnectionInfoFromConfig the catalog port uses. Sources whose connection can't be resolved project without a quoter — still searchable, and querying them fails earlier at executeQuery.

Tests

  • Projection regression: dotted + CJK physical columns → aliased names, dialect-quoted expr, aliased grain, and the projected source passes toResolvedWire (the exact contract Source contract rejects column/grain names containing "." — blocks sl_query for the source #337 tripped). Fails on main, passes here.
  • Deterministic collision handling (a.b alongside a_ba_b_2), dotted pk → aliased grain.
  • Quoter resolution per dialect ("…" postgres/snowflake, `…` mysql), driver-form input, and null for MongoDB/unknown.
  • End-to-end check against the Python engine: a source shaped exactly like the fixed projection output (alias + quoted expr) compiles on both quoting families —
    postgres: SELECT ("注文.金額") AS 注文_金額, SUM(("注文.金額")) AS total_amount FROM analytics.sales AS sales …
    mysql: SELECT (`注文.金額`) AS 注文_金額, …
  • type-check, Biome, dead-code (knip default + production) pass; CLI suite has no new failures vs main on my machine (same pre-existing Windows environment failures noted in fix(scan): exclude per-scan extractedAt and warnings from enrichment stage hashes #352).

Known limits (called out, not silently skipped)

  • Auto-generated manifest joins[].on clauses would still carry a dotted key column verbatim; relationship keys with dotted names should be rare (pk/fk detection), and it's a separable follow-up.
  • Two Windows-environment findings from testing, filed separately if useful: the ktx-sl YAML loader reads files with the platform default encoding (cp1252 on Windows), which fails on non-ASCII sources; and the same sqlite EBUSY test-cleanup issue noted in fix(scan): exclude per-scan extractedAt and warnings from enrichment stage hashes #352.

Warehouse column names may legally contain '.' (quoted identifiers, MongoDB
fields), and projectManifestEntry copied them verbatim into columns[].name
and grain — where the TS/Python contract rejects them, so every sl_query
against the source failed with ComposeContractError.

Manifest projection now aliases dotted physical names ('.' -> '_', deduped
deterministically) and threads the physical reference through the column's
expr, quoted with the connection dialect's identifier character so the
Python planner compiles it as a computed column on every dialect. Non-SQL
connections alias without expr; their sources are searchable but never
SQL-queried. The authoring contract stays strict: a dot in an agent-written
source still means a mistaken table qualification.

Fixes Kaelio#337

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 12, 2026

Copy link
Copy Markdown

@sricursion is attempting to deploy a commit to the Kaelio Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Source contract rejects column/grain names containing "." — blocks sl_query for the source

1 participant