fix(sl): project dotted physical columns to contract-valid aliases#353
Open
sricursion wants to merge 1 commit into
Open
fix(sl): project dotted physical columns to contract-valid aliases#353sricursion wants to merge 1 commit into
sricursion wants to merge 1 commit into
Conversation
Warehouse column names may legally contain '.' (quoted identifiers, MongoDB
fields), and projectManifestEntry copied them verbatim into columns[].name
and grain — where the TS/Python contract rejects them, so every sl_query
against the source failed with ComposeContractError.
Manifest projection now aliases dotted physical names ('.' -> '_', deduped
deterministically) and threads the physical reference through the column's
expr, quoted with the connection dialect's identifier character so the
Python planner compiles it as a computed column on every dialect. Non-SQL
connections alias without expr; their sources are searchable but never
SQL-queried. The authoring contract stays strict: a dot in an agent-written
source still means a mistaken table qualification.
Fixes Kaelio#337
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@sricursion is attempting to deploy a commit to the Kaelio Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #337
Makes manifest-backed sources with dotted physical column names queryable instead of hard-failing the TS/Python contract on every
sl_query.Warehouse column names may legally contain
.(quoted identifiers on Postgres/Snowflake — the telemetry case had CJK names like注文.金額— and MongoDB field names).projectManifestEntrycopied them verbatim intocolumns[].nameandgrain, andtoResolvedWirethen rejected the source withComposeContractError: must be unqualified (no '.')— ktx generated a source it refused to query, on every request that touched it (root-cause possibility 1 from the issue).Approach
Manifest projection now emits contract-valid output names and keeps the physical reference in
expr:.→_, deduped against sibling columns with a numeric suffix), incolumns[].nameandgrainboth.expr, quoted with the connection dialect's identifier character (reusing each connector'squoteIdentifier). The Python planner parses columnexprin the connection's native dialect (the convention fix(sl): valid reserved-word and week-grain SQL on BigQuery/MySQL #340 enforced), so a hardcoded postgres-style quote would break backtick dialects — this is why the quoter is threaded per connection rather than hardcoded.expr: their sources are searchable, and they are never SQL-queried (assertSqlQueryableConnectionblocks that earlier).table.columnqualification. Only the manifest projection — where names are physical by definition — normalizes.Plumbing:
manifestIdentifierQuoterForConnectionTyperesolves the quoter from the catalog'sconnectionType(accepting bothPOSTGRESQLdescriptor form andpostgresdriver form via a reverse map derived from the existingDRIVER_TO_CONNECTION_TYPE), cached per connection inSemanticLayerService;local-sl.tsresolves it from project config through the samelocalConnectionInfoFromConfigthe catalog port uses. Sources whose connection can't be resolved project without a quoter — still searchable, and querying them fails earlier atexecuteQuery.Tests
expr, aliased grain, and the projected source passestoResolvedWire(the exact contract Source contract rejects column/grain names containing "." — blocks sl_query for the source #337 tripped). Fails onmain, passes here.a.balongsidea_b→a_b_2), dotted pk → aliased grain."…"postgres/snowflake,`…`mysql), driver-form input, andnullfor MongoDB/unknown.expr) compiles on both quoting families —postgres:
SELECT ("注文.金額") AS 注文_金額, SUM(("注文.金額")) AS total_amount FROM analytics.sales AS sales …mysql:
SELECT (`注文.金額`) AS 注文_金額, …type-check, Biome,dead-code(knip default + production) pass; CLI suite has no new failures vsmainon my machine (same pre-existing Windows environment failures noted in fix(scan): exclude per-scan extractedAt and warnings from enrichment stage hashes #352).Known limits (called out, not silently skipped)
joins[].onclauses would still carry a dotted key column verbatim; relationship keys with dotted names should be rare (pk/fk detection), and it's a separable follow-up.EBUSYtest-cleanup issue noted in fix(scan): exclude per-scan extractedAt and warnings from enrichment stage hashes #352.