feat(connectors): add Trino data source connector#322
Open
bazooka720 wants to merge 2 commits into
Open
Conversation
Add a Trino connector alongside the existing warehouse connectors (Snowflake, BigQuery, ClickHouse, Postgres, MySQL, SQL Server, SQLite). Trino federates many catalogs behind one endpoint, so the connector discovers every non-system catalog (or an explicit `catalogs` allowlist) and introspects each via information_schema, emitting catalog-qualified (catalog.schema.table) tables. SQL goes through the read-only gate and maps to the sqlglot 'trino' dialect for analysis. - connectors/trino: KtxTrinoScanConnector + KtxTrinoDialect (three-part, ANSI quoting, TABLESAMPLE BERNOULLI), injectable trino-client factory - register in driver/dialect registries, scan driver union, zod schema, sql-analysis dialect map, setup wizard (catalog scope, port 8080) - tests: dialect + connector suites (fake client); update driver fixtures Verified: tsc (src+test), biome, full vitest suite green.
|
@bazooka720 is attempting to deploy a commit to the Kaelio Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
|
Thanks @bazooka720 , we'll review and get back to you soon! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds Trino as a data source, alongside the existing warehouse connectors (Snowflake, BigQuery, ClickHouse, Postgres, MySQL, SQL Server, SQLite).
Because Trino federates many catalogs behind a single endpoint, the connector treats all catalogs as first-class: it discovers every non-system catalog (or an explicit
catalogsallowlist) viasystem.metadata.catalogs, introspects each throughinformation_schema, and emits catalog-qualifiedcatalog.schema.tabletables.How
connectors/trino/KtxTrinoScanConnector—introspect(per-catalog),listSchemas/listTables(catalog-namespaced),testConnection,sampleTable,sampleColumn,executeReadOnly,cleanup. Talks to Trino via an injectableKtxTrinoClientfactory backed bytrino-client(lazy import), so it is unit-testable with a fake.KtxTrinoDialect—three-partidentifier shape, ANSI double-quoting,TABLESAMPLE BERNOULLI, Trino-to-dimension type mapping.KtxConnectionDriverunion,KtxScopeConfigKey(catalogs), zod warehouse schema, sql-analysistrino(sqlglot) mapping, and the setup wizard (catalog scope discovery, default port 8080).SELECT-able system tables (noSHOW).Notes / scope
hasHistoricSqlReader: false— query-history ingestion deferred (same as ClickHouse/MySQL/SQL Server today).BasicAuth); OAuth2/Kerberos not wired yet.estimatedRowsleft null (no portable cheap row-count in Trino).Testing
tsc -p tsconfig.jsonandtsconfig.test.json— cleantest/connectors/trino/{dialect,connector}.test.tscover all-catalogs discovery, allowlist,tableScope, sampling, read-only enforcement, and config/URL/ssl parsingdrivers.test.tsfixtures for the new driver