Skip to content

feat(data-sources): add Cloudflare R2 SQL source#291

Merged
lentil32 merged 1 commit into
mainfrom
feat/cloudflare-r2-sql-source
Jun 25, 2026
Merged

feat(data-sources): add Cloudflare R2 SQL source#291
lentil32 merged 1 commit into
mainfrom
feat/cloudflare-r2-sql-source

Conversation

@lentil32

Copy link
Copy Markdown
Contributor

One-Line Summary

Add Cloudflare R2 SQL as a queryable data source for R2 Data Catalog Iceberg tables.

User-Facing Changes

  • Users can connect a cloudflare_r2_sql source with accountId, bucketName, apiToken, and optional apiBaseUrl.
  • The data source catalog, dashboard provider picker, CLI connect flow, shared connection guide, and landing docs now list Cloudflare R2 SQL.
  • SQL query execution can run read-only SELECT queries against Cloudflare R2 SQL and return normalized row objects.

Why This Changed

R2 Data Catalog tables were not directly queryable through OneQuery. Users could store data in R2 and query it externally, but OneQuery had no provider key, credential schema, query driver, or UI metadata for Cloudflare R2 SQL.

How It Changed

  • Added CloudflareR2SqlCredentialsSchema and registered cloudflare_r2_sql as a database/warehouse provider.
  • Added a Cloudflare R2 SQL REST query driver that posts to /api/v1/accounts/{accountId}/r2-sql/query/{bucketName}, sends the derived warehouse name, normalizes result.rows, masks API tokens in errors, and supports connection testing.
  • Wired the driver into the query registry and read-only SQL validation with the Athena dialect.
  • Added provider icons, shared connection guide content, source provider docs, CLI credential injection, and provider snapshots.

Bug Fixes

N/A

Performance Notes (Optional)

N/A

Extra Context / Decisions (Optional)

  • The first version uses the existing JSON credential form instead of adding a dedicated R2 SQL form.
  • Query stats/cost persistence is intentionally not enabled yet because R2 SQL billing and stats semantics are still separate from the existing BigQuery/Athena cost models.

Verification

  • rtk bunx turbo test --json --filter=@onequery/db --filter=@onequery/server --filter=@onequery/cli-server --filter=@onequery/dashboard --filter=@onequery/ui
  • rtk bunx turbo check:fix
  • rtk bunx turbo typecheck --json --filter=@onequery/db --filter=@onequery/server --filter=@onequery/cli-server --filter=@onequery/dashboard --filter=@onequery/ui --filter=@onequery/landing
  • Pre-push hook ran type-aware lint: oxlint . --type-aware --type-check

Video / Screenshot (Optional)

  • N/A

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
onequery-landing 1c878ce Commit Preview URL

Branch Preview URL
Jun 25 2026, 07:24 AM

@lentil32 lentil32 merged commit 8953d7b into main Jun 25, 2026
9 checks passed
@lentil32 lentil32 deleted the feat/cloudflare-r2-sql-source branch June 25, 2026 07:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1c878ceb75

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const TRANSIENT_CLOUDFLARE_R2_SQL_STATUS_CODES = new Set([
429, 500, 502, 503, 504,
]);
const CONNECTION_TEST_QUERY = "SELECT 1 AS onequery_connection_test";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use a supported R2 SQL smoke query

When users run source test or click the dashboard test action for a Cloudflare R2 SQL source, this sends SELECT 1 AS onequery_connection_test to the R2 SQL endpoint. Cloudflare's R2 SQL reference documents SELECT queries as requiring FROM namespace_name.table_name (schema discovery commands such as SHOW DATABASES are separate), so valid R2 credentials can be reported as failed and the source marked error even though real table queries would work. Use a table-independent supported discovery command, or leave this provider non-testable until a table-aware smoke query is available.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant