Skip to content

Update Databricks integration setup docs (DOIO-94)#38129

Open
charlesmyu wants to merge 1 commit into
masterfrom
charlesyu/doio-94-update-databricks-integration-docs
Open

Update Databricks integration setup docs (DOIO-94)#38129
charlesmyu wants to merge 1 commit into
masterfrom
charlesyu/doio-94-update-databricks-integration-docs

Conversation

@charlesmyu

@charlesmyu charlesmyu commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Addresses user-reported friction in the Databricks integration setup docs (DOIO-94).

Files touched:

  • content/en/data_observability/jobs_monitoring/databricks/_index.md
  • content/en/data_observability/quality_monitoring/data_warehouses/databricks.md
  • content/en/cloud_cost_management/setup/saas_costs.md

Changes, mapped to the ticket items:

  • Item 1 — SP entitlements: Under Use a Service Principal for OAuth, folded the required workspace entitlements (Workspace access, Databricks SQL access, Admin access) into the service-principal creation step, replacing the separate "add to admins group" steps (Admin access is equivalent). Added a note that granular access (per the Permissions section under Advanced Configuration) is the fallback when Admin access can't be granted.
  • Item 3 — warehouse ID location: The SQL Warehouse setup is now a single step with ordered substeps (select a Pro/Serverless warehouse → copy the ID from the overview page → grant access). Also cross-links Quality Monitoring as something the warehouse ID powers.
  • Item 4 — ordering: Warehouse selection (including the Pro/Serverless requirement and dedicated-2XS recommendation) now comes before the access grants.
  • Item 5 — SQL Editor: Added an explicit "run in the SQL Editor" instruction before the system-table GRANT block (jobs-monitoring, quality-monitoring, saas_costs). (Screenshot not added — docs-repo limitation; follow-up.)
  • Item 6 — grant syntax: Replaced <service_principal> / <token_principal> with the backticked `<client-id>` and a "use the client ID, not the display name" note.
  • Surfaced the MANAGE on CATALOG system prerequisite as a warning callout next to the grant commands.

Intentionally not changed (per ticket-owner feedback during review):

  • Item 2 (SP MANAGE on itself): kept as-is.
  • Item 7 (required vs recommended): left the wording vague; the definitive "optional but recommended" framing belongs to the in-tile UI copy, not the docs.
  • Item 8 (env-var quotes): kept the DATABRICKS_WORKSPACE="..." quotes and the "enclose in double quotes if it contains whitespace" note, since quotes are needed for workspace names containing whitespace.

Out of scope for this docs repo (follow-ups for the owning team): the in-tile UI copy for items 3/7 and the SQL Editor screenshot for item 5.

Motivation

DOIO-94 — reduce onboarding friction for the Databricks integration.

Additional Notes

Draft — pending review, not intended to merge yet.

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Jul 13, 2026

Copy link
Copy Markdown

DOIO-94

@charlesmyu charlesmyu force-pushed the charlesyu/doio-94-update-databricks-integration-docs branch from a75c7c2 to 2b1b494 Compare July 13, 2026 18:45
Reduce onboarding friction in the Databricks setup docs:

- Fold the required workspace entitlements (Workspace access, Databricks
  SQL access, Admin access) into service principal creation, replacing the
  separate "add to admins group" steps; note the granular-permissions
  fallback when Admin access cannot be granted.
- Restructure SQL Warehouse setup into a single step with ordered
  substeps: select a Pro/Serverless warehouse (recommend a dedicated 2XS),
  copy the ID, grant CAN USE, and grant system-table read access.
- Document where to find the SQL Warehouse ID, and note it powers Quality
  Monitoring, with a cross-link to that page.
- Tell users to run the system-table GRANT commands in the SQL Editor, and
  fix the grant syntax to use the backticked client ID instead of the
  service principal display name.
- Surface the MANAGE-on-CATALOG-system prerequisite as a warning callout.

Also updates the Cloud Cost Management (saas_costs) and Quality Monitoring
Databricks docs to use the SQL Editor and backticked client ID for the
system-table grants.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@charlesmyu charlesmyu marked this pull request as ready for review July 13, 2026 22:01
@charlesmyu charlesmyu requested review from a team as code owners July 13, 2026 22:01
@charlesmyu charlesmyu force-pushed the charlesyu/doio-94-update-databricks-integration-docs branch from 2b1b494 to fd25825 Compare July 13, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant