Skip to content

docs: Add Databricks integration documentation#1301

Merged
nrichers merged 13 commits intomainfrom
nrichers/sc-14813/documentation---databricks-integ
May 5, 2026
Merged

docs: Add Databricks integration documentation#1301
nrichers merged 13 commits intomainfrom
nrichers/sc-14813/documentation---databricks-integ

Conversation

@nrichers
Copy link
Copy Markdown
Collaborator

@nrichers nrichers commented Apr 30, 2026

What and why?

This PR:

  • Documents the Databricks Unity Catalog integration (sc-14813).
  • Adds connection configuration details, updates the supported integrations list, and provides a step-by-step guide for configuring connections and linking inventory records to Databricks resources.
  • Splits the existing integration examples into separate files, as the integrations examples were already one super long file.

How to test

Try the live preview:

What needs special review?

These docs mention linking to models, datasets, and agents in Databricks Unity Catalog, but I only see Link Model in the UI. Is Dataset/Agent linking planned for this release, or should I update the docs to clarify that only model linking is currently supported?

image

I figured out these steps by inspecting the code changes and testing in the UI where I could, but I can't actually set up a real Databricks connection. If you have some credentials for testing, I'd be grateful (or please review the steps here carefully!).

Dependencies, breaking changes, and deployment notes

Depends on validmind/validmind-library#495 for the Databricks notebook link to work.

Release notes

Added documentation for configuring the Databricks Unity Catalog integration, including connection setup and linking models.

Checklist

  • What and why
  • How to test
  • Labels applied
  • PR linked to Shortcut

Add documentation for the new Databricks integration that enables
linking ValidMind inventory records to Databricks Unity Catalog
resources including models, datasets, and agents.

- Add Databricks to supported connections in configure-connections.qmd
- Add Databricks to model inventory integrations in managing-integrations.qmd
- Add step-by-step connection and linking examples in integrations-examples.qmd

Closes sc-14813
@github-actions
Copy link
Copy Markdown
Contributor

Pull requests must include at least one of the required labels: internal, highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

@nrichers nrichers added the documentation Improvements or additions to documentation label Apr 30, 2026
- databricks host → workspace url
- client id → sql warehouse id
- client secret → personal access token
@nrichers nrichers changed the title [SC-14813] Add Databricks Unity Catalog integration documentation [SC-14813] Add Databricks integration documentation Apr 30, 2026
@nrichers nrichers requested a review from nateshim April 30, 2026 09:35
@nrichers nrichers changed the title [SC-14813] Add Databricks integration documentation docs: Add Databricks integration documentation Apr 30, 2026
Comment thread site/guide/integrations/configure-connections.qmd Outdated
Comment thread site/guide/integrations/configure-connections.qmd
- client id: Update to describe SQL Warehouse ID
- client secret: Update to describe PAT token secret
@nrichers
Copy link
Copy Markdown
Collaborator Author

@nateshim thank you for reviewing! I've addressed your comments, could you approve, please?

@nrichers nrichers requested a review from nateshim April 30, 2026 23:34
@nrichers
Copy link
Copy Markdown
Collaborator Author

nrichers commented May 1, 2026

Reminder for Nik: @nateshim has a very nice demo video that shows how to run a databricks notebook in validmind/validmind-library#495. Might be good to borrow a bit from that for the docs.

nrichers added 4 commits May 1, 2026 14:09
Add a new section explaining how to run validation notebooks against
Databricks-hosted data, with a link to the quickstart notebook and a
mermaid diagram showing the data flow between platforms.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

Lighthouse check results

⚠️ WARN: Average accessibility score is 0.87 (required: >0.9) — Check the workflow run

Show Lighthouse scores

Folder depth level checked: 0

Commit SHA: 1a58d73

Modify the workflow to check a different depth:

  • 0: Top-level navigation only — /index.html, /guide/guides.html, ...
  • 1: All first-level subdirectories — /guide/.html, /developer/.html, ...
  • 2: All second-level subdirectories — /guide/attestation/*.html, ...
Page Accessibility Performance Best Practices SEO
/developer/validmind-library.html 0.85 0.69 1.00 0.82
/get-started/get-started.html 0.85 0.71 1.00 0.73
/guide/guides.html 0.85 0.68 1.00 0.82
/index.html 0.93 0.63 1.00 0.82
/releases/all-releases.html 0.86 0.69 1.00 0.73
/support/support.html 0.91 0.69 1.00 0.82
/training/training.html 0.85 0.62 0.96 0.73

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

PR Summary

This PR updates the integration documentation and examples by enhancing the sidebar structure and adding detailed integration guides. In the sidebar configuration file, the single reference to the integrations examples page has been replaced by a structured file reference that includes multiple subpages. These subpages offer step-by-step instructions for setting up and linking various integrations such as Jira, ServiceNow, AWS SageMaker, AWS Bedrock, GitLab, and Databricks, as well as using webhooks with workflows.

Key functional changes include:

  • Updating the sidebar configuration to list multiple integration example files, which makes it easier for users to navigate directly to the specific guides they need.

  • Introducing grid listings (with custom parameters like grid-columns and max-description-length) for the integrations examples view, providing a more visual and organized presentation.

  • Adding new detailed markdown documentation files (in QMD format) for each integration scenario. These include:

    • Creating Jira tickets via HTTP request workflows.
    • Creating ServiceNow incidents via HTTP request workflows.
    • Synchronizing models with AWS SageMaker and AWS Bedrock.
    • Synchronizing models with GitLab.
    • Integrating with Databricks including a diagram of the data flow between Databricks and the platform.
    • Using webhooks to both start workflows and trigger paused ones.
  • Minor stylistic improvements and consistency updates to markdown content (e.g., consistent formatting of apostrophes and keys in headers) that do not affect functionality but improve readability.

Overall, these changes are aimed at improving the clarity, usability, and navigability of the integration documentation, while also providing enriched examples for practical setup scenarios.

Test Suggestions

  • Verify that the updated sidebar correctly displays the integration example listing with all new files and nested contents.
  • Ensure that each newly added integration guide page (Jira, ServiceNow, AWS SageMaker, AWS Bedrock, GitLab, Databricks, webhooks) renders correctly with all images, diagrams, and code snippets displaying as expected.
  • Test navigation between the integration examples and other parts of the documentation to confirm that links in the sidebar and the grid listing work without error.
  • Manually follow a few of the integration steps (e.g., creating a Jira ticket and linking a model) to ensure the instructions are clear and the described processes work in the application.
  • Check that authentication instructions (using webhook secrets and tokens) are correctly documented and that no sensitive information is exposed in the documentation.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

Validate docs site

✓ INFO: A live preview of the docs site is available — Open the preview

@nrichers nrichers merged commit 470d5f6 into main May 5, 2026
6 of 7 checks passed
@nrichers nrichers deleted the nrichers/sc-14813/documentation---databricks-integ branch May 5, 2026 16:42
nrichers added a commit that referenced this pull request May 6, 2026
* docs: Add Databricks Unity Catalog integration documentation

Add documentation for the new Databricks integration that enables
linking ValidMind inventory records to Databricks Unity Catalog
resources including models, datasets, and agents.

- Add Databricks to supported connections in configure-connections.qmd
- Add Databricks to model inventory integrations in managing-integrations.qmd
- Add step-by-step connection and linking examples in integrations-examples.qmd

Closes sc-14813

* Add placeholder columns to integrations grid layout

* Fix Databricks connection field names to match UI

- databricks host → workspace url
- client id → sql warehouse id
- client secret → personal access token

* Fix Databricks link model steps to match UI

* docs: Fix Databricks connection field descriptions

- client id: Update to describe SQL Warehouse ID
- client secret: Update to describe PAT token secret

* docs: Add developer workflow for running validations on Databricks data

Add a new section explaining how to run validation notebooks against
Databricks-hosted data, with a link to the quickstart notebook and a
mermaid diagram showing the data flow between platforms.

* Split examples up into separate files.

* File rename

* Remove terminology ambiguity

* Edits

* Add missing headings for steps

* Add configuration screenshot
nrichers added a commit that referenced this pull request May 6, 2026
* docs: Map and assess evidence (#1299)

* docs: Add autogenerate validation reports documentation

Document the AI-powered features for validators:
- Auto-Link Evidence for single/bulk guideline evidence linking
- Risk Assessment Generation for single/bulk assessments
- Understanding the structured AI-generated assessment format

Closes sc-14812.

* fix: Rename to map-and-assess-evidence with accurate UI terminology

- Rename file from autogenerate-validation-reports.qmd to map-and-assess-evidence.qmd
- Update title to "Map and assess evidence"
- Fix UI element names to match actual product (Map Evidence, Assess Evidence)
- Describe Evidence Assessment output structure accurately
- Add relevance threshold slider documentation

* Add screenshots for Map Evidence and Assess Evidence panels

- Add map-evidence-panel.png showing evidence type toggles and relevance threshold
- Add assess-evidence-panel.png showing compliance assessment option
- Display both in column-margin divs alongside their respective sections

* fix: Add Approve All / Reject All workflow for Map and Assess Evidence

- Update Map Evidence steps to describe approve/reject workflow
- Update Assess Evidence steps to describe approve/reject workflow
- Clarify that editing is available after approving assessments

* Add individual approve/reject, relevance analysis, and reassess options

- Add individual Approve/Reject buttons for mapped evidence
- Add See Relevance Analysis option with relevance score explanation
- Add individual Approve/Reject buttons for evidence assessments
- Add Reassess Evidence option to regenerate assessments
- Add new "Understand mapped evidence" section explaining relevance scores and analysis

* Distinguish report-level vs section-level approve/reject workflows

- Report overview: Approve All / Reject All for entire report
- Section-level: Individual Approve / Reject per evidence item or assessment
- Clarify the two-level review structure for both Map Evidence and Assess Evidence

* Consolidate intro into benefits-focused overview section

Replace detailed "Understand mapped evidence" and "Understand evidence
assessments" sections with a concise "How do evidence mapping and
assessment work?" section that explains the benefits for new users.

* Format overview section as definition list

* Remove What's next section

* Remove unused evidence-assessment-example.png

* Add What's next section with listing to assess-compliance

Links to the manual compliance assessment workflow for users who
prefer traditional evidence linking and assessment selection.

* Document Remap Evidence button in report overview

* Replace 'report overview' with visible UI terminology

Changed 'From the report overview' to 'For the entire report' and
removed references to non-existent 'Overview page' label.

* Simplify navigation: click Validation link directly

* Address review comments from Fernanda

* Fix link warning

* docs: Add Databricks integration documentation (#1301) (#1316)

* docs: Add Databricks Unity Catalog integration documentation

Add documentation for the new Databricks integration that enables
linking ValidMind inventory records to Databricks Unity Catalog
resources including models, datasets, and agents.

- Add Databricks to supported connections in configure-connections.qmd
- Add Databricks to model inventory integrations in managing-integrations.qmd
- Add step-by-step connection and linking examples in integrations-examples.qmd

Closes sc-14813

* Add placeholder columns to integrations grid layout

* Fix Databricks connection field names to match UI

- databricks host → workspace url
- client id → sql warehouse id
- client secret → personal access token

* Fix Databricks link model steps to match UI

* docs: Fix Databricks connection field descriptions

- client id: Update to describe SQL Warehouse ID
- client secret: Update to describe PAT token secret

* docs: Add developer workflow for running validations on Databricks data

Add a new section explaining how to run validation notebooks against
Databricks-hosted data, with a link to the quickstart notebook and a
mermaid diagram showing the data flow between platforms.

* Split examples up into separate files.

* File rename

* Remove terminology ambiguity

* Edits

* Add missing headings for steps

* Add configuration screenshot

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@validbeck validbeck mentioned this pull request May 7, 2026
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants