Skip to content

terraform_dabs_map: make DABsPathToTerraform a true inverse of TerraformPathToDABs#5621

Merged
denik merged 11 commits into
mainfrom
denik/fix-dabs-to-terraform-path
Jun 23, 2026
Merged

terraform_dabs_map: make DABsPathToTerraform a true inverse of TerraformPathToDABs#5621
denik merged 11 commits into
mainfrom
denik/fix-dabs-to-terraform-path

Conversation

@denik

@denik denik commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Make DABsPathToTerraform the exact inverse of TerraformPathToDABs:

  • Prepend the TF wrapper segment (e.g. spec) only for fields that actually live under it, driven by a generated DABsToTerraformWrapperFields set. Root-level fields (status, timestamps, IDs) and unknowns now round-trip correctly.
  • Map the mlflow model's numeric ID across engines: model_id (direct) ↔ registered_model_id (Terraform). It's a state-computed field the codegen heuristics can't derive, so it's wired up via a small manualRenames table; registered_model_id is no longer classified Terraform-only.

A reference to registered_model_id now resolves on both the terraform and direct engines, covered by a new acceptance test.

Needed for #5399.

This pull request and its description were written by Isaac.

@denik denik temporarily deployed to test-trigger-is June 16, 2026 18:09 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 16, 2026 18:09 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 5b1f10c

Run: 27845764480

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 7 13 265 1012 5:14
💚​ aws windows 7 13 267 1010 7:02
💚​ aws-ucws linux 7 13 361 926 6:16
💚​ aws-ucws windows 7 13 363 924 8:02
💚​ azure linux 1 15 268 1010 4:54
💚​ azure windows 1 15 270 1008 6:52
💚​ azure-ucws linux 1 15 366 922 6:42
🔄​ azure-ucws windows 2 1 15 366 920 9:17
💚​ gcp linux 1 15 264 1013 5:55
💚​ gcp windows 1 15 266 1011 7:36
22 interesting tests: 13 SKIP, 7 RECOVERED, 2 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestFetchRepositoryInfoAPI_FromRepo/root ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestFetchRepositoryInfoAPI_FromRepo/subdir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
Top 20 slowest tests (at least 2 minutes):
duration env testname
4:45 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:32 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:03 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:59 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:22 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:20 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:18 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:11 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:08 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:06 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:03 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:02 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:53 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:51 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:48 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:47 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:35 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:33 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:32 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:26 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@denik denik force-pushed the denik/fix-dabs-to-terraform-path branch from 4ca0527 to c64a6f3 Compare June 17, 2026 01:14
@denik denik temporarily deployed to test-trigger-is June 17, 2026 01:15 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 17, 2026 01:15 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 17, 2026 01:24 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 17, 2026 01:24 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 17, 2026 21:24 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 17, 2026 21:24 — with GitHub Actions Inactive
denik added 8 commits June 18, 2026 15:39
…level TF fields

For postgres groups with a spec wrapper, DABsPathToTerraform was
unconditionally prepending "spec" to all paths. Root-level TF fields
(name, status.*, create_time, project_id, etc.) live outside the wrapper
and must not receive the prefix.

Extend the codegen to emit DABsToTerraformRootFields: for each wrapper
group, the set of first-level TF field names that are at the resource
root. DABsPathToTerraform now only prepends the wrapper when the path's
first segment is absent from that set.

Remove noRoundtrip: true from the affected translate_test.go cases; add
new roundtrip cases for project_id and name.

Co-authored-by: Isaac
…st.go

All TerraformPathToDABs test cases now round-trip correctly through
DABsPathToTerraform, so the field and its guard are no longer needed.

Co-authored-by: Isaac
TerraformPathToDABs should be idempotent: a DABs-format path passed
through a second time must come back unchanged. Assert this for every
non-error test case.

Co-authored-by: Isaac
Rebase onto main picked up the new databricks_postgres_role resource;
regenerate generated.go to include it.

Co-authored-by: Isaac
Replace DABsToTerraformRootFields (exception-based: fields NOT under the
wrapper) with DABsToTerraformWrapperFields (positive: fields that ARE under
the wrapper). DABsPathToTerraform now prepends the wrapper only when the
first path segment appears in this set; unknown and root-level fields pass
through unchanged without requiring explicit enumeration.

Co-authored-by: Isaac
@denik denik force-pushed the denik/fix-dabs-to-terraform-path branch from 0fdd7cc to f2e6aee Compare June 18, 2026 22:41
@denik denik temporarily deployed to test-trigger-is June 18, 2026 22:41 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 18, 2026 22:41 — with GitHub Actions Inactive
The mlflow model's numeric ID is a state-computed field named model_id in the
direct engine and registered_model_id in Terraform. It lives in RemoteType, not
the config struct, and the two names are lexically unrelated, so the codegen's
heuristic matcher cannot derive the rename. Add a small manualRenames table so
the rename flows into both translation maps and registered_model_id is no longer
classified Terraform-only.

This lets a reference to registered_model_id resolve on both engines, covered by
a new acceptance test.

Co-authored-by: Isaac
@denik denik temporarily deployed to test-trigger-is June 18, 2026 22:48 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 18, 2026 22:48 — with GitHub Actions Inactive
@denik denik changed the title terraform_dabs_map: fix DABsPathToTerraform to skip wrapper for TF root-level fields terraform_dabs_map: make DABsPathToTerraform a true inverse of TerraformPathToDABs Jun 18, 2026
@denik denik temporarily deployed to test-trigger-is June 19, 2026 19:46 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 19, 2026 19:46 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 19, 2026 19:56 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 19, 2026 19:56 — with GitHub Actions Inactive
@denik denik enabled auto-merge June 19, 2026 20:07
Comment thread NEXT_CHANGELOG.md

### Bundles
* `bundle run` now prints the modern job run URL (`/jobs/<id>/runs/<id>`) so that non-admin users permitted to view the run are taken to the run instead of the workspace homepage.
* References to a registered model's `registered_model_id` now resolve under the direct engine, matching Terraform behavior ([#5621](https://github.com/databricks/cli/pull/5621)).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is registered_model_id? I don't see it here in our public documentation:
https://docs.databricks.com/api/workspace/registeredmodels/create

@denik denik Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the field that DABs on terraform supports. So we translate it for compat between terraform & direct engine.

@denik denik requested a review from shreyas-goenka June 22, 2026 14:35
@denik denik added this pull request to the merge queue Jun 23, 2026
Merged via the queue into main with commit 633245f Jun 23, 2026
29 checks passed
@denik denik deleted the denik/fix-dabs-to-terraform-path branch June 23, 2026 08:44
denik added a commit that referenced this pull request Jun 23, 2026
main's DABsToTerraformRenameMap now maps models.model_id ->
registered_model_id (PR #5621), so the manual tfStateFieldAliases map
and the alias-fallback branch in LookupTFField are redundant.
LookupTFField now translates once and navigates.

Co-authored-by: Isaac
denik added a commit that referenced this pull request Jun 24, 2026
main's DABsToTerraformRenameMap now maps models.model_id ->
registered_model_id (PR #5621), so the manual tfStateFieldAliases map
and the alias-fallback branch in LookupTFField are redundant.
LookupTFField now translates once and navigates.

Co-authored-by: Isaac
denik added a commit that referenced this pull request Jun 24, 2026
main's DABsToTerraformRenameMap now maps models.model_id ->
registered_model_id (PR #5621), so the manual tfStateFieldAliases map
and the alias-fallback branch in LookupTFField are redundant.
LookupTFField now translates once and navigates.

Co-authored-by: Isaac
deco-sdk-tagging Bot added a commit that referenced this pull request Jun 24, 2026
## Release v1.5.0

### CLI
* `workspace export-dir` no longer aborts when a workspace object's name is not a legal local filename (e.g. a notebook named `New Notebook 2026-05-04 13:54:24` whose `:` is illegal on Windows). Such files are now exported under a sanitized name with a warning and the export completes ([#5171](#5171)).
* `ssh connect` now opens an interactive `bash` login shell by default instead of the compute image's default `/bin/sh`, falling back gracefully when `bash` is unavailable. Passing an explicit remote command (`-- <cmd>`) is unaffected ([#5687](#5687)).
* `ssh connect` interactive sessions now start in the user's workspace home folder (`/Workspace/Users/<email>`) instead of the OS home directory, falling back to the OS home when that folder is unavailable ([#5688](#5688)).

### Bundles
* Add documentation for the common bundle resource fields `permissions`, `lifecycle`, and `grants` in the JSON schema, so they surface in editor completions and the docs.
* `bundle run` now prints the modern job run URL (`/jobs/<id>/runs/<id>`) so that non-admin users permitted to view the run are taken to the run instead of the workspace homepage.
* References to a registered model's `registered_model_id` now resolve under the direct engine, matching Terraform behavior ([#5621](#5621)).
* Fix missing field descriptions in the bundle JSON schema for fields whose upstream API docs arrived after the field was first annotated (e.g. `vector_search_endpoints.*.target_qps`); stale placeholder markers no longer hide them ([#5588](#5588)).
* Fix `bundle deploy --plan` dropping a `postgres_role`'s `role_id`, which caused the role to be recreated on the next deploy ([#5672](#5672)).
* direct: Fix spurious cluster recreate when `apply_policy_default_values: true` is set ([#5693](#5693)).
* direct: New 'deployment migrate' implementation that parses terraform state instead of fetching resources state from the backend ([#5399](#5399)).

### Dependency updates
* Bump `github.com/databricks/databricks-sdk-go` from v0.141.0 to v0.147.0 ([#5636](#5636)).
* Bump Terraform provider from v1.117.0 to v1.118.0 ([#5637](#5637)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants