From f05dbba3163b4418bd1326a3c5d012a18b3e779c Mon Sep 17 00:00:00 2001 From: Jan Rose Date: Wed, 17 Jun 2026 12:16:36 +0200 Subject: [PATCH 1/4] acceptance: consolidate test-authoring best practices Document the conventions agents most often miss when writing acceptance tests, so the rules file is the single source of truth instead of relying on precedent in existing tests: - print_requests.py: document --unique/--oneline, and add a rule against hand-rolling a local print_requests() jq wrapper (the most common copy-pasted anti-pattern) or inline `jq select out.requests.txt`. - Built-in shell helpers: document trace/title/errcode/musterr (and the rest of acceptance/script.prepare) with rules to assert expected failures via musterr (not `! cmd`) and tolerated failures via errcode (not `|| true`). - Test server: prefer modeling behavior in libs/testserver/ over per-test [[Server]] response stubs, which only help one test and never converge on the real API contract. Also surface musterr/title in acceptance/README.md next to the existing errcode/trace helpers. Co-authored-by: Isaac --- .agent/rules/testing.md | 57 ++++++++++++++++++++++++++++++++++++++++- acceptance/README.md | 4 +++ 2 files changed, 60 insertions(+), 1 deletion(-) diff --git a/.agent/rules/testing.md b/.agent/rules/testing.md index 77173ff6506..27630f7b611 100644 --- a/.agent/rules/testing.md +++ b/.agent/rules/testing.md @@ -103,6 +103,24 @@ If the only reason for divergence is a server-side default that one engine sets - Run tests on cloud: `deco env run -i -n aws-prod-ucws -- ` (requires `deco` tool and access to test env). - `script.prepare` files from parent directories are concatenated into the test script. Use them for shared bash helpers. +### Built-in shell helpers + +`acceptance/script.prepare` defines shell helpers that are in scope for every `script`. Prefer them over hand-rolled equivalents — they keep `output.txt` consistent across tests and make intent obvious to the next reader. + +- `trace CMD...`: print `>>> CMD` to stderr, then run CMD. Wrap commands whose invocation should appear in `output.txt` so the captured output explains what produced it. Leading `KEY=value` arguments are exported for the command (e.g. `trace FOO=bar $CLI ...`). +- `title "TEXT"`: print a `=== TEXT` section header. Use `\n` escapes for spacing. Use it to label the phases of a multi-step script. +- `errcode CMD...`: run CMD; if it exits non-zero, append `Exit code: N` to the output but let the script continue (scripts run under `bash -e`, which would otherwise abort). Use when a command is *allowed* to fail and later steps must still run. +- `musterr CMD...`: run CMD and fail the whole test if it *succeeds*; on the expected failure the script continues. Use to assert a command must error. +- `withdir DIR CMD...`: run CMD with the working directory set to DIR, restoring it afterwards. +- `git-repo-init`: initialize a deterministic git repo (fixed user/email, no hooks) and commit `databricks.yml`. +- `uuid`, `sethome DIR`, `venv_activate`, `as-test-sp CMD...`, `readplanarg FILE`, `envsubst`: see `acceptance/script.prepare` for the full list and exact semantics. + +**RULE: Wrap commands whose invocation should be visible in `output.txt` with `trace`.** The `>>> ...` line ties each block of output to the command that produced it; without it the output is an unlabeled wall of text. + +**RULE: Assert an expected failure with `musterr`, not `! cmd` or a bare `errcode`.** Only `musterr` fails the test when the command unexpectedly *succeeds*. `! cmd` is exempt from `set -e` and silently passes on success; `errcode` is for *tolerated* failures, not required ones. + +**RULE: Capture a tolerated non-zero exit with `errcode`, not `cmd || true`.** `errcode` records `Exit code: N` in the output so the failure stays visible and asserted; `|| true` hides it entirely. + ### Helper scripts **RULE: Use the `acceptance/bin/` helpers before reaching for inline `jq` or `grep` pipelines.** When a test needs to filter recorded requests, assert a substring is or isn't present, or register a dynamic replacement, the helpers handle sorting, URL query normalization, redaction hooks, and cross-platform path issues. Inline `jq` in an acceptance script is brittle and hard to read. @@ -124,7 +142,7 @@ BAD: Available on `PATH` during test execution (from `acceptance/bin/`): - `contains.py SUBSTR [!SUBSTR_NOT]`: passthrough filter (stdin→stdout) that checks substrings are present (or absent with `!` prefix). Errors are reported on stderr. -- `print_requests.py //path [^//exclude] [--get] [--sort] [--keep]`: print recorded HTTP requests matching path filters. Requires `RecordRequests=true` in `test.toml`. Clears `out.requests.txt` afterwards unless `--keep`. Use `--get` to include GET requests (excluded by default). Use `^` prefix to exclude paths. +- `print_requests.py //path [^//exclude] [--get] [--sort] [--unique] [--oneline] [--keep]`: print recorded HTTP requests matching path filters. Requires `RecordRequests = true` in `test.toml`. Excludes GET by default (`--get` includes them); clears `out.requests.txt` afterwards (`--keep` retains it). `^` prefix excludes a path; multiple positive filters are OR'd together. `--sort` orders output deterministically (use when the request set is order-independent), `--unique` collapses consecutive duplicates (e.g. repeated polls), `--oneline` prints one request per line. - `replace_ids.py [-t TARGET]`: read deployment state and add `[NAME_ID]` replacements for all resource IDs. - `read_id.py [-t TARGET] NAME`: read ID of a single resource from state, print it, and add a `[NAME_ID]` replacement. - `add_repl.py VALUE REPLACEMENT`: add a custom replacement (VALUE will be replaced with `[REPLACEMENT]` in output). @@ -140,8 +158,45 @@ Available on `PATH` during test execution (from `acceptance/bin/`): **RULE: Don't pass `--keep` to `print_requests.py` if a later `print_requests.py` call follows.** The buffer accumulates, so the second call double-prints the earlier requests. +**RULE: Filter recorded requests with `print_requests.py`, never with a hand-written `jq 'select(...)' out.requests.txt` pipeline** — inline, or hidden inside a local `print_requests()` shell function. The helper already excludes GET, normalizes query strings, optionally sorts, and deletes `out.requests.txt` afterwards. A copy-pasted `jq` wrapper reimplements all of that, drifts from the canonical output format, and is the single most common acceptance-test anti-pattern in this repo. Wrapping `print_requests.py` *itself* in a local function is fine — e.g. to send each variant's output to its own `out.requests..json`. Reach for `jq` on `out.requests.txt` only for what `print_requests.py` genuinely can't express: filtering on request *body* content, or deleting noisy body fields (prefer a `Repls` entry in `test.toml` even then). + +GOOD: + +```bash +trace print_requests.py //pipelines --sort +``` + +BAD: + +```bash +print_requests() { + jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt + rm out.requests.txt +} +trace print_requests +``` + **RULE: Route noisy or non-deterministic command output to `LOG.` instead of `output.txt` or `/dev/null`.** `LOG.*` files are visible under `go test -v` but excluded from the diff — see `acceptance/selftest/log/`. Use `&> LOG.` to capture both streams (then `contains.py` to assert invariants like `'!panic' '!internal error'`), or `2>>LOG.` for cleanup-step stderr you'd otherwise drop to `/dev/null`. +### Test server + +Acceptance tests run against an in-process fake of the Databricks API in `libs/testserver/` (`FakeWorkspace` and the per-service handler files). The fake keeps real in-memory state and returns the same errors the backend does: 404 on a missing parent, 409 on a duplicate create, 400 on a missing required field, and so on. `test.toml` can also stub a single route with a canned response: + +```toml +[[Server]] +Pattern = "POST /api/2.2/jobs/create" +Response.StatusCode = 400 +Response.Body = '''{"error_code": "INVALID_PARAMETER_VALUE", "message": "..."}''' +``` + +**RULE: Model API behavior in `libs/testserver/`, not in per-test `[[Server]]` response stubs.** When a test needs the fake server to validate input or return an error, add or extend the handler in `libs/testserver/` so the behavior is stateful and shared by every test. A `[[Server]]` stub hijacks the route with a static response that ignores request state, diverges from the real API, and only helps the one test that declares it — so the next test re-stubs the same error and the fake never converges on the real contract. + +GOOD: teach the create handler in `libs/testserver/postgres.go` to return 404 when the referenced role does not exist, so every test that creates a database against a missing role observes the real error. + +BAD: add `[[Server]]` with `Pattern = "POST .../databases"` and `Response.StatusCode = 404` to a single test's `test.toml` to fake that same error. + +Reserve `[[Server]]` for routes the testserver does not model at all (a one-off endpoint exercised by a single test) and for injecting a response a stateful handler genuinely can't express (for transient faults and forced disconnects, prefer the `fault.py` / kill helpers instead). + ### Update workflow **RULE: Run `./task test-update` to regenerate outputs, then `./task fmt` and `./task lint`.** If fmt or lint modify files in `acceptance/`, there's an issue in the source files. Fix the source, regenerate, and verify fmt/lint pass cleanly. diff --git a/acceptance/README.md b/acceptance/README.md index 8e33d273a70..7277481327b 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -15,9 +15,13 @@ The scripts are run with `bash -e` so any errors will be propagated. They are ca For more complex tests one can also use: - `errcode` helper: if the command fails with non-zero code, it appends `Exit code: N` to the output but returns success to caller (bash), allowing continuation of script. +- `musterr` helper: runs the command and fails the test if it *succeeds*; on the expected failure the script continues. Use it to assert a command must error. - `trace` helper: prints the arguments before executing the command. +- `title` helper: prints a `=== ` section header to label a phase of the script. - custom output files: redirect output to custom file (it must start with `out`), e.g. `$CLI bundle validate > out.txt 2> out.error.txt`. +The complete set of shell helpers (the above plus `withdir`, `git-repo-init`, `uuid`, `sethome`, and others) is defined in `acceptance/script.prepare`. + Any file starting with "LOG" will be logged to test log (visible with go test -v). See [selftest](./selftest) for more examples. From 9d9e57fd1e2f3009c6db4dec35a2140398f517cc Mon Sep 17 00:00:00 2001 From: Jan Rose Date: Wed, 17 Jun 2026 12:35:37 +0200 Subject: [PATCH 2/4] acceptance: backfill print_requests.py over existing tests Replace hand-rolled `jq 'select(...)' out.requests.txt` request filtering with the print_requests.py helper across 23 existing tests, so the examples agents copy from match the rule added in the previous commit. Each local print_requests()/print_sorted_requests()/print_app_requests() wrapper (and a couple of inline jq pipelines) that reimplemented the helper now calls print_requests.py directly (--sort for the previously shell-sorted variants). The helper puts method/path first and deletes out.requests.txt itself, so the redundant `rm` lines are dropped. Behavior-preserving: the regenerated request output is identical except that method/path now lead each request ahead of the body; most output files are byte-identical. Left as jq: pipelines that filter on request body content or extract/delete specific body fields, which print_requests.py can't express (and which the rule explicitly permits). Co-authored-by: Isaac --- acceptance/bundle/plan/no_upload/output.txt | 2 +- acceptance/bundle/plan/no_upload/script | 4 +--- .../apps/lifecycle-started-omitted/output.txt | 18 +++++++------- .../apps/lifecycle-started-omitted/script | 3 +-- .../genie_spaces/current_can_manage/script | 3 +-- .../permissions/jobs/deleted_remotely/script | 3 +-- .../resources/permissions/jobs/update/script | 3 +-- .../models/current_can_manage/script | 3 +-- .../permissions/pipelines/update/script | 6 ++--- .../permissions/target_permissions/script | 6 ++--- .../allow-duplicate-names/output.txt | 12 +++++----- .../pipelines/allow-duplicate-names/script | 2 +- .../resources/pipelines/update/output.txt | 6 ++--- .../bundle/resources/pipelines/update/script | 3 +-- .../recreate/out.requests.create.txt | 10 ++++---- .../recreate/out.requests.update.txt | 10 ++++---- .../postgres_projects/recreate/script | 3 +-- .../bundle/resources/schemas/recreate/script | 3 +-- .../resources/volumes/change-comment/script | 3 +-- .../resources/volumes/change-name/script | 3 +-- .../volumes/change-schema-name/script | 3 +-- .../volumes/set-storage-location/script | 3 +-- .../bundle/run/jobs/partial_run/output.txt | 24 +++++++++---------- acceptance/bundle/run/jobs/partial_run/script | 3 +-- .../bundle/run/refresh-flags/output.txt | 24 +++++++++---------- acceptance/bundle/run/refresh-flags/script | 3 +-- .../dry-run/dry-run-pipeline/output.txt | 12 +++++----- .../pipelines/dry-run/dry-run-pipeline/script | 3 +-- .../pipelines/dry-run/restart/output.txt | 6 ++--- acceptance/pipelines/dry-run/restart/script | 3 +-- .../pipelines/run/refresh-flags/output.txt | 24 +++++++++---------- acceptance/pipelines/run/refresh-flags/script | 3 +-- acceptance/pipelines/run/restart/output.txt | 4 ++-- acceptance/pipelines/run/restart/script | 3 +-- acceptance/pipelines/stop/script | 3 +-- 35 files changed, 101 insertions(+), 126 deletions(-) diff --git a/acceptance/bundle/plan/no_upload/output.txt b/acceptance/bundle/plan/no_upload/output.txt index e6cc86840a8..9711d59171a 100644 --- a/acceptance/bundle/plan/no_upload/output.txt +++ b/acceptance/bundle/plan/no_upload/output.txt @@ -4,4 +4,4 @@ create jobs.my_job Plan: 1 to add, 0 to change, 0 to delete, 0 unchanged ->>> jq -s .[] | select(.method != "GET") out.requests.txt +>>> print_requests.py diff --git a/acceptance/bundle/plan/no_upload/script b/acceptance/bundle/plan/no_upload/script index 02d44d657ba..03bcb61fde8 100644 --- a/acceptance/bundle/plan/no_upload/script +++ b/acceptance/bundle/plan/no_upload/script @@ -1,6 +1,4 @@ trace $CLI bundle plan # Expect no non-GET requests -trace jq -s '.[] | select(.method != "GET")' out.requests.txt - -rm out.requests.txt +trace print_requests.py diff --git a/acceptance/bundle/resources/apps/lifecycle-started-omitted/output.txt b/acceptance/bundle/resources/apps/lifecycle-started-omitted/output.txt index c4b88e6634a..66fe33bfacd 100644 --- a/acceptance/bundle/resources/apps/lifecycle-started-omitted/output.txt +++ b/acceptance/bundle/resources/apps/lifecycle-started-omitted/output.txt @@ -8,14 +8,14 @@ Deployment complete! >>> print_app_requests { - "body": { - "description": "my_app_description", - "name": "[UNIQUE_NAME]" - }, "method": "POST", "path": "/api/2.0/apps", "q": { "no_compute": "true" + }, + "body": { + "description": "my_app_description", + "name": "[UNIQUE_NAME]" } } @@ -46,17 +46,17 @@ Deployment complete! >>> print_app_requests { - "body": {}, "method": "POST", - "path": "/api/2.0/apps/[UNIQUE_NAME]/start" + "path": "/api/2.0/apps/[UNIQUE_NAME]/start", + "body": {} } { + "method": "POST", + "path": "/api/2.0/apps/[UNIQUE_NAME]/deployments", "body": { "mode": "SNAPSHOT", "source_code_path": "/Workspace/Users/[USERNAME]/.bundle/lifecycle-started-omitted-[UNIQUE_NAME]/default/files/app" - }, - "method": "POST", - "path": "/api/2.0/apps/[UNIQUE_NAME]/deployments" + } } === started: true -> (started omitted) -> deploy: no start/stop requests (compute stays as-is) diff --git a/acceptance/bundle/resources/apps/lifecycle-started-omitted/script b/acceptance/bundle/resources/apps/lifecycle-started-omitted/script index ca97563ffaf..c836adff587 100644 --- a/acceptance/bundle/resources/apps/lifecycle-started-omitted/script +++ b/acceptance/bundle/resources/apps/lifecycle-started-omitted/script @@ -7,8 +7,7 @@ cleanup() { trap cleanup EXIT print_app_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/apps")))' < out.requests.txt - rm out.requests.txt + print_requests.py //apps } title "(started omitted) -> deploy: app created with no_compute=true, no start/stop requests" diff --git a/acceptance/bundle/resources/permissions/genie_spaces/current_can_manage/script b/acceptance/bundle/resources/permissions/genie_spaces/current_can_manage/script index 1b20af07543..21c43825868 100644 --- a/acceptance/bundle/resources/permissions/genie_spaces/current_can_manage/script +++ b/acceptance/bundle/resources/permissions/genie_spaces/current_can_manage/script @@ -4,8 +4,7 @@ rm out.requests.txt $CLI bundle plan -o json > out.plan.$DATABRICKS_BUNDLE_ENGINE.json print_requests() { - jq -c < out.requests.txt | jq 'select(.method != "GET" and (.path | contains("permissions")))' - rm out.requests.txt + print_requests.py //permissions } rm out.requests.txt diff --git a/acceptance/bundle/resources/permissions/jobs/deleted_remotely/script b/acceptance/bundle/resources/permissions/jobs/deleted_remotely/script index a21c8f9b11e..fa2f6c82b50 100644 --- a/acceptance/bundle/resources/permissions/jobs/deleted_remotely/script +++ b/acceptance/bundle/resources/permissions/jobs/deleted_remotely/script @@ -1,6 +1,5 @@ print_requests() { - jq 'select(.path | contains("/jobs/")) | select(.method != "GET")' < out.requests.txt - rm out.requests.txt + print_requests.py //jobs/ } trace $CLI bundle plan -o json > out.plan_create.$DATABRICKS_BUNDLE_ENGINE.json diff --git a/acceptance/bundle/resources/permissions/jobs/update/script b/acceptance/bundle/resources/permissions/jobs/update/script index d13f5336ed7..696ddf2491e 100644 --- a/acceptance/bundle/resources/permissions/jobs/update/script +++ b/acceptance/bundle/resources/permissions/jobs/update/script @@ -1,6 +1,5 @@ print_requests() { - jq 'select(.path | contains("/jobs/")) | select(.method != "GET")' < out.requests.txt - rm out.requests.txt + print_requests.py //jobs/ } cp databricks.yml databricks.yml.saved diff --git a/acceptance/bundle/resources/permissions/models/current_can_manage/script b/acceptance/bundle/resources/permissions/models/current_can_manage/script index 19146a993f3..037a9fddcd1 100644 --- a/acceptance/bundle/resources/permissions/models/current_can_manage/script +++ b/acceptance/bundle/resources/permissions/models/current_can_manage/script @@ -4,8 +4,7 @@ rm out.requests.txt $CLI bundle plan -o json > out.plan.$DATABRICKS_BUNDLE_ENGINE.json print_requests() { - jq -c < out.requests.txt | jq 'select(.method != "GET" and (.path | contains("permissions")))' - rm out.requests.txt + print_requests.py //permissions } rm out.requests.txt diff --git a/acceptance/bundle/resources/permissions/pipelines/update/script b/acceptance/bundle/resources/permissions/pipelines/update/script index 430521b2918..3498a4d6e49 100644 --- a/acceptance/bundle/resources/permissions/pipelines/update/script +++ b/acceptance/bundle/resources/permissions/pipelines/update/script @@ -1,11 +1,9 @@ print_requests() { - jq 'select(.path | contains("/pipeline")) | select(.method != "GET")' < out.requests.txt - rm out.requests.txt + print_requests.py //pipeline } print_sorted_requests() { - jq -c < out.requests.txt | sort | jq --sort-keys 'select(.path | contains("/pipeline")) | select(.method != "GET")' - rm out.requests.txt + print_requests.py //pipeline --sort } diff --git a/acceptance/bundle/resources/permissions/target_permissions/script b/acceptance/bundle/resources/permissions/target_permissions/script index 2caf8dc733c..d28c2a39e49 100644 --- a/acceptance/bundle/resources/permissions/target_permissions/script +++ b/acceptance/bundle/resources/permissions/target_permissions/script @@ -1,11 +1,9 @@ print_requests() { - jq 'select(.path | contains("/jobs/")) | select(.method != "GET")' < out.requests.txt - rm out.requests.txt + print_requests.py //jobs/ } print_sorted_requests() { - jq -c < out.requests.txt | sort | jq --sort-keys 'select(.path | contains("/jobs/")) | select(.method != "GET")' - rm out.requests.txt + print_requests.py //jobs/ --sort } trace $CLI bundle plan diff --git a/acceptance/bundle/resources/pipelines/allow-duplicate-names/output.txt b/acceptance/bundle/resources/pipelines/allow-duplicate-names/output.txt index 278c60dabbc..5bbca4998bc 100644 --- a/acceptance/bundle/resources/pipelines/allow-duplicate-names/output.txt +++ b/acceptance/bundle/resources/pipelines/allow-duplicate-names/output.txt @@ -7,6 +7,8 @@ Deployment complete! >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines", "body": { "libraries": [ { @@ -16,11 +18,11 @@ Deployment complete! } ], "name": "test-pipeline-same-name-[UNIQUE_NAME]" - }, - "method": "POST", - "path": "/api/2.0/pipelines" + } } { + "method": "POST", + "path": "/api/2.0/pipelines", "body": { "allow_duplicate_names": true, "channel": "CURRENT", @@ -37,9 +39,7 @@ Deployment complete! } ], "name": "test-pipeline-same-name-[UNIQUE_NAME]" - }, - "method": "POST", - "path": "/api/2.0/pipelines" + } } >>> [CLI] bundle destroy --auto-approve diff --git a/acceptance/bundle/resources/pipelines/allow-duplicate-names/script b/acceptance/bundle/resources/pipelines/allow-duplicate-names/script index 6db327deec3..b288e561838 100644 --- a/acceptance/bundle/resources/pipelines/allow-duplicate-names/script +++ b/acceptance/bundle/resources/pipelines/allow-duplicate-names/script @@ -17,6 +17,6 @@ export PIPELINE_ID trace $CLI bundle deploy print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt + print_requests.py //pipelines } trace print_requests diff --git a/acceptance/bundle/resources/pipelines/update/output.txt b/acceptance/bundle/resources/pipelines/update/output.txt index 597df75da99..213e8a0cc64 100644 --- a/acceptance/bundle/resources/pipelines/update/output.txt +++ b/acceptance/bundle/resources/pipelines/update/output.txt @@ -7,6 +7,8 @@ Deployment complete! >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines", "body": { "channel": "CURRENT", "deployment": { @@ -22,9 +24,7 @@ Deployment complete! } ], "name": "test-pipeline-[UNIQUE_NAME]" - }, - "method": "POST", - "path": "/api/2.0/pipelines" + } } pipelines my id='[MY_ID]' name='test-pipeline-[UNIQUE_NAME]' diff --git a/acceptance/bundle/resources/pipelines/update/script b/acceptance/bundle/resources/pipelines/update/script index d801e604793..255dea2e00d 100644 --- a/acceptance/bundle/resources/pipelines/update/script +++ b/acceptance/bundle/resources/pipelines/update/script @@ -4,8 +4,7 @@ touch bar.py trace $CLI bundle deploy print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt - rm out.requests.txt + print_requests.py //pipelines read_state.py pipelines my id name } diff --git a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt index e0bb397af98..4ee3402fd27 100644 --- a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt +++ b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt @@ -1,4 +1,9 @@ { + "method": "POST", + "path": "/api/2.0/postgres/projects", + "q": { + "project_id": "test-pg-old-[UNIQUE_NAME]" + }, "body": { "spec": { "default_endpoint_settings": { @@ -10,10 +15,5 @@ "history_retention_duration": "604800s", "pg_version": 16 } - }, - "method": "POST", - "path": "/api/2.0/postgres/projects", - "q": { - "project_id": "test-pg-old-[UNIQUE_NAME]" } } diff --git a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt index a15cfde30f8..fc8b1d408cf 100644 --- a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt +++ b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt @@ -3,6 +3,11 @@ "path": "/api/2.0/postgres/[MY_PROJECT_ID]" } { + "method": "POST", + "path": "/api/2.0/postgres/projects", + "q": { + "project_id": "test-pg-new-[UNIQUE_NAME]" + }, "body": { "spec": { "default_endpoint_settings": { @@ -14,10 +19,5 @@ "history_retention_duration": "604800s", "pg_version": 16 } - }, - "method": "POST", - "path": "/api/2.0/postgres/projects", - "q": { - "project_id": "test-pg-new-[UNIQUE_NAME]" } } diff --git a/acceptance/bundle/resources/postgres_projects/recreate/script b/acceptance/bundle/resources/postgres_projects/recreate/script index 422b6257001..0c23dae366f 100644 --- a/acceptance/bundle/resources/postgres_projects/recreate/script +++ b/acceptance/bundle/resources/postgres_projects/recreate/script @@ -23,8 +23,7 @@ project_id_1=`read_id.py my_project` print_requests() { local name=$1 - jq --sort-keys 'select(.method != "GET" and (.path | contains("/postgres")))' < out.requests.txt > out.requests.${name}.txt - rm -f out.requests.txt + print_requests.py //postgres > out.requests.${name}.txt } print_requests create diff --git a/acceptance/bundle/resources/schemas/recreate/script b/acceptance/bundle/resources/schemas/recreate/script index cadbfa29a7f..2fc7ac43d02 100644 --- a/acceptance/bundle/resources/schemas/recreate/script +++ b/acceptance/bundle/resources/schemas/recreate/script @@ -2,8 +2,7 @@ echo "*" > .gitignore trace $CLI bundle deploy print_requests() { - jq 'select(.method != "GET" and (.path | contains("/unity")))' < out.requests.txt - rm out.requests.txt + print_requests.py //unity } trace print_requests diff --git a/acceptance/bundle/resources/volumes/change-comment/script b/acceptance/bundle/resources/volumes/change-comment/script index aa237582ac3..6fa1cacf797 100644 --- a/acceptance/bundle/resources/volumes/change-comment/script +++ b/acceptance/bundle/resources/volumes/change-comment/script @@ -7,8 +7,7 @@ trace musterr $CLI volumes read main.myschema.myvolume trace $CLI bundle deploy print_requests() { - jq 'select(.method != "GET" and (.path | contains("/unity")))' < out.requests.txt - rm out.requests.txt + print_requests.py //unity } trace print_requests diff --git a/acceptance/bundle/resources/volumes/change-name/script b/acceptance/bundle/resources/volumes/change-name/script index bc012c0a396..8a5e19240ba 100644 --- a/acceptance/bundle/resources/volumes/change-name/script +++ b/acceptance/bundle/resources/volumes/change-name/script @@ -1,8 +1,7 @@ trace $CLI bundle deploy print_requests() { - jq 'select(.method != "GET" and (.path | contains("/unity")))' < out.requests.txt - rm out.requests.txt + print_requests.py //unity } trace print_requests diff --git a/acceptance/bundle/resources/volumes/change-schema-name/script b/acceptance/bundle/resources/volumes/change-schema-name/script index 088998d8b0f..df17b011f76 100644 --- a/acceptance/bundle/resources/volumes/change-schema-name/script +++ b/acceptance/bundle/resources/volumes/change-schema-name/script @@ -1,8 +1,7 @@ trace $CLI bundle deploy print_requests() { - jq 'select(.method != "GET" and (.path | contains("/unity")))' < out.requests.txt - rm out.requests.txt + print_requests.py //unity } trace print_requests diff --git a/acceptance/bundle/resources/volumes/set-storage-location/script b/acceptance/bundle/resources/volumes/set-storage-location/script index 3ae0d89ae63..ce16f98e896 100644 --- a/acceptance/bundle/resources/volumes/set-storage-location/script +++ b/acceptance/bundle/resources/volumes/set-storage-location/script @@ -1,6 +1,5 @@ print_requests() { - jq 'select(.method != "GET" and (.path | contains("/unity")))' < out.requests.txt - rm out.requests.txt + print_requests.py //unity } cleanup() { diff --git a/acceptance/bundle/run/jobs/partial_run/output.txt b/acceptance/bundle/run/jobs/partial_run/output.txt index c9cc043cfe7..7cc61635bf2 100644 --- a/acceptance/bundle/run/jobs/partial_run/output.txt +++ b/acceptance/bundle/run/jobs/partial_run/output.txt @@ -22,14 +22,14 @@ Hello from notebook2! >>> print_requests { + "method": "POST", + "path": "/api/2.2/jobs/run-now", "body": { "job_id": [NUMID], "only": [ "task_1" ] - }, - "method": "POST", - "path": "/api/2.2/jobs/run-now" + } } >>> [CLI] bundle run my_job --only task_1,task_2 @@ -49,15 +49,15 @@ Hello from notebook2! >>> print_requests { + "method": "POST", + "path": "/api/2.2/jobs/run-now", "body": { "job_id": [NUMID], "only": [ "task_1", "task_2" ] - }, - "method": "POST", - "path": "/api/2.2/jobs/run-now" + } } >>> [CLI] bundle run my_job @@ -77,11 +77,11 @@ Hello from notebook2! >>> print_requests { + "method": "POST", + "path": "/api/2.2/jobs/run-now", "body": { "job_id": [NUMID] - }, - "method": "POST", - "path": "/api/2.2/jobs/run-now" + } } >>> musterr [CLI] bundle run my_job --only non_existent_task @@ -107,6 +107,8 @@ Hello from notebook2! >>> print_requests { + "method": "POST", + "path": "/api/2.2/jobs/run-now", "body": { "job_id": [NUMID], "only": [ @@ -114,7 +116,5 @@ Hello from notebook2! "task2>", "task3>table3" ] - }, - "method": "POST", - "path": "/api/2.2/jobs/run-now" + } } diff --git a/acceptance/bundle/run/jobs/partial_run/script b/acceptance/bundle/run/jobs/partial_run/script index 833ada53062..bc858589858 100644 --- a/acceptance/bundle/run/jobs/partial_run/script +++ b/acceptance/bundle/run/jobs/partial_run/script @@ -1,6 +1,5 @@ print_requests() { - jq --sort-keys 'select(.path | contains("/jobs/run-now"))' < out.requests.txt - rm out.requests.txt + print_requests.py //jobs/run-now } trace $CLI bundle deploy diff --git a/acceptance/bundle/run/refresh-flags/output.txt b/acceptance/bundle/run/refresh-flags/output.txt index 34d23f77407..d3f92276703 100644 --- a/acceptance/bundle/run/refresh-flags/output.txt +++ b/acceptance/bundle/run/refresh-flags/output.txt @@ -13,14 +13,14 @@ Update ID: [UUID] >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "refresh_selection": [ "table1", "table2" ] - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } === Running pipeline with --full-refresh-all flag @@ -31,11 +31,11 @@ Update ID: [UUID] >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "full_refresh": true - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } === Running pipeline with --full-refresh flag and specific tables @@ -46,14 +46,14 @@ Update ID: [UUID] >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "full_refresh_selection": [ "table1", "table2" ] - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } === Running pipeline with --full-refresh flag and --refresh flag @@ -64,6 +64,8 @@ Update ID: [UUID] >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "full_refresh_selection": [ "table1", @@ -73,7 +75,5 @@ Update ID: [UUID] "table3", "table4" ] - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } diff --git a/acceptance/bundle/run/refresh-flags/script b/acceptance/bundle/run/refresh-flags/script index ba3de195693..eafe6c494aa 100644 --- a/acceptance/bundle/run/refresh-flags/script +++ b/acceptance/bundle/run/refresh-flags/script @@ -1,6 +1,5 @@ print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt - rm out.requests.txt + print_requests.py //pipelines } trace $CLI bundle deploy diff --git a/acceptance/pipelines/dry-run/dry-run-pipeline/output.txt b/acceptance/pipelines/dry-run/dry-run-pipeline/output.txt index 159397f0f9a..c9799262bdd 100644 --- a/acceptance/pipelines/dry-run/dry-run-pipeline/output.txt +++ b/acceptance/pipelines/dry-run/dry-run-pipeline/output.txt @@ -16,11 +16,11 @@ Update URL: [DATABRICKS_URL]/#joblist/pipelines/[UUID]/updates/[UUID] Update ID: [UUID] { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "validate_only": true - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } === Dry running pipeline with KEY, expect same output as without KEY @@ -33,9 +33,9 @@ Update URL: [DATABRICKS_URL]/#joblist/pipelines/[UUID]/updates/[UUID] Update ID: [UUID] { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "validate_only": true - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } diff --git a/acceptance/pipelines/dry-run/dry-run-pipeline/script b/acceptance/pipelines/dry-run/dry-run-pipeline/script index 8afe4968c51..1c16b84db43 100755 --- a/acceptance/pipelines/dry-run/dry-run-pipeline/script +++ b/acceptance/pipelines/dry-run/dry-run-pipeline/script @@ -1,6 +1,5 @@ print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt - rm out.requests.txt + print_requests.py //pipelines } trace $CLI pipelines deploy diff --git a/acceptance/pipelines/dry-run/restart/output.txt b/acceptance/pipelines/dry-run/restart/output.txt index f6764e55d6d..1d4c504a498 100644 --- a/acceptance/pipelines/dry-run/restart/output.txt +++ b/acceptance/pipelines/dry-run/restart/output.txt @@ -22,9 +22,9 @@ Update ID: [UUID] "path": "/api/2.0/pipelines/[UUID]/stop" } { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "validate_only": true - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } diff --git a/acceptance/pipelines/dry-run/restart/script b/acceptance/pipelines/dry-run/restart/script index b60de9760d0..3350aa55b77 100644 --- a/acceptance/pipelines/dry-run/restart/script +++ b/acceptance/pipelines/dry-run/restart/script @@ -1,6 +1,5 @@ print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt - rm out.requests.txt + print_requests.py //pipelines } trace $CLI pipelines deploy diff --git a/acceptance/pipelines/run/refresh-flags/output.txt b/acceptance/pipelines/run/refresh-flags/output.txt index b7296296577..9c1c6b86771 100644 --- a/acceptance/pipelines/run/refresh-flags/output.txt +++ b/acceptance/pipelines/run/refresh-flags/output.txt @@ -23,14 +23,14 @@ Pipeline configurations for this update: >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "refresh_selection": [ "table1", "table2" ] - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } === Running pipeline with --full-refresh-all flag @@ -50,11 +50,11 @@ Pipeline configurations for this update: >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "full_refresh": true - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } === Running pipeline with --full-refresh flag and specific tables @@ -74,14 +74,14 @@ Pipeline configurations for this update: >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "full_refresh_selection": [ "table1", "table2" ] - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } === Running pipeline with --full-refresh flag and --refresh flag @@ -101,6 +101,8 @@ Pipeline configurations for this update: >>> print_requests { + "method": "POST", + "path": "/api/2.0/pipelines/[UUID]/updates", "body": { "full_refresh_selection": [ "table1", @@ -110,7 +112,5 @@ Pipeline configurations for this update: "table3", "table4" ] - }, - "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + } } diff --git a/acceptance/pipelines/run/refresh-flags/script b/acceptance/pipelines/run/refresh-flags/script index 58fea998ccc..7c2286b3aea 100644 --- a/acceptance/pipelines/run/refresh-flags/script +++ b/acceptance/pipelines/run/refresh-flags/script @@ -1,6 +1,5 @@ print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt - rm out.requests.txt + print_requests.py //pipelines } trace $CLI pipelines deploy diff --git a/acceptance/pipelines/run/restart/output.txt b/acceptance/pipelines/run/restart/output.txt index 1ba6202cc03..7baadb6e36f 100644 --- a/acceptance/pipelines/run/restart/output.txt +++ b/acceptance/pipelines/run/restart/output.txt @@ -27,7 +27,7 @@ Pipeline configurations for this update: "path": "/api/2.0/pipelines/[UUID]/stop" } { - "body": {}, "method": "POST", - "path": "/api/2.0/pipelines/[UUID]/updates" + "path": "/api/2.0/pipelines/[UUID]/updates", + "body": {} } diff --git a/acceptance/pipelines/run/restart/script b/acceptance/pipelines/run/restart/script index a470005c3b0..a184a10f836 100644 --- a/acceptance/pipelines/run/restart/script +++ b/acceptance/pipelines/run/restart/script @@ -1,6 +1,5 @@ print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt - rm out.requests.txt + print_requests.py //pipelines } trace $CLI pipelines deploy diff --git a/acceptance/pipelines/stop/script b/acceptance/pipelines/stop/script index 8eed3686432..674a8aec7b7 100644 --- a/acceptance/pipelines/stop/script +++ b/acceptance/pipelines/stop/script @@ -1,6 +1,5 @@ print_requests() { - jq --sort-keys 'select(.method != "GET" and (.path | contains("/pipelines")))' < out.requests.txt - rm out.requests.txt + print_requests.py //pipelines } trace $CLI pipelines deploy From 0b2d9b45f0b78394ffb415b8fcf13359792db04a Mon Sep 17 00:00:00 2001 From: Jan Rose Date: Wed, 17 Jun 2026 13:02:51 +0200 Subject: [PATCH 3/4] acceptance: use musterr for must-fail tests instead of errcode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit errcode tolerates both success and failure, so a regression that makes a "this must fail" command start succeeding slips through. musterr fails the test on unexpected success, which is the correct assertion for these error-path tests. Convert standalone errcode to musterr across 35 tests whose command must fail (invalid args/flags, not-found, unknown subcommand, missing auth, client-side validation errors: select/missing, paths, multi_profile, duplicate keys, ...) — each confirmed by an `Exit code:` line in its output.txt. The regenerated output drops that line (musterr doesn't print it; the asserted error message stays). Left as errcode: piped `errcode ... | jq` (musterr's failure is swallowed by the pipe without pipefail), commands that may legitimately succeed (e.g. `secrets list-scopes`, the default_profile resolution paths, `--var a=one` in arg-repeat), engine-dependent failures (*-terraform-error, select/rejected), WAL/crash side-effect tests, and cloud-recording selftests where the failure is genuinely tolerated. Co-authored-by: Isaac --- acceptance/apps/deploy/no-bundle-no-args/output.txt | 2 -- acceptance/apps/deploy/no-bundle-no-args/script | 2 +- acceptance/bundle/artifacts/nil_artifacts/output.txt | 2 -- acceptance/bundle/artifacts/nil_artifacts/script | 2 +- .../generate/alert_existing_id_not_found/output.txt | 4 +--- .../bundle/generate/alert_existing_id_not_found/script | 2 +- .../bundle/multi_profile/no_workspace_profiles/output.txt | 2 -- .../bundle/multi_profile/no_workspace_profiles/script | 2 +- .../bundle/multi_profile/non_interactive_error/output.txt | 2 -- .../bundle/multi_profile/non_interactive_error/script | 2 +- .../paths/pipeline_root_path_doesnotexist/output.txt | 2 -- .../bundle/paths/pipeline_root_path_doesnotexist/script | 2 +- .../with_file_variable_interpolation_error/output.txt | 2 -- .../alerts/with_file_variable_interpolation_error/script | 2 +- .../duplicate_resource_and_script_name_root/output.txt | 4 ---- .../duplicate_resource_and_script_name_root/script | 4 ++-- .../output.txt | 4 ---- .../script | 4 ++-- .../duplicate_script_names_in_subconfiguration/output.txt | 4 ---- .../duplicate_script_names_in_subconfiguration/script | 4 ++-- acceptance/bundle/select/missing/output.txt | 4 ---- acceptance/bundle/select/missing/script | 4 ++-- acceptance/bundle/syncroot/dotdot-git/output.txt | 2 -- acceptance/bundle/syncroot/dotdot-git/script | 2 +- acceptance/bundle/variables/arg-repeat/output.txt | 4 +--- acceptance/bundle/variables/arg-repeat/script | 2 +- acceptance/cmd/auth/logout/error-cases/output.txt | 4 ---- acceptance/cmd/auth/logout/error-cases/script | 4 ++-- .../token/force-refresh-invalid-refresh-token/output.txt | 2 -- .../auth/token/force-refresh-invalid-refresh-token/script | 2 +- .../cmd/auth/token/force-refresh-no-cache/output.txt | 2 -- acceptance/cmd/auth/token/force-refresh-no-cache/script | 2 +- acceptance/cmd/auth/token/no-args-no-profiles/output.txt | 2 -- acceptance/cmd/auth/token/no-args-no-profiles/script | 2 +- .../cmd/auth/token/no-args-with-profiles/output.txt | 2 -- acceptance/cmd/auth/token/no-args-with-profiles/script | 2 +- acceptance/cmd/fs/cp/input-validation/output.txt | 8 ++------ acceptance/cmd/fs/cp/input-validation/script | 4 ++-- .../cmd/sandbox/config/idle-timeout-bounds/output.txt | 4 ---- acceptance/cmd/sandbox/config/idle-timeout-bounds/script | 4 ++-- acceptance/cmd/sandbox/config/no-flags/output.txt | 2 -- acceptance/cmd/sandbox/config/no-flags/script | 2 +- acceptance/cmd/sandbox/default/not-found/output.txt | 2 -- acceptance/cmd/sandbox/default/not-found/script | 2 +- .../cmd/sandbox/delete/no-tty-no-auto-approve/output.txt | 2 -- .../cmd/sandbox/delete/no-tty-no-auto-approve/script | 2 +- acceptance/cmd/sandbox/delete/not-found/output.txt | 2 -- acceptance/cmd/sandbox/delete/not-found/script | 2 +- acceptance/cmd/sandbox/status/not-found/output.txt | 2 -- acceptance/cmd/sandbox/status/not-found/script | 2 +- acceptance/cmd/sync-without-args/output.txt | 4 +--- acceptance/cmd/sync-without-args/script | 2 +- acceptance/cmd/unknown-subcommand/output.txt | 4 +--- acceptance/cmd/unknown-subcommand/script | 2 +- .../experimental/genie/ask-endpoint-gone/output.txt | 2 -- acceptance/experimental/genie/ask-endpoint-gone/script | 2 +- .../experimental/genie/ask-request-drift/output.txt | 2 -- acceptance/experimental/genie/ask-request-drift/script | 2 +- acceptance/experimental/genie/ask/output.txt | 4 ---- acceptance/experimental/genie/ask/script | 4 ++-- acceptance/experimental/open/output.txt | 2 -- acceptance/experimental/open/script | 2 +- acceptance/pipelines/generate/bad-path/output.txt | 4 +--- acceptance/pipelines/generate/bad-path/script | 2 +- acceptance/pipelines/generate/fail-overwrite/output.txt | 4 +--- acceptance/pipelines/generate/fail-overwrite/script | 2 +- acceptance/pipelines/init/error-cases/output.txt | 8 ++------ acceptance/pipelines/init/error-cases/script | 4 ++-- acceptance/workspace/repos/get_errors/output.txt | 4 ---- acceptance/workspace/repos/get_errors/script | 4 ++-- 70 files changed, 55 insertions(+), 145 deletions(-) diff --git a/acceptance/apps/deploy/no-bundle-no-args/output.txt b/acceptance/apps/deploy/no-bundle-no-args/output.txt index 0401016dacf..9a96c4a49b8 100644 --- a/acceptance/apps/deploy/no-bundle-no-args/output.txt +++ b/acceptance/apps/deploy/no-bundle-no-args/output.txt @@ -5,5 +5,3 @@ Usage: databricks apps deploy APP_NAME APP_NAME is the name of the Databricks app to operate on. Alternatively, run this command from a project directory containing databricks.yml to auto-detect the app name. - -Exit code: 1 diff --git a/acceptance/apps/deploy/no-bundle-no-args/script b/acceptance/apps/deploy/no-bundle-no-args/script index 7161d5f742f..11862a782d2 100644 --- a/acceptance/apps/deploy/no-bundle-no-args/script +++ b/acceptance/apps/deploy/no-bundle-no-args/script @@ -1,3 +1,3 @@ # Test: apps deploy without databricks.yml and no APP_NAME # Expected: Error about missing argument -errcode $CLI apps deploy +musterr $CLI apps deploy diff --git a/acceptance/bundle/artifacts/nil_artifacts/output.txt b/acceptance/bundle/artifacts/nil_artifacts/output.txt index f4bf0a6f400..a47dca01faf 100644 --- a/acceptance/bundle/artifacts/nil_artifacts/output.txt +++ b/acceptance/bundle/artifacts/nil_artifacts/output.txt @@ -12,5 +12,3 @@ Workspace: Path: /Workspace/Users/[USERNAME]/.bundle/nil_artifacts_test/default Found 1 error - -Exit code: 1 diff --git a/acceptance/bundle/artifacts/nil_artifacts/script b/acceptance/bundle/artifacts/nil_artifacts/script index f52b452ee67..277355fe918 100644 --- a/acceptance/bundle/artifacts/nil_artifacts/script +++ b/acceptance/bundle/artifacts/nil_artifacts/script @@ -1 +1 @@ -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate diff --git a/acceptance/bundle/generate/alert_existing_id_not_found/output.txt b/acceptance/bundle/generate/alert_existing_id_not_found/output.txt index 3f841a92798..20e75bf0ff7 100644 --- a/acceptance/bundle/generate/alert_existing_id_not_found/output.txt +++ b/acceptance/bundle/generate/alert_existing_id_not_found/output.txt @@ -1,5 +1,3 @@ ->>> errcode [CLI] bundle generate alert --existing-id f00dcafe +>>> musterr [CLI] bundle generate alert --existing-id f00dcafe Error: alert with ID f00dcafe not found - -Exit code: 1 diff --git a/acceptance/bundle/generate/alert_existing_id_not_found/script b/acceptance/bundle/generate/alert_existing_id_not_found/script index 85edf127c61..b84a42bd8c2 100644 --- a/acceptance/bundle/generate/alert_existing_id_not_found/script +++ b/acceptance/bundle/generate/alert_existing_id_not_found/script @@ -1,2 +1,2 @@ # Test that bundle generate alert fails when the existing ID is not found -trace errcode $CLI bundle generate alert --existing-id f00dcafe +trace musterr $CLI bundle generate alert --existing-id f00dcafe diff --git a/acceptance/bundle/multi_profile/no_workspace_profiles/output.txt b/acceptance/bundle/multi_profile/no_workspace_profiles/output.txt index 02a544efb52..5ab1e747db4 100644 --- a/acceptance/bundle/multi_profile/no_workspace_profiles/output.txt +++ b/acceptance/bundle/multi_profile/no_workspace_profiles/output.txt @@ -8,5 +8,3 @@ Workspace: Host: [DATABRICKS_URL] Found 1 error - -Exit code: 1 diff --git a/acceptance/bundle/multi_profile/no_workspace_profiles/script b/acceptance/bundle/multi_profile/no_workspace_profiles/script index 97dda0e2afc..0ab4a8e8365 100644 --- a/acceptance/bundle/multi_profile/no_workspace_profiles/script +++ b/acceptance/bundle/multi_profile/no_workspace_profiles/script @@ -14,4 +14,4 @@ unset DATABRICKS_HOST unset DATABRICKS_TOKEN # No workspace-compatible profiles → original multi-profile error returned. -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate diff --git a/acceptance/bundle/multi_profile/non_interactive_error/output.txt b/acceptance/bundle/multi_profile/non_interactive_error/output.txt index d3621f1274b..f5fd72ff443 100644 --- a/acceptance/bundle/multi_profile/non_interactive_error/output.txt +++ b/acceptance/bundle/multi_profile/non_interactive_error/output.txt @@ -19,5 +19,3 @@ Workspace: Host: [DATABRICKS_URL] Found 1 error - -Exit code: 1 diff --git a/acceptance/bundle/multi_profile/non_interactive_error/script b/acceptance/bundle/multi_profile/non_interactive_error/script index 85003ad4710..d7e5c4102ad 100644 --- a/acceptance/bundle/multi_profile/non_interactive_error/script +++ b/acceptance/bundle/multi_profile/non_interactive_error/script @@ -14,4 +14,4 @@ unset DATABRICKS_HOST unset DATABRICKS_TOKEN # Multiple workspace profiles, non-interactive → error with guidance. -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate diff --git a/acceptance/bundle/paths/pipeline_root_path_doesnotexist/output.txt b/acceptance/bundle/paths/pipeline_root_path_doesnotexist/output.txt index 043526e6692..85ffdf9201e 100644 --- a/acceptance/bundle/paths/pipeline_root_path_doesnotexist/output.txt +++ b/acceptance/bundle/paths/pipeline_root_path_doesnotexist/output.txt @@ -9,5 +9,3 @@ Workspace: Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default Found 1 error - -Exit code: 1 diff --git a/acceptance/bundle/paths/pipeline_root_path_doesnotexist/script b/acceptance/bundle/paths/pipeline_root_path_doesnotexist/script index f52b452ee67..277355fe918 100644 --- a/acceptance/bundle/paths/pipeline_root_path_doesnotexist/script +++ b/acceptance/bundle/paths/pipeline_root_path_doesnotexist/script @@ -1 +1 @@ -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate diff --git a/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/output.txt b/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/output.txt index 8b63705221d..554f9cf1d7e 100644 --- a/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/output.txt +++ b/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/output.txt @@ -13,5 +13,3 @@ Workspace: Path: /Workspace/Users/[USERNAME]/.bundle/alerts-variable-error/default Found 1 error - -Exit code: 1 diff --git a/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/script b/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/script index f52b452ee67..277355fe918 100755 --- a/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/script +++ b/acceptance/bundle/resources/alerts/with_file_variable_interpolation_error/script @@ -1 +1 @@ -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate diff --git a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/output.txt b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/output.txt index ccc96861549..a2bae4a4f69 100644 --- a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/output.txt +++ b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/output.txt @@ -10,8 +10,6 @@ Name: test Found 1 error -Exit code: 1 - >>> [CLI] bundle run foo Error: multiple resources or scripts have been defined with the same key: foo at resources.jobs.foo @@ -19,5 +17,3 @@ Error: multiple resources or scripts have been defined with the same key: foo in databricks.yml:9:5 databricks.yml:15:7 - -Exit code: 1 diff --git a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/script b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/script index 4c87f36259e..2f13ca05f0f 100644 --- a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/script +++ b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_name_root/script @@ -1,3 +1,3 @@ -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate -errcode trace $CLI bundle run foo +musterr trace $CLI bundle run foo diff --git a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/output.txt b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/output.txt index f3076886e60..9c80bfbd622 100644 --- a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/output.txt +++ b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/output.txt @@ -10,8 +10,6 @@ Name: test Found 1 error -Exit code: 1 - >>> [CLI] bundle run foo Error: multiple resources or scripts have been defined with the same key: foo at resources.pipelines.foo @@ -19,5 +17,3 @@ Error: multiple resources or scripts have been defined with the same key: foo in resource.yml:4:7 script.yml:3:5 - -Exit code: 1 diff --git a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/script b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/script index 4c87f36259e..2f13ca05f0f 100644 --- a/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/script +++ b/acceptance/bundle/run/scripts/unique_keys/duplicate_resource_and_script_subconfigurations/script @@ -1,3 +1,3 @@ -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate -errcode trace $CLI bundle run foo +musterr trace $CLI bundle run foo diff --git a/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/output.txt b/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/output.txt index 4834307e9f8..9cad499cb40 100644 --- a/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/output.txt +++ b/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/output.txt @@ -9,13 +9,9 @@ Name: test Found 1 error -Exit code: 1 - >>> [CLI] bundle run foo Error: multiple resources or scripts have been defined with the same key: foo at scripts.foo in script1.yml:3:5 script2.yml:3:5 - -Exit code: 1 diff --git a/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/script b/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/script index 4c87f36259e..2f13ca05f0f 100644 --- a/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/script +++ b/acceptance/bundle/run/scripts/unique_keys/duplicate_script_names_in_subconfiguration/script @@ -1,3 +1,3 @@ -errcode trace $CLI bundle validate +musterr trace $CLI bundle validate -errcode trace $CLI bundle run foo +musterr trace $CLI bundle run foo diff --git a/acceptance/bundle/select/missing/output.txt b/acceptance/bundle/select/missing/output.txt index f70c74476ba..de64d7bb511 100644 --- a/acceptance/bundle/select/missing/output.txt +++ b/acceptance/bundle/select/missing/output.txt @@ -3,10 +3,6 @@ Error: no such resource: no_such_resource -Exit code: 1 - >>> [CLI] bundle plan --select jobs.no_such_job Error: no such resource: jobs.no_such_job - -Exit code: 1 diff --git a/acceptance/bundle/select/missing/script b/acceptance/bundle/select/missing/script index 0c3193764ef..878e547cec7 100644 --- a/acceptance/bundle/select/missing/script +++ b/acceptance/bundle/select/missing/script @@ -2,7 +2,7 @@ # engine is known, so these errors are identical on both engines. # Unqualified name that matches no resource. -errcode trace $CLI bundle plan --select no_such_resource +musterr trace $CLI bundle plan --select no_such_resource # Qualified name whose resource does not exist. -errcode trace $CLI bundle plan --select jobs.no_such_job +musterr trace $CLI bundle plan --select jobs.no_such_job diff --git a/acceptance/bundle/syncroot/dotdot-git/output.txt b/acceptance/bundle/syncroot/dotdot-git/output.txt index fe3091b4bde..d21e1f25839 100644 --- a/acceptance/bundle/syncroot/dotdot-git/output.txt +++ b/acceptance/bundle/syncroot/dotdot-git/output.txt @@ -7,5 +7,3 @@ Workspace: Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default Found 1 error - -Exit code: 1 diff --git a/acceptance/bundle/syncroot/dotdot-git/script b/acceptance/bundle/syncroot/dotdot-git/script index 278e7710109..2cfcbcdb3c3 100644 --- a/acceptance/bundle/syncroot/dotdot-git/script +++ b/acceptance/bundle/syncroot/dotdot-git/script @@ -3,6 +3,6 @@ mkdir myrepo cd myrepo cp ../databricks.yml . git-repo-init -errcode $CLI bundle validate +musterr $CLI bundle validate cd .. rm -fr myrepo diff --git a/acceptance/bundle/variables/arg-repeat/output.txt b/acceptance/bundle/variables/arg-repeat/output.txt index 4b97d70a153..dafadc8e4aa 100644 --- a/acceptance/bundle/variables/arg-repeat/output.txt +++ b/acceptance/bundle/variables/arg-repeat/output.txt @@ -7,12 +7,10 @@ } } ->>> errcode [CLI] bundle validate --var a=one --var a=two +>>> musterr [CLI] bundle validate --var a=one --var a=two Error: failed to assign two to a: variable has already been assigned value: one Name: arg-repeat Target: default Found 1 error - -Exit code: 1 diff --git a/acceptance/bundle/variables/arg-repeat/script b/acceptance/bundle/variables/arg-repeat/script index 3e03dbcb121..d74ff35c362 100644 --- a/acceptance/bundle/variables/arg-repeat/script +++ b/acceptance/bundle/variables/arg-repeat/script @@ -1,2 +1,2 @@ trace errcode $CLI bundle validate --var a=one -o json | jq .variables -trace errcode $CLI bundle validate --var a=one --var a=two +trace musterr $CLI bundle validate --var a=one --var a=two diff --git a/acceptance/cmd/auth/logout/error-cases/output.txt b/acceptance/cmd/auth/logout/error-cases/output.txt index b61328b5d11..3d00fa795d8 100644 --- a/acceptance/cmd/auth/logout/error-cases/output.txt +++ b/acceptance/cmd/auth/logout/error-cases/output.txt @@ -2,9 +2,5 @@ === Logout of non-existent profile Error: profile "nonexistent" not found. Available profiles: dev -Exit code: 1 - === Logout without --profile in non-interactive mode Error: the command is being run in a non-interactive environment, please specify a profile using the PROFILE argument or --profile flag - -Exit code: 1 diff --git a/acceptance/cmd/auth/logout/error-cases/script b/acceptance/cmd/auth/logout/error-cases/script index 603a5ecec73..8fff9ffb80f 100644 --- a/acceptance/cmd/auth/logout/error-cases/script +++ b/acceptance/cmd/auth/logout/error-cases/script @@ -10,7 +10,7 @@ auth_type = databricks-cli EOF title "Logout of non-existent profile\n" -errcode $CLI auth logout --profile nonexistent --auto-approve +musterr $CLI auth logout --profile nonexistent --auto-approve title "Logout without --profile in non-interactive mode\n" -errcode $CLI auth logout --auto-approve +musterr $CLI auth logout --auto-approve diff --git a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/output.txt b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/output.txt index f1a1b8cadba..e3862605170 100644 --- a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/output.txt +++ b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/output.txt @@ -1,4 +1,2 @@ Error: A new access token could not be retrieved because the refresh token is invalid. To reauthenticate, run the following command: $ databricks auth login --profile test-profile - -Exit code: 1 diff --git a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/script b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/script index b5f5c297bc5..b1a76cafb1e 100644 --- a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/script +++ b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/script @@ -1,4 +1,4 @@ setup_test_profile setup_test_token_cache -errcode $CLI auth token --profile test-profile --force-refresh +musterr $CLI auth token --profile test-profile --force-refresh diff --git a/acceptance/cmd/auth/token/force-refresh-no-cache/output.txt b/acceptance/cmd/auth/token/force-refresh-no-cache/output.txt index bd35319a5a8..148424a83a4 100644 --- a/acceptance/cmd/auth/token/force-refresh-no-cache/output.txt +++ b/acceptance/cmd/auth/token/force-refresh-no-cache/output.txt @@ -1,3 +1 @@ Error: cache: databricks OAuth is not configured for this host. no cached credentials; run `databricks auth login` to sign in - -Exit code: 1 diff --git a/acceptance/cmd/auth/token/force-refresh-no-cache/script b/acceptance/cmd/auth/token/force-refresh-no-cache/script index e561647987f..b40fc0fa97c 100644 --- a/acceptance/cmd/auth/token/force-refresh-no-cache/script +++ b/acceptance/cmd/auth/token/force-refresh-no-cache/script @@ -1,3 +1,3 @@ setup_test_profile -errcode $CLI auth token --profile test-profile --force-refresh +musterr $CLI auth token --profile test-profile --force-refresh diff --git a/acceptance/cmd/auth/token/no-args-no-profiles/output.txt b/acceptance/cmd/auth/token/no-args-no-profiles/output.txt index 5af1fb20e1b..ed92d886a2d 100644 --- a/acceptance/cmd/auth/token/no-args-no-profiles/output.txt +++ b/acceptance/cmd/auth/token/no-args-no-profiles/output.txt @@ -1,3 +1 @@ Error: no profiles configured. Run 'databricks auth login' to create a profile - -Exit code: 1 diff --git a/acceptance/cmd/auth/token/no-args-no-profiles/script b/acceptance/cmd/auth/token/no-args-no-profiles/script index 82089f4aa80..a4a01b25ef0 100644 --- a/acceptance/cmd/auth/token/no-args-no-profiles/script +++ b/acceptance/cmd/auth/token/no-args-no-profiles/script @@ -5,4 +5,4 @@ unset DATABRICKS_TOKEN unset DATABRICKS_CONFIG_PROFILE # No config file, non-interactive: should error with login hint -errcode $CLI auth token +musterr $CLI auth token diff --git a/acceptance/cmd/auth/token/no-args-with-profiles/output.txt b/acceptance/cmd/auth/token/no-args-with-profiles/output.txt index 116c4de17ea..4a174243bef 100644 --- a/acceptance/cmd/auth/token/no-args-with-profiles/output.txt +++ b/acceptance/cmd/auth/token/no-args-with-profiles/output.txt @@ -1,3 +1 @@ Error: no profile specified. Use --profile to specify which profile to use - -Exit code: 1 diff --git a/acceptance/cmd/auth/token/no-args-with-profiles/script b/acceptance/cmd/auth/token/no-args-with-profiles/script index ea632a7b52b..9f4ea50cbb3 100644 --- a/acceptance/cmd/auth/token/no-args-with-profiles/script +++ b/acceptance/cmd/auth/token/no-args-with-profiles/script @@ -12,4 +12,4 @@ auth_type = databricks-cli ENDCFG # No arguments, non-interactive: should error with profile hint -errcode $CLI auth token +musterr $CLI auth token diff --git a/acceptance/cmd/fs/cp/input-validation/output.txt b/acceptance/cmd/fs/cp/input-validation/output.txt index febe55b74e8..8588023d8f5 100644 --- a/acceptance/cmd/fs/cp/input-validation/output.txt +++ b/acceptance/cmd/fs/cp/input-validation/output.txt @@ -1,10 +1,6 @@ ->>> errcode [CLI] fs cp src dst --concurrency -1 +>>> musterr [CLI] fs cp src dst --concurrency -1 Error: --concurrency must be at least 1 -Exit code: 1 - ->>> errcode [CLI] fs cp src dst --concurrency 0 +>>> musterr [CLI] fs cp src dst --concurrency 0 Error: --concurrency must be at least 1 - -Exit code: 1 diff --git a/acceptance/cmd/fs/cp/input-validation/script b/acceptance/cmd/fs/cp/input-validation/script index a5e8cec8621..0b5afdc5b8a 100644 --- a/acceptance/cmd/fs/cp/input-validation/script +++ b/acceptance/cmd/fs/cp/input-validation/script @@ -1,3 +1,3 @@ # Invalid concurrency values should fail. -trace errcode $CLI fs cp src dst --concurrency -1 -trace errcode $CLI fs cp src dst --concurrency 0 +trace musterr $CLI fs cp src dst --concurrency -1 +trace musterr $CLI fs cp src dst --concurrency 0 diff --git a/acceptance/cmd/sandbox/config/idle-timeout-bounds/output.txt b/acceptance/cmd/sandbox/config/idle-timeout-bounds/output.txt index 00e16383f48..da5dfe76648 100644 --- a/acceptance/cmd/sandbox/config/idle-timeout-bounds/output.txt +++ b/acceptance/cmd/sandbox/config/idle-timeout-bounds/output.txt @@ -1,6 +1,2 @@ Error: idle-timeout must be 0 (clear) or between 1m and 24h, got 30s - -Exit code: 1 Error: idle-timeout must be 0 (clear) or between 1m and 24h, got 48h - -Exit code: 1 diff --git a/acceptance/cmd/sandbox/config/idle-timeout-bounds/script b/acceptance/cmd/sandbox/config/idle-timeout-bounds/script index 252f2fba2a0..30f141cd7c3 100644 --- a/acceptance/cmd/sandbox/config/idle-timeout-bounds/script +++ b/acceptance/cmd/sandbox/config/idle-timeout-bounds/script @@ -1,6 +1,6 @@ # Below the 60s lower bound — client-side validation rejects before # any API call, so no [[Server]] stub needed. -errcode $CLI sandbox config happy-panda-1234 --idle-timeout 30s +musterr $CLI sandbox config happy-panda-1234 --idle-timeout 30s # Above the 24h upper bound. -errcode $CLI sandbox config happy-panda-1234 --idle-timeout 48h +musterr $CLI sandbox config happy-panda-1234 --idle-timeout 48h diff --git a/acceptance/cmd/sandbox/config/no-flags/output.txt b/acceptance/cmd/sandbox/config/no-flags/output.txt index 52eacb32dd3..56d78059977 100644 --- a/acceptance/cmd/sandbox/config/no-flags/output.txt +++ b/acceptance/cmd/sandbox/config/no-flags/output.txt @@ -1,3 +1 @@ Error: nothing to update — pass --name, --idle-timeout, and/or --no-autostop - -Exit code: 1 diff --git a/acceptance/cmd/sandbox/config/no-flags/script b/acceptance/cmd/sandbox/config/no-flags/script index 7752926ba3d..0058167b0b6 100644 --- a/acceptance/cmd/sandbox/config/no-flags/script +++ b/acceptance/cmd/sandbox/config/no-flags/script @@ -1 +1 @@ -errcode $CLI sandbox config happy-panda-1234 +musterr $CLI sandbox config happy-panda-1234 diff --git a/acceptance/cmd/sandbox/default/not-found/output.txt b/acceptance/cmd/sandbox/default/not-found/output.txt index 7e1cb205b8d..235a49bc669 100644 --- a/acceptance/cmd/sandbox/default/not-found/output.txt +++ b/acceptance/cmd/sandbox/default/not-found/output.txt @@ -1,3 +1 @@ Error: no sandbox named "no-such-box" — `databricks sandbox list` shows available IDs - -Exit code: 1 diff --git a/acceptance/cmd/sandbox/default/not-found/script b/acceptance/cmd/sandbox/default/not-found/script index 6f6628de43c..2155f9f651e 100644 --- a/acceptance/cmd/sandbox/default/not-found/script +++ b/acceptance/cmd/sandbox/default/not-found/script @@ -1 +1 @@ -errcode $CLI sandbox default no-such-box +musterr $CLI sandbox default no-such-box diff --git a/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/output.txt b/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/output.txt index cf81ce5c0f6..fb62b25b406 100644 --- a/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/output.txt +++ b/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/output.txt @@ -1,3 +1 @@ Error: `databricks sandbox delete` permanently destroys the sandbox; pass --auto-approve to confirm in non-interactive contexts - -Exit code: 1 diff --git a/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/script b/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/script index 89c75a93e11..b596bf74dc6 100644 --- a/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/script +++ b/acceptance/cmd/sandbox/delete/no-tty-no-auto-approve/script @@ -1,3 +1,3 @@ # No --auto-approve and no TTY → must fail fast pointing at the flag, # not hang on a read from a closed stdin. -errcode $CLI sandbox delete happy-panda-1234 +musterr $CLI sandbox delete happy-panda-1234 diff --git a/acceptance/cmd/sandbox/delete/not-found/output.txt b/acceptance/cmd/sandbox/delete/not-found/output.txt index 7e1cb205b8d..235a49bc669 100644 --- a/acceptance/cmd/sandbox/delete/not-found/output.txt +++ b/acceptance/cmd/sandbox/delete/not-found/output.txt @@ -1,3 +1 @@ Error: no sandbox named "no-such-box" — `databricks sandbox list` shows available IDs - -Exit code: 1 diff --git a/acceptance/cmd/sandbox/delete/not-found/script b/acceptance/cmd/sandbox/delete/not-found/script index 380b1c6be7d..deed8686976 100644 --- a/acceptance/cmd/sandbox/delete/not-found/script +++ b/acceptance/cmd/sandbox/delete/not-found/script @@ -1 +1 @@ -errcode $CLI sandbox delete no-such-box --auto-approve +musterr $CLI sandbox delete no-such-box --auto-approve diff --git a/acceptance/cmd/sandbox/status/not-found/output.txt b/acceptance/cmd/sandbox/status/not-found/output.txt index 7e1cb205b8d..235a49bc669 100644 --- a/acceptance/cmd/sandbox/status/not-found/output.txt +++ b/acceptance/cmd/sandbox/status/not-found/output.txt @@ -1,3 +1 @@ Error: no sandbox named "no-such-box" — `databricks sandbox list` shows available IDs - -Exit code: 1 diff --git a/acceptance/cmd/sandbox/status/not-found/script b/acceptance/cmd/sandbox/status/not-found/script index 376d41cb3f3..26c78d32db7 100644 --- a/acceptance/cmd/sandbox/status/not-found/script +++ b/acceptance/cmd/sandbox/status/not-found/script @@ -1 +1 @@ -errcode $CLI sandbox status no-such-box +musterr $CLI sandbox status no-such-box diff --git a/acceptance/cmd/sync-without-args/output.txt b/acceptance/cmd/sync-without-args/output.txt index ce0fae213c3..6c757495507 100644 --- a/acceptance/cmd/sync-without-args/output.txt +++ b/acceptance/cmd/sync-without-args/output.txt @@ -1,5 +1,5 @@ ->>> errcode [CLI] sync +>>> musterr [CLI] sync Error: accepts 2 arg(s), received 0 Usage: @@ -22,5 +22,3 @@ Global Flags: -p, --profile string ~/.databrickscfg profile -t, --target string bundle target to use (if applicable) - -Exit code: 1 diff --git a/acceptance/cmd/sync-without-args/script b/acceptance/cmd/sync-without-args/script index 8528cd34ab0..0841a0d4ca8 100644 --- a/acceptance/cmd/sync-without-args/script +++ b/acceptance/cmd/sync-without-args/script @@ -1 +1 @@ -trace errcode $CLI sync +trace musterr $CLI sync diff --git a/acceptance/cmd/unknown-subcommand/output.txt b/acceptance/cmd/unknown-subcommand/output.txt index 77ef660d74e..acaad967bdb 100644 --- a/acceptance/cmd/unknown-subcommand/output.txt +++ b/acceptance/cmd/unknown-subcommand/output.txt @@ -2,7 +2,7 @@ >>> errcode [CLI] secrets list-scopes Scope Backend Type ->>> errcode [CLI] secrets unknown +>>> musterr [CLI] secrets unknown Error: unknown command "unknown" for "databricks secrets" Usage: @@ -33,5 +33,3 @@ Global Flags: Use "databricks secrets [command] --help" for more information about a command. - -Exit code: 1 diff --git a/acceptance/cmd/unknown-subcommand/script b/acceptance/cmd/unknown-subcommand/script index 9e7cda66c7a..c36d69ac010 100644 --- a/acceptance/cmd/unknown-subcommand/script +++ b/acceptance/cmd/unknown-subcommand/script @@ -1,2 +1,2 @@ trace errcode $CLI secrets list-scopes -trace errcode $CLI secrets unknown +trace musterr $CLI secrets unknown diff --git a/acceptance/experimental/genie/ask-endpoint-gone/output.txt b/acceptance/experimental/genie/ask-endpoint-gone/output.txt index e16f43614e8..9862aba8e86 100644 --- a/acceptance/experimental/genie/ask-endpoint-gone/output.txt +++ b/acceptance/experimental/genie/ask-endpoint-gone/output.txt @@ -2,5 +2,3 @@ === a removed endpoint tells the user to update the CLI >>> [CLI] experimental genie ask What are total sales by franchise? Error: the Genie API is not available on this workspace: No API found for 'POST /data-rooms/tools/onechat/responses'; the endpoint may have moved since this CLI release: update the Databricks CLI to the latest version (run 'databricks version --check') - -Exit code: 1 diff --git a/acceptance/experimental/genie/ask-endpoint-gone/script b/acceptance/experimental/genie/ask-endpoint-gone/script index 71522bd23c0..7dfda2348e7 100644 --- a/acceptance/experimental/genie/ask-endpoint-gone/script +++ b/acceptance/experimental/genie/ask-endpoint-gone/script @@ -1,2 +1,2 @@ title "a removed endpoint tells the user to update the CLI" -errcode trace $CLI experimental genie ask "What are total sales by franchise?" +musterr trace $CLI experimental genie ask "What are total sales by franchise?" diff --git a/acceptance/experimental/genie/ask-request-drift/output.txt b/acceptance/experimental/genie/ask-request-drift/output.txt index adc75fec7d8..e5d0e7bd29a 100644 --- a/acceptance/experimental/genie/ask-request-drift/output.txt +++ b/acceptance/experimental/genie/ask-request-drift/output.txt @@ -2,5 +2,3 @@ === a 500 with no message points at a possible request format change >>> [CLI] experimental genie ask What are total sales by franchise? Error: the Genie backend could not process the request (500 with no details); if this keeps happening, the request format may have changed since this CLI release: update the Databricks CLI to the latest version (run 'databricks version --check') - -Exit code: 1 diff --git a/acceptance/experimental/genie/ask-request-drift/script b/acceptance/experimental/genie/ask-request-drift/script index c180e2cfeb3..a0aaffdb615 100644 --- a/acceptance/experimental/genie/ask-request-drift/script +++ b/acceptance/experimental/genie/ask-request-drift/script @@ -1,2 +1,2 @@ title "a 500 with no message points at a possible request format change" -errcode trace $CLI experimental genie ask "What are total sales by franchise?" +musterr trace $CLI experimental genie ask "What are total sales by franchise?" diff --git a/acceptance/experimental/genie/ask/output.txt b/acceptance/experimental/genie/ask/output.txt index 9cc81d57bde..5ef02778861 100644 --- a/acceptance/experimental/genie/ask/output.txt +++ b/acceptance/experimental/genie/ask/output.txt @@ -54,9 +54,5 @@ Alpha leads with 1,250 total sales. >>> [CLI] experimental genie ask q --raw --output json Error: --raw cannot be used with --output json -Exit code: 1 - >>> [CLI] experimental genie ask q --raw --include-sql Error: --include-sql cannot be used with --raw - -Exit code: 1 diff --git a/acceptance/experimental/genie/ask/script b/acceptance/experimental/genie/ask/script index cad599a974c..ef0081f1455 100644 --- a/acceptance/experimental/genie/ask/script +++ b/acceptance/experimental/genie/ask/script @@ -11,5 +11,5 @@ title "ask with --raw dumps raw SSE events" trace $CLI experimental genie ask "What are total sales by franchise?" --raw title "incompatible flags are rejected" -errcode trace $CLI experimental genie ask "q" --raw --output json -errcode trace $CLI experimental genie ask "q" --raw --include-sql +musterr trace $CLI experimental genie ask "q" --raw --output json +musterr trace $CLI experimental genie ask "q" --raw --include-sql diff --git a/acceptance/experimental/open/output.txt b/acceptance/experimental/open/output.txt index c395bb4967e..f634d630fc8 100644 --- a/acceptance/experimental/open/output.txt +++ b/acceptance/experimental/open/output.txt @@ -11,8 +11,6 @@ >>> [CLI] experimental open --url unknown 123 Error: unknown resource type "unknown", must be one of: alerts, apps, catalogs, clusters, dashboards, database_catalogs, database_instances, experiments, genie_spaces, jobs, model_serving_endpoints, models, notebooks, pipelines, postgres_catalogs, postgres_synced_tables, quality_monitors, queries, registered_models, schemas, synced_database_tables, vector_search_endpoints, vector_search_indexes, volumes, warehouses -Exit code: 1 - === test auto-completion handler >>> [CLI] __complete experimental open , alerts diff --git a/acceptance/experimental/open/script b/acceptance/experimental/open/script index 820175db8de..596378fc178 100644 --- a/acceptance/experimental/open/script +++ b/acceptance/experimental/open/script @@ -5,7 +5,7 @@ title "print URL for a notebook" trace $CLI experimental open --url notebooks 12345 title "unknown resource type" -errcode trace $CLI experimental open --url unknown 123 +musterr trace $CLI experimental open --url unknown 123 title "test auto-completion handler" trace $CLI __complete experimental open , diff --git a/acceptance/pipelines/generate/bad-path/output.txt b/acceptance/pipelines/generate/bad-path/output.txt index 7b12955c2f5..2a88941a230 100644 --- a/acceptance/pipelines/generate/bad-path/output.txt +++ b/acceptance/pipelines/generate/bad-path/output.txt @@ -1,5 +1,3 @@ ->>> errcode [CLI] pipelines generate --existing-pipeline-dir /tmp/foo +>>> musterr [CLI] pipelines generate --existing-pipeline-dir /tmp/foo Error: please make sure the directory is located in side 'src/' (for example 'src/my_pipeline'), got: [path] - -Exit code: 1 diff --git a/acceptance/pipelines/generate/bad-path/script b/acceptance/pipelines/generate/bad-path/script index c00adf6ce5e..b61b7c886f3 100644 --- a/acceptance/pipelines/generate/bad-path/script +++ b/acceptance/pipelines/generate/bad-path/script @@ -1,2 +1,2 @@ -trace errcode $CLI pipelines generate --existing-pipeline-dir /tmp/foo +trace musterr $CLI pipelines generate --existing-pipeline-dir /tmp/foo diff --git a/acceptance/pipelines/generate/fail-overwrite/output.txt b/acceptance/pipelines/generate/fail-overwrite/output.txt index ec2a53fffc8..bba32160e25 100644 --- a/acceptance/pipelines/generate/fail-overwrite/output.txt +++ b/acceptance/pipelines/generate/fail-overwrite/output.txt @@ -1,5 +1,3 @@ ->>> errcode [CLI] pipelines generate --existing-pipeline-dir src/my_pipeline +>>> musterr [CLI] pipelines generate --existing-pipeline-dir src/my_pipeline Error: resources/my_pipeline.pipeline.yml already exists. Use --force to overwrite - -Exit code: 1 diff --git a/acceptance/pipelines/generate/fail-overwrite/script b/acceptance/pipelines/generate/fail-overwrite/script index 1eee25565cf..82bedcaf66f 100644 --- a/acceptance/pipelines/generate/fail-overwrite/script +++ b/acceptance/pipelines/generate/fail-overwrite/script @@ -1,2 +1,2 @@ -trace errcode $CLI pipelines generate --existing-pipeline-dir src/my_pipeline +trace musterr $CLI pipelines generate --existing-pipeline-dir src/my_pipeline diff --git a/acceptance/pipelines/init/error-cases/output.txt b/acceptance/pipelines/init/error-cases/output.txt index f3394fb34ea..fc266593a14 100644 --- a/acceptance/pipelines/init/error-cases/output.txt +++ b/acceptance/pipelines/init/error-cases/output.txt @@ -1,12 +1,8 @@ === Test with invalid project name (contains uppercase letters) ->>> errcode [CLI] pipelines init --config-file ./invalid_input.json --output-dir invalid-output +>>> musterr [CLI] pipelines init --config-file ./invalid_input.json --output-dir invalid-output Error: failed to load config from file ./invalid_input.json: invalid value for project_name: "InvalidProjectName". Name must consist of lower case letters, numbers, and underscores. -Exit code: 1 - === Test with non-existent config file ->>> errcode [CLI] pipelines init --config-file ./nonexistent.json --output-dir invalid-output-2 +>>> musterr [CLI] pipelines init --config-file ./nonexistent.json --output-dir invalid-output-2 Error: failed to load config from file ./nonexistent.json: open ./nonexistent.json: no such file or directory - -Exit code: 1 diff --git a/acceptance/pipelines/init/error-cases/script b/acceptance/pipelines/init/error-cases/script index 4baddca6799..25fba63fffe 100644 --- a/acceptance/pipelines/init/error-cases/script +++ b/acceptance/pipelines/init/error-cases/script @@ -1,5 +1,5 @@ title "Test with invalid project name (contains uppercase letters)" -trace errcode $CLI pipelines init --config-file ./invalid_input.json --output-dir invalid-output +trace musterr $CLI pipelines init --config-file ./invalid_input.json --output-dir invalid-output title "Test with non-existent config file" -trace errcode $CLI pipelines init --config-file ./nonexistent.json --output-dir invalid-output-2 +trace musterr $CLI pipelines init --config-file ./nonexistent.json --output-dir invalid-output-2 diff --git a/acceptance/workspace/repos/get_errors/output.txt b/acceptance/workspace/repos/get_errors/output.txt index 33084a05ce4..5eda2b1b643 100644 --- a/acceptance/workspace/repos/get_errors/output.txt +++ b/acceptance/workspace/repos/get_errors/output.txt @@ -2,9 +2,5 @@ >>> [CLI] repos get /Repos/me@databricks.com/doesnotexist -o json Error: failed to look up repo by path: Workspace path not found -Exit code: 1 - >>> [CLI] repos get /not-a-repo -o json Error: object at path "/not-a-repo" is not a repo - -Exit code: 1 diff --git a/acceptance/workspace/repos/get_errors/script b/acceptance/workspace/repos/get_errors/script index a2a0b525e8e..b813fe7b4d4 100644 --- a/acceptance/workspace/repos/get_errors/script +++ b/acceptance/workspace/repos/get_errors/script @@ -1,4 +1,4 @@ -errcode trace $CLI repos get /Repos/me@databricks.com/doesnotexist -o json +musterr trace $CLI repos get /Repos/me@databricks.com/doesnotexist -o json $CLI workspace mkdirs /not-a-repo -errcode trace $CLI repos get /not-a-repo -o json +musterr trace $CLI repos get /not-a-repo -o json From b3a25b7880c2f5f19ff83875f369168ffa56b4e8 Mon Sep 17 00:00:00 2001 From: Jan Rose Date: Wed, 17 Jun 2026 15:31:15 +0200 Subject: [PATCH 4/4] acceptance: keep jq in postgres_projects/recreate (Windows fix) The bundle is named deploy-postgres-recreate, so its state-file paths (.bundle/deploy-postgres-recreate/.../state/*.json) contain the substring "postgres". The backfill converted the request filter to print_requests.py //postgres, but on Windows Git-Bash/MSYS strips the leading slash the helper relies on, degrading the filter to a bare "postgres" that also matches those workspace-files writes. The direct variant then diverged from the committed golden (which was generated on macOS, where the filter stays /postgres). Only this test collides because its bundle name contains the filter keyword. Revert this wrapper to jq: contains("/postgres") matches only the API paths and, being inside the jq program rather than an argv path, isn't subject to MSYS mangling. Consistent with the sibling postgres_branches/recreate and postgres_endpoints/recreate tests. The recorded output reverts to its pre-backfill form. Co-authored-by: Isaac --- .../postgres_projects/recreate/out.requests.create.txt | 10 +++++----- .../postgres_projects/recreate/out.requests.update.txt | 10 +++++----- .../bundle/resources/postgres_projects/recreate/script | 8 +++++++- 3 files changed, 17 insertions(+), 11 deletions(-) diff --git a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt index 4ee3402fd27..e0bb397af98 100644 --- a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt +++ b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.create.txt @@ -1,9 +1,4 @@ { - "method": "POST", - "path": "/api/2.0/postgres/projects", - "q": { - "project_id": "test-pg-old-[UNIQUE_NAME]" - }, "body": { "spec": { "default_endpoint_settings": { @@ -15,5 +10,10 @@ "history_retention_duration": "604800s", "pg_version": 16 } + }, + "method": "POST", + "path": "/api/2.0/postgres/projects", + "q": { + "project_id": "test-pg-old-[UNIQUE_NAME]" } } diff --git a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt index fc8b1d408cf..a15cfde30f8 100644 --- a/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt +++ b/acceptance/bundle/resources/postgres_projects/recreate/out.requests.update.txt @@ -3,11 +3,6 @@ "path": "/api/2.0/postgres/[MY_PROJECT_ID]" } { - "method": "POST", - "path": "/api/2.0/postgres/projects", - "q": { - "project_id": "test-pg-new-[UNIQUE_NAME]" - }, "body": { "spec": { "default_endpoint_settings": { @@ -19,5 +14,10 @@ "history_retention_duration": "604800s", "pg_version": 16 } + }, + "method": "POST", + "path": "/api/2.0/postgres/projects", + "q": { + "project_id": "test-pg-new-[UNIQUE_NAME]" } } diff --git a/acceptance/bundle/resources/postgres_projects/recreate/script b/acceptance/bundle/resources/postgres_projects/recreate/script index 0c23dae366f..1f9ca852cf5 100644 --- a/acceptance/bundle/resources/postgres_projects/recreate/script +++ b/acceptance/bundle/resources/postgres_projects/recreate/script @@ -23,7 +23,13 @@ project_id_1=`read_id.py my_project` print_requests() { local name=$1 - print_requests.py //postgres > out.requests.${name}.txt + # Keep jq here, not print_requests.py: the bundle name contains "postgres", + # so the state-file paths under .bundle/deploy-postgres-recreate/ also match + # a bare "postgres" filter. jq's contains("/postgres") matches only the API + # paths, and unlike print_requests.py's //postgres arg it isn't subject to + # Git-Bash/MSYS slash mangling on Windows. Matches the sibling recreate tests. + jq --sort-keys 'select(.method != "GET" and (.path | contains("/postgres")))' < out.requests.txt > out.requests.${name}.txt + rm -f out.requests.txt } print_requests create