Add pipelines preview-datasets command

[tanishgupta-db]

Changes

Adds a hidden databricks pipelines preview-datasets command that lists the datasets a pipeline defines. It reads the datasets from a pipeline's latest validate-only dry-run; if none exists it triggers one and waits. Flags: --force-dry-run always triggers a fresh dry-run, --no-dry-run never triggers one, --timeout bounds the wait.

The command talks directly to the pipeline entities REST endpoints (PUBLIC_UNDOCUMENTED in staging), so it is hidden.

Why

Users want to see a pipeline's resolved datasets without opening the workspace UI. Reading them requires a dry-run's computed graph, so the command wraps the trigger/poll/read flow behind one invocation.

Key decisions:

• Billed-cluster safety. A dry-run starts a cluster, so every trigger path prints a notice up front, and an interrupted or errored run best-effort stops the cluster to avoid leaks.
• A completed dry-run always wins. If a completed dry-run exists it is used even when an unrelated update is running; we only refuse to trigger a new one while another update is active.
• Canceled vs failed are reported distinctly (a canceled dry-run has no diagnostics).

Future work (not in this PR): consider a shell-completion function for the pipeline key; revisit the poll interval for very long dry-runs; add a harness for all preview (dry-run based) commands common functionality (flags, triggering, etc.); Isolate all http logic (types, calls, etc) so SDK can be switched in easily.

Tests

Unit tests for the dry-run resolution state machine (all modes/states), diagnostics formatting, and rendering; full-flow tests over an in-process fake of the pipeline API (trigger→poll→fetch, pagination, failed-dry-run diagnostics, continuous/active-update errors); and acceptance tests exercising the real CLI against the fake on both deployment engines.