feat(dbt): add keboola_snowflake profile target and keboola adapter env vars

[Matovidlo]

Release Notes

• dbt generate profile and dbt init now generate two outputs in profiles.yml: {target_name} (direct Snowflake access, default) and keboola_{target_name} (Keboola keboola_snowflake adapter via Query Service)
• dbt generate env and dbt init now write DBT_KBC_*_BASE_URL, _BRANCH_ID, _WORKSPACE_ID, and KEBOOLA_TOKEN to .env.local for use with the keboola_snowflake adapter target (only for SQL/editor-session workspaces)
• Private key is now written to a separate .dbt_private_key_<target>.p8 file (mode 0600) referenced via DBT_KBC_*_PRIVATE_KEY_PATH; .env.local and the key file are both added to .gitignore automatically
• Introduces WorkspaceDetails struct in the dbt/generate/env package, replacing the dependency on *sandbox.SandboxWorkspace
• Breaking change: dbt init removes the --key-pair flag. Key-pair auth is now always used (password auth removed); Snowflake deprecated password auth for service accounts.

Plans for customer communication

None.

Impact analysis

• pkg/lib/operation/dbt/generate/profile/operation.go — new keboola_{target_name} output added to profiles.yml; existing {target_name} output is unchanged; private_key → private_key_path (⚠ profile format change)
• pkg/lib/operation/dbt/generate/env/operation.go — Options.Workspace type changed from *sandbox.SandboxWorkspace to new WorkspaceDetails struct; callers updated accordingly; .env.local gains keboola adapter vars and KEBOOLA_TOKEN when workspace has an editor session; private key written to .p8 file instead of inline
• pkg/lib/operation/dbt/init/operation.go — new BaseURL field on DbtInitOptions; --key-pair flag removed (key-pair always used)
• internal/pkg/keboola/sandbox bridge package — not touched; workspace list/create/delete operations unchanged
• ⚠ Breaking change: --key-pair flag removed from dbt init

Change type

Feature — Enables the keboola_snowflake dbt adapter target alongside the existing direct-Snowflake profile

Justification

The keboola_snowflake dbt adapter routes queries through the Keboola Query Service instead of connecting directly to Snowflake. To use it, dbt needs base_url, branch_id, workspace_id, and KEBOOLA_TOKEN env vars alongside the existing connection vars. This PR adds those vars to .env.local and a second profile target to profiles.yml so users can switch adapters with dbt run --target keboola_{target_name} without any manual config changes.

The --key-pair flag is removed because Snowflake has deprecated password authentication for service accounts; key-pair auth is always used now.

Extracted from PR #2562 — only the dbt profile/env generation changes, without the workspace/data-science refactoring.

Deployment

Merge & automatic deploy.

Rollback plan

Revert of this PR.

Post release support plan

None.

[Matovidlo]

@claude review

[claude]

Review summary

Overall the approach is clean — introducing WorkspaceDetails to decouple the operation layer from *sandbox.SandboxWorkspace is a good call, and the two-phase comment block in init/operation.go makes the workflow much easier to follow.

A few issues worth addressing before merge:

Blocking

keboola_snowflake profile target causes dbt startup failures when env vars are absent

profiles.yml now always includes a keboola_{target} output block with bare env_var() references (no defaults). dbt evaluates all env_var() calls at startup regardless of the active target, so any environment that lacks DBT_KBC_*_BASE_URL, _BRANCH_ID, _WORKSPACE_ID (Python/R workspaces, CI environments that source .env.local selectively, etc.) will abort on dbt run. See inline comment on profile/operation.go:120-133.

keboola_snowflake target written for all workspace types

BigQuery (and any future non-Snowflake) workspace will get a keboola_snowflake output in profiles.yml. The adapter name makes it non-functional there. See inline on profile/operation.go:113.

Non-blocking but should fix

baseURLFromHost duplicated

Identical function defined in both dbtinit/cmd.go and generate/env/cmd.go. Should live in one place. See inline on dbtinit/cmd.go:81.

deref closure duplicated

Same inline closure in dialog.go and init/operation.go. Minor but worth consolidating.

Observation

The baseURLFromHost logic (strip connection., prepend query.) produces a wrong URL for hosts that do not start with connection. (e.g. staging or on-premise stacks). This may be intentional if all Keboola stacks follow the connection.* convention, but it should be covered by a unit test to document and lock in the expected inputs/outputs.

[Copilot]

Pull request overview

Adds support for generating a second dbt profile target for the keboola_snowflake adapter (Query Service) alongside the existing direct Snowflake target, and extends .env.local generation to emit the Query Service adapter environment variables (base URL, branch ID, workspace ID) for editor-session (SQL) workspaces.

Changes:

• dbt generate profile / dbt init now generate an additional keboola_{target_name} output in profiles.yml for the keboola_snowflake adapter.
• dbt generate env / dbt init now populate DBT_KBC_{TARGET}_BASE_URL, DBT_KBC_{TARGET}_BRANCH_ID, and DBT_KBC_{TARGET}_WORKSPACE_ID in .env.local when editor-session coordinates are available.
• Introduces a new WorkspaceDetails struct for env generation, removing the dependency on *sandbox.SandboxWorkspace in the dbt env/profile flow.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
pkg/lib/operation/dbt/init/operation.go	Builds combined workspace details (editor session coords + storage creds) and passes them into profile/env generation.
pkg/lib/operation/dbt/generate/profile/operation.go	Adds the keboola_{target} profile output using keboola_snowflake and env-var based configuration.
pkg/lib/operation/dbt/generate/env/operation.go	Switches to WorkspaceDetails and writes additional Query Service adapter env vars when available.
internal/pkg/service/cli/cmd/dbt/generate/env/dialog.go	Updates CLI option gathering to populate WorkspaceDetails for both Python/R and SQL workspaces.
internal/pkg/service/cli/cmd/dbt/generate/env/cmd.go	Derives and injects Query Service base URL into generated env options.
internal/pkg/service/cli/cmd/dbt/dbtinit/cmd.go	Derives and injects Query Service base URL into dbt init options.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[claude]

Review

Good overall structure — the two-target design (direct Snowflake + keboola_snowflake adapter), private key extraction to a .p8 file, and automatic .gitignore entries are all solid improvements.

Issues found

Bug – non-deterministic linked-bucket log message (pkg/lib/operation/dbt/generate/env/operation.go, lines 185-188)

The linkedBucketEnvsMap keys are iterated in map order before being joined into the log message. This produces a different message each run, violating the codebase's "Deterministic output" rule (CLAUDE.md). These lines predate this PR but the PR reshapes the surrounding code — worth fixing here.

var linkedBucketEnvs []string
for envName := range linkedBucketEnvsMap {
    linkedBucketEnvs = append(linkedBucketEnvs, envName)
}
sort.Strings(linkedBucketEnvs) // add this
l.Infof(ctx, "  have been generated: \"%s\"", strings.Join(linkedBucketEnvs, `", "`))

---

See also the three inline comments for:

• database/schema/warehouse missing empty-string defaults in the keboola_snowflake profile target (would cause dbt startup failures when direct-Snowflake vars are absent)
• strings.Replace → strings.ReplaceAll
• Table-driven test missing t.Run sub-tests

[Copilot]

Pull request overview

Copilot reviewed 17 out of 35 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 39 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[claude]

Review: feat(dbt) keboola_snowflake profile target + keboola adapter env vars

Overall the change is well-structured and the motivations are clear. A few issues to address.

Non-deterministic log output (pre-existing, worth fixing here)

In pkg/lib/operation/dbt/generate/env/operation.go lines 185–188, the linked-bucket env var names are collected from a map and joined without sorting. CLAUDE.md requires sorting map iterations before output. Add sort.Strings(linkedBucketEnvs) before the l.Infof call.

Inline comments posted

1. dbtutil/util.go:21 — Doc comment references a --base-url override flag that doesn't exist on either command. Fix or remove the sentence.
2. operation.go:6 — "os" imported for O_* constants; check whether the filesystem package has a higher-level write-with-mode API to avoid this.
3. operation.go:119–129 — File handle not managed with defer; current pattern leaks on panic. Suggested alternative collapses both error paths cleanly.
4. operation.go:137–144 — KEBOOLA_TOKEN is an unscoped global key; if a user manages multiple targets or stacks in the same project, repeated dbt generate env runs will silently overwrite it. Consider target-scoping it or at minimum documenting the intentional collision behaviour.
5. dialog.go:87–112 — Identical nil-unpacking block duplicated in init/operation.go:80–95. Two-site reuse warrants a small conversion helper.
6. profile/operation.go:128–139 — database/schema/warehouse in the keboola_ target lack default values while all other keboola_-specific vars have "" defaults; add a comment explaining the intentional asymmetry (they are shared with the direct-Snowflake target so are always populated when .env.local is sourced).

Minor

linkedBucketEnvsMap is maintained in parallel with envVars solely to drive the final warning message. Collecting just the names ([]string) as entries are added to envVars removes the duplicate map.

[jachym-tousek-keboola]

No issues with the code but this really isn't my area of expertise. 😅