fix(sqlmesh): resolve DuckDB connection-config conflict blocking state init#32
Merged
Conversation
Two changes that landed together because they were found together: 1. analytics.py: drop `from __future__ import annotations` and import `AssetExecutionContext` directly instead of via `dg.AssetExecutionContext`. Dagster's op-definition validator does an identity check against the imported class; PEP 563 turns annotations into strings and the `dg.X` attribute form doesn't compare equal. Result: `defs` wouldn't load at all since the prior strict-mypy commit introduced the typed `context` parameter. 2. Taskfile.yaml: `dagster asset materialize -f <path>` → `-m databox.orchestration.definitions`. `dagster dev` drops `-f` entirely — it auto-discovers workspace.yaml at the repo root. workspace.yaml itself switches from `python_file` to `python_module` so both entry points target the same import path. Validated with `dagster definitions validate` against both `-m` and `-w workspace.yaml`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
37 lines of 1Password source code do not justify a top-level directory. Inlined into docs/secrets.md alongside the existing wiring example — same "copy-adapt" intent, fewer directories. CLAUDE.md reference dropped. Reconciled closed loom ticket secrets-pluggable with a follow-up note so the graph stays truthful — the file was present at close time but has since been removed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…e init
Two-part fix for the md: URL connection-config conflict that was crashing
`sqlmesh_project` on first run under both backends:
1. `config/settings.py::sqlmesh_config()` now registers only the gateway
matching `settings.backend`. SQLMesh's `Context.engine_adapters` eagerly
builds an EngineAdapter for every gateway in `Config.gateways`, so
registering both made `DATABOX_BACKEND=local` still open a MotherDuck
connection. A dedicated `state_connection` at `data/sqlmesh_state.duckdb`
is retained on both paths.
2. `orchestration/_factories.py::ensure_motherduck_databases()` now opens
`duckdb.connect(database=..., config={"custom_user_agent": f"SQLMesh/{__version__}"})`
so the md: URL config matches SQLMesh's later open. DuckDB caches a
process-global handle per `md:?motherduck_token=...` URL and rejects
subsequent opens with mismatched config dicts.
Verified on a clean `data/` dir: `task verify` ends with RUN_SUCCESS under
both `DATABOX_BACKEND=local` and `DATABOX_BACKEND=motherduck`. Ruff + mypy
clean. See `.loom/evidence/20260423-sqlmesh-state-conn-conflict-fix.md`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes
ticket:sqlmesh-state-conn-conflict. Thesqlmesh_projectasset was crashing on first run with "Can't open a connection to same database file with a different configuration than existing connections" before any model materialized. Two-part fix inpackages/databox/:config/settings.py::sqlmesh_config()— register only the gateway matchingsettings.backend. SQLMesh'sContext.engine_adapterseagerly builds anEngineAdapterfor every gateway inConfig.gateways, so registering both madeDATABOX_BACKEND=localstill open a MotherDuck connection at snapshot-evaluation time. Dedicatedstate_connectionatdata/sqlmesh_state.duckdbis retained on both paths.orchestration/_factories.py::ensure_motherduck_databases()— openduckdb.connect(database=..., config={"custom_user_agent": f"SQLMesh/{__version__}"})so the md: URL config matches SQLMesh's later open. DuckDB caches a process-global handle permd:?motherduck_token=...URL and rejects subsequent opens with mismatched config dicts.Also sweeps in two pre-existing local-main commits:
fix(orchestration): restore Dagster loading + switch to -m entry points(surfaced this bug) andchore: remove examples/ top-level dir.Test plan
task verify(smoke) ends withRUN_SUCCESSunderDATABOX_BACKEND=localon a cleandata/dirtask verify(smoke) ends withRUN_SUCCESSunderDATABOX_BACKEND=motherduckon a cleandata/diruv run ruff check packages/ transforms/cleanuv run mypy packages/— 50 source files, no issuestests/test_motherduck_autocreate.pyupdated for new kwargs-basedduckdb.connectcall signatureEvidence:
.loom/evidence/20260423-sqlmesh-state-conn-conflict-fix.md.🤖 Generated with Claude Code