Skip to content

AGENTS.md

Latest commit

 

History

History
118 lines (88 loc) · 5.41 KB

AGENTS.md

File metadata and controls

118 lines (88 loc) · 5.41 KB

Sidemantic

Universal SQL-first semantic layer in Python. Imports from Cube/LookML/dbt/Hex/Rill/Malloy, queries against DuckDB/Postgres/BigQuery/Snowflake/etc.

Usage Preference (Important)

  • Sidemantic usage is overwhelmingly CLI-first in practice (roughly 95% CLI usage).
  • Default to sidemantic CLI workflows (validate, info, query, migrator, etc.).
  • Treat Python API examples as secondary, advanced automation paths, not the default onboarding path.

Codebase Structure

Main Python package: sidemantic/

  • core/ - SemanticLayer, Model, Dimension, Metric, SemanticGraph
  • adapters/ - Format parsers (Cube, LookML, MetricFlow, Hex, Rill, Malloy, etc.)
  • db/ - Database adapters
  • sql/ - SQL generation and query rewriting
  • widget/, server/, workbench/, mcp_server.py - Optional features with lazy imports

Separate experimental implementations (not the main codebase):

  • sidemantic-rs/ - Rust rewrite (WIP)
  • sidemantic-duckdb/ - DuckDB extension wrapping sidemantic-rs

These don't share code with Python. CI runs them only when their dirs change.

Version Management

Update BOTH when releasing:

  • pyproject.toml: version = "X.Y.Z"
  • sidemantic/__init__.py: __version__ = "X.Y.Z"

CRITICAL: Before Every Commit

ALWAYS run the EXACT same commands CI runs before committing:

# Run these in order:
uv run ruff check . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run ruff format --check . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run pytest -v

If any fail, fix them:

# Fix ruff issues
uv run ruff check --fix . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run ruff format . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar

This is NON-NEGOTIABLE. You MUST run these BEFORE every commit.

PRs: Do not include test commands in PR bodies unless explicitly requested.

Why this matters:

  • CI runs these exact commands and will fail if they don't pass
  • You keep pushing broken code because you don't run these locally
  • Ruff must be in [project.optional-dependencies] dev for CI
  • NOT in [dependency-groups] (that's uv-specific, CI uses optional-dependencies)

Dependency Management

  • Use uv for all Python package management
  • Ruff should be in dev dependencies ([dependency-groups] dev)
  • DO NOT add dev tools to main dependencies unless explicitly requested
  • Optional features use [project.optional-dependencies]:
    • workbench - textual, plotext, textual-plotext (for TUI, NOT Pyodide compatible)
    • serve - mcp[cli], riffq, pyarrow (for PostgreSQL server, NOT Pyodide compatible)
    • dev - pytest, ruff, pandas, numpy (for development)

CRITICAL: Pyodide Compatibility Rules

  • Core dependencies (in main dependencies list) MUST work in Pyodide/WASM
  • Optional dependencies can use packages incompatible with Pyodide (textual, riffq, pyarrow, mcp)
  • ALL imports of optional deps MUST be lazy (inside functions, NOT at module level)
  • Test: from sidemantic import Model, Dimension, Metric must work without any optional deps

Pyodide Compatibility

Import Structure:

  • sidemantic/__init__.py - Only imports core classes (Model, Dimension, etc.)
  • sidemantic/cli.py - Imports typer (core dep) at top, workbench/server imports inside command functions
  • sidemantic/workbench/__init__.py - Lazy imports textual inside run_workbench() function
  • sidemantic/server/ - Never imported unless sidemantic serve command is run

Pyodide typing-extensions issue:

  • Pyodide has typing-extensions==4.11.0
  • Some deps (pydantic>=2.10, inflect>=7.2) require typing-extensions>=4.12+
  • Dashboard handles this by installing pydantic<2.10 with deps=False
  • inflect<7.2 constraint in core deps marked with # PYODIDE: comment

Version constraints with "PYODIDE:" comments are REQUIRED:

  • Heavy deps (textual, riffq, mcp) are optional to avoid Pyodide conflicts
  • Pyodide CI builds local wheel and installs it with deps=False
  • This ensures code changes are tested in Pyodide before publish
  • If adding new core deps, check they work in Pyodide or make them optional

If Pyodide CI fails:

  • Check if a dep version changed that requires newer typing-extensions
  • Either downgrade that dep OR add workaround in dashboard.qmd install
  • CI installs local wheel to test current code, not PyPI version

Testing

Run tests before committing significant changes:

uv run pytest -v

Publishing to PyPI

gh workflow run publish.yml

Common Mistakes to Avoid

  1. Breaking ruff - Always ensure ruff is installed in dev dependencies
  2. Pyodide compatibility - Keep heavy deps (textual, pygments) optional
  3. Not linting - Format and lint EVERY TIME before commit