DEV-1643: Import OSI (Open Semantic Interchange) configs into SLayer

CLAUDE.md

@@ -77,7 +77,7 @@ poetry run ruff check slayer/ tests/
 - **Window functions in filters**: filter strings and `ModelMeasure.formula` cannot contain raw `OVER (...)` SQL — SLayer's formula parser is Python-AST-based and rejects with an actionable error pointing at the `rank()` / `first()` / `last()` / `lag()` / `lead()` transforms. Filtering on a `Column` whose `sql` contains a window function is also rejected (DEV-1369; the prior auto-promotion escape hatch from DEV-1336 is removed). For top-N use the inline `rank(<measure>) <= N` transform (or `dense_rank` / `percent_rank` / `ntile(n=<N>)`); for non-standard window expressions, factor them into an earlier stage of a multi-stage `source_queries` model.
 - **Bare same-model derived refs in `Column.sql`** (DEV-1410): A bare identifier inside a `Column.sql` that names a sibling **derived** column on the host model is inlined parenthesised, identical to the qualified form (`A.foo` and bare `foo` produce the same SQL). Inlining is scope-guarded — refs inside a nested scope (sub-query, set-op branch, CTE, `VALUES`) are left alone because they belong to the inner rowset. Cycles in the derived-ref graph (`c1.sql = "c2 + 1"`, `c2.sql = "c1 - 1"`) raise `slayer.core.errors.ColumnCycleError` at `storage.save_model` time, before the model reaches a query. `ColumnCycleError` subclasses both `SlayerError` and `ValueError` so existing `except ValueError` call sites keep working. `StorageBackend.save_model` is a template method that runs the cycle validator before delegating to the backend's `_save_model_impl`; the migration write-back at `_migrate_and_refine_on_load` passes `_validate=False` so legacy cyclic data remains loadable. Save-time validation stays within the model's `data_source`; unresolved join targets are silently skipped (best-effort) — the compile-time guard in `slayer/engine/column_expansion.py` remains authoritative.
 - Filters support `{variable}` placeholders substituted from `query.variables: Dict[str, Any]`. Values must be str/number, inserted as-is. `{{`/`}}` for literal braces. Undefined variables raise errors.
-- Models can have explicit `joins` to other models (LEFT JOINs). Cross-model measures use dotted syntax with colon aggregation (`customers.revenue:sum`) and multi-hop (`customers.regions.name`). Joins are auto-resolved by walking the join graph. Transforms work on cross-model measures (`cumsum(customers.revenue:sum)`)
+- Models can have explicit `joins` to other models (LEFT JOINs). Cross-model measures use dotted syntax with colon aggregation (`customers.revenue:sum`) and multi-hop (`customers.regions.name`). Joins are auto-resolved by walking the join graph. Transforms work on cross-model measures (`cumsum(customers.revenue:sum)`). `ModelJoin` also carries optional `description` and `meta` fields (DEV-1643, used to carry OSI relationship `ai_context` on import); they are purely additive/optional, so **no** `SlayerModel` version bump — old v7 join data validates unchanged.
 - **Path-based table aliases**: Joined tables use `__`-delimited path aliases in SQL to disambiguate diamond joins. In queries, dots denote paths (`customers.regions.name`); in model SQL definitions, `__` denotes the table alias (`customers__regions.name`). For diamond joins (same table reached via different paths, e.g., `orders → customers → regions` AND `orders → warehouses → regions`), each path gets a unique alias (`customers__regions` vs `warehouses__regions`). Auto-ingestion creates only direct joins (one per FK on the source table); multi-hop paths are resolved at query time by walking each intermediate model's own joins
 - `SlayerQuery.source_model` accepts a model name, inline `SlayerModel`, or `ModelExtension` (extends a model with extra `columns`/`measures` formulas/`joins`). `create_model_from_query()` saves a query as a permanent model
 - Models can have `filters` (always-applied WHERE conditions, e.g., `"deleted_at IS NULL"`)
@@ -181,6 +181,7 @@ poetry run ruff check slayer/ tests/
 - `slayer serve --ingest-on-startup` and `slayer mcp --ingest-on-startup` (DEV-1392) — opt-in boot-time idempotent auto-ingestion across every configured datasource, sync-before-listen (uvicorn/mcp.run don't start until ingest finishes). Continue-on-failure: per-datasource errors are friendly-formatted to stderr and never abort startup; `storage.list_datasources()` raising is the only thing that prevents the server from starting. `to_delete` drift entries are printed but **never auto-applied** — destructive cleanup stays gated behind `slayer validate-models --force-clean [--yes]`. Composes freely with `--demo` (demo first, then the ingest pass over every datasource including the freshly-created demo). Also exposed via `SLAYER_INGEST_ON_STARTUP=1` env var (flag wins when both set) and the `ingest_on_startup=True` kwarg on `create_app` / `create_mcp_server`. All output goes to stderr — `slayer mcp` stdio JSON-RPC remains protocol-safe. Orchestrator: `slayer/engine/ingestion.py::ingest_all_datasources_idempotent`. **Memory embeddings** (DEV-1416): each per-datasource pass also re-embeds every memory whose canonical entities are rooted at the datasource, so a stale `embeddings.db` is repaired by the next `--ingest-on-startup` without extra steps. See [docs/concepts/ingestion.md](docs/concepts/ingestion.md#ingesting-at-startup).
 - `slayer validate-models [--datasource X] [--force-clean] [--yes]` (DEV-1356) — read-only diff against live schemas; with `--force-clean`, prompts to apply each delete via `engine.apply_drift_deletes`. See [docs/concepts/schema-drift.md](docs/concepts/schema-drift.md).
 - `slayer recommend-root-model ITEM... [--data-source X] [--root-hint M] [--format json|text]` (DEV-1626) — given `model.column` / `model.metric` items, introspect the join graph and recommend the query `source_model` (root) plus each item's join-qualified path from it. Core is `engine.recommend_root_model(items, *, data_source=None, root_hint=None) -> RootModelRecommendation` (+ `_sync`), delegating to the pure `slayer/engine/join_graph.py::JoinGraph` primitive (LEFT = directed edge, INNER = undirected via storage symmetry; `reachable_from` + lexicographic-tie-broken `shortest_path`) and reusing `resolve_entity` for input normalization + `_all_models_in_datasource` for candidate loading. A valid root reaches every mentioned owning model; selection minimizes total hops over **distinct** owning models, then prefers a mentioned model, then the lexicographically smallest name. Items must be columns or metrics (aggregations/bare-model/datasource refs raise `ValueError`; unresolvable → `EntityResolutionError`; cross-datasource mix → `ValueError`). Aggregation suffixes are preserved verbatim (the engine resolves agg kwargs relative to the aggregated column's owning model, invariant under root choice). **`root_hint`** (optional, `_resolve_root_hint`) names the intended root — a bare model name or `<data_source>.<model>` within the resolved datasource; resolved **after** the datasource is fixed from items, so it can't pick the datasource. When the hint is a **feasible** root (reaches every item) it is honored outright, overriding the min-hops pick — this lets a caller force a **bridge** model that owns none of the items but matches the intended grain (the descriptions-first workflow). When **infeasible** the auto-pick is used and a warning names the rejected hint (echoed verbatim as typed), the unreachable owning model(s), and the fallback root; honored-vs-fallback is conveyed via `message`/`warnings` only (**no structured result field**). A non-existent / wrong-kind (bare column) / 3-part / cross-datasource hint raises `ValueError`; an empty/whitespace hint is a no-op. When no single model reaches everything, returns a structured `reachable=False` result whose `coverage` is the **Pareto frontier** of partial roots (dominance: strictly-larger reach set, or equal reach set with no longer and ≥1 shorter path), **plus the `root_hint`'s own row when given** (force-included even if dominated / zero-reach, never duplicated) — `_build_recommend_coverage(..., force_include=...)`. `_expand_join_graph` (schema-drift error path) delegates its directed reachability to `JoinGraph.reachable_from` (one reachability impl). Result models in `slayer/core/recommend.py` (`RootModelRecommendation`/`ItemPath`/`CandidateCoverage`). Surfaced on MCP `recommend_root_model`, REST `POST /recommend-root-model` (400 on `ValueError`/`SlayerError`), CLI, and `SlayerClient.recommend_root_model(_sync)`. See [docs/concepts/queries.md](docs/concepts/queries.md#choosing-a-root-model).
+- `slayer import-osi <path> --datasource X [--dialect ANSI_SQL] [--storage ...]` (DEV-1643) — import OSI (Open Semantic Interchange) configs (YAML/JSON, file or dir) into SLayer models, mirroring `import-dbt`'s CLI-only shape (parse → convert → `storage.save_model` per model → printed report). OSI maps DIRECTLY to SLayer (not via the dbt intermediate — SLayer's dbt converter clean-fails measure-less metrics and infers joins from entity names, both structural mismatches). Package `slayer/osi/` mirrors `slayer/dbt/`: `models.py` (Pydantic port of the OSI schema; versions 1.0/0.1.0/0.1.1/0.2.0.dev0 are structurally identical, accepted; unknown → warn+parse), `source.py` (`parse_source` splits `[cat.]db.schema.table`; `resolve_datasource(database, default)` is the stubbed per-database routing hook, drops db for now), `expression.py` (`convert_expression` — sqlglot-AST → SLayer colon formula: agg leaves, arithmetic+constants, `SCALAR_PASSTHROUGH` funcs, non-bare operands materialized as hidden derived columns, percentile → `col:percentile(p=…)`/`median`; validates via `parse_formula`; everything else clean-fails), `converter.py` (`OsiToSlayerConverter`). Live introspection (`introspect_table_to_model`) provides real column types/PK; OSI overlays label/description/is_time/`ai_context`→description+`meta['osi_ai_context']`/`custom_extensions`→`meta`/`unique_keys`→`meta`. Relationships → LEFT `ModelJoin` (composite-safe, length-mismatch clean-fails). Metrics → `ModelMeasure` on an anchor model chosen via the shared `slayer/engine/join_graph.py::min_hops_root` (also now used by `recommend_root_model`); cross-dataset refs emit anchor-relative dotted paths; a column-less `COUNT(*)` anchors on the semantic model's **unique fact table** (the one dataset never used as a join target) — 0-or-many candidates → clean-failed as an orphan (ambiguous grain), never guessed. The conversion-report types (`ConversionResult`/`ConversionWarning`) were extracted to the neutral `slayer/ingest_report.py` and re-exported from `slayer/dbt/converter.py`. Query-source (`sql`-mode) datasets are live-introspected too via `slayer/sql/client.py::_get_column_types_sync` (the same LIMIT-0 / cursor-metadata probe used elsewhere), driven by the datasource type. CLI-only — no MCP/REST. See [docs/osi/osi_import.md](docs/osi/osi_import.md).
 - `slayer storage migrate-types [--data-source X] [--dry-run]` (DEV-1361) — refine `DOUBLE → INT` on base columns whose live SQL type is integer for every persisted model, then write the refined v5 dict back. Hard-fails if a datasource is unreachable. The same refinement runs transparently inside `storage.get_model` on first load; this CLI is a batch / inspectable alternative.
 - `slayer search [--entity ENT ...] [--query JSON_OR_@FILE] [--question TEXT] [--datasource DS] [--cypher-filter CYPHER] [--max-results N] [--format json|text] [--verbose]` (DEV-1375 / DEV-1386 / DEV-1409 / DEV-1464 / DEV-1532 / DEV-1549) — up to three-channel semantic search over memories + canonical entities (BM25 over memory entity tags + tantivy full-text + optional dense embedding similarity). Returns a single flat ranked `results` list. `--datasource` scopes the corpus to one datasource. `--cypher-filter` pre-filters all channels: full openCypher when `advanced_search` is installed; simple `MATCH (n:Label) RETURN n.id AS id` kind-filter without it. **DEV-1549**: compact-by-default. Each `SearchHit` carries a `description` field; under the default `compact=True` (everywhere — MCP / REST `compact: bool = True` / CLI / `SlayerClient`) memory hits surface `Memory.description` (or the first non-empty paragraph of `learning`, capped at 500 chars) and entity hits surface `entity.description`, with `hit.text == ""`. CLI flips compact off via `--verbose`; REST/MCP/SlayerClient via `compact=False`. Drill-in pattern: `search(entities=["memory:<id>"], max_results=1, compact=False)`. See [docs/concepts/search.md](docs/concepts/search.md).
 - `slayer search refresh-samples [--data-source X] [--model M ...]` (DEV-1375) — re-profile and persist `Column.sampled` for table-backed models. Best-effort: per-column failures are reported but don't abort.

docs/interfaces/cli.md

@@ -124,6 +124,21 @@ slayer import-dbt ./my_dbt_project --datasource my_postgres --include-hidden-mod
 | `dbt_project_path` | Yes | Path to the dbt project root (or a models directory) |
 | `--datasource` | Yes | SLayer datasource name for the imported models |
 | `--include-hidden-models` | No | Also import regular dbt models (those not wrapped by a `semantic_model`) as hidden SLayer models via SQL introspection. Requires the `dbt` extra. |
+
+### `slayer import-osi`
+
+Import OSI (Open Semantic Interchange) configs into SLayer. See [Importing OSI configs](../osi/osi_import.md).
+
+```bash
+slayer import-osi ./osi_configs --datasource my_postgres
+slayer import-osi ./model.yaml --datasource my_postgres --dialect SNOWFLAKE
+```
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `osi_path` | Yes | Path to an OSI file or directory (`.yaml`/`.yml`/`.json`) |
+| `--datasource` | Yes | SLayer datasource name (must be reachable — types come from live introspection) |
+| `--dialect` | No | OSI expression dialect to read (default `ANSI_SQL`); falls back to another SQL dialect when absent |
 | `--storage` | No | Storage path |
 
 ### `slayer models`

docs/osi/osi_import.md

@@ -0,0 +1,50 @@
+# Importing OSI (Open Semantic Interchange) Configs
+
+SLayer can import [Open Semantic Interchange](https://github.com/open-semantic-interchange/OSI) (OSI) semantic-model configs and convert them into SLayer models. OSI is a vendor-neutral YAML/JSON standard for semantic models, datasets, relationships, and metrics.
+
+## Quick Start
+
+```bash
+slayer import-osi ./osi_configs --datasource my_postgres --storage ./slayer_data
+```
+
+This reads every `.yaml`/`.yml`/`.json` file in the path (a single file or a directory), converts each OSI dataset into a SLayer model, and saves the models to storage. A **reachable datasource is required** — column data types come from live table introspection (OSI carries no column types), so the importer connects to `--datasource` and overlays OSI's semantic metadata on top.
+
+Spec versions `1.0`, `0.1.0`, `0.1.1`, and `0.2.0.dev0` are all accepted (they are structurally identical); an unknown version is warned about but still parsed.
+
+## What Gets Converted
+
+| OSI construct | SLayer target |
+|---|---|
+| `semantic_model[].datasets[]` | one `SlayerModel` each (name = dataset name) |
+| dataset `source` (`db.schema.table`) | `sql_table` + `schema`; a query source → `sql` mode |
+| dataset `fields[]` | `Column`s (real types from introspection) |
+| field `expression` (bare) | overlaid onto the introspected column |
+| field `expression` (derived, e.g. `UPPER(x)`) | a derived `Column` with `sql` set |
+| field `dimension.is_time` | column typed temporal; sets `default_time_dimension` |
+| dataset `primary_key` | `Column.primary_key = true` |
+| `relationships[]` (`from` → `to`) | a LEFT `ModelJoin` on the `from` model |
+| `metrics[]` (raw SQL aggregation) | a `ModelMeasure` formula |
+| `ai_context` (instructions + synonyms) | entity `description` + `meta["osi_ai_context"]` |
+| `unique_keys` / `custom_extensions` | model/column `meta` |
+
+### Metrics
+
+An OSI metric holds a raw SQL aggregation expression. SLayer parses it into colon-syntax formulas:
+
+- `SUM(amount)` → `amount:sum`, `COUNT(*)` → `*:count`, `COUNT(DISTINCT id)` → `id:count_distinct`
+- arithmetic + constants + scalar functions pass through: `SUM(a) / NULLIF(COUNT(*), 0)` → `a:sum / nullif(*:count, 0)`
+- a non-bare aggregate operand is materialized as a hidden derived column: `SUM(quantity * amount)` → a hidden column `quantity * amount` plus `<col>:sum`
+- `PERCENTILE_CONT(0.9) WITHIN GROUP (ORDER BY x)` → `x:percentile(p=0.9)` (`0.5` → `x:median`)
+
+A metric that references columns from more than one dataset is attached to an **anchor** model — the model that reaches every referenced dataset over the relationship-derived joins (chosen via the same logic as [`recommend_root_model`](../concepts/queries.md#choosing-a-root-model)). Cross-dataset columns are emitted as join-qualified dotted refs (`customers.regions.population:sum`).
+
+## Dialect Selection
+
+OSI expressions are multi-dialect. `--dialect` (default `ANSI_SQL`) picks which one to read; when it is absent, the importer falls back to another SQL-compatible dialect (`SNOWFLAKE`, `DATABRICKS`). An expression available only in a non-SQL dialect (`MDX`, `MAQL`, `TABLEAU`) is clean-failed to the report.
+
+## Clean-Fail Report
+
+Anything that cannot be expressed exactly is reported (never silently dropped) and the raw construct is preserved in `meta`. Examples: a metric with a `CASE`/window expression, a relationship with mismatched key lengths or an unknown target, a dataset whose table cannot be introspected, an illegal name (containing `.`/`:`), a metric whose referenced datasets are not connected by any join path, or an orphan `COUNT(*)` (a column-less metric in a semantic model with no unique fact table, so its grain is ambiguous). The CLI prints a grouped report and a `models / unconverted / dropped` tally at the end.
+
+A query source is introspected live (a `LIMIT 0` / cursor-metadata probe) for real column types, the same way table sources are — no connection-less heuristics.

docs/reference/cli.md

@@ -109,6 +109,21 @@ slayer import-dbt ./my_dbt_project --datasource my_postgres --include-hidden-mod
 | `dbt_project_path` | Yes | Path to the dbt project root (or a models directory) |
 | `--datasource` | Yes | SLayer datasource name for the imported models |
 | `--include-hidden-models` | No | Also import regular dbt models (those not wrapped by a `semantic_model`) as hidden SLayer models via SQL introspection. Requires the `dbt` extra (`pip install 'motley-slayer[dbt]'`). See [dbt Import](../dbt/dbt_import.md#regular-dbt-models-hidden-import). |
+
+### `slayer import-osi`
+
+Import OSI (Open Semantic Interchange) configs into SLayer. See [Importing OSI configs](../osi/osi_import.md).
+
+```bash
+slayer import-osi ./osi_configs --datasource my_postgres
+slayer import-osi ./model.yaml --datasource my_postgres --dialect SNOWFLAKE
+```
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `osi_path` | Yes | Path to an OSI file or directory (`.yaml`/`.yml`/`.json`) |
+| `--datasource` | Yes | SLayer datasource name (must be reachable — column types come from live introspection) |
+| `--dialect` | No | OSI expression dialect to read (default `ANSI_SQL`); falls back to another SQL dialect when the requested one is absent |
 | `--storage` | No | Storage path |
 
 ### `slayer models`