docs: headline the 1.17 pairs()/align ladder; demote dataset()/research() to legacy

README.md

@@ -10,7 +10,7 @@
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 [![Docs](https://img.shields.io/badge/docs-mostlyright.md-blue.svg)](https://mostlyright.md/docs/sdk/)
 
-`mostlyright` gives you one clean interface to public data sources — **general-first**. Today it ships weather (live observations, forecasts, climate history) as source-identified domain tables, plus `dataset()`, a leakage-safe training-table builder that aligns them to any label you choose. Prediction markets (Kalshi, Polymarket) are a thin convenience layer on top. Local-first, no hosted backend, no API key.
+`mostlyright` gives you one clean interface to public data sources — **general-first**. Today it ships weather (live observations, forecasts, climate history) and economic indicators as source-identified domain tables, a per-domain `pairs()` quickstart, and `mr.align(spine, *sources)` — one leakage-guarded composition operator that as-of-joins any sources onto any label spine. Prediction markets (Kalshi, Polymarket) are a thin convenience layer on top. Local-first, no hosted backend, no API key.
 
 ---
 
@@ -30,26 +30,45 @@ Python 3.11+. Node 18+. No API key required for any package below.
 
 ```python
 # Python
-import mostlyright
+from mostlyright import weather
 
-# dataset() is the alignment engine; research() is a fully working alias.
-# Pass an explicit label= (here the NWS CLI settlement extremes).
-df = mostlyright.dataset("KNYC", "2025-01-06", "2025-01-12", label="cli")
+# pairs() is the capped quickstart (v0.14.1 heritage: client.pairs()).
+# label="cli" (the NWS CLI settlement extremes) is the documented default.
+df = weather.pairs("KNYC", "2025-01-06", "2025-01-12")
 print(df.head())
 # pandas DataFrame: one row per LST settlement date
 # Columns: date, station, cli_high_f, cli_low_f, obs_high_f, obs_low_f,
-#          obs_mean_f, obs_count, fcst_*, market_close_utc
+#          obs_mean_f, obs_count, ..., market_close_utc
+```
+
+When you outgrow the one-liner, you don't add kwargs — you graduate one rung
+down to the same call it desugars to:
+
+```python
+import mostlyright as mr
+from mostlyright import weather
+
+spine = weather.label.cli("KNYC", "2025-01-06", "2025-01-12")   # the target (y)
+frame = mr.align(spine, weather.obs("KNYC"), weather.forecasts("KNYC"))
+# align() as-of-joins each source onto the spine and enforces the leakage
+# guard (source.knowledge_time <= spine.decision_time) in exactly one place.
 ```
 
 ```ts
-// TypeScript
+// TypeScript — the TS SDK keeps research()/dataset() unchanged on 1.17
+// (align/spine/pairs are Python-only for now; parity ticket PT-3401).
 import { research } from "mostlyright";
 
 const rows = await research("KNYC", "2025-01-06", "2025-01-12");
 console.log(rows[0]);
 // ReadonlyArray<PairsRow>: same schema as Python, JSON-serializable
 ```
 
+> **Coming from `dataset()` / `research()`?** Both still work and return the
+> same bytes — they are deprecation shims now, removed at 2.0. The
+> [1.17 migration guide](docs/migration/v117-align-spine-sources.md) maps all
+> 26 old kwargs to their new homes.
+
 First call writes a parquet (Python) or JSON-envelope (Node) cache to `~/.mostlyright/cache/`. Subsequent calls in the same window are local-only — no network.
 
 Full quickstart with concepts at <https://mostlyright.md/docs/sdk/>.
@@ -60,7 +79,7 @@ Full quickstart with concepts at <https://mostlyright.md/docs/sdk/>.
 
 | Package | Description | Downloads | Status |
 |---|---|---|---|
-| [`mostlyrightmd`](https://pypi.org/project/mostlyrightmd/) | Core types, schemas, validators, the `research()` join, and snapshot primitives. Imports as `mostlyright`. | [![monthly downloads](https://badgen.net/pypi/dm/mostlyrightmd?label=monthly)](https://pypistats.org/packages/mostlyrightmd) | stable |
+| [`mostlyrightmd`](https://pypi.org/project/mostlyrightmd/) | Core types, schemas, validators, the `align()`/`spine()` composition layer, and snapshot primitives. Imports as `mostlyright`. | [![monthly downloads](https://badgen.net/pypi/dm/mostlyrightmd?label=monthly)](https://pypistats.org/packages/mostlyrightmd) | stable |
 | [`mostlyrightmd-weather`](https://pypi.org/project/mostlyrightmd-weather/) | Weather data fetchers — live METAR (AWC), ASOS archive (IEM), historical observations (GHCNh), and NWS climate text products (CLI). Direct public-API access. | [![monthly downloads](https://badgen.net/pypi/dm/mostlyrightmd-weather?label=monthly)](https://pypistats.org/packages/mostlyrightmd-weather) | stable |
 | [`mostlyrightmd-markets`](https://pypi.org/project/mostlyrightmd-markets/) | Prediction-market data — Kalshi NHIGH/NLOW weather-contract resolvers, Polymarket discovery + settlement, and Kalshi + Polymarket trade history. | [![monthly downloads](https://badgen.net/pypi/dm/mostlyrightmd-markets?label=monthly)](https://pypistats.org/packages/mostlyrightmd-markets) | stable |
 | `mostlyrightmd-edgar` | SEC filings (10-K, 10-Q, 8-K) — direct EDGAR full-text + facts access. | n/a | planned |
@@ -74,7 +93,7 @@ Full quickstart with concepts at <https://mostlyright.md/docs/sdk/>.
 | Package | Description | Downloads | Status |
 |---|---|---|---|
 | [`mostlyright`](https://www.npmjs.com/package/mostlyright) | Meta package — one `import { research } from "mostlyright"` for weather data + prediction-market settlements + the core join. | [![monthly downloads](https://img.shields.io/npm/dm/mostlyright?label=monthly)](https://www.npmjs.com/package/mostlyright) | stable |
-| [`@mostlyrightmd/core`](https://www.npmjs.com/package/@mostlyrightmd/core) | Core types, schemas, validators, temporal-safety primitives, and the `research()` join. | [![monthly downloads](https://img.shields.io/npm/dm/%40mostlyrightmd%2Fcore?label=monthly)](https://www.npmjs.com/package/@mostlyrightmd/core) | stable |
+| [`@mostlyrightmd/core`](https://www.npmjs.com/package/@mostlyrightmd/core) | Core types, schemas, validators, and temporal-safety primitives (`research()` itself is exported by the `mostlyright` meta package). | [![monthly downloads](https://img.shields.io/npm/dm/%40mostlyrightmd%2Fcore?label=monthly)](https://www.npmjs.com/package/@mostlyrightmd/core) | stable |
 | [`@mostlyrightmd/weather`](https://www.npmjs.com/package/@mostlyrightmd/weather) | Weather data fetchers — live METAR (AWC), ASOS archive (IEM), historical observations (GHCNh), and NWS climate text products (CLI). | [![monthly downloads](https://img.shields.io/npm/dm/%40mostlyrightmd%2Fweather?label=monthly)](https://www.npmjs.com/package/@mostlyrightmd/weather) | stable |
 | [`@mostlyrightmd/markets`](https://www.npmjs.com/package/@mostlyrightmd/markets) | Prediction-market data — Kalshi NHIGH/NLOW weather-contract resolvers, Polymarket discovery + settlement, and Kalshi + Polymarket trade history. | [![monthly downloads](https://img.shields.io/npm/dm/%40mostlyrightmd%2Fmarkets?label=monthly)](https://www.npmjs.com/package/@mostlyrightmd/markets) | stable |
 | `@mostlyrightmd/edgar` | SEC filings (10-K, 10-Q, 8-K) — direct EDGAR full-text + facts access. | n/a | planned |
@@ -89,7 +108,7 @@ Full quickstart with concepts at <https://mostlyright.md/docs/sdk/>.
 
 The domain tables are the raw ingredients — each row carries a durable **source
 identity** so a model trained on one provider never silently reconciles against
-another. Fetch them directly and join by hand, or feed them to `dataset()`.
+another. Fetch them directly and join by hand, or hand them to `mr.align()`.
 
 ```python
 from mostlyright.weather import obs, climate
@@ -98,40 +117,53 @@ from mostlyright.weather import obs, climate
 o = obs("KNYC", "2024-01-01", "2024-12-31", granularity="observation")
 c = climate("KNYC", "2024-01-01", "2024-12-31")   # NWS CLI settlement labels
 
-# Forecasts compose into dataset() via features=["forecasts"]; the raw NWP
-# table is mostlyright.forecasts.forecast_nwp (needs the [nwp] extra).
+# weather.forecasts(...) is the registered forecast source for align(); the
+# raw NWP table is mostlyright.forecasts.forecast_nwp (needs the [nwp] extra).
 ```
 
-### Build a leakage-safe training table with `dataset()`
+### Compose a leakage-safe training table — the `align` ladder
 
-`dataset()` is the **alignment engine** — a training-table builder, not a fixed
-recipe. Give it a station, a window, a **label axis**, and the features you want;
-it returns one leakage-safe row per LST settlement day, every column aligned to the
-settlement calendar and joined as-of so nothing from the future leaks backward.
-`research()` is a fully working alias in this release.
+Sources (features / X) and label spines (targets / y) are **orthogonal**;
+`mr.align(spine, *sources)` is the one composition operator that joins them.
+Every intermediate product is a plain, inspectable `pd.DataFrame`, and every
+rung is supported — pick how much you want the SDK to do:
 
 ```python
-import mostlyright
-
-# A general feature table — works for ANY catalog station worldwide, no market
-# required. label=None composes features only (no settlement label columns).
-df = mostlyright.dataset("EGLL", "2025-01-06", "2025-01-12", label=None)
-
-# Attach the WU/NOAA-WRH daily extremes as the label (native Celsius names).
-df = mostlyright.dataset("KNYC", "2025-01-06", "2025-01-12", label="daily_extremes")
-
-# Or bring your own label frame — the SDK aligns it to the settlement calendar.
-df = mostlyright.dataset("KNYC", "2025-01-06", "2025-01-12", label=my_label_frame)
+import mostlyright as mr
+from mostlyright import weather
+
+# Rung 3 — the per-domain quickstart one-liner (capped kwargs, never grows).
+pairs = weather.pairs("KNYC", "2025-01-01", "2025-01-31")           # label="cli" default
+
+# Rung 2 — spine + align: the SDK does ONLY the dangerous step (the as-of join).
+spine = weather.label.daily_extremes("KNYC", "2025-01-01", "2025-01-31")
+frame = mr.align(spine, weather.obs("KNYC"), weather.forecasts("KNYC"))
+
+# Rung 1 — raw source frames; build your own spine, or re-enter with mr.spine().
+raw_obs = weather.obs("KNYC", "2025-01-01", "2025-01-31")
+# BYO label frame: keep a `date` column (the ISO settlement day) — it is the
+# key weather sources attach on. mr.spine() maps YOUR column names explicitly.
+spine   = mr.spine(my_label_frame,
+                   entity="station", decision_time="cutoff", y="settle_high")
+frame   = mr.align(spine, weather.obs("KNYC"))
 ```
 
-> Calling `dataset()` **without** an explicit `label=` still returns the NWS CLI
-> settlement columns byte-identically, but emits a `FutureWarning`: the default
-> flips to `label=None` at 2.0. Pass `label="cli"` or `label=None` to silence it.
-> Full walkthrough in [`docs/dataset.md`](docs/dataset.md).
+Features-only (no settlement label)? Align sources onto the bare settlement
+calendar: `mr.align(weather.days("KSEA", d1, d2), weather.obs("KSEA"))`.
+
+> **Legacy surface.** `dataset()` / `research()` — the previous 26-kwarg
+> composer — still work and return identical bytes, as deprecation shims
+> removed at 2.0. No `FutureWarning` fires on the old default anymore in
+> Python (the planned label-default flip was cancelled; TS `dataset()` still
+> emits it pending the ladder port). The
+> [1.17 migration guide](docs/migration/v117-align-spine-sources.md) maps
+> every old kwarg to its new home; the legacy walkthrough lives in
+> [`docs/dataset.md`](docs/dataset.md).
 
 ### Train models with train/infer source parity
 
-`dataset()` (and its `research()` alias) stamps a source identity on every row.
+Every composed frame — `pairs()`, `align()`, and the legacy shims alike —
+stamps a source identity on every row.
 Pinning `source=` on `obs()` returns single-source coverage; the validator catches
 train/infer mismatches at load time, instead of silently corrupting predictions in
 production.
@@ -155,21 +187,23 @@ validate_dataframe(train, schema_id="schema.observation.v1")
 
 ### Backtest prediction markets (thin convenience layer)
 
-The prediction-market entry points are thin delegators to the venue-free
-`dataset()` — they own **all** venue knowledge (station resolution, strike parsing,
-outcome targets) and route to the **venue-correct** label: Kalshi settles on NWS CLI
+The prediction-market quickstarts are thin, venue-aware delegators — they own
+**all** venue knowledge (station resolution, strike parsing, outcome targets)
+and route to the **venue-correct** label: Kalshi settles on NWS CLI
 (`label="cli"`), Polymarket on WU/NOAA-WRH extremes (`label="daily_extremes"`).
 `outcome=True` appends a binary `label_outcome` by binarizing that venue's label
-against its parsed strike (inclusive `>=`).
+against its parsed strike (inclusive `>=`). Venue label spines
+(`markets.kalshi.label.settlement(...)`) compose with `mr.align()` like any other.
 
 ```python
 from mostlyright.markets import kalshi, polymarket
 
 # Kalshi → NWS CLI settlement. outcome=True adds the binary YES/NO target.
-k = kalshi.dataset("KXHIGHNY-25MAY26-T79", "2025-05-26", "2025-05-26", outcome=True)
+k = kalshi.pairs("KXHIGHNY-25MAY26-T79", "2025-05-26", "2025-05-26", outcome=True)
 
 # Polymarket → WU/NOAA-WRH extremes (the correct ground truth — NOT cli_*).
-p = polymarket.dataset("<event_id>", "2025-05-26", "2025-05-26", outcome=True)
+# The event id comes from Polymarket discovery (alphanumeric slug).
+p = polymarket.pairs("your-event-id", "2025-05-26", "2025-05-26", outcome=True)
 ```
 
 ### Feed AI agents structured public data
@@ -244,13 +278,14 @@ table, coverage map, auto-routing, cheap-CONUS steering, DSRF gating, the 28 TB
 - **No hosted backend.** Direct calls to public APIs (NOAA, NWS, IEM, Kalshi, Polymarket). No proxy. No vendor account. No rate-limited tier.
 - **Local-first cache.** Parquet (Python) or JSON envelope (Node) at `~/.mostlyright/cache/`. Byte-stable across runs — deterministic backtests.
 - **Schema-versioned outputs.** Every response carries a stable `schema.*.v1` URI. Train/infer source mismatches fail loudly instead of silently corrupting models.
-- **Python + TypeScript peers.** Same `research()` shape, byte-equivalent on the parity fixtures. Use whichever runtime your stack prefers.
+- **Python + TypeScript peers.** Same training-table shape (`weather.pairs()` in Python, `research()` in TS), byte-equivalent on the parity fixtures. Use whichever runtime your stack prefers.
 - **MIT licensed.** Use it commercially. Fork it. Ship it.
 
 ## Documentation
 
 - **Quickstart + concepts:** <https://mostlyright.md/docs/sdk/>
-- **`dataset()` — the alignment engine:** [`docs/dataset.md`](docs/dataset.md) — the label axis, the contributor protocol, panels, and the honest with/without comparison.
+- **The `align` / `spine` / `sources` ladder (the 1.17 surface):** [`docs/migration/v117-align-spine-sources.md`](docs/migration/v117-align-spine-sources.md) — the three rungs, per-domain `pairs()`, and the composition model.
+- **`dataset()` — the legacy composer (deprecated, removed at 2.0):** [`docs/dataset.md`](docs/dataset.md) — the label axis, the contributor protocol, panels; kept for shim users.
 - **Source identity — the cross-vertical kwarg contract:** [`docs/source-identity.md`](docs/source-identity.md) — `source=` / `delivery=` / grain, the labels-only firewall, and the fetch-provenance ledger.
 - **Migrating to 1.17.0 — the `align` / `spine` / `sources` ladder:** [`docs/migration/v117-align-spine-sources.md`](docs/migration/v117-align-spine-sources.md) — the where-did-each-`dataset()`-kwarg-go table (all 26), `label`'s 5 jobs → 4 call shapes, per-domain `pairs()`, the namespace-inversion symbol map, TypeScript-on-1.17, and the econ-unchanged note. Additive-plus-shims; the D-22 `FutureWarning` is gone.
 - **Migrating to 1.15.0:** [`docs/migration-1.15.md`](docs/migration-1.15.md) — the D-22 default-label `FutureWarning`, `include_*` → `features=`, and `contract=` → the markets layer.

docs/dataset.md

@@ -1,5 +1,12 @@
 # `dataset()` — the alignment engine
 
+> **Legacy surface (1.17+).** `dataset()` and `research()` are deprecation
+> shims as of 1.17.0 — they keep returning identical bytes but are removed at
+> 2.0. The current surface is the per-domain `pairs()` quickstart and the
+> `mr.align(spine, *sources)` ladder; see the
+> [1.17 migration guide](migration/v117-align-spine-sources.md) for the
+> complete kwarg-by-kwarg mapping. This document is kept for shim users.
+
 `dataset()` is the one thing in mostlyright that is genuinely hard to get right
 by hand: it takes a station, a window, and a set of feature ingredients, and
 returns **one leakage-safe row per LST settlement day** — every column aligned to

docs/ts-quickstart.md

@@ -1,5 +1,14 @@
 # TypeScript SDK quickstart (`@mostlyrightmd/*`)
 
+> **TS on 1.17:** the TS SDK surface is unchanged — `research()` and
+> `dataset()` keep working exactly as in 1.16, including `dataset()`'s
+> one-time D-22 `FutureWarning` on an omitted `label=` (pass `label` explicitly
+> to silence it; the Python side cancelled that flip in 1.17 and the TS
+> reconciliation is tracked with the ladder port). Python's new
+> `align`/`spine`/`pairs()` ladder is Python-only for now (parity ticket
+> PT-3401); see the
+> ["TypeScript on 1.17" section of the migration guide](migration/v117-align-spine-sources.md).
+
 The TS SDK mirrors the Python public surface and ships four npm packages:
 
 | npm | Path | Use case |