Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/run-weather-backfill.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ name: Run weather backfill fleet (28-21)
# ROLLOUT GATE (operator sequence): the DEFAULT run is a 1-STATION PILOT
# (task_count=1) — cheap, proves the read→reduce→R2-upload loop + the big-bytes
# firewall end to end. The FULL fleet shards station×year (28-28): one array task
# per (station, year), so task_count = 65 stations × N years (the Kalshi∪Polymarket
# per (station, year), so task_count = 64 stations × N years (the Kalshi∪Polymarket
# roster minus the non-satellite HKO, D-28.8; N = current UTC year - 2017 + 1). It
# reduces ~28 TB of raw imagery in-region and is the phase's
# largest spend — it is BLOCKED unless the operator sets mode=full AND
Expand Down Expand Up @@ -40,7 +40,7 @@ on:
default: "latest"
type: string
mode:
description: "pilot = 1 station, 1 monolithic task (cheap smoke test, default). station = 1+ stations sharded across ALL years (task_count = stations × years; the fast full-single-station cost/speed run, e.g. KNYC 2017..now ACMC+DSRF — NOT cost-gated, bounded by station count). full = the 65-station×year fleet (~28 TB reduce; needs cost sign-off; non-GOES shards no-op, native ring is 28-26)."
description: "pilot = 1 station, 1 monolithic task (cheap smoke test, default). station = 1+ stations sharded across ALL years (task_count = stations × years; the fast full-single-station cost/speed run, e.g. KNYC 2017..now ACMC+DSRF — NOT cost-gated, bounded by station count). full = the 64-station×year fleet (~28 TB reduce; needs cost sign-off; non-GOES shards no-op, native ring is 28-26)."
required: true
default: "pilot"
type: choice
Expand Down Expand Up @@ -116,7 +116,7 @@ jobs:
- name: Enforce the cost sign-off gate for a full run
run: |
if [ "${{ inputs.mode }}" = "full" ] && [ "${{ inputs.confirm_cost_signoff }}" != "true" ]; then
echo "::error::mode=full runs the 65-shard ~28 TB fleet (the phase's largest spend)."
echo "::error::mode=full runs the 64-shard ~28 TB fleet (the phase's largest spend)."
echo "::error::Set confirm_cost_signoff=true (H5 pilot cost sign-off) to proceed, or use mode=pilot."
exit 1
fi
Expand Down Expand Up @@ -154,16 +154,16 @@ jobs:
IMAGE="${AR_HOST}/${AR_PROJECT}/${AR_REPO}/${IMAGE_NAME}:${IMAGE_TAG}"
if [ "$MODE" = "full" ]; then
# Station×year sharding (28-28): one array task per (station, year), so
# TASK_COUNT = stations × years. Stations = 65 (the Kalshi∪Polymarket
# union (66) minus the one non-satellite station HKO,
# TASK_COUNT = stations × years. Stations = 64 (the Kalshi∪Polymarket
# union (65) minus the one non-satellite station HKO,
# _roster._NON_SATELLITE_STATIONS — one shard base per satellite-
# resolvable station). Years = the roster default window
# [2017 .. current UTC year] inclusive. year_start MUST match the CLI's
# _DEFAULT_ROSTER_YEAR_START (2017); year_end is pinned to today's UTC
# year and passed EXPLICITLY so the CLI's derived num_years matches this
# TASK_COUNT exactly (else the station×year decomposition would not
# trigger and shards would round-robin over the full range).
STATIONS=65
STATIONS=64
FULL_YEAR_START=2017
FULL_YEAR_END=$(date -u +%Y)
NUM_YEARS=$(( FULL_YEAR_END - FULL_YEAR_START + 1 ))
Expand All @@ -175,7 +175,7 @@ jobs:
PARALLELISM=48
CPU_MILLI=16000
MEM_MIB=65536
# Roster mode: the CLI resolves the 65-station roster and decomposes
# Roster mode: the CLI resolves the 64-station roster and decomposes
# BATCH_TASK_INDEX into (station, year). --year-start/--year-end pin the
# window so num_years == this NUM_YEARS. --max-workers 16 matches the
# n2-standard-16 fleet machine (16 vCPU). Only override the roster's
Expand Down Expand Up @@ -230,7 +230,7 @@ jobs:
# Mid-tier machine: n2-standard-8 (8 vCPU / 32 GiB) — DSRF full-disk
# decode is CPU/mem heavy (--executor process), so bigger than the
# pilot's n2-standard-4, but one station is far cheaper than the
# 65-station fleet. Run all year-shards concurrently (parallelism
# 64-station fleet. Run all year-shards concurrently (parallelism
# capped at 48 to match the fleet's Spot-availability ceiling).
# n2-standard-16 (compute budget approved 2026-07-05): with the
# spawn process pool + the probe-raised rate cap, decode scales
Expand Down
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,27 @@ All notable changes to `mostlyright`. The format follows [Keep a Changelog](http

## [Unreleased]

### Fixed

- **Settlement-station registry: the Kalshi daily-temp universe is 20 cities,
not 21 (live-universe audit, 2026-07-15).** The v0.1.0 scope doc drifted from
the live exchange: DTW/CVG/BNA/SLC never had daily-temperature markets (every
series probe 404s) and three real market cities were missing. Parity-critical —
a backtest keyed to the phantom four settled against data no market resolves on.
- `KALSHI_SETTLEMENT_STATIONS`: dropped `DTW`/`CVG`/`BNA`/`SLC`; added
`NOLA` → KMSY (NOT Lakefront KNEW), `OKC` → KOKC, `SATX` → KSAT (NOT
Stinson KSSF). `KNOWN_WRONG_STATIONS` gains KNEW + KSSF.
- Core catalog `kalshi` venue tags follow (KMSY/KOKC/KSAT tagged;
KDTW/KCVG/KBNA/KSLC remain as bare weather stations, records kept).
- `_KALSHI_TICKER_ALIASES` now resolves the live `KXHIGHT<CITY>` /
`KXLOWT<CITY>` series family (`KXHIGHTATL`, `KXLOWTNYC`, …, plus
`KXHIGHPHIL`/`KXHIGHNYD`/`KXHIGHTEMPDEN`), which previously failed
city-ticker lookup for every city except Las Vegas.
- Satellite backfill roster: union is 65 stations (satellite roster 64;
`batch.tf`/workflow task math updated in lockstep).
- Schemas + generated TS data modules re-exported; Python and TS contract
tests re-pinned to the 20-city universe.

## [1.17.0] — 2026-07-14 — Registry API redesign: the `align` / `spine` / `sources` ladder + per-domain `pairs()`

Minor release, dual-SDK (PyPI 1.17.0 + npm 1.17.0 in lockstep). Phase 34 answers
Expand Down
6 changes: 3 additions & 3 deletions infra/batch.tf
Original file line number Diff line number Diff line change
Expand Up @@ -83,15 +83,15 @@ resource "google_storage_bucket_iam_member" "backfill_progress_rw" {
#
# task_group:
# # Station×year sharding (28-28): ONE array task per (station, year), so
# # task_count = 65 stations × N years.
# task_count = 65 * N # 65 = Kalshi∪Polymarket roster (66) MINUS the one non-
# # task_count = 64 stations × N years.
# task_count = 64 * N # 64 = Kalshi∪Polymarket roster (65) MINUS the one non-
# # satellite station HKO (_roster._NON_SATELLITE_STATIONS;
# # HKO has no satellite StationInfo → a shard for it would
# # resolve to zero partitions). One shard BASE per satellite-
# # resolvable station (D-28.8), fanned out across N years.
# # N = current_UTC_year - 2017 + 1 (roster default window).
# # run-weather-backfill.yml computes the SAME
# # TASK_COUNT = 65 * NUM_YEARS and pins --year-start 2017
# # TASK_COUNT = 64 * NUM_YEARS and pins --year-start 2017
# # / --year-end <current year> so the CLI's derived num_years
# # matches — keep them in LOCKSTEP with the roster + window.
# # NOTE: the container passes no --satellites, so each shard
Expand Down
28 changes: 13 additions & 15 deletions packages-ts/core/src/data/generated/stations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -222,9 +222,7 @@ export const STATIONS: ReadonlyArray<StationInfo> = [
longitude: -86.6782,
name: "Nashville International",
tz: "America/Chicago",
venues: [
"kalshi",
],
venues: [],
},
{
code: "BOS",
Expand All @@ -248,9 +246,7 @@ export const STATIONS: ReadonlyArray<StationInfo> = [
longitude: -84.6678,
name: "Cincinnati/Northern Kentucky International",
tz: "America/New_York",
venues: [
"kalshi",
],
venues: [],
},
{
code: "DAL",
Expand Down Expand Up @@ -313,9 +309,7 @@ export const STATIONS: ReadonlyArray<StationInfo> = [
longitude: -83.3534,
name: "Detroit Metropolitan Wayne County",
tz: "America/New_York",
venues: [
"kalshi",
],
venues: [],
},
{
code: "HOU",
Expand Down Expand Up @@ -432,7 +426,9 @@ export const STATIONS: ReadonlyArray<StationInfo> = [
longitude: -90.258,
name: "New Orleans Louis Armstrong International",
tz: "America/Chicago",
venues: [],
venues: [
"kalshi",
],
},
{
code: "NYC",
Expand All @@ -456,7 +452,9 @@ export const STATIONS: ReadonlyArray<StationInfo> = [
longitude: -97.6007,
name: "Oklahoma City Will Rogers World",
tz: "America/Chicago",
venues: [],
venues: [
"kalshi",
],
},
{
code: "ORD",
Expand Down Expand Up @@ -506,7 +504,9 @@ export const STATIONS: ReadonlyArray<StationInfo> = [
longitude: -98.4698,
name: "San Antonio International",
tz: "America/Chicago",
venues: [],
venues: [
"kalshi",
],
},
{
code: "SEA",
Expand Down Expand Up @@ -545,9 +545,7 @@ export const STATIONS: ReadonlyArray<StationInfo> = [
longitude: -111.9778,
name: "Salt Lake City International",
tz: "America/Denver",
venues: [
"kalshi",
],
venues: [],
},
{
code: "LEBL",
Expand Down
23 changes: 13 additions & 10 deletions packages-ts/core/tests/stations.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@ import { describe, expect, it } from "vitest";
import { STATIONS } from "../src/data/generated/stations.js";
import { CATALOG, type Station, StationCatalog } from "../src/index.js";

// The 21 Kalshi NHIGH/NLOW settlement ICAOs (mirrors the markets citations).
// The 20 Kalshi NHIGH/NLOW settlement ICAOs (mirrors the markets citations).
// 2026-07 live-universe audit: KDTW/KCVG/KBNA/KSLC out (no live markets),
// KMSY/KOKC/KSAT in (NOLA/OKC/SATX series are live).
const KALSHI_ICAOS = new Set([
"KATL",
"KAUS",
Expand All @@ -21,20 +23,21 @@ const KALSHI_ICAOS = new Set([
"KSEA",
"KSFO",
"KIAH",
"KDTW",
"KCVG",
"KBNA",
"KSLC",
"KLAS",
"KMSY",
"KOKC",
"KSAT",
]);

// Registry stations no prediction-market venue settles against. Phase 23 grew
// this from 4 to 29: 3 US bare (KHOU moved TO polymarket) + 26 intl (roster
// this from 4 to 29; the 2026-07 audit swapped the US membership (4 bare US
// stations now: the ex-Kalshi KDTW/KCVG/KBNA/KSLC) + 26 intl (roster
// drops + the London/Moscow/Taipei/HK/Paris move-sources).
const NO_VENUE_ICAOS = new Set([
"KMSY",
"KOKC",
"KSAT",
"KBNA",
"KCVG",
"KDTW",
"KSLC",
"EDDB",
"EDDF",
"EGKK",
Expand Down Expand Up @@ -95,7 +98,7 @@ describe("StationCatalog", () => {

it("kalshi venue equals the settlement universe, NOT every US station", () => {
expect(icaosOf(CATALOG.filterByVenue("kalshi"))).toEqual(KALSHI_ICAOS);
expect(CATALOG.filterByVenue("kalshi").length).toBe(21);
expect(CATALOG.filterByVenue("kalshi").length).toBe(20);
});

it("polymarket venue is the explicit 50-station roster, NOT every intl", () => {
Expand Down
30 changes: 14 additions & 16 deletions packages-ts/markets/src/data/generated/kalshi-stations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@ export const KALSHI_SETTLEMENT_STATIONS: Readonly<Record<string, KalshiStation>>
citation: "https://kalshi.com/markets/khighaus (Austin-Bergstrom; the only Austin station Kalshi cites)",
station: "KAUS",
},
BNA: {
citation: "https://kalshi.com/markets/khighbna (Nashville International)",
station: "KBNA",
},
BOS: {
citation: "https://kalshi.com/markets/khighbos (Boston Logan)",
station: "KBOS",
Expand All @@ -28,10 +24,6 @@ export const KALSHI_SETTLEMENT_STATIONS: Readonly<Record<string, KalshiStation>>
citation: "https://kalshi.com/markets/khighchi (Midway, NOT ORD)",
station: "KMDW",
},
CVG: {
citation: "https://kalshi.com/markets/khighcvg (Cincinnati/Northern Kentucky International)",
station: "KCVG",
},
DAL: {
citation: "https://kalshi.com/markets/khighdal (DFW, NOT Love Field)",
station: "KDFW",
Expand All @@ -44,10 +36,6 @@ export const KALSHI_SETTLEMENT_STATIONS: Readonly<Record<string, KalshiStation>>
citation: "https://kalshi.com/markets/khighden (Denver International)",
station: "KDEN",
},
DTW: {
citation: "https://kalshi.com/markets/khighdtw (Detroit Metropolitan)",
station: "KDTW",
},
HOU: {
citation: "https://kalshi.com/markets/khighhou (Intercontinental, NOT Hobby; Kalshi cites IAH)",
station: "KIAH",
Expand All @@ -64,10 +52,18 @@ export const KALSHI_SETTLEMENT_STATIONS: Readonly<Record<string, KalshiStation>>
citation: "https://kalshi.com/markets/khighmsp (Minneapolis-St. Paul International)",
station: "KMSP",
},
NOLA: {
citation: "https://kalshi.com/markets/kxhightnola (Louis Armstrong Intl, NOT Lakefront KNEW; settles vs NWS CLIMSY)",
station: "KMSY",
},
NYC: {
citation: "https://kalshi.com/markets/khighny (Central Park, NOT LGA/JFK)",
station: "KNYC",
},
OKC: {
citation: "https://kalshi.com/markets/kxhightokc (Will Rogers World Airport)",
station: "KOKC",
},
PHL: {
citation: "https://kalshi.com/markets/khighphl (Philadelphia International)",
station: "KPHL",
Expand All @@ -76,6 +72,10 @@ export const KALSHI_SETTLEMENT_STATIONS: Readonly<Record<string, KalshiStation>>
citation: "https://kalshi.com/markets/khighphx (Sky Harbor International)",
station: "KPHX",
},
SATX: {
citation: "https://kalshi.com/markets/kxhightsatx (San Antonio Intl, NOT Stinson KSSF)",
station: "KSAT",
},
SEA: {
citation: "https://kalshi.com/markets/khighsea (SeaTac, NOT BFI)",
station: "KSEA",
Expand All @@ -84,10 +84,6 @@ export const KALSHI_SETTLEMENT_STATIONS: Readonly<Record<string, KalshiStation>>
citation: "https://kalshi.com/markets/khighsfo (San Francisco International, NOT OAK)",
station: "KSFO",
},
SLC: {
citation: "https://kalshi.com/markets/khighslc (Salt Lake City International)",
station: "KSLC",
},
TLV: {
citation: "https://kalshi.com/markets/kxhightlv (Harry Reid/McCarran; settles vs NWS CLILAS)",
station: "KLAS",
Expand All @@ -102,6 +98,8 @@ export const KNOWN_WRONG_STATIONS: ReadonlySet<string> = new Set<string>([
"KIAD",
"KJFK",
"KLGA",
"KNEW",
"KOAK",
"KORD",
"KSSF",
]);
32 changes: 30 additions & 2 deletions packages/core/src/mostlyright/_compose.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,36 @@
#: ROADMAP example) would fail to resolve.
_KALSHI_TICKER_ALIASES: dict[str, str] = {
"NY": "NYC",
# All other Kalshi cities use the canonical 3-letter slug as their
# ticker suffix verbatim (identity mapping is implicit).
"NYD": "NYC", # KXHIGHNYD — NYC daily series variant
"PHIL": "PHL", # KXHIGHPHIL / KXLOWPHIL
# The modern "KXHIGHT<CITY>" / "KXLOWT<CITY>" series family (verified
# against the live Kalshi series list, 2026-07-15): after the KX→K
# normalization the parser sees KHIGHT<CITY>, so the extracted suffix
# carries a leading "T" that must be stripped to reach the canonical
# catalog key. (TLV is NOT aliased — it is itself the catalog key for
# Las Vegas, from exactly this parsing of KXHIGHTLV.)
"TATL": "ATL",
"TAUS": "AUS",
"TBOS": "BOS",
"TCHI": "CHI",
"TDAL": "DAL",
"TDC": "DCA",
"TDEN": "DEN",
"TEMPDEN": "DEN", # KXHIGHTEMPDEN — legacy Denver series
"THOU": "HOU",
"TLAX": "LAX",
"TMIA": "MIA",
"TMIN": "MSP",
"TNOLA": "NOLA",
"TNYC": "NYC",
"TOKC": "OKC",
"TPHIL": "PHL",
"TPHX": "PHX",
"TSATX": "SATX",
"TSEA": "SEA",
"TSFO": "SFO",
# All other Kalshi cities use the canonical slug as their ticker
# suffix verbatim (identity mapping is implicit).
}


Expand Down
Loading
Loading