From 1c451e99050eb0881e47ecc46054dd2e4991f5c2 Mon Sep 17 00:00:00 2001 From: EiffL Date: Mon, 25 May 2026 21:21:39 +0200 Subject: [PATCH 01/17] =?UTF-8?q?refactor:=20Strategy=20A=20=E2=80=94=20My?= =?UTF-8?q?STRA=20becomes=20a=20stock-MyST=20plugin?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Invert the architecture: instead of generating a whole document from astra.yaml and serving it through a bespoke content server, MySTRA is now a single MyST plugin (directives + roles + transforms) that imports/cites ASTRA components into an author-written Markdown report and runs on the stock `myst` CLI + themes. Net -6k lines. Removed (generate-everything era): - src/server/ (Express content server + ws + watcher), src/cli.ts, src/theme/launcher.ts, src/transform/index.ts (whole-page orchestration), src/types/content-server.ts, src/utils/hash.ts - DOI/paper subsystem (src/doi/, src/papers/) — citations now lean on MyST - unused render-* helpers; old server/page-shape tests Added / reshaped: - the plugin is the package entry: src/index.ts (default export = plugin) - recognition markers (astra-* classes) + per-page resolved data store (hidden div.astra-store) for rich themes - deterministic, lazy result-artifact resolution (lightcone path convention) — no filesystem scanning - tests/plugin-emission.test.ts Lean on @astra-spec/sdk: - data-model types imported by their SDK names (no local types/ module) - runtime reuse: loadYaml, resolveAnalysisTree, isConditionMet (js-yaml dropped) Flattened to src/index.ts + src/loader.ts + src/transform/*. Docs (README, SPEC) rewritten; rationale in STRATEGY-A-REFACTOR.md. Co-Authored-By: Claude Opus 4.7 (1M context) --- .gitignore | 2 + README.md | 248 ++- SPEC.md | 880 +++------- STRATEGY-A-REFACTOR.md | 297 ++++ package-lock.json | 1668 ++---------------- package.json | 29 +- src/cli.ts | 75 - src/doi/cache.ts | 45 - src/doi/fetcher.ts | 32 - src/doi/resolver.ts | 177 -- src/index.ts | 928 +++++++++- src/loader.ts | 87 + src/loader/index.ts | 36 - src/loader/result-scanner.ts | 99 -- src/loader/universe-loader.ts | 50 - src/loader/yaml-loader.ts | 39 - src/papers/index.ts | 180 -- src/server/index.ts | 219 --- src/server/routes/astra.ts | 340 ---- src/server/routes/config.ts | 40 - src/server/routes/content.ts | 50 - src/server/routes/xref.ts | 22 - src/server/watcher.ts | 47 - src/server/websocket.ts | 34 - src/theme/launcher.ts | 159 -- src/transform/index.ts | 347 ---- src/transform/narrative-parser.ts | 39 +- src/transform/parse-table-data.ts | 5 +- src/transform/render-data-sources.ts | 6 +- src/transform/render-evidence.ts | 189 +- src/transform/render-findings.ts | 44 +- src/transform/render-methods.ts | 87 +- src/transform/render-narrative.ts | 88 - src/transform/render-output-provenance.ts | 143 -- src/transform/render-output-recipe.ts | 191 -- src/transform/render-prior-insights.ts | 68 - src/transform/render-sub-analyses.ts | 63 - src/transform/render-universe-banner.ts | 68 - src/transform/resolve-output.ts | 22 +- src/transform/resolved-store.ts | 338 ++++ src/types/astra.ts | 289 --- src/types/content-server.ts | 111 -- src/types/papers.ts | 15 - src/utils/hash.ts | 5 - tests/fixtures/schema-v0.0.7/README.md | 23 - tests/fixtures/schema-v0.0.7/analysis.yaml | 654 ------- tests/fixtures/schema-v0.0.7/insight.yaml | 136 -- tests/fixtures/schema-v0.0.7/universe.yaml | 80 - tests/narrative-parser.test.ts | 16 +- tests/page-shape.test.ts | 1833 -------------------- tests/plugin-emission.test.ts | 248 +++ tests/schema-coverage.test.ts | 201 --- tests/server-routes.test.ts | 92 - 53 files changed, 2586 insertions(+), 8598 deletions(-) create mode 100644 STRATEGY-A-REFACTOR.md delete mode 100644 src/cli.ts delete mode 100644 src/doi/cache.ts delete mode 100644 src/doi/fetcher.ts delete mode 100644 src/doi/resolver.ts create mode 100644 src/loader.ts delete mode 100644 src/loader/index.ts delete mode 100644 src/loader/result-scanner.ts delete mode 100644 src/loader/universe-loader.ts delete mode 100644 src/loader/yaml-loader.ts delete mode 100644 src/papers/index.ts delete mode 100644 src/server/index.ts delete mode 100644 src/server/routes/astra.ts delete mode 100644 src/server/routes/config.ts delete mode 100644 src/server/routes/content.ts delete mode 100644 src/server/routes/xref.ts delete mode 100644 src/server/watcher.ts delete mode 100644 src/server/websocket.ts delete mode 100644 src/theme/launcher.ts delete mode 100644 src/transform/index.ts delete mode 100644 src/transform/render-narrative.ts delete mode 100644 src/transform/render-output-provenance.ts delete mode 100644 src/transform/render-output-recipe.ts delete mode 100644 src/transform/render-prior-insights.ts delete mode 100644 src/transform/render-sub-analyses.ts delete mode 100644 src/transform/render-universe-banner.ts create mode 100644 src/transform/resolved-store.ts delete mode 100644 src/types/astra.ts delete mode 100644 src/types/content-server.ts delete mode 100644 src/types/papers.ts delete mode 100644 src/utils/hash.ts delete mode 100644 tests/fixtures/schema-v0.0.7/README.md delete mode 100644 tests/fixtures/schema-v0.0.7/analysis.yaml delete mode 100644 tests/fixtures/schema-v0.0.7/insight.yaml delete mode 100644 tests/fixtures/schema-v0.0.7/universe.yaml delete mode 100644 tests/page-shape.test.ts create mode 100644 tests/plugin-emission.test.ts delete mode 100644 tests/schema-coverage.test.ts delete mode 100644 tests/server-routes.test.ts diff --git a/.gitignore b/.gitignore index f5b624f..26b4c1f 100644 --- a/.gitignore +++ b/.gitignore @@ -11,3 +11,5 @@ _build/ snapshot.md # local fibers / notes are not part of the public repo .felt/ + +prototype/ \ No newline at end of file diff --git a/README.md b/README.md index 0de6c5b..80d6420 100644 --- a/README.md +++ b/README.md @@ -1,142 +1,204 @@ # MySTRA -**Live ASTRA document rendering via MyST.** +**A MyST plugin that imports/cites [ASTRA](https://github.com/LightconeResearch/ASTRA) analysis components into Markdown reports.** -MySTRA turns an [ASTRA](https://github.com/LightconeResearch/ASTRA) analysis specification into a browsable, interactive web document — with collapsible method decisions, tabbed option comparisons, citation hover previews, and live reload on file changes. +You write a normal [MyST](https://mystmd.org/) Markdown report and pull in ASTRA +components — decisions, outputs, findings, prior insights, data tables — *by +reference*. They stay single-sourced in `astra.yaml`. MySTRA is a MyST plugin +(directives + roles + transforms) that reads `astra.yaml` at build time and +emits standard MyST AST, so it runs on the **stock `myst` CLI and themes** — no +custom server. -It works by generating [MyST](https://mystmd.org/) AST directly from the ASTRA data model and serving it to the unmodified MyST book-theme. No intermediate markdown is produced. +> This is the "Strategy A" architecture. It replaces an earlier +> generate-everything content server; see +> [`STRATEGY-A-REFACTOR.md`](./STRATEGY-A-REFACTOR.md) for the full rationale and +> [`SPEC.md`](./SPEC.md) for the design. + +## The idea: three single sources + +| Concern | Single source of truth | +|---|---| +| **Data** — what a decision/output/finding *is* | `astra.yaml` (+ `universes/`, `results/`) | +| **Composition** — what appears, where, in what order | your `index.md` | +| **Presentation** — how it looks | the theme | + +The plugin is a pure **projector**: it reads the data and emits (a) rendered, +neutral semantic AST for the elements you placed (the baseline any theme shows) +and (b) — for rich themes — the resolved ASTRA data, once, keyed by id. It makes +no authoring or styling decisions. ## Quick start ```bash npm install -npm run build - -# Render an ASTRA project -npx mystra path/to/astra-project/ +npm run build # compiles the plugin to dist/ -# Open http://localhost:3000 +cd prototype # a worked DESI DR1 BAO example +ASTRA_PROJECT_ROOT="$PWD" npx mystmd start # → http://localhost:3000 ``` -## How it works +Register the plugin in your project's `myst.yml` and point `ASTRA_PROJECT_ROOT` +at the ASTRA project (defaults to the working directory; pick a universe with +`ASTRA_UNIVERSE`): + +```yaml +version: 1 +project: + plugins: + - mystra # or a local path to dist/index.js + toc: + - file: index.md +site: + template: book-theme # the clean baseline; swap for a rich ASTRA theme later +``` +## Two render modes + +- **Basic — plugin only.** Registering the plugin is all you need. On the stock + `book-theme` with no stylesheet the document is clean and readable: decisions + render as dropdowns with tabbed options, outputs as real figures/tables, + findings and prior insights as cards, interpolated numbers show their value, + and inline references show a plain label. Preview cards are hidden by default, + so a bare viewer never spills card content inline. **No user CSS required.** +- **Rich — a `lightcone-astra` theme.** Glyphs, per-kind colours, hover preview + cards, and "powerful patterns" (e.g. a product-dependency graph) are + appearance keyed on the `astra-*` classes the plugin emits, driven from the + resolved store. The only change is the `template:` line. (The theme is a + separate deliverable; until it ships, `book-theme` is the baseline. The + prototype's `custom.css` is a reference stylesheet that previews the rich mode + on book-theme via `site.options.style`.) + +## Authoring vocabulary + +**Block "import" directives** (one component, by id): + +```markdown +:::{astra:decision} covariance_source +::: +:::{astra:output} bao_fit_plot +::: +:::{astra:finding} bao_detected_post_recon +::: +:::{astra:prior-insight} recon_sharpens_bao_peak +::: +:::{astra:inputs} +::: # full inputs registry table (root scope) +:::{astra:outputs} clustering +::: # outputs table for the clustering sub-analysis +:::{astra:subanalysis} reconstruction +::: # nav card to the sub-analysis page ``` -astra.yaml + universes/ + results/ - │ - ▼ - ASTRA → MyST AST transform - │ - ▼ - Content Server (:3100) Theme Server (:3000) - serves JSON AST per page ◀── fetches & renders via - config, xrefs, citations myst-to-react (unmodified) - │ │ - └──────────────────────────────────▶ Browser + +`:::{astra:output}` figures carry the `output-` anchor and a collapsed +provenance disclosure (type, upstream products, decisions, recipe); tables +render as a clean numbered `container[table]`. `:::{astra:finding}` accepts +`:compact:` to render claim + notes + scope only. + +**Inline "cite" roles** — a neutral token (label) carrying a hidden preview card: + +```markdown +{astra:decision}`covariance_source` +{astra:output}`hubble_diagram_plot` +{astra:finding}`subpercent_alpha_iso_precision` +{astra:prior-insight}`recon_sharpens_bao_peak|the recovered peak` # |display override +{astra:analysis}`reconstruction` ``` -The MyST book-theme doesn't care where its AST comes from. MySTRA replaces the content server with one that transforms ASTRA directly into MyST AST JSON. The theme works identically to a standard MyST site. +**Inline value interpolation** — never hard-type a measured number; pull it live +from a result product at build time: -### Why direct AST generation (not markdown) +```markdown +{astra:value}`bao_distance_table tracer=lrg3_elg1 col=DV_over_rd pm` → 19.88 ± 0.17 +{astra:value}`bao_alpha_values tracer=elg1 recon=Pre col=alpha1_std` → 0.0696 +``` -- **No syntax fragility** — nested MyST directives require careful fence-depth management. AST nodes are just objects; nesting is trivial. -- **Tree-to-tree mapping** — ASTRA is a tree (Analysis > Decision > Option > Insight > Evidence). The MyST AST is a tree. The transform is a direct structural mapping. -- **Extensible** — custom AST node types (`details`, `cite`, `tabSet`) integrate seamlessly with the theme's renderers. +Grammar: ` col= [= …] [pm] [err=] [sig=N]`. It +reads the materialised CSV/JSON, filters rows by `key=val`, and renders the cell +(with `± std` via `pm`/`err=`). -## Features +**Anchor grammar** — `[text](#decisions.x)`, `#outputs.y`, `#analyses.sub.…` +resolve to cross-references, alongside plain MyST `[](#output-bao_fit_plot)`. -- **Flat addressable elements** — every finding, decision, prior-insight, input, output, and narrative chunk is emitted as a top-level block with a stable `-` identifier. Themes and downstream renderers compose layout from those carriers; MySTRA imposes no section structure of its own. -- **Structured ASTRA sidecar** — `/astra/.json` exposes resolved inputs/outputs, recipes, and inline metric/table payloads for renderer-native gallery/detail views. -- **Findings** as h3 blocks with author notes, scope, and evidence (figures, tables, citations). -- **Decisions** as collapsible dropdowns with tabbed option comparisons (selected option marked with **●**). -- **Prior insights** as flat blocks; option tabs cross-reference them rather than expanding inline. -- **Universe banner** summarising active decision selections with links to each decision. -- **Narrative anchor grammar** — `[text](#path.to.element)` resolves to a `crossReference` everywhere prose appears (narrative sections, claims, rationales, descriptions, captions, excluded reasons). -- **Live reload** — edits to the root spec, nested `analyses/**/astra.yaml`, or result artifacts under `results/` and `analyses/**/results/` trigger an automatic page refresh. -- **DOI + paper-cache enrichment** — disk-cached citation metadata, optional cached-PDF links, and insight→decision backlinks for cited papers. -- **Recursive sub-analyses** rendered as separate pages with their own universe scoping. +**Scoping** — a path is `` (root analysis) or `.` (sub-analysis), +e.g. `reconstruction.algorithm`, `clustering.xi_multipoles_plot`. Each +sub-analysis is its own page (`reconstruction.md`, `clustering.md`). -## Usage +## Recognition markers & the resolved store -``` -mystra [project-dir] [options] -``` +Every placed block carries a stable `astra-` class (`astra-decision`, +`astra-output`/`--figure`, `astra-finding`, `astra-prior-insight`, +`astra-inputs`/`astra-outputs`, `astra-subanalysis`) on the node bearing its +`-` identifier. Inline tokens are neutral +(`span.astra-ref--` + label + a hidden `span.astra-card`). -| Option | Default | Description | -|--------|---------|-------------| -| `-p, --port ` | `3000` | Theme server port | -| `--content-port ` | `3100` | Content server port | -| `-u, --universe ` | first found | Universe to render | -| `--no-theme` | | Content server only (API mode) | +For rich themes, the plugin bakes a **resolved store** onto a hidden +`div.astra-store` carrier's `data` (per page scope): the resolved outputs +(project-relative paths, parsed table/metric values, recipes, provenance), +inputs, decisions (selected option), findings, prior insights, and +sub-analyses — all keyed by id. A theme selects a placed node by +`identifier`/class and joins it to the store; it never reads `astra.yaml`. ## Project structure ``` src/ -├── transform/ ASTRA → MyST AST conversion -│ ├── index.ts Main orchestrator + page builder -│ ├── ast-helpers.ts Pure AST node constructors -│ ├── narrative-parser.ts myst-parser wrapper + anchor grammar resolver -│ ├── render-narrative.ts Narrative chunks (summary/findings/methods/inputs/outputs) -│ ├── render-findings.ts Findings as flat per-finding blocks -│ ├── render-prior-insights.ts Prior insights as flat per-insight blocks -│ ├── render-methods.ts Decisions as details/summary + tabbed options -│ ├── render-evidence.ts Artifact rendering driven by Output.type; cites + quotes -│ ├── render-universe-banner.ts Collapsible decision summary table -│ ├── render-data-sources.ts Inputs and outputs tables -│ └── render-sub-analyses.ts Sub-analysis cards -├── loader/ ASTRA source loading (YAML, universes, results) -├── server/ Express content server + WebSocket live reload -├── doi/ DOI resolution, caching, citation formatting -├── papers/ Cached-paper enrichment + DOI insight backlinks -├── theme/ MyST book-theme launcher -├── types/ TypeScript interfaces (ASTRA, content-server API) -└── cli.ts CLI entry point +├── index.ts The MyST plugin + package entry (default export = the plugin) +├── loader.ts Load a project for one universe (via the SDK) + resolve result files +└── transform/ Per-component renderers used by the plugin + ├── ast-helpers.ts Pure AST node constructors + ├── narrative-parser.ts myst-parser wrapper + anchor-grammar resolver + ├── parse-table-data.ts CSV/JSON table parser + ├── resolve-output.ts Resolves `from:` output/alias chains + ├── resolved-store.ts Builds the resolved data store for rich themes + ├── render-methods.ts renderDecision (details/summary + tabbed options) + ├── render-findings.ts renderFinding (claim + notes + scope + evidence) + ├── render-evidence.ts renderOneOutput + evidence/table rendering (Output.type) + └── render-data-sources.ts Inputs/outputs registry tables ``` -## ASTRA project layout +Data-model types are imported directly from **`@astra-spec/sdk`** (`Analysis`, +`Decision`, `Output`, …) — MySTRA defines none of its own. -MySTRA expects: +## ASTRA project layout ``` my-analysis/ -├── astra.yaml Analysis specification (decisions, findings, evidence) +├── astra.yaml Analysis specification (decisions, findings, outputs, …) ├── universes/ -│ ├── baseline.yaml Decision selections for the baseline universe -│ └── variant.yaml Alternative universe -└── results/ - ├── baseline/ Outputs produced under the baseline universe - │ ├── figure.png - │ └── data.json - └── variant/ +│ └── baseline.yaml Decision selections for the baseline universe +├── results/ +│ └── baseline//.png Materialised result artifacts +├── myst.yml Registers the plugin; lists pages +└── index.md (+ sub-analysis pages) Your report ``` -Nested analyses typically live under `analyses//astra.yaml`; MySTRA also -scans `analyses/**/results//` when resolving artifacts and serving -`/static/*` URLs. +A sub-analysis that declares `path: ./analyses/` roots its own +`results//` there. MySTRA never scans the results tree: it computes +each output's directory deterministically from this convention (the analysis's +`path:` + universe + output id) and resolves the artifact file lazily, on render. -## Content API +## What MyST does for us -When running with `--no-theme`, the content server exposes: +We lean on the stock `myst` engine for everything it already does: building, +serving, asset hashing/copying (it rewrites the plugin's project-relative image +paths into hashed assets), live reload of Markdown, numbering, cross-references, +and search. We write only the ASTRA→AST bridge. -| Endpoint | Description | -|----------|-------------| -| `GET /config.json` | Site manifest + table of contents | -| `GET /content/*.json` | Page AST + frontmatter + references | -| `GET /myst.xref.json` | Cross-reference index | -| `GET /astra/*.json` | Structured ASTRA sidecar with resolved inputs/outputs, recipes, metric/table payloads | -| `GET /doi-metadata/:doi(*)` | Enriched DOI metadata, including cached-PDF links and insight backlinks when available | -| `GET /papers/*` | Cached paper PDFs from the local ASTRA paper cache | -| `GET /static/*` | Result artifacts from root or nested sub-analysis results | -| `WS /socket` | Live reload notifications | +**Citations** are delegated to MyST. DOI evidence renders as a plain `doi.org` +link; MySTRA carries no DOI resolver or cache of its own. Author–year labels and +a linked reference list come for free once a project bibliography is wired — a +clean follow-up (see the citation note in the spec). ## Development ```bash -npm run dev -- path/to/astra-project/ # Run with tsx (no build step) -npm run build # Compile TypeScript -npm test # Run tests +npm run build # compile the plugin +npm test # plugin-emission + store + parser tests (vitest) ``` +`astra.yaml` is parsed once and cached; `myst start` watches Markdown, not +`astra.yaml`, so editing the spec needs a server restart. + ## License MIT diff --git a/SPEC.md b/SPEC.md index 0ebf27a..2ab050e 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,664 +1,236 @@ -# MySTRA — Live ASTRA Document Rendering via MyST - -> Tracks **astra-spec v0.0.7** (commit `ed13f48`). Notable v0.0.6→v0.0.7 -> deltas reflected in the transform's type surface (`src/types/astra.ts`): -> `Output.inputs` and `Output.decisions` now carry the per-output -> provenance contract (PR #19); `Recipe` shrinks to pure *how* -> (`command`, `resources`, `container`); `Resources` gains `disk`. The -> Recipe template grammar (`{inputs.}`, `{decisions.}`, -> `{output}`) is the runner's substitution surface, not MySTRA's. -> Earlier v0.0.5→v0.0.6 deltas — structured `Analysis.narrative` with -> tree-path anchor grammar, `container_build` collapsed into -> `container`, `from_ref` renamed to `from`, optional `label` on -> `Input`/`Output`/`Insight`, reserved-keyword ID exclusions — remain -> in force. +# MySTRA — ASTRA components as a MyST plugin (Strategy A) -## 1. Goal - -Render an ASTRA analysis (`astra.yaml` + `universes/` + `results/`) as a live, browsable structured document using MyST's rendering infrastructure. The document updates automatically when the spec, universe selections, or results change on disk (typically because an agent modified them). - -### Guiding principle - -Reuse existing MyST ecosystem packages wherever possible. The MyST project (MIT-licensed) provides well-tested utilities for AST types, citation resolution, React rendering, and theming. We should import and use these directly rather than reimplementing. Custom code should be limited to the ASTRA-specific transform and the thin content server. - -## 2. Architecture - -### How MyST works internally - -MyST uses a content/theme separation: - -``` -[Content Server :3100] ←──── [Theme Server :3000] ────→ Browser - serves JSON AST fetches JSON AST - per-page content renders via myst-to-react - config, xrefs sidebar, navigation, styling -``` - -The **content server** exposes: -- `GET /config.json` — site metadata + table of contents -- `GET /content/*.json` — page AST + frontmatter + references -- `GET /myst.xref.json` — cross-reference index -- `GET /astra/*.json` — structured ASTRA sidecar for renderer-native views -- `GET /doi-metadata/:doi(*)` — enriched DOI metadata (including cached-PDF links when present) -- `GET /papers/*` — cached paper PDFs from the local ASTRA paper cache -- `GET /static/*` — result artifacts from root or nested sub-analyses -- `WS /socket` — WebSocket for live reload notifications +> This spec describes the current architecture. It supersedes the earlier +> generate-everything content server. The motivation, the alternatives weighed, +> and the keep/refactor/remove record live in +> [`STRATEGY-A-REFACTOR.md`](./STRATEGY-A-REFACTOR.md). -The **theme server** (book-theme) is a Remix app that fetches JSON from the content server and renders it with `myst-to-react`. It has no knowledge of the source format. - -### The key insight - -The theme doesn't care where the JSON AST came from. We replace the content server with one that transforms ASTRA directly into MyST AST JSON. The theme works identically. - -### Architecture - -``` - ┌──────────────────────────────┐ - │ File System Watcher │ - │ watches: astra.yaml + │ - │ analyses/**/astra.yaml, │ - │ universes/*.yaml, │ - │ results/**/* + │ - │ analyses/**/results/**/* │ - └──────────┬───────────────────┘ - │ on change - ▼ -┌─────────────┐ ┌─────────────────────────┐ ┌──────────────┐ -│ astra.yaml │────▶│ ASTRA → AST Transform │────▶│ Content API │ -│ universes/ │ │ │ │ :3100 │ -│ results/ │ │ Reads ASTRA spec │ │ │ -│ analyses/ │ │ Reads universe selections │ │ /config.json │ -│ │ │ Reads result artifacts │ │ /content/*.json -│ │ │ Produces MyST AST JSON │ │ /myst.xref.json -│ │ │ │ │ /astra/*.json -│ │ │ │ │ /doi-metadata/* -│ │ │ │ │ /papers/* │ -└─────────────┘ └─────────────────────────────┘ └──────┬───────┘ - │ - │ fetch JSON - ▼ - ┌──────────────┐ - │ Theme :3000 │ - │ (book-theme) │ - │ unmodified │ - │ │ - │ myst-to-react│ - │ renders AST │ - │ as React │ - └──────┬───────┘ - │ - ▼ - Browser -``` - -### Why direct AST generation (not markdown) - -We generate MyST AST JSON directly rather than generating MyST markdown because: - -1. **No syntax fragility** — Nested MyST directives require careful fence-depth management (`:::` vs `::::` vs `::::::`). AST nodes are just objects; nesting is trivial. -2. **Tree-to-tree is natural** — ASTRA is a tree (Analysis → Decision → Option → Insight → Evidence). The MyST AST is a tree. The transform is a direct structural mapping. -3. **The content server API is still small** — a handful of JSON/document endpoints. The page-content endpoint is still just a JSON object, and the extra sidecars stay thin. -4. **Extensible** — We can add custom AST node types if needed and register renderers for them. - -DOI auto-resolution (which MyST's markdown parser provides for free) is handled by fetching citation metadata in the content server as a background enrichment step. +## 1. Goal -## 3. The ASTRA → MyST AST transform +Let a human or agent author a normal **MyST Markdown report** that *imports and +cites* [ASTRA](https://github.com/LightconeResearch/ASTRA) components — +decisions, outputs, findings, prior insights, data tables — which stay +single-sourced in `astra.yaml`. Render it on the **stock `myst` CLI and themes**, +with no custom server. -### Node type mapping +### Guiding principle: three single sources -| ASTRA Concept | MyST AST Node(s) | +| Concern | Single source of truth | |---|---| -| Analysis (root) | `root` + flat children carrying `-` identifiers | -| Narrative section (summary, findings, methods, inputs, outputs) | block-level mdast carrying `narrative-
` on its first child | -| Narrative anchor `[t](#path.to.element)` | `crossReference` (resolved) or `link` (unresolved / parent-escape) | -| Universe banner | `details` + `summary` + decision-summary `table` | -| Finding | `heading` (h3) carrying `finding-` + author notes + evidence | -| Finding evidence (DOI) | `blockquote` + `paragraph` with `cite` (or plain `link` fallback) | -| Finding evidence (artifact, Output.type=figure) | `container` (kind: figure) + `image` + `caption` (caption parses Output.description) | -| Finding evidence (artifact, Output.type=table) | `details` + `summary` + `table` (JSON / CSV body) | -| Finding evidence (artifact, Output.type=metric/data/report) | inline labelled reference + optional quote | -| Prior insight | `container` (kind: prior-insight) carrying `prior_insight-` + `data.{id,label,scope,tags,derived}` + claim/evidence children | -| Decision | `heading` (h4) carrying `decision-` + `details` + `summary` | -| Decision options | `tabSet` + `tabItem` per option | -| Option supporting insights | `crossReference` to `prior_insight-` (no inline expansion) | -| Insight quote | `blockquote` + `paragraph` | -| DOI reference | `cite` / `citeGroup` (or `link` fallback when uncached) | -| Input | `tableRow` carrying `input-` | -| Output | `tableRow` carrying `output-` | -| Output provenance (Output.inputs / Output.decisions) | `container` (kind: output-provenance) carrying `output--provenance` + `data.{outputId, inputs, decisions, from, unresolved}` + inline `crossReference` chips for each input / decision | -| Sub-analysis | Separate page + `card` (carrying `analysis-`) in parent | - -### Document structure - -The transform produces a **flat sequence of addressable blocks** for -each analysis page. There are no programmatic h2 section headings -("Findings", "Methods", "Data Sources", "Sub-Analyses"); narrative -sections, structural elements, and sub-analysis cards all sit at the -same depth. Themes and downstream renderers (paper view, dashboard, -DAG, …) compose layouts however they like by looking up -`identifier` attributes — MySTRA imposes no narrative around the -data. - -Block-emission order is the spec-declared default: +| **Data** — what a decision/output/finding *is* | `astra.yaml` (+ `universes/`, `results/`) | +| **Composition** — what appears, where, in what order | the author's `index.md` | +| **Presentation** — how it looks | the theme | -``` -Root -├── narrative.summary block-level mdast, first child id=narrative-summary -├── narrative.findings block-level mdast, first child id=narrative-findings -├── narrative.methods … -├── narrative.inputs … -├── narrative.outputs … -├── Universe banner details/summary + decision-summary table -├── Findings (flat) one h3 per finding; tags ride on heading.data.tags -├── Prior insights (flat) one `container.kind=prior-insight` per prior_insight -├── Decisions (flat) one h4 + details + tabSet per rendered decision -├── Inputs table one row per input, carrying input- -├── Outputs table one row per output, carrying output- -├── Output provenance one container per Output with non-empty -│ inputs/decisions (after `from:` resolution), -│ carrying output--provenance -└── Sub-analysis cards one card per nested analysis, carrying analysis- -``` - -A decision drops out of the page (and the xref index) if it's a bare -`from`-reference or its `when` predicate is unmet under the active -universe. The xref contract is "every published id has a real -carrier in the rendered AST"; collectIdentifiers and the renderers -agree on which ids are live. - -### AST examples - -**A finding with inline figure and methodology cross-references:** - -```json -[ - { - "type": "heading", - "depth": 3, - "identifier": "finding-1", - "label": "finding-1", - "data": { "tags": ["trgb", "magnitude"] }, - "children": [ - { "type": "text", "value": "1. " }, - { "type": "text", "value": "B-sequence SARGs are the best TRGB standard candles" } - ] - }, - { - "type": "paragraph", - "children": [{ "type": "text", "value": "The TRGB magnitude hierarchy is consistent across both galaxies..." }] - }, - { - "type": "container", - "kind": "figure", - "children": [ - { "type": "image", "url": "/static/trgb_hierarchy_figure.png", "alt": "TRGB hierarchy" }, - { - "type": "caption", - "children": [ - { - "type": "paragraph", - "children": [ - { "type": "text", "value": "M_I vs mean (V-I)_0 for all samples in " }, - { "type": "crossReference", "identifier": "input-lmc", - "children": [{ "type": "text", "value": "LMC" }] }, - { "type": "text", "value": " and SMC." } - ] - } - ] - } - ] - } -] -``` - -The figure caption parses through myst-parser with the v0.0.6 -narrative anchor grammar — `[LMC](#inputs.lmc)` becomes a -`crossReference`, not glued text. The figure container itself -carries no `identifier`; the structural `output-` carrier -lives on the per-output row in the outputs table. Renderer-imposed -"Methodology" admonitions and "This finding depends on…" glue are -gone; explicit relations route through anchor grammar in the -author's notes / claim / methods narrative. - -**A decision as a collapsible dropdown with option tabs:** - -```json -{ - "type": "details", - "open": false, - "children": [ - { - "type": "summary", - "children": [ - { "type": "strong", "children": [{ "type": "text", "value": "R_V for SMC" }] }, - { "type": "text", "value": " — selected: R_V = 2.7" } - ] - }, - { "type": "paragraph", "children": [{ "type": "text", "value": "R_V controls extinction coefficients..." }] }, - { - "type": "tabSet", - "children": [ - { - "type": "tabItem", - "title": "R_V = 2.7 ●", - "children": [ - { "type": "paragraph", "children": [{ "type": "text", "value": "SMC average from Bouchet+1985..." }] }, - { - "type": "details", - "children": [ - { "type": "summary", "children": [{ "type": "text", "value": "Evidence (3 insights)" }] }, - { "type": "paragraph", "children": [ - { "type": "strong", "children": [{ "type": "text", "value": "Gordon et al. (2003)" }] }, - { "type": "text", "value": " — " }, - { "type": "link", "url": "https://doi.org/10.1086/376774", "children": [{ "type": "text", "value": "10.1086/376774" }] } - ]}, - { "type": "blockquote", "children": [ - { "type": "paragraph", "children": [{ "type": "text", "value": "For the SMC Bar, we find that RV = 2.74 ± 0.13..." }] } - ]} - ] - } - ] - }, - { - "type": "tabItem", - "title": "R_V = 3.3 ○", - "children": [ "..." ] - } - ] - } - ] -} -``` - -### Transform implementation - -```typescript -interface ASTRASource { - analysis: ASTRAAnalysis // parsed astra.yaml - universe: ASTRAUniverse // active universe selections - results: Map // output_id → file path (if produced) - projectDir: string // root of the ASTRA project (DOI cache lives here) - slug: string // the host page's slug (anchor resolution context) -} - -function astraToMystAST(source: ASTRASource): Root { - const { analysis, universe, results, projectDir, slug } = source - - // Bound once per page: prose parser threads anchor resolution - // into every render-* helper; tabItem factory mints stable keys - // per transform pass; doiCacheDir replaces the prior module- - // global; outputsById feeds artifact-evidence dispatch. - const prose = makeProseParser({ analysis, slug }) - const tabItem = makeTabItem() - const doiCacheDir = join(projectDir, '.mystra-cache', 'doi') - const outputsById = new Map((analysis.outputs ?? []).map(o => [o.id, o])) - - return { - type: 'root', - children: [ - blockBreak(), - - // Narrative chunks — each section is an addressable block at - // narrative-
; first child of the parsed mdast carries - // the identifier. Spec-declared order (summary → outputs). - ...renderNarrativeChunks(analysis, slug).flatMap(c => c.mdast), - - // Universe banner — orientation for the active selections. - renderUniverseBanner(universe, analysis.decisions, prose), - - // Flat structural elements — no surrounding section headings. - ...renderFindings(analysis.findings, results, outputsById, prose, doiCacheDir), - ...renderPriorInsights(analysis.prior_insights, prose, doiCacheDir), - ...renderMethodsSections(analysis.decisions, analysis.prior_insights, - universe, prose, tabItem, doiCacheDir), - ...(analysis.inputs?.length ? [renderInputsTable(analysis.inputs, prose)] : []), - ...(analysis.outputs?.length ? [renderOutputsTable(analysis.outputs, prose)] : []), - ...(analysis.analyses ? renderSubAnalysisCards(analysis.analyses, slug) : []), - ] - } -} -``` - -**`renderFindings`** — flat per-finding blocks. Each finding gets an -h3 heading carrying `finding-` (with tags on `data.tags`), -notes prose parsed via myst-parser, scope, and evidence blocks. -No tag-overlap-derived crossReferences and no "depends on" glue; -explicit relations are the author's job through narrative anchors. - -**`renderEvidenceBlock`** — for DOI evidence, emits citation + -optional quote. For artifact evidence, looks up the referenced -output by id and dispatches on `Output.type`: `figure` → -image+caption (caption parses Output.description with anchor -resolution); `table` → JSON/CSV table render; metric/data/report → -labelled inline reference. Broken artifact references emit a -`console.warn`. - -**`renderPriorInsights`** — flat per-insight blocks parallel to -findings. Each prior_insight gets an h3 carrier identified by -`prior_insight-` so it's addressable from anywhere on the -page (option tabs cross-reference back to it instead of expanding -inline). - -**`renderMethodsSections`** — flat per-decision blocks. Each -decision renders as an h4 heading (carrying `decision-`) -followed by a `details` dropdown with rationale and a `tabSet` of -options. The selected option (from the active universe) is marked -with ●. Option supporting-insight references emit -`crossReference` nodes pointing at the prior_insight flat-block -carrier, not inline expansions. Decision tags survive on the -heading's `data.tags` slot. - -## 4. Content server - -### Endpoints - -``` -GET /config.json Site manifest + table of contents -GET /content/*.json Page AST + frontmatter + references -GET /myst.xref.json Cross-reference index -GET /astra/*.json Structured ASTRA sidecar for renderer-native views -GET /doi-metadata/:doi(*) Enriched DOI metadata -GET /papers/* Cached paper PDFs -GET /static/* Result artifacts from root or nested sub-analyses -WS /socket Live reload notifications -``` +MySTRA is the **projector** between data and AST. It makes no authoring or +styling decisions: it renders the elements the author placed (neutral semantic +AST, the baseline any theme shows) and bakes the resolved data for rich themes. -**`/config.json`:** - -```json -{ - "version": 1, - "myst": "1.0.0", - "id": "mystra", - "title": "Analysis Name", - "projects": [{ - "slug": "", - "index": "index", - "title": "Analysis Name", - "pages": [ - { - "slug": "preprocessing", - "title": "Preprocessing", - "level": 2, - "description": "Feature extraction and normalization." - } - ] - }] -} -``` - -**`/content/*.json`:** - -```json -{ - "kind": "Article", - "sha256": "content-hash-for-cache-invalidation", - "slug": "index", - "mdast": { "type": "root", "children": [...] }, - "frontmatter": { - "title": "Analysis Name", - "authors": [{ "name": "Author Name" }], - "tags": ["tag1", "tag2"], - "description": "First paragraph of narrative.summary, plain text." - }, - "references": {}, - "dependencies": ["/static/figure.png"] -} -``` - -**`/myst.xref.json`:** - -```json -{ - "version": "1", - "references": [ - { "identifier": "narrative-summary", "kind": "heading", "data": "/content/index.json", "url": "/" }, - { "identifier": "decision-scaling", "kind": "heading", "data": "/content/index.json", "url": "/" } - ] -} -``` - -**`/astra/*.json`:** - -```json -{ - "inputs": [ - { "id": "catalog", "type": "data", "source": "s3://bucket/catalog.parquet" } - ], - "outputs": [ - { - "id": "accuracy_plot", - "type": "figure", - "resolved_path": "/static/accuracy_plot.png", - "inputs": ["catalog"], - "decisions": ["scaling"], - "recipe": { - "command": "snakemake results/baseline/accuracy_plot.png" - } - } - ] -} -``` - -`/doi-metadata/:doi(*)` returns the resolved citation record for a DOI, -enriched with cached-paper metadata (`pdf_url`, `version`, `cache_key`) and -insight backlinks when the local ASTRA paper cache has that paper. `/papers/*` -streams the corresponding cached PDF. - -### Static file serving - -Result artifacts are served from the recursive scanner's `output_id → absolute -path` map. The scanner covers both the root `results//` directory and -nested `analyses/**/results//` directories, so `/static/` -works for root outputs and sub-analysis outputs alike. The server resolves a -basename match from that map first, then falls back to the root -`results//` static directory for legacy callers. - -```typescript -app.use('/static', (req, res, next) => { - const rel = decodeURIComponent((req.url ?? '/').split('?')[0]).replace(/^\/+/, ''); - for (const absPath of resultsByOutputId.values()) { - if (basename(absPath) === rel) { - createReadStream(absPath).pipe(res); - return; - } - } - next(); -}); -app.use('/static', express.static(join(projectDir, 'results', activeUniverseId))); -``` - -Image URLs in the mdast and `resolved_path` values in `/astra/*.json` both -point at this mount. - -### File watching and live reload - -```typescript -const watcher = chokidar.watch([ - `${projectDir}/astra.yaml`, - `${projectDir}/analyses/**/astra.yaml`, - `${projectDir}/universes/*.yaml`, - `${projectDir}/universes/*.yml`, - `${projectDir}/results/**/*.{png,jpg,jpeg,svg,csv,json,md}`, - `${projectDir}/analyses/**/results/**/*.{png,jpg,jpeg,svg,csv,json,md}`, -], { ignoreInitial: true }); - -watcher.on('all', () => { - reload(); - wsBroadcast({ type: 'reload' }); -}); -``` - -On any watched file change, the server reloads the source, rebuilds the page -AST + ASTRA sidecars, refreshes DOI metadata asynchronously, and broadcasts a -WebSocket reload so connected browsers refetch. - -## 5. Sub-analyses - -ASTRA's self-similar structure maps to a multi-page MyST site. Each analysis node becomes its own page. - -**URL structure:** -``` -/ → root analysis -/preprocessing → sub-analysis "preprocessing" -/training → sub-analysis "training" -/training/validation → nested sub-analysis -``` - -**Page generation is recursive:** - -```typescript -function buildAllPages(analysis, universe, results, projectDir, basePath = '') { - const slug = basePath || 'index'; - const pages = [ - page(slug, astraToMystAST({ analysis, universe, results, projectDir, slug })), - ]; - - for (const [id, sub] of Object.entries(analysis.analyses ?? {})) { - const subSlug = basePath ? `${basePath}/${id}` : id; - const subUniverseNode = universe.analyses?.[id]; - const subUniverse = { - id: universe.id, - description: universe.description, - decisions: subUniverseNode?.decisions ?? {}, - analyses: subUniverseNode?.analyses, - }; - pages.push(...buildAllPages(sub, subUniverse, results, projectDir, subSlug)); - } - - return pages; -} -``` - -In the parent page, sub-analyses appear as clickable cards showing the sub-analysis name and its narrative-summary prose. MySTRA deliberately does not synthesize stat strings onto those cards. - -## 6. Live reload flow - -### Agent edits astra.yaml - -``` -1. Agent writes to astra.yaml (or analyses//astra.yaml) -2. chokidar detects the change -3. Content server re-loads the ASTRA source and rebuilds page AST + sidecars -4. DOI metadata refresh is kicked off in the background -5. WebSocket broadcasts { type: "reload" } -6. Browser refetches /content/index.json (and any sidecars it uses) -7. myst-to-react re-renders the updated AST -``` - -### New result produced - -``` -1. A script produces results/baseline/smoothing_stability_figure.png - (or analyses//results/baseline/... for a sub-analysis) -2. chokidar detects the new file -3. Content server re-runs the transform + sidecar builders -4. `/static/...` and `/astra/.json` now resolve to the produced artifact -5. WebSocket reload → browser shows the new figure / table / metric payload inline -``` - -## 7. Technology +## 2. Architecture -| Component | Technology | +### How MyST works + +MyST is two-stage: the **engine** (`myst` CLI) parses source Markdown into an +`mdast` AST and writes content JSON; the **theme** renders that JSON in the +browser. The theme never sees the source files — and in particular **cannot +read `astra.yaml`**. Therefore anything a theme needs must be baked into the +build output by the only thing that reads ASTRA: the plugin. + +Plugins run *in the engine, at build time*. MyST supports three plugin units, +all of which MySTRA uses: + +- **directives** — block-level; one per placed ASTRA component. +- **roles** — inline; references and value interpolation. +- **transforms** — AST mutations run at a named stage. + +The plugin "renderers" unit (shipping interactive render behaviour from a +`.mjs`) is *not implemented* in MyST, so rich *interactive* rendering belongs in +a **theme**, not the plugin. + +``` +astra.yaml + universes/ + results/ index.md (+ sub-analysis pages) + │ (data, single source) │ (composition, single source) + └──────────────┬───────────────────────────┘ + ▼ + MySTRA plugin + · directives → block ASTRA components as stock MyST AST (book-theme baseline) + · roles → inline reference tokens (neutral: class + id + label) + value interpolation + · transforms → anchor grammar; emit the resolved ASTRA store (keyed by id) + ▼ + stock `myst` engine → content JSON → stock content server + ▼ + book-theme (baseline, readable) OR a rich ASTRA theme + · NodeRenderers keyed on astra-* classes + · reads the resolved store → cards, graphs, layouts +``` + +## 3. The ASTRA → MyST AST projection + +The plugin reads the project once per `(root, universe)` (cached) and resolves a +**scope** for each placed component path: it splits leading sub-analysis +segments (`reconstruction.algorithm` → walk into `analyses.reconstruction`) from +the trailing id, narrows the universe to each sub-analysis's selections, and +binds a prose parser so anchor grammar resolves within rendered prose. + +### Directive node mapping + +| Directive | Emitted MyST AST | Identifier carrier | Recognition class | +|---|---|---|---| +| `astra:decision ` | `heading`(h4) + `details`/`summary` + `tabSet`/`tabItem` per option (selected ●) | `decision-` (heading) | `astra-decision` | +| `astra:output ` (figure) | `container`(figure) + `image` + `caption`, then a collapsed provenance `details` | `output-` (container) | `astra-output astra-output--figure` | +| `astra:output ` (table) | `container`(table) + `table` + `caption`, then provenance | `output-` | `astra-output astra-output--table` | +| `astra:output ` (metric/data/report) | inline labelled reference + provenance | `output-` | `astra-output astra-output--` | +| `astra:finding ` | `heading`(h3) + notes + scope + evidence (`:compact:` = claim/notes/scope only) | `finding-` | `astra-finding` | +| `astra:prior-insight ` | `admonition`(seealso): claim + evidence | `prior_insight-` | `astra-prior-insight` | +| `astra:inputs [scope]` | inputs registry `table` (rows carry `input-`) | — | `astra-inputs` | +| `astra:outputs [scope]` | outputs registry `table` (row ids stripped — the rich block owns `output-`) | — | `astra-outputs` | +| `astra:subanalysis ` | `card` linking to the sub-page (`/`) | `analysis-` | `astra-subanalysis` | + +Every block carries its `astra-` class on the node bearing its +`-` identifier (the **recognition marker** — see §5). All AST is +**neutral**: only semantic classes, text, and identifiers — no glyphs, colours, +or inline styles. (Where a `style` is set it is a JS object; a string crashes +the React renderer.) + +### Role node mapping + +| Role | Emitted AST | |---|---| -| Content server | TypeScript + Express | -| ASTRA parsing | `js-yaml` | -| AST construction | TypeScript with `myst-spec` types | -| Citation resolution | `citation-js-utils` (from MyST ecosystem) | -| File watcher | `chokidar` | -| Theme | `myst-theme/book-theme` (unmodified) | -| CSV parsing | `papaparse` | -| Static files | Recursive result basename resolver + Express static fallback | - -```json -{ - "dependencies": { - "js-yaml": "^4.1.0", - "chokidar": "^3.6.0", - "express": "^4.18.0", - "papaparse": "^5.4.0", - "ws": "^8.16.0", - "myst-spec": "^0.0.5", - "citation-js-utils": "^1.2.0" - } -} -``` - -The MyST book-theme is fetched automatically when the theme server starts. - -## 8. CLI - -```bash -mystra [project-dir] # Start MySTRA for the given ASTRA project (default: .) -mystra --port 4000 # Custom theme server port -mystra --universe u001 # View a specific universe (default: first in universes/) -``` - -MySTRA starts two processes: -1. Content server on port 3100 -2. MyST book-theme on port 3000 - -It watches the project directory for changes and keeps the document live. - -## 9. Implementation - -**Transform flow.** `loadASTRASource(projectDir)` parses `astra.yaml`, picks an active universe from `universes/`, and scans both `results//` and nested `analyses/**/results//` directories for produced artifacts. `buildAllPages` walks the analysis tree recursively — one MyST page per node — and `astraToMystAST(source)` produces each page's `root`. The root's `children` are emitted as a flat sequence of addressable blocks: narrative chunks first (in spec-declared order summary → findings → methods → inputs → outputs), then the universe banner, then findings, prior_insights, decisions, the inputs/outputs tables, the per-output provenance + recipe carriers, and sub-analysis cards. There are no programmatic h2 section headings — every structural element sits at the same depth, identified by `-` so themes and downstream renderers compose layout from carriers rather than from spatial position. - -**Render helpers.** Each ASTRA concept has one helper, all in `src/transform/`: - -- `renderNarrativeChunks` (`render-narrative.ts`) — parses each non-empty narrative section to mdast and attaches `narrative-
` to the section's first node. -- `renderUniverseBanner` — `details`/`summary` over a decision-summary table; the universe id and description form the summary line. -- `renderFindings` — flat per-finding blocks. Each finding gets an h3 heading carrying `finding-` (with tags on `data.tags`), notes prose, scope, and evidence. -- `renderPriorInsights` — flat per-insight `container` carriers (kind `prior-insight`, identifier `prior_insight-`, structured `data`, children `[claim, …evidence]`). Minimal carriers — no heading, no separators — because how to surface prior_insights is a renderer's call. -- `renderMethodsSections` — flat per-decision blocks. Each rendered decision is an h4 heading carrying `decision-` followed by a `details` dropdown with rationale and a `tabSet` of options. The selected option (from the active universe) is marked ●; option supporting-insight references emit `crossReference` nodes pointing at the prior_insight carrier. -- `renderInputsTable` / `renderOutputsTable` — one table each; every row carries `input-` / `output-` so anchors land regardless of evidence references. -- `renderOutputProvenance` — one `container.kind=output-provenance` per Output with non-empty resolved `inputs` / `decisions`; closes the provenance leak so renderers never read `astra.yaml` directly. -- `renderOutputRecipes` — one `container.kind=output-recipe` per Output with a non-empty resolved recipe; renderers can pattern-match on `data` or fall back to the shipped `details` block. -- `renderSubAnalysisCards` — one `card` per nested analysis carrying `analysis-` and the sub-analysis's narrative summary. -- `renderEvidenceBlock` (`render-evidence.ts`) — DOI evidence becomes a `cite` (or fallback `link`) with optional quote blockquote; artifact evidence dispatches on the referenced output's `Output.type` (figure → image+caption, table → JSON/CSV table, metric/data/report → labelled inline reference). Broken artifact references emit a `console.warn`. - -**Prose and anchor grammar.** All Markdown content (narrative sections, claims, rationales, descriptions, captions, excluded reasons, finding notes) flows through `myst-parser` via the `ProseParser` interface (`src/transform/narrative-parser.ts`). `parseProseBlocks` returns block-level mdast; `parseProseInline` extracts inline phrasing for table cells, captions, and headings. A `ProseParser` is bound once per page to `(analysis, slug)` and threaded into every render helper, so the v0.0.6 anchor grammar `[t](#path.to.element)` resolves everywhere prose appears: `resolveNarrativeAnchors` walks the parsed tree and rewrites in-scope `link` nodes with `#…` URLs into `crossReference` nodes against the corresponding `-` carrier. - -**Stable id-anchor convention.** Every structural element and narrative chunk gets a deterministic identifier: `decision-`, `finding-`, `prior_insight-`, `input-`, `output-`, `output--provenance`, `output--recipe`, `analysis-`, `narrative-
`. The same identifier is published in `myst.xref.json` and used by the resolver; cross-page anchors (`#analyses..outputs.`) translate to the destination page's URL with the corresponding fragment. - -**The xref contract.** Every identifier published by `collectIdentifiers` has a real carrier in the rendered AST, and vice versa. Decisions that drop out of the page (bare `from`-references, `when`-unmet under the active universe) are filtered with the same predicate the renderer uses; unreferenced prior_insights still get a carrier; outputs that no evidence cites still get a row; provenance/recipe ids publish only when the resolved Output actually has that content. Anchors never land on nothing. - -## 10. DOI enrichment and citations - -MyST's markdown parser auto-resolves DOIs to full citations via doi.org. Since we bypass the parser, we handle this in the content server. - -**Approach:** Import `citation-js-utils` from the MyST ecosystem (MIT-licensed) for citation parsing and rendering. Write a thin DOI fetcher (~30 lines) that requests metadata from `https://doi.org/{doi}` with content negotiation (`Accept: application/x-bibtex`, fallback `application/vnd.citationstyles.csl+json`). Cache results as CSL-JSON on disk. - -```typescript -import { getCitationRenderers } from 'citation-js-utils' - -// At startup: -// 1. Collect all DOIs from prior_insights + findings evidence -// 2. For each DOI not in cache: fetch from doi.org, save as CSL-JSON -// 3. Use getCitationRenderers() to produce formatted HTML -// 4. Build the references object for the page response - -const references = { - cite: { - order: ["Gordon_2003", "Rizzi_2007", ...], - data: { - "Gordon_2003": { - label: "Gordon_2003", - enumerator: "1", - doi: "10.1086/376774", - html: "Gordon, K. D., Clayton, G. C., ... (2003). ApJ, 594(1), 279–293." - } - } - } -} -``` - -This gives us the auto-generated References section and proper citation formatting that the book-theme renders at the bottom of each page. - -**Dependencies:** `citation-js-utils` (from MyST monorepo, published on npm). - -## 11. Open questions - -1. **Tab AST nodes**: Verify that `tabSet`/`tabItem` nodes work correctly when produced programmatically (vs. parsed from MyST markdown). If not, fall back to nested `details`/`summary` elements. - -2. **Public contract for the sidecars**: `/astra/*.json` and `/doi-metadata/:doi(*)` are now real downstream surfaces, not just internal glue. If multiple renderers start depending on them independently, decide whether to version those sidecars explicitly or keep them as adjuncts to the mdast contract. +| `astra:decision|output|finding|prior-insight|analysis [\|display]` | `span.astra-ref.astra-ref--` → `span.astra-ref__label` (label) + `span.astra-card.astra-card--` (hidden preview card) | +| `astra:value col= [k=v …] [pm] [err=] [sig=N]` | `span.astra-ref--value` token whose label is the **interpolated cell** read from the result file, with a card naming source/column/row | + +The preview card is emitted with an inline `display:none`, so a bare viewer +(plugin only, no theme CSS) shows just the clean label and never spills the card +content inline. A theme/stylesheet reveals it on hover. + +The `astra:value` role guarantees **no measured number is hard-typed**: it reads +the materialised CSV/JSON for the output, filters rows by `key=val`, formats the +cell to `sig` significant figures, and appends `± ` when `pm` (uses +`_std`) or `err=` is given. A missing column/row/file renders a clear +inline-code error, never a silent wrong number. + +### Transforms + +- **`astra-anchor-grammar`** (document stage) rewrites ASTRA tree-path anchor + links the author writes directly (`[t](#decisions.x)`, `#outputs.y`, + `#analyses.sub.outputs.z`) into `crossReference`s (same page) or sub-page + links, reusing the narrative parser's resolver. Page scope is derived from the + file basename (`index` → root; `` → the `` sub-analysis). +- **`astra-resolved-store`** (document stage) emits the resolved store (§5). + +## 4. Composition & scope + +The author owns composition: the directive/role vocabulary *is* the degrees of +freedom. The plugin never injects an element the author didn't place. MyST +handles prose, math, figures, numbering, cross-references, the table of +contents, and search. + +Sub-analyses are **separate pages** (`reconstruction.md`, `clustering.md`), +mirroring ASTRA's recursive analysis tree. A component path is `` (root) or +`.` (nested, may chain `a.b.id`); the resolver walks the analyses tree +and the matching sub-universe, accumulating the results base from each +sub-analysis's `path:`. + +## 5. The resolved data store (for rich themes) + +Because the theme cannot read `astra.yaml`, the plugin bakes a **resolved** +projection of each page's analysis scope into the build, keyed by id, on a +hidden carrier node: + +``` +{ type: 'div', class: 'astra-store', identifier: 'astra-store', + style: { display: 'none' }, data: { astra: ResolvedStore }, children: [] } +``` + +`ResolvedStore` (`src/transform/resolved-store.ts`) contains, all keyed by id: + +- **outputs** — resolved `type`, `label`, `description`, **project-relative** + `resolved_path`, `recipe` (command/container), `inputs`, `decisions`, `from`, + and inlined `table_data` (parsed rows for tables) / `metric` (scalar/tuple/ + object for metrics). `from:` chains are walked, so the view is resolved. +- **inputs** — with aliased (`from:`) inputs resolved against ancestor scopes. +- **decisions** — `rationale`, all `options`, and the `selected` option under + the active universe. +- **findings**, **prior_insights** (claim + first DOI/quote), **subanalyses** + (name, summary, page url, decision/output counts). + +A rich theme selects a placed node by its `identifier`/`astra-*` class and joins +it to the matching store entry (the proven `mergeRenderers` + +`unist-util-select` pattern) — recognition + data without re-reading +`astra.yaml` and without per-node duplication. + +**Delivery is verified.** Node `data` survives the engine's serialization into +content JSON (confirmed by building the `prototype/` project: the `astra-store` +carrier and its `data.astra` appear intact in `content/index.json`, scoped per +page). MyST's asset pipeline rewrites the plugin's project-relative image paths +into hashed copied assets. + +## 6. Citations + +Delegated to MyST, which resolves DOIs and renders the reference list natively. +MySTRA carries **no DOI subsystem** of its own: DOI evidence renders as a plain +`doi.org` link, and inline cards show the bare DOI as their citation hint. +Wiring a project bibliography — so MyST renders author–year labels and a linked +reference list — is the clean follow-up. + +## 7. The ASTRA data model + +MySTRA leans on the official **`@astra-spec/sdk`** package, not just for the +data model but for the spec mechanics it would otherwise re-implement: +- **types** — imported directly by their SDK names (`Analysis`, `Decision`, + `Output`, …; `import type`, erased at runtime). MySTRA defines none of its own + and has no `types/` module; +- **YAML + tree resolution** — `src/loader.ts` parses `astra.yaml` and inlines + `path:` sub-analyses with the SDK's `loadYaml` + `resolveAnalysisTree` (which + preserves each sub's `path:`, so its results base stays computable); +- **`when`-condition evaluation** — `isConditionMet` (used to decide whether a + decision renders) is the SDK's, not a local copy. + +The SDK is the single source of truth — bump it with `npm update` rather than +editing by hand. (MySTRA keeps `js-yaml` out entirely; the SDK's `yaml` is the +one parser.) An **Analysis** is recursive +(`analyses`), with `inputs`, `outputs`, `decisions`, `prior_insights`, +`findings`, and a five-section `narrative`. **Output** is the unit of provenance +(`inputs`, `decisions`, `recipe`; `from:` re-export aliases inherit the source's +fields). **Findings** and **prior_insights** share the `Insight` model (`claim`, +`evidence`, `scope`, `label`). A **Universe** selects one option per decision and +nests selections for sub-analyses; `src/loader.ts` loads one (by `name`, else the +first under `universes/`). Result artifacts are **not** scanned — each output's file is +resolved on demand from the deterministic convention +`[/]results///`, preferring `.` (the +recipe-chosen extension is the only part not fixed by the spec). See §7.1. + +### 7.1 Output-path resolution + +astra-spec leaves an output's on-disk path to the runner (`{output}` is a recipe +placeholder). lightcone-cli, the runner, fixes the *directory*: +`[/]results///` — so MySTRA computes it +deterministically from the analysis's `path:` chain, the universe, and the +output id, and never scans the results tree or guesses ids from filenames. The +one thing the spec doesn't fix is the file *name* inside that directory (the +recipe writes it), so `resolveArtifact` (`src/loader.ts`) reads that one +directory, preferring `.`, else the first regular file (dotfiles — +including lightcone's `.lightcone-manifest.json` — skipped). Resolution is lazy: +each scope carries an `ArtifactResolver` bound to its results base + universe, +and an output is resolved only when actually rendered. + +## 8. Non-goals + +- No bespoke content server, live data sidecar, or per-node data duplication — + one store, referenced by id; `myst start` provides serving and reload. +- No baked presentation in the AST (beyond the minimal default-hide for the + theme-only preview card). +- No speculative pattern widgets (DAG, gallery) until an author needs the + directive; the store is what makes them possible. +- Rich interactive rendering lives in a separate `lightcone-astra` theme, not + the plugin. + +## 9. Open questions + +- **`astra.yaml` live reload.** `myst start` watches Markdown, not `astra.yaml`, + and the plugin caches the parse; editing the spec needs a restart. A watch + hook or cache invalidation is a possible follow-up. +- **Citation bibliography.** Emit a generated `references.bib` / project + bibliography so MyST links the reference list. +- **Packaging & the theme.** Publish the plugin (working name + `@lightcone/astra-myst`) and build the `lightcone-astra` theme (its stylesheet + seeded by `prototype/custom.css`). diff --git a/STRATEGY-A-REFACTOR.md b/STRATEGY-A-REFACTOR.md new file mode 100644 index 0000000..c8738b2 --- /dev/null +++ b/STRATEGY-A-REFACTOR.md @@ -0,0 +1,297 @@ +# MySTRA → Strategy A: refactor plan + +> Status: **landed** (everything except the separate `lightcone-astra` theme, +> §9). Supersedes the architecture in `SPEC.md` (the generate-everything content +> server); `SPEC.md` has been rewritten to the implemented design. This document +> remains the rationale and the keep/refactor/remove record. + +## 1. Context & motivation + +MySTRA today (the `main` branch) is a **generate-everything content server**: +it loads an ASTRA project, transforms the *entire* analysis into MyST AST +(`transform/index.ts: buildAllPages`/`astraToMystAST`), and runs a **bespoke +Express content server** (`src/server/`) that serves `config.json`, +`content/*.json`, `myst.xref.json`, plus custom sidecars (`/astra/*.json`, +`/doi-metadata`, `/papers`, `/static`) and a WebSocket reload, to an unmodified +book-theme. + +A review early in this effort found that approach **partly overengineered for +its stated goal** ("AST any theme server can render"): the custom container +kinds and the `/astra` sidecar are a *parallel data contract* aimed at a +specific bespoke renderer, and the custom server re-implements a slice of what +the MyST engine already does. It works, but it fights MyST's grain and carries +a large surface area. + +**Strategy A** inverts the model. Instead of generating the document, the +human/agent writes a normal **MyST Markdown report** and *imports/cites* ASTRA +components by reference. A single **MyST plugin** (directives + roles + a +transform) reads `astra.yaml` at build time and emits standard MyST AST. It runs +on the **stock `myst` CLI and themes** — no custom server. We validated this +end-to-end on a real DESI DR1 BAO analysis (`prototype/`). + +### The governing principle: three single-sources + +| Concern | Single source of truth | +|---|---| +| **Data** — what a decision/output/finding *is* | `astra.yaml` (+ `universes/`, `results/`) | +| **Composition** — what appears, where, in what order, which view | the author's `index.md` | +| **Presentation** — how it looks | the theme | + +The plugin is a pure **projector**: it reads the data and emits (a) rendered +semantic AST for the elements the author placed (the baseline any theme shows) +and (b) — for rich themes — the resolved ASTRA data. It makes no authoring or +styling decisions. Everything in the build output is a *derived projection*, not +an authored duplicate. + +## 2. What we learned (decisions this plan rests on) + +1. **MyST is two-stage.** The engine parses source → AST; the theme renders the + AST. The content server only *serves JSON* and the theme **cannot read + `astra.yaml`**. ⇒ anything a theme needs must be baked into the build output + by the plugin (the only thing that reads ASTRA). +2. **Plugins run in the engine at build time.** Directives/roles/transforms are + supported and sufficient. The plugin "**renderers**" type (shipping render + behavior from a `.mjs`) is *not implemented* ⇒ custom *interactive* rendering + must live in a **theme**, not the plugin. +3. **Block directives degrade for free; inline cards do not.** `:::{astra:*}` + directives emit stock nodes (`container[figure]`, `table`, `details`, + `admonition`, `card`) that every theme renders. Inline hover *cards* are a + presentation behavior ⇒ they belong to the theme, not the plugin/AST. +4. **Keep the AST neutral.** Emit only **semantic classes + text + identifiers** + (no glyphs, colours, or inline styles baked in). Appearance is the theme's. + (`style` must be a JS object if ever set — a string crashes the React + renderer.) +5. **Normalize to avoid duplication.** Placed nodes carry an **identifier + + kind** (and the author's view options); the resolved ASTRA data is conveyed + **once**, keyed by id; a rich theme joins node-id → data. The only remaining + duplication is the intentional rendered-fallback for the baseline theme, + single-sourced in one build pass. +6. **Authoring control = placement.** The directive/role vocabulary *is* the + author's degrees of freedom. The theme may change *how* a placed element + renders and may *fill* author-placed pattern directives (`:::{astra:dag}`) + from the data — but it must never inject elements the author didn't place. +7. **Recognition needs an explicit marker.** A theme recognizes an ASTRA element + via a stable `class`/`identifier` (e.g. `output-`, `class="astra-output"`) + — not a fragile id-prefix convention alone — and then can re-render it + dynamically *if* it can reach the data (point 5). +8. **Numbers must be sourced, not typed.** The `{astra:value}` role interpolates + real cells from result files at build time; no measured number is hand-typed + in prose. +9. **MyST already does the rest.** `myst start` gives build + content server + + theme + live reload. MyST's asset pipeline copies/【hashes】 result images. + MyST resolves DOIs natively ⇒ the bespoke DOI/paper subsystem can largely go. +10. **The Vellum mocks are the design north-star** (narrative-first prose, inline + glyph tokens, focused preview cards — graphs optional, author-placed). + +## 3. Target architecture + +``` +astra.yaml + universes/ + results/ index.md (+ sub-analysis pages) + │ (data, single source) │ (composition, single source) + └──────────────┬───────────────────────────┘ + ▼ + @lightcone/astra-myst (the plugin, npm) + · directives → block ASTRA components as stock MyST AST (book-theme baseline) + · roles → inline reference tokens (neutral: class + id + label) + · transform → emit the resolved ASTRA data store, once, keyed by id + ▼ + stock `myst` engine → content JSON → stock content server + ▼ + book-theme (baseline, readable) OR lightcone-astra theme (rich) + · NodeRenderers keyed on astra classes + · reads the resolved store → cards, DAGs, widgets +``` + +Two render modes, by theme choice only: + +- **Basic** — register the plugin in `myst.yml`, `template: book-theme`. Clean, + readable: real figures/tables/dropdowns, live numbers, plain-text inline + references. **Nothing else required** (no user CSS). +- **Rich** — `template: lightcone-astra`. Glyphs, colours, hover preview cards, + and "powerful patterns" (e.g. a product-dependency graph), all driven from the + resolved store. The user's only change is the one `template:` line. + +## 4. The refactor — keep / refactor / remove + +The lower layers are reused; the server/orchestration layer is removed. + +### Keep (becomes the core of the package) +- `src/loader.ts` — one file. Loads a project for one universe via the SDK + (`loadYaml` + `resolveAnalysisTree`), picks the universe, and resolves result + artifacts deterministically on demand from the lightcone-cli path convention + `[/]results///` (no scanning). Replaces the old + three-file `src/loader/` + `result-scanner`. +- **No data-model types of our own.** They're imported directly by their + `@astra-spec/sdk` names (`Analysis`, `Decision`, …; `import type`). The + hand-mirrored `src/types/astra.ts`, the alias module that briefly replaced it, + and the `schema-coverage` guard test are all gone — the SDK is the single + source of truth, pinned in `package.json`. +- `src/transform/` per-component renderers actually used by the plugin: + `ast-helpers`, `narrative-parser` (prose + anchor grammar), `parse-table-data`, + `resolve-output`, `render-methods` (renderDecision), `render-findings` + (renderFinding), `render-evidence` (renderOneOutput / evidence / tables), + `render-data-sources` (inputs/outputs tables). +- `src/index.ts` — the plugin itself *is* the package entry (default export); no + `plugin/` subfolder, no separate thin entry. + +### Refactor +- **Move scope-resolution into the plugin** (already done there): the recursive + sub-analysis walk + sub-universe selection currently in `transform/index.ts: + buildAllPages` is replaced by the plugin's `resolveScope`. Delete the + whole-page orchestration; keep only the per-component helpers. +- **The `/astra` sidecar logic → the resolved store builder.** Reuse the + `SerializedOutput`/`SerializedInput` resolution (from `server/routes/astra.ts`: + `resolveOutputs`, `table_data`, `readMetric`, input alias resolution) but emit + it as a **build artifact** (see §5), not a server endpoint. This is the one + substantive piece of `src/server/` worth salvaging. +- **Package shape.** Ship as an npm package (working name `@lightcone/astra-myst`) + whose entry is the compiled plugin `.mjs`. `myst.yml` references it by name + once published, or by local path during development. Optionally a tiny CLI + remains only for *offline* tasks (warm DOI cache / pre-generate the store) — + never for serving. + +### Remove (remnants of the generate-everything approach) +- `src/server/` — the entire bespoke content server: `index.ts`, `routes/*`, + `watcher.ts`, `websocket.ts`. Replaced by the stock `myst` engine + server + + live reload. +- `src/theme/launcher.ts` — `myst start` launches the theme. +- `src/cli.ts` — the `mystra` two-server boot CLI. (Optionally replaced by a + minimal offline-only CLI; otherwise gone — users run `myst`.) +- `src/transform/index.ts` — `buildAllPages` / `astraToMystAST` whole-document + orchestration and its flat-block page assembly. +- `src/types/content-server.ts` — `PageData` / `SiteManifest` / `PageContent` + (the server API surface). +- `src/utils/hash.ts` — content sha256 for the server's cache invalidation. +- **Audit-and-remove** per-component renderers that only served the generated + document and aren't invoked by any directive: `render-narrative` (the author + now writes narrative as Markdown), `render-universe-banner`, + `render-sub-analyses`, `render-output-recipe`, `render-output-provenance`, + `render-prior-insights` (the plugin builds the prior-insight admonition itself). + Remove each once `tsc`/tests confirm it has no remaining caller. Their + *semantics* (provenance, recipe) live on in the resolved store, not as bespoke + container kinds. +- `src/papers/` and most of `src/doi/` — see §6 (citations). +- Tests tied to the old surface: `tests/server-routes.test.ts`, + `tests/page-shape.test.ts`. Replace with plugin-emission tests (§8). + +## 5. The resolved data layer (for rich themes) + +Goal: give a theme the **complete, resolved** ASTRA model so it can do anything +(recognition, cards, dependency graphs, alternative layouts) **without reading +`astra.yaml` and without per-node duplication**. + +- **Content:** the plugin's *resolved* model (universes applied, `from:` chains + resolved, result paths, parsed table/metric values) — i.e. the + `SerializedOutput`/`SerializedInput` shape, extended to decisions/findings/ + prior-insights/sub-analyses, **keyed by id**. Resolved, not raw YAML, so the + theme never re-implements ASTRA semantics. +- **Delivery (decide during implementation; prefer the simplest that survives + the MyST pipeline):** + 1. a single **site-global static JSON** emitted by the build (closest to the + old `/astra/*.json`, but baked, not served live); the theme loads it once + into a provider — **preferred**, because powerful patterns are global; or + 2. attach the resolved tree to **node `data`** on a dedicated root-level + carrier (we know `data` survives to the theme); or + 3. page **frontmatter** scoped per page (verify custom-key passthrough first). +- **Recognition markers (do now, cheap, harmless to book-theme):** every placed + ASTRA node carries a stable `class` (`astra-output`/`astra-output--figure`, + `astra-decision`, …) and keeps its `identifier` (`output-`). A theme + selects on these (the proven `mergeRenderers` + `unist-util-select` pattern) and + joins id → store. +- **Non-goal:** do **not** rebuild a live sidecar server, and do **not** embed a + full data copy on every node. One store, referenced by id. + +## 6. Citations — lean into MyST + +MyST resolves DOIs natively. Retire MySTRA's bespoke DOI resolver/fetcher/cache +and the paper-backlink subsystem (`src/papers/`, `src/doi/`): emit citations in a +form MyST resolves (DOIs / a generated `references.bib`) and let the theme render +the reference list. **As implemented, `src/doi/` was removed entirely** — the +cache-read hedge had no writer left (the fetcher was gone), so DOI evidence now +renders as a plain `doi.org` link and inline cards show the bare DOI. This +removed a whole subsystem and cleared the "Could not link citation" warnings the +prototype's stale cache had been producing. Author–year text + a linked +reference list return once a project bibliography is wired (the follow-up). + +## 7. Authoring contract (the directive/role vocabulary) + +This vocabulary *is* the author's compositional surface; extend it to add freedom +rather than adding theme magic. + +- **Block "import":** `astra:decision`, `astra:output`, `astra:finding`, + `astra:prior-insight`, `astra:inputs`, `astra:outputs`, `astra:subanalysis` + (+ future patterns like `astra:dag`, `astra:gallery` — empty author-placed + hooks the theme fills from the store). +- **Inline "cite":** `astra:decision|output|finding|prior-insight|analysis` — + neutral glyph-free tokens (`span.astra-ref--` + label + id), optional + `|display text`; and `astra:value` for sourced numbers. +- **View options** on directives (`:as:`, `:view:`, `:compact:`) express the + author's chosen rendering; the theme honours them. +- All Markdown prose, anchor grammar (`[t](#decisions.x)`), math, figures, TOC, + multi-page sub-analyses are plain MyST. + +## 8. Simplicity principles / explicit non-goals + +- **Lean on MyST for everything it already does** — serving, asset hashing, live + reload, numbering, xref, search, citations. Write only the ASTRA→AST bridge. +- **Prefer stock node types.** Use `container`/`details`/`table`/`admonition`/ + `card`/`span`; introduce a custom node type only when a stock one cannot carry + the intent. Custom *kinds* are fine as long as children render on book-theme. +- **No baked presentation in the AST** (no glyphs/colours/inline styles, beyond + the minimal default-hide for any theme-only affordance). +- **No parallel live data server, no per-node data duplication.** +- **Don't pre-build speculative patterns** (DAG, gallery) until an author needs + the directive; ship the data layer that makes them possible, not the widgets. +- Delete aggressively: every file in §4-Remove should be gone, with `tsc` and + tests green, before calling the refactor done. + +## 9. The `lightcone-astra` theme (separate deliverable) + +Out of scope for the package refactor, tracked here for completeness. A MyST +theme (extends `book`/`article` theme) that: registers `NodeRenderer`s keyed on +the `astra-*` classes/identifiers; reveals the inline preview cards (built from +the store, not from hidden AST spans); renders the rich figure/decision/insight +treatments and any author-placed patterns. Start as a **light** theme (base theme ++ bundled stylesheet) since block content already renders; graduate to custom +React renderers for true popovers/graphs. The prototype's `custom.css` is the +seed of its stylesheet. Until it exists, book-theme is the (clean) baseline. + +## 10. Suggested phasing + +1. **Snapshot** the prototype as the reference behaviour; capture screenshots. +2. **Restructure** the package around the plugin; add §5 recognition markers + (classes) to every emitted element. +3. **Remove** the server, CLI, theme launcher, whole-page transform, and + content-server types (§4-Remove); make `tsc` + tests green. +4. **Salvage** the `/astra` resolution into the resolved-store builder (§5) and + emit the store; verify the delivery channel. +5. **Prune** unused render-* and `utils/hash`; audit with the compiler. +6. **Citations**: retire the DOI/paper subsystem in favour of MyST-native (§6). +7. **Docs**: rewrite `README.md` + `SPEC.md` to Strategy A; document the + authoring vocabulary and the theme contract. +8. **Theme**: build `lightcone-astra` (separate package). + +## 11. Verification + +- `npm run build` clean (no dead imports after removals). +- New tests assert **plugin emission** per directive/role (decision → + `details`+`tabSet`; output figure → `container[figure]` + relative image url + + `output-` + `astra-output` class; `astra:value` → correct interpolated + cell; sub-analysis scope resolution) and the **resolved store** shape. +- `cd prototype && npx mystmd start` (book-theme, **no `custom.css`**) → + document is clean and readable; figures load; numbers are live; no inline + clutter; no `astra.yaml` read by the server/theme. +- Confirm a theme can select `container[identifier^="output-"]` / + `.astra-output` and reach the store by id (recognition guarantee). + +## 12. Open questions + +- ~~**Store delivery channel**~~ **(resolved)** — a hidden `div.astra-store` + carrier with the store on node `data` survives the engine's content-JSON + serialization intact (verified by building `prototype/`: `data.astra` is + present in `content/index.json`, scoped per page). This is the channel. +- **`astra.yaml` live reload**: the plugin caches the parse; `myst start` watches + Markdown, not `astra.yaml`. Decide between a watch hook, cache invalidation, or + "restart to pick up data changes" (acceptable for now). +- **Package/name/publishing** for the plugin and the theme template. +- How much of `SPEC.md` to preserve vs rewrite (lean: rewrite). diff --git a/package-lock.json b/package-lock.json index bacc251..2bf2046 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,104 +1,39 @@ { "name": "mystra", - "version": "0.1.0", + "version": "0.2.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "mystra", - "version": "0.1.0", + "version": "0.2.0", "license": "MIT", "dependencies": { - "chokidar": "^3.6.0", - "citation-js-utils": "^1.2.8", - "commander": "^12.1.0", - "cors": "^2.8.5", - "express": "^4.21.0", - "js-yaml": "^4.1.0", + "@astra-spec/sdk": "^0.0.3", "myst-parser": "^1.7.1", "myst-spec": "^0.0.5", - "papaparse": "^5.4.0", - "ws": "^8.18.0" - }, - "bin": { - "mystra": "dist/cli.js" + "papaparse": "^5.4.0" }, "devDependencies": { - "@types/cors": "^2.8.0", - "@types/express": "^4.17.0", - "@types/js-yaml": "^4.0.0", "@types/node": "^20.0.0", "@types/papaparse": "^5.3.0", - "@types/ws": "^8.5.0", "tsx": "^4.0.0", "typescript": "^5.5.0", "vitest": "4.1.6" } }, - "node_modules/@citation-js/core": { - "version": "0.7.21", - "resolved": "https://registry.npmjs.org/@citation-js/core/-/core-0.7.21.tgz", - "integrity": "sha512-Vobv2/Yfnn6C6BVO/pvj7madQ7Mfzl83/jAWwixbemGF6ZThhGMz8++FD9hWHyHXDMYuLGa6fK68c2VsolZmTA==", - "license": "MIT", - "dependencies": { - "@citation-js/date": "^0.5.0", - "@citation-js/name": "^0.4.2", - "fetch-ponyfill": "^7.1.0", - "sync-fetch": "^0.4.1" - }, - "engines": { - "node": ">=16.0.0" - } - }, - "node_modules/@citation-js/date": { - "version": "0.5.1", - "resolved": "https://registry.npmjs.org/@citation-js/date/-/date-0.5.1.tgz", - "integrity": "sha512-1iDKAZ4ie48PVhovsOXQ+C6o55dWJloXqtznnnKy6CltJBQLIuLLuUqa8zlIvma0ZigjVjgDUhnVaNU1MErtZw==", - "license": "MIT", - "engines": { - "node": ">=10.0.0" - } - }, - "node_modules/@citation-js/name": { - "version": "0.4.2", - "resolved": "https://registry.npmjs.org/@citation-js/name/-/name-0.4.2.tgz", - "integrity": "sha512-brSPsjs2fOVzSnARLKu0qncn6suWjHVQtrqSUrnqyaRH95r/Ad4wPF5EsoWr+Dx8HzkCGb/ogmoAzfCsqlTwTQ==", - "license": "MIT", - "engines": { - "node": ">=6" - } - }, - "node_modules/@citation-js/plugin-bibtex": { - "version": "0.7.21", - "resolved": "https://registry.npmjs.org/@citation-js/plugin-bibtex/-/plugin-bibtex-0.7.21.tgz", - "integrity": "sha512-O008pSsJgiYKn4+7gAWrbNpNdUH++aMeYmZaJ2oFQ8X1tcY5jNBxJcr0zZojNtUi5CVOaXXHQ0yIifoUhuF2Vg==", - "license": "MIT", - "dependencies": { - "@citation-js/date": "^0.5.0", - "@citation-js/name": "^0.4.2", - "moo": "^0.5.1" - }, - "engines": { - "node": ">=16.0.0" - }, - "peerDependencies": { - "@citation-js/core": "^0.7.0" - } - }, - "node_modules/@citation-js/plugin-csl": { - "version": "0.7.22", - "resolved": "https://registry.npmjs.org/@citation-js/plugin-csl/-/plugin-csl-0.7.22.tgz", - "integrity": "sha512-/rGdtbeP3nS4uZDdEbQUHT8PrUcIs0da2t+sWMKYXoOhXQqfw3oJJ7p4tUD+R8lptyIR5Eq20/DFk/kQDdLpYg==", - "license": "MIT", + "node_modules/@astra-spec/sdk": { + "version": "0.0.3", + "resolved": "https://registry.npmjs.org/@astra-spec/sdk/-/sdk-0.0.3.tgz", + "integrity": "sha512-WboQVD5v+520IFbTh7YWZwEDYe4vf7LW3ABpd+anXO1onjy0ibpsYI8Imi2rJonaEcERF8jwUDcWJlNODYN0hA==", + "license": "BSD-3-Clause", "dependencies": { - "@citation-js/date": "^0.5.0", - "citeproc": "^2.4.6" + "ajv": "^8.17.1", + "ajv-formats": "^3.0.1", + "yaml": "^2.6.1" }, "engines": { - "node": ">=16.0.0" - }, - "peerDependencies": { - "@citation-js/core": "^0.7.0" + "node": ">=18" } }, "node_modules/@emnapi/core": { @@ -913,17 +848,6 @@ "tslib": "^2.4.0" } }, - "node_modules/@types/body-parser": { - "version": "1.19.6", - "resolved": "https://registry.npmjs.org/@types/body-parser/-/body-parser-1.19.6.tgz", - "integrity": "sha512-HLFeCYgz89uk22N5Qg3dvGvsv46B8GLvKKo1zKG4NybA8U2DiEO3w9lqGg29t/tfLRJpJ6iQxnVw4OnB7MoM9g==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/connect": "*", - "@types/node": "*" - } - }, "node_modules/@types/chai": { "version": "5.2.3", "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz", @@ -935,26 +859,6 @@ "assertion-error": "^2.0.1" } }, - "node_modules/@types/connect": { - "version": "3.4.38", - "resolved": "https://registry.npmjs.org/@types/connect/-/connect-3.4.38.tgz", - "integrity": "sha512-K6uROf1LD88uDQqJCktA4yzL1YYAK6NgfsI0v/mTgyPKWsX1CnJ0XPSDhViejru1GcRkLWb8RlzFYJRqGUbaug==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/node": "*" - } - }, - "node_modules/@types/cors": { - "version": "2.8.19", - "resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.19.tgz", - "integrity": "sha512-mFNylyeyqN93lfe/9CSxOGREz8cpzAhH+E93xJ4xWQf62V8sQ/24reV2nyzUWM6H6Xji+GGHpkbLe7pVoUEskg==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/node": "*" - } - }, "node_modules/@types/deep-eql": { "version": "4.0.2", "resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz", @@ -969,46 +873,6 @@ "dev": true, "license": "MIT" }, - "node_modules/@types/express": { - "version": "4.17.25", - "resolved": "https://registry.npmjs.org/@types/express/-/express-4.17.25.tgz", - "integrity": "sha512-dVd04UKsfpINUnK0yBoYHDF3xu7xVH4BuDotC/xGuycx4CgbP48X/KF/586bcObxT0HENHXEU8Nqtu6NR+eKhw==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/body-parser": "*", - "@types/express-serve-static-core": "^4.17.33", - "@types/qs": "*", - "@types/serve-static": "^1" - } - }, - "node_modules/@types/express-serve-static-core": { - "version": "4.19.8", - "resolved": "https://registry.npmjs.org/@types/express-serve-static-core/-/express-serve-static-core-4.19.8.tgz", - "integrity": "sha512-02S5fmqeoKzVZCHPZid4b8JH2eM5HzQLZWN2FohQEy/0eXTq8VXZfSN6Pcr3F6N9R/vNrj7cpgbhjie6m/1tCA==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/node": "*", - "@types/qs": "*", - "@types/range-parser": "*", - "@types/send": "*" - } - }, - "node_modules/@types/http-errors": { - "version": "2.0.5", - "resolved": "https://registry.npmjs.org/@types/http-errors/-/http-errors-2.0.5.tgz", - "integrity": "sha512-r8Tayk8HJnX0FztbZN7oVqGccWgw98T/0neJphO91KkmOzug1KkofZURD4UaD5uH8AqcFLfdPErnBod0u71/qg==", - "dev": true, - "license": "MIT" - }, - "node_modules/@types/js-yaml": { - "version": "4.0.9", - "resolved": "https://registry.npmjs.org/@types/js-yaml/-/js-yaml-4.0.9.tgz", - "integrity": "sha512-k4MGaQl5TGo/iipqb2UDG2UwjXziSWkh0uysQelTlJpX1qGlpUZYm8PnO4DxG1qBomtJUdYJ6qR6xdIah10JLg==", - "dev": true, - "license": "MIT" - }, "node_modules/@types/linkify-it": { "version": "3.0.5", "resolved": "https://registry.npmjs.org/@types/linkify-it/-/linkify-it-3.0.5.tgz", @@ -1031,13 +895,6 @@ "integrity": "sha512-6L6VymKTzYSrEf4Nev4Xa1LCHKrlTlYCBMTlQKFuddo1CvQcE52I0mwfOJayueUC7MJuXOeHTcIU683lzd0cUA==", "license": "MIT" }, - "node_modules/@types/mime": { - "version": "1.3.5", - "resolved": "https://registry.npmjs.org/@types/mime/-/mime-1.3.5.tgz", - "integrity": "sha512-/pyBZWSLD2n0dcHE3hq8s8ZvcETHtEuF+3E7XVt0Ig2nvsVQXdghHVcEkIWjy9A0wKfTn97a/PSDYohKIlnP/w==", - "dev": true, - "license": "MIT" - }, "node_modules/@types/node": { "version": "20.19.37", "resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.37.tgz", @@ -1058,69 +915,12 @@ "@types/node": "*" } }, - "node_modules/@types/qs": { - "version": "6.15.0", - "resolved": "https://registry.npmjs.org/@types/qs/-/qs-6.15.0.tgz", - "integrity": "sha512-JawvT8iBVWpzTrz3EGw9BTQFg3BQNmwERdKE22vlTxawwtbyUSlMppvZYKLZzB5zgACXdXxbD3m1bXaMqP/9ow==", - "dev": true, - "license": "MIT" - }, - "node_modules/@types/range-parser": { - "version": "1.2.7", - "resolved": "https://registry.npmjs.org/@types/range-parser/-/range-parser-1.2.7.tgz", - "integrity": "sha512-hKormJbkJqzQGhziax5PItDUTMAM9uE2XXQmM37dyd4hVM+5aVl7oVxMVUiVQn2oCQFN/LKCZdvSM0pFRqbSmQ==", - "dev": true, - "license": "MIT" - }, - "node_modules/@types/send": { - "version": "1.2.1", - "resolved": "https://registry.npmjs.org/@types/send/-/send-1.2.1.tgz", - "integrity": "sha512-arsCikDvlU99zl1g69TcAB3mzZPpxgw0UQnaHeC1Nwb015xp8bknZv5rIfri9xTOcMuaVgvabfIRA7PSZVuZIQ==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/node": "*" - } - }, - "node_modules/@types/serve-static": { - "version": "1.15.10", - "resolved": "https://registry.npmjs.org/@types/serve-static/-/serve-static-1.15.10.tgz", - "integrity": "sha512-tRs1dB+g8Itk72rlSI2ZrW6vZg0YrLI81iQSTkMmOqnqCaNr/8Ek4VwWcN5vZgCYWbg/JJSGBlUaYGAOP73qBw==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/http-errors": "*", - "@types/node": "*", - "@types/send": "<1" - } - }, - "node_modules/@types/serve-static/node_modules/@types/send": { - "version": "0.17.6", - "resolved": "https://registry.npmjs.org/@types/send/-/send-0.17.6.tgz", - "integrity": "sha512-Uqt8rPBE8SY0RK8JB1EzVOIZ32uqy8HwdxCnoCOsYrvnswqmFZ/k+9Ikidlk/ImhsdvBsloHbAlewb2IEBV/Og==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/mime": "^1", - "@types/node": "*" - } - }, "node_modules/@types/unist": { "version": "2.0.11", "resolved": "https://registry.npmjs.org/@types/unist/-/unist-2.0.11.tgz", "integrity": "sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA==", "license": "MIT" }, - "node_modules/@types/ws": { - "version": "8.18.1", - "resolved": "https://registry.npmjs.org/@types/ws/-/ws-8.18.1.tgz", - "integrity": "sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/node": "*" - } - }, "node_modules/@vitest/expect": { "version": "4.1.6", "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.6.tgz", @@ -1234,30 +1034,37 @@ "url": "https://opencollective.com/vitest" } }, - "node_modules/accepts": { - "version": "1.3.8", - "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz", - "integrity": "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw==", + "node_modules/ajv": { + "version": "8.20.0", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz", + "integrity": "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA==", "license": "MIT", "dependencies": { - "mime-types": "~2.1.34", - "negotiator": "0.6.3" + "fast-deep-equal": "^3.1.3", + "fast-uri": "^3.0.1", + "json-schema-traverse": "^1.0.0", + "require-from-string": "^2.0.2" }, - "engines": { - "node": ">= 0.6" + "funding": { + "type": "github", + "url": "https://github.com/sponsors/epoberezkin" } }, - "node_modules/anymatch": { - "version": "3.1.3", - "resolved": "https://registry.npmjs.org/anymatch/-/anymatch-3.1.3.tgz", - "integrity": "sha512-KMReFUr0B4t+D+OBkjR3KYqvocp2XaSzO55UcB6mgQMd3KbcE+mWTyvVV7D/zsdEbNnV6acZUutkiHQXvTr1Rw==", - "license": "ISC", + "node_modules/ajv-formats": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/ajv-formats/-/ajv-formats-3.0.1.tgz", + "integrity": "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ==", + "license": "MIT", "dependencies": { - "normalize-path": "^3.0.0", - "picomatch": "^2.0.4" + "ajv": "^8.0.0" }, - "engines": { - "node": ">= 8" + "peerDependencies": { + "ajv": "^8.0.0" + }, + "peerDependenciesMeta": { + "ajv": { + "optional": true + } } }, "node_modules/argparse": { @@ -1266,12 +1073,6 @@ "integrity": "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==", "license": "Python-2.0" }, - "node_modules/array-flatten": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz", - "integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==", - "license": "MIT" - }, "node_modules/assertion-error": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz", @@ -1292,142 +1093,12 @@ "url": "https://github.com/sponsors/wooorm" } }, - "node_modules/base64-js": { - "version": "1.5.1", - "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", - "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/feross" - }, - { - "type": "patreon", - "url": "https://www.patreon.com/feross" - }, - { - "type": "consulting", - "url": "https://feross.org/support" - } - ], - "license": "MIT" - }, - "node_modules/binary-extensions": { - "version": "2.3.0", - "resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.3.0.tgz", - "integrity": "sha512-Ceh+7ox5qe7LJuLHoY0feh3pHuUDHAcRUeyL2VYghZwfpkNIy/+8Ocg0a3UuSoYzavmylwuLWQOf3hl0jjMMIw==", - "license": "MIT", - "engines": { - "node": ">=8" - }, - "funding": { - "url": "https://github.com/sponsors/sindresorhus" - } - }, - "node_modules/body-parser": { - "version": "1.20.4", - "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.4.tgz", - "integrity": "sha512-ZTgYYLMOXY9qKU/57FAo8F+HA2dGX7bqGc71txDRC1rS4frdFI5R7NhluHxH6M0YItAP0sHB4uqAOcYKxO6uGA==", - "license": "MIT", - "dependencies": { - "bytes": "~3.1.2", - "content-type": "~1.0.5", - "debug": "2.6.9", - "depd": "2.0.0", - "destroy": "~1.2.0", - "http-errors": "~2.0.1", - "iconv-lite": "~0.4.24", - "on-finished": "~2.4.1", - "qs": "~6.14.0", - "raw-body": "~2.5.3", - "type-is": "~1.6.18", - "unpipe": "~1.0.0" - }, - "engines": { - "node": ">= 0.8", - "npm": "1.2.8000 || >= 1.4.16" - } - }, "node_modules/boolbase": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz", "integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==", "license": "ISC" }, - "node_modules/braces": { - "version": "3.0.3", - "resolved": "https://registry.npmjs.org/braces/-/braces-3.0.3.tgz", - "integrity": "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==", - "license": "MIT", - "dependencies": { - "fill-range": "^7.1.1" - }, - "engines": { - "node": ">=8" - } - }, - "node_modules/buffer": { - "version": "5.7.1", - "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz", - "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/feross" - }, - { - "type": "patreon", - "url": "https://www.patreon.com/feross" - }, - { - "type": "consulting", - "url": "https://feross.org/support" - } - ], - "license": "MIT", - "dependencies": { - "base64-js": "^1.3.1", - "ieee754": "^1.1.13" - } - }, - "node_modules/bytes": { - "version": "3.1.2", - "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz", - "integrity": "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/call-bind-apply-helpers": { - "version": "1.0.2", - "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz", - "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==", - "license": "MIT", - "dependencies": { - "es-errors": "^1.3.0", - "function-bind": "^1.1.2" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/call-bound": { - "version": "1.0.4", - "resolved": "https://registry.npmjs.org/call-bound/-/call-bound-1.0.4.tgz", - "integrity": "sha512-+ys997U96po4Kx/ABpBCqhA9EuxJaQWDQg7295H4hBphv3IZg0boBKuwYpt4YXp6MZ5AmZQnU/tyMTlRpaSejg==", - "license": "MIT", - "dependencies": { - "call-bind-apply-helpers": "^1.0.2", - "get-intrinsic": "^1.3.0" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, "node_modules/chai": { "version": "6.2.2", "resolved": "https://registry.npmjs.org/chai/-/chai-6.2.2.tgz", @@ -1438,88 +1109,12 @@ "node": ">=18" } }, - "node_modules/chokidar": { - "version": "3.6.0", - "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-3.6.0.tgz", - "integrity": "sha512-7VT13fmjotKpGipCW9JEQAusEPE+Ei8nl6/g4FBAmIm0GOOLMua9NDDo/DWp0ZAxCr3cPq5ZpBqmPAQgDda2Pw==", - "license": "MIT", - "dependencies": { - "anymatch": "~3.1.2", - "braces": "~3.0.2", - "glob-parent": "~5.1.2", - "is-binary-path": "~2.1.0", - "is-glob": "~4.0.1", - "normalize-path": "~3.0.0", - "readdirp": "~3.6.0" - }, - "engines": { - "node": ">= 8.10.0" - }, - "funding": { - "url": "https://paulmillr.com/funding/" - }, - "optionalDependencies": { - "fsevents": "~2.3.2" - } - }, - "node_modules/citation-js-utils": { - "version": "1.2.8", - "resolved": "https://registry.npmjs.org/citation-js-utils/-/citation-js-utils-1.2.8.tgz", - "integrity": "sha512-/aHhuTHuVIFisjxRM1bB5xb6dRkpsNBYO5HuANbf5VFYZ8fRJCf4bZlYo8EHv+95pbjevJ9ITl7FJvhM1ARmBA==", - "license": "MIT", - "dependencies": { - "@citation-js/core": "^0.7.18", - "@citation-js/plugin-bibtex": "^0.7.18", - "@citation-js/plugin-csl": "^0.7.18", - "sanitize-html": "^2.7.0" - }, - "engines": { - "node": ">=20", - "npm": ">=10" - } - }, - "node_modules/citeproc": { - "version": "2.4.63", - "resolved": "https://registry.npmjs.org/citeproc/-/citeproc-2.4.63.tgz", - "integrity": "sha512-68F95Bp4UbgZU/DBUGQn0qV3HDZLCdI9+Bb2ByrTaNJDL5VEm9LqaiNaxljsvoaExSLEXe1/r6n2Z06SCzW3/Q==", - "license": "CPAL-1.0 OR AGPL-1.0" - }, "node_modules/classnames": { "version": "2.5.1", "resolved": "https://registry.npmjs.org/classnames/-/classnames-2.5.1.tgz", "integrity": "sha512-saHYOzhIQs6wy2sVxTM6bUDsQO4F50V9RQ22qBpEdCW+I+/Wmke2HOl6lS6dTpdxVhb88/I6+Hs+438c3lfUow==", "license": "MIT" }, - "node_modules/commander": { - "version": "12.1.0", - "resolved": "https://registry.npmjs.org/commander/-/commander-12.1.0.tgz", - "integrity": "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA==", - "license": "MIT", - "engines": { - "node": ">=18" - } - }, - "node_modules/content-disposition": { - "version": "0.5.4", - "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz", - "integrity": "sha512-FveZTNuGw04cxlAiWbzi6zTAL/lhehaWbTtgluJh4/E95DqMwTmha3KZN1aAWA8cFIhHzMZUvLevkw5Rqk+tSQ==", - "license": "MIT", - "dependencies": { - "safe-buffer": "5.2.1" - }, - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/content-type": { - "version": "1.0.5", - "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.5.tgz", - "integrity": "sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, "node_modules/convert-source-map": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-2.0.0.tgz", @@ -1527,38 +1122,6 @@ "dev": true, "license": "MIT" }, - "node_modules/cookie": { - "version": "0.7.2", - "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.2.tgz", - "integrity": "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/cookie-signature": { - "version": "1.0.7", - "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.7.tgz", - "integrity": "sha512-NXdYc3dLr47pBkpUCHtKSwIOQXLVn8dZEuywboCOJY/osA0wFSLlSawr3KN8qXJEyX66FcONTH8EIlVuK0yyFA==", - "license": "MIT" - }, - "node_modules/cors": { - "version": "2.8.6", - "resolved": "https://registry.npmjs.org/cors/-/cors-2.8.6.tgz", - "integrity": "sha512-tJtZBBHA6vjIAaF6EnIaq6laBBP9aq/Y3ouVJjEfoHbRBcHBAHYcMh/w8LDrk2PvIMMq8gmopa5D4V8RmbrxGw==", - "license": "MIT", - "dependencies": { - "object-assign": "^4", - "vary": "^1" - }, - "engines": { - "node": ">= 0.10" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/express" - } - }, "node_modules/credit-roles": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/credit-roles/-/credit-roles-2.1.0.tgz", @@ -1577,51 +1140,14 @@ "integrity": "sha512-l3nz3euub2QMg5ouu5U09Ew9Wf6/wQ8I++ch1loQ0ljmzhmfZYrH9fflS22i/PQEvsPvxCwxgz5q7UB8K1JO4Q==", "license": "MIT" }, - "node_modules/debug": { - "version": "2.6.9", - "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz", - "integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==", - "license": "MIT", - "dependencies": { - "ms": "2.0.0" - } - }, - "node_modules/deepmerge": { - "version": "4.3.1", - "resolved": "https://registry.npmjs.org/deepmerge/-/deepmerge-4.3.1.tgz", - "integrity": "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A==", - "license": "MIT", - "engines": { - "node": ">=0.10.0" - } - }, - "node_modules/depd": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz", - "integrity": "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/destroy": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.2.0.tgz", - "integrity": "sha512-2sJGJTaXIIaR1w4iJSNoN0hnMY7Gpc/n8D4qSCJw8QqFWXf7cuAgnEHxBpweaVcPevC2l3KpjYCx3NypQQgaJg==", - "license": "MIT", - "engines": { - "node": ">= 0.8", - "npm": "1.2.8000 || >= 1.4.16" - } - }, - "node_modules/detect-libc": { - "version": "2.1.2", - "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", - "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", - "dev": true, - "license": "Apache-2.0", - "engines": { - "node": ">=8" + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=8" } }, "node_modules/doi-utils": { @@ -1630,132 +1156,6 @@ "integrity": "sha512-QkaDmWbr5lIugDgl9fCxDMos2OJJ9Rp4c9XB7wgRGm6hJQ+hgeqI78DqToU8+vmtU+1XAyyVYJoLZUAgBdsFtg==", "license": "MIT" }, - "node_modules/dom-serializer": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz", - "integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==", - "license": "MIT", - "dependencies": { - "domelementtype": "^2.3.0", - "domhandler": "^5.0.2", - "entities": "^4.2.0" - }, - "funding": { - "url": "https://github.com/cheeriojs/dom-serializer?sponsor=1" - } - }, - "node_modules/dom-serializer/node_modules/entities": { - "version": "4.5.0", - "resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz", - "integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/domelementtype": { - "version": "2.3.0", - "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz", - "integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/fb55" - } - ], - "license": "BSD-2-Clause" - }, - "node_modules/domhandler": { - "version": "5.0.3", - "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz", - "integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==", - "license": "BSD-2-Clause", - "dependencies": { - "domelementtype": "^2.3.0" - }, - "engines": { - "node": ">= 4" - }, - "funding": { - "url": "https://github.com/fb55/domhandler?sponsor=1" - } - }, - "node_modules/domutils": { - "version": "3.2.2", - "resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz", - "integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==", - "license": "BSD-2-Clause", - "dependencies": { - "dom-serializer": "^2.0.0", - "domelementtype": "^2.3.0", - "domhandler": "^5.0.3" - }, - "funding": { - "url": "https://github.com/fb55/domutils?sponsor=1" - } - }, - "node_modules/dunder-proto": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz", - "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==", - "license": "MIT", - "dependencies": { - "call-bind-apply-helpers": "^1.0.1", - "es-errors": "^1.3.0", - "gopd": "^1.2.0" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/ee-first": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz", - "integrity": "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==", - "license": "MIT" - }, - "node_modules/encodeurl": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-2.0.0.tgz", - "integrity": "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/entities": { - "version": "7.0.1", - "resolved": "https://registry.npmjs.org/entities/-/entities-7.0.1.tgz", - "integrity": "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/es-define-property": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz", - "integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==", - "license": "MIT", - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/es-errors": { - "version": "1.3.0", - "resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz", - "integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==", - "license": "MIT", - "engines": { - "node": ">= 0.4" - } - }, "node_modules/es-module-lexer": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.1.0.tgz", @@ -1763,18 +1163,6 @@ "dev": true, "license": "MIT" }, - "node_modules/es-object-atoms": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz", - "integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==", - "license": "MIT", - "dependencies": { - "es-errors": "^1.3.0" - }, - "engines": { - "node": ">= 0.4" - } - }, "node_modules/esbuild": { "version": "0.27.4", "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.4.tgz", @@ -1817,24 +1205,6 @@ "@esbuild/win32-x64": "0.27.4" } }, - "node_modules/escape-html": { - "version": "1.0.3", - "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz", - "integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==", - "license": "MIT" - }, - "node_modules/escape-string-regexp": { - "version": "4.0.0", - "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz", - "integrity": "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA==", - "license": "MIT", - "engines": { - "node": ">=10" - }, - "funding": { - "url": "https://github.com/sponsors/sindresorhus" - } - }, "node_modules/estree-walker": { "version": "3.0.3", "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz", @@ -1845,15 +1215,6 @@ "@types/estree": "^1.0.0" } }, - "node_modules/etag": { - "version": "1.8.1", - "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz", - "integrity": "sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, "node_modules/expect-type": { "version": "1.3.0", "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.3.0.tgz", @@ -1864,119 +1225,39 @@ "node": ">=12.0.0" } }, - "node_modules/express": { - "version": "4.22.1", - "resolved": "https://registry.npmjs.org/express/-/express-4.22.1.tgz", - "integrity": "sha512-F2X8g9P1X7uCPZMA3MVf9wcTqlyNp7IhH5qPCI0izhaOIYXaW9L535tGA3qmjRzpH+bZczqq7hVKxTR4NWnu+g==", - "license": "MIT", - "dependencies": { - "accepts": "~1.3.8", - "array-flatten": "1.1.1", - "body-parser": "~1.20.3", - "content-disposition": "~0.5.4", - "content-type": "~1.0.4", - "cookie": "~0.7.1", - "cookie-signature": "~1.0.6", - "debug": "2.6.9", - "depd": "2.0.0", - "encodeurl": "~2.0.0", - "escape-html": "~1.0.3", - "etag": "~1.8.1", - "finalhandler": "~1.3.1", - "fresh": "~0.5.2", - "http-errors": "~2.0.0", - "merge-descriptors": "1.0.3", - "methods": "~1.1.2", - "on-finished": "~2.4.1", - "parseurl": "~1.3.3", - "path-to-regexp": "~0.1.12", - "proxy-addr": "~2.0.7", - "qs": "~6.14.0", - "range-parser": "~1.2.1", - "safe-buffer": "5.2.1", - "send": "~0.19.0", - "serve-static": "~1.16.2", - "setprototypeof": "1.2.0", - "statuses": "~2.0.1", - "type-is": "~1.6.18", - "utils-merge": "1.0.1", - "vary": "~1.1.2" - }, - "engines": { - "node": ">= 0.10.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/express" - } - }, "node_modules/extend": { "version": "3.0.2", "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz", "integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==", "license": "MIT" }, - "node_modules/fetch-ponyfill": { - "version": "7.1.0", - "resolved": "https://registry.npmjs.org/fetch-ponyfill/-/fetch-ponyfill-7.1.0.tgz", - "integrity": "sha512-FhbbL55dj/qdVO3YNK7ZEkshvj3eQ7EuIGV2I6ic/2YiocvyWv+7jg2s4AyS0wdRU75s3tA8ZxI/xPigb0v5Aw==", - "license": "MIT", - "dependencies": { - "node-fetch": "~2.6.1" - } - }, - "node_modules/fill-range": { - "version": "7.1.1", - "resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz", - "integrity": "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==", - "license": "MIT", - "dependencies": { - "to-regex-range": "^5.0.1" - }, - "engines": { - "node": ">=8" - } - }, - "node_modules/finalhandler": { - "version": "1.3.2", - "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz", - "integrity": "sha512-aA4RyPcd3badbdABGDuTXCMTtOneUCAYH/gxoYRTZlIJdF0YPWuGqiAsIrhNnnqdXGswYk6dGujem4w80UJFhg==", - "license": "MIT", - "dependencies": { - "debug": "2.6.9", - "encodeurl": "~2.0.0", - "escape-html": "~1.0.3", - "on-finished": "~2.4.1", - "parseurl": "~1.3.3", - "statuses": "~2.0.2", - "unpipe": "~1.0.0" - }, - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/forwarded": { - "version": "0.2.0", - "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz", - "integrity": "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } + "node_modules/fast-deep-equal": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", + "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==", + "license": "MIT" }, - "node_modules/fresh": { - "version": "0.5.2", - "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz", - "integrity": "sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } + "node_modules/fast-uri": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.2.tgz", + "integrity": "sha512-rVjf7ArG3LTk+FS6Yw81V1DLuZl1bRbNrev6Tmd/9RaroeeRRJhAt7jg/6YFxbvAQXUCavSoZhPPj6oOx+5KjQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fastify" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fastify" + } + ], + "license": "BSD-3-Clause" }, "node_modules/fsevents": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, "hasInstallScript": true, "license": "MIT", "optional": true, @@ -1987,52 +1268,6 @@ "node": "^8.16.0 || ^10.6.0 || >=11.0.0" } }, - "node_modules/function-bind": { - "version": "1.1.2", - "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz", - "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==", - "license": "MIT", - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/get-intrinsic": { - "version": "1.3.0", - "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz", - "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==", - "license": "MIT", - "dependencies": { - "call-bind-apply-helpers": "^1.0.2", - "es-define-property": "^1.0.1", - "es-errors": "^1.3.0", - "es-object-atoms": "^1.1.1", - "function-bind": "^1.1.2", - "get-proto": "^1.0.1", - "gopd": "^1.2.0", - "has-symbols": "^1.1.0", - "hasown": "^2.0.2", - "math-intrinsics": "^1.1.0" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/get-proto": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz", - "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==", - "license": "MIT", - "dependencies": { - "dunder-proto": "^1.0.1", - "es-object-atoms": "^1.0.0" - }, - "engines": { - "node": ">= 0.4" - } - }, "node_modules/get-tsconfig": { "version": "4.13.7", "resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.13.7.tgz", @@ -2046,54 +1281,6 @@ "url": "https://github.com/privatenumber/get-tsconfig?sponsor=1" } }, - "node_modules/glob-parent": { - "version": "5.1.2", - "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", - "integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==", - "license": "ISC", - "dependencies": { - "is-glob": "^4.0.1" - }, - "engines": { - "node": ">= 6" - } - }, - "node_modules/gopd": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz", - "integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==", - "license": "MIT", - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/has-symbols": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz", - "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==", - "license": "MIT", - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/hasown": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz", - "integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==", - "license": "MIT", - "dependencies": { - "function-bind": "^1.1.2" - }, - "engines": { - "node": ">= 0.4" - } - }, "node_modules/he": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/he/-/he-1.2.0.tgz", @@ -2103,104 +1290,6 @@ "he": "bin/he" } }, - "node_modules/htmlparser2": { - "version": "10.1.0", - "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz", - "integrity": "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==", - "funding": [ - "https://github.com/fb55/htmlparser2?sponsor=1", - { - "type": "github", - "url": "https://github.com/sponsors/fb55" - } - ], - "license": "MIT", - "dependencies": { - "domelementtype": "^2.3.0", - "domhandler": "^5.0.3", - "domutils": "^3.2.2", - "entities": "^7.0.1" - } - }, - "node_modules/http-errors": { - "version": "2.0.1", - "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz", - "integrity": "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==", - "license": "MIT", - "dependencies": { - "depd": "~2.0.0", - "inherits": "~2.0.4", - "setprototypeof": "~1.2.0", - "statuses": "~2.0.2", - "toidentifier": "~1.0.1" - }, - "engines": { - "node": ">= 0.8" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/express" - } - }, - "node_modules/iconv-lite": { - "version": "0.4.24", - "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz", - "integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==", - "license": "MIT", - "dependencies": { - "safer-buffer": ">= 2.1.2 < 3" - }, - "engines": { - "node": ">=0.10.0" - } - }, - "node_modules/ieee754": { - "version": "1.2.1", - "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz", - "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/feross" - }, - { - "type": "patreon", - "url": "https://www.patreon.com/feross" - }, - { - "type": "consulting", - "url": "https://feross.org/support" - } - ], - "license": "BSD-3-Clause" - }, - "node_modules/inherits": { - "version": "2.0.4", - "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz", - "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", - "license": "ISC" - }, - "node_modules/ipaddr.js": { - "version": "1.9.1", - "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz", - "integrity": "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==", - "license": "MIT", - "engines": { - "node": ">= 0.10" - } - }, - "node_modules/is-binary-path": { - "version": "2.1.0", - "resolved": "https://registry.npmjs.org/is-binary-path/-/is-binary-path-2.1.0.tgz", - "integrity": "sha512-ZMERYes6pDydyuGidse7OsHxtbI7WVeUEozgR/g7rd0xUimYNlvZRE/K2MgZTjWy725IfelLeVcEM97mmtRGXw==", - "license": "MIT", - "dependencies": { - "binary-extensions": "^2.0.0" - }, - "engines": { - "node": ">=8" - } - }, "node_modules/is-buffer": { "version": "2.0.5", "resolved": "https://registry.npmjs.org/is-buffer/-/is-buffer-2.0.5.tgz", @@ -2224,36 +1313,6 @@ "node": ">=4" } }, - "node_modules/is-extglob": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz", - "integrity": "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ==", - "license": "MIT", - "engines": { - "node": ">=0.10.0" - } - }, - "node_modules/is-glob": { - "version": "4.0.3", - "resolved": "https://registry.npmjs.org/is-glob/-/is-glob-4.0.3.tgz", - "integrity": "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg==", - "license": "MIT", - "dependencies": { - "is-extglob": "^2.1.1" - }, - "engines": { - "node": ">=0.10.0" - } - }, - "node_modules/is-number": { - "version": "7.0.0", - "resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz", - "integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==", - "license": "MIT", - "engines": { - "node": ">=0.12.0" - } - }, "node_modules/is-plain-obj": { "version": "4.1.0", "resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-4.1.0.tgz", @@ -2266,15 +1325,6 @@ "url": "https://github.com/sponsors/sindresorhus" } }, - "node_modules/is-plain-object": { - "version": "5.0.0", - "resolved": "https://registry.npmjs.org/is-plain-object/-/is-plain-object-5.0.0.tgz", - "integrity": "sha512-VRSzKkbMm5jMDoKLbltAkFQ5Qr7VDiTFGXxYFXXowVj387GeGNOCsOH6Msy00SGZ3Fp84b1Naa1psqgcCIEP5Q==", - "license": "MIT", - "engines": { - "node": ">=0.10.0" - } - }, "node_modules/js-yaml": { "version": "4.1.1", "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz", @@ -2287,6 +1337,12 @@ "js-yaml": "bin/js-yaml.js" } }, + "node_modules/json-schema-traverse": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz", + "integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==", + "license": "MIT" + }, "node_modules/json5": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/json5/-/json5-1.0.2.tgz", @@ -2692,92 +1748,23 @@ "engines": { "node": ">=0.12" }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/math-intrinsics": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz", - "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==", - "license": "MIT", - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/mdast": { - "version": "3.0.0", - "resolved": "https://registry.npmjs.org/mdast/-/mdast-3.0.0.tgz", - "integrity": "sha512-xySmf8g4fPKMeC07jXGz971EkLbWAJ83s4US2Tj9lEdnZ142UP5grN73H1Xd3HzrdbU5o9GYYP/y8F9ZSwLE9g==", - "deprecated": "`mdast` was renamed to `remark`", - "license": "MIT" - }, - "node_modules/mdurl": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/mdurl/-/mdurl-1.0.1.tgz", - "integrity": "sha512-/sKlQJCBYVY9Ers9hqzKou4H6V5UWc/M59TH2dvkt+84itfnq7uFOMLpOiOS4ujvHP4etln18fmIxA5R5fll0g==", - "license": "MIT" - }, - "node_modules/media-typer": { - "version": "0.3.0", - "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz", - "integrity": "sha512-dq+qelQ9akHpcOl/gUVRTxVIOkAJ1wR3QAvb4RsVjS8oVoFjDGTc679wJYmUmknUF5HwMLOgb5O+a3KxfWapPQ==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/merge-descriptors": { - "version": "1.0.3", - "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.3.tgz", - "integrity": "sha512-gaNvAS7TZ897/rVaZ0nMtAyxNyi/pdbjbAwUpFQpN70GqnVfOiXpeUUMKRBmzXaSQ8DdTX4/0ms62r2K+hE6mQ==", - "license": "MIT", - "funding": { - "url": "https://github.com/sponsors/sindresorhus" - } - }, - "node_modules/methods": { - "version": "1.1.2", - "resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz", - "integrity": "sha512-iclAHeNqNm68zFtnZ0e+1L2yUIdvzNoauKU4WBA3VvH/vPFieF7qfRlwUZU+DA9P9bPXIS90ulxoUoCH23sV2w==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/mime": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz", - "integrity": "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==", - "license": "MIT", - "bin": { - "mime": "cli.js" - }, - "engines": { - "node": ">=4" - } - }, - "node_modules/mime-db": { - "version": "1.52.0", - "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", - "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/mime-types": { - "version": "2.1.35", - "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", - "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", - "license": "MIT", - "dependencies": { - "mime-db": "1.52.0" - }, - "engines": { - "node": ">= 0.6" + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" } }, + "node_modules/mdast": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/mdast/-/mdast-3.0.0.tgz", + "integrity": "sha512-xySmf8g4fPKMeC07jXGz971EkLbWAJ83s4US2Tj9lEdnZ142UP5grN73H1Xd3HzrdbU5o9GYYP/y8F9ZSwLE9g==", + "deprecated": "`mdast` was renamed to `remark`", + "license": "MIT" + }, + "node_modules/mdurl": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/mdurl/-/mdurl-1.0.1.tgz", + "integrity": "sha512-/sKlQJCBYVY9Ers9hqzKou4H6V5UWc/M59TH2dvkt+84itfnq7uFOMLpOiOS4ujvHP4etln18fmIxA5R5fll0g==", + "license": "MIT" + }, "node_modules/minimist": { "version": "1.2.8", "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz", @@ -2787,18 +1774,6 @@ "url": "https://github.com/sponsors/ljharb" } }, - "node_modules/moo": { - "version": "0.5.3", - "resolved": "https://registry.npmjs.org/moo/-/moo-0.5.3.tgz", - "integrity": "sha512-m2fmM2dDm7GZQsY7KK2cme8agi+AAljILjQnof7p1ZMDe6dQ4bdnSMx0cPppudoeNv5hEFQirN6u+O4fDE0IWA==", - "license": "BSD-3-Clause" - }, - "node_modules/ms": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", - "integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==", - "license": "MIT" - }, "node_modules/myst-common": { "version": "1.9.5", "resolved": "https://registry.npmjs.org/myst-common/-/myst-common-1.9.5.tgz", @@ -2951,6 +1926,7 @@ "version": "3.3.11", "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz", "integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==", + "dev": true, "funding": [ { "type": "github", @@ -2965,44 +1941,6 @@ "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" } }, - "node_modules/negotiator": { - "version": "0.6.3", - "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz", - "integrity": "sha512-+EUsqGPLsM+j/zdChZjsnX51g4XrHFOIXwfnCVPGlQk/k5giakcKsuxCObBRu6DSm9opw/O6slWbJdghQM4bBg==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/node-fetch": { - "version": "2.6.13", - "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.13.tgz", - "integrity": "sha512-StxNAxh15zr77QvvkmveSQ8uCQ4+v5FkvNTj0OESmiHu+VRi/gXArXtkWMElOsOUNLtUEvI4yS+rdtOHZTwlQA==", - "license": "MIT", - "dependencies": { - "whatwg-url": "^5.0.0" - }, - "engines": { - "node": "4.x || >=6.0.0" - }, - "peerDependencies": { - "encoding": "^0.1.0" - }, - "peerDependenciesMeta": { - "encoding": { - "optional": true - } - } - }, - "node_modules/normalize-path": { - "version": "3.0.0", - "resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz", - "integrity": "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA==", - "license": "MIT", - "engines": { - "node": ">=0.10.0" - } - }, "node_modules/nth-check": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz", @@ -3015,27 +1953,6 @@ "url": "https://github.com/fb55/nth-check?sponsor=1" } }, - "node_modules/object-assign": { - "version": "4.1.1", - "resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz", - "integrity": "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==", - "license": "MIT", - "engines": { - "node": ">=0.10.0" - } - }, - "node_modules/object-inspect": { - "version": "1.13.4", - "resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz", - "integrity": "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew==", - "license": "MIT", - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, "node_modules/obug": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/obug/-/obug-2.1.1.tgz", @@ -3047,18 +1964,6 @@ ], "license": "MIT" }, - "node_modules/on-finished": { - "version": "2.4.1", - "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz", - "integrity": "sha512-oVlzkg3ENAhCk2zdv7IJwd/QUD4z2RxRwpkcGY8psCVcCYZNq4wYnVWALHM+brtuJjePWiYF/ClmuDr8Ch5+kg==", - "license": "MIT", - "dependencies": { - "ee-first": "1.1.1" - }, - "engines": { - "node": ">= 0.8" - } - }, "node_modules/orcid": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/orcid/-/orcid-1.0.0.tgz", @@ -3074,27 +1979,6 @@ "integrity": "sha512-5QvjGxYVjxO59MGU2lHVYpRWBBtKHnlIAcSe1uNFCkkptUh63NFRj0FJQm7nR67puEruUci/ZkjmEFrjCAyP4A==", "license": "MIT" }, - "node_modules/parse-srcset": { - "version": "1.0.2", - "resolved": "https://registry.npmjs.org/parse-srcset/-/parse-srcset-1.0.2.tgz", - "integrity": "sha512-/2qh0lav6CmI15FzA3i/2Bzk2zCgQhGMkvhOhKNcBVQ1ldgpbfiNTVslmooUmWJcADi1f1kIeynbDRVzNlfR6Q==", - "license": "MIT" - }, - "node_modules/parseurl": { - "version": "1.3.3", - "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz", - "integrity": "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/path-to-regexp": { - "version": "0.1.13", - "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.13.tgz", - "integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==", - "license": "MIT" - }, "node_modules/pathe": { "version": "2.0.3", "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz", @@ -3106,24 +1990,14 @@ "version": "1.1.1", "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", + "dev": true, "license": "ISC" }, - "node_modules/picomatch": { - "version": "2.3.2", - "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz", - "integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==", - "license": "MIT", - "engines": { - "node": ">=8.6" - }, - "funding": { - "url": "https://github.com/sponsors/jonschlinkert" - } - }, "node_modules/postcss": { "version": "8.5.14", "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.14.tgz", "integrity": "sha512-SoSL4+OSEtR99LHFZQiJLkT59C5B1amGO1NzTwj7TT1qCUgUO6hxOvzkOYxD+vMrXBM3XJIKzokoERdqQq/Zmg==", + "dev": true, "funding": [ { "type": "opencollective", @@ -3148,68 +2022,13 @@ "node": "^10 || ^12 || >=14" } }, - "node_modules/proxy-addr": { - "version": "2.0.7", - "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz", - "integrity": "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==", - "license": "MIT", - "dependencies": { - "forwarded": "0.2.0", - "ipaddr.js": "1.9.1" - }, - "engines": { - "node": ">= 0.10" - } - }, - "node_modules/qs": { - "version": "6.14.2", - "resolved": "https://registry.npmjs.org/qs/-/qs-6.14.2.tgz", - "integrity": "sha512-V/yCWTTF7VJ9hIh18Ugr2zhJMP01MY7c5kh4J870L7imm6/DIzBsNLTXzMwUA3yZ5b/KBqLx8Kp3uRvd7xSe3Q==", - "license": "BSD-3-Clause", - "dependencies": { - "side-channel": "^1.1.0" - }, - "engines": { - "node": ">=0.6" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/range-parser": { - "version": "1.2.1", - "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz", - "integrity": "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/raw-body": { - "version": "2.5.3", - "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.5.3.tgz", - "integrity": "sha512-s4VSOf6yN0rvbRZGxs8Om5CWj6seneMwK3oDb4lWDH0UPhWcxwOWw5+qk24bxq87szX1ydrwylIOp2uG1ojUpA==", - "license": "MIT", - "dependencies": { - "bytes": "~3.1.2", - "http-errors": "~2.0.1", - "iconv-lite": "~0.4.24", - "unpipe": "~1.0.0" - }, - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/readdirp": { - "version": "3.6.0", - "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-3.6.0.tgz", - "integrity": "sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==", + "node_modules/require-from-string": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", + "integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==", "license": "MIT", - "dependencies": { - "picomatch": "^2.2.1" - }, "engines": { - "node": ">=8.10.0" + "node": ">=0.10.0" } }, "node_modules/resolve-pkg-maps": { @@ -3256,169 +2075,6 @@ "@rolldown/binding-win32-x64-msvc": "1.0.0" } }, - "node_modules/safe-buffer": { - "version": "5.2.1", - "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", - "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/feross" - }, - { - "type": "patreon", - "url": "https://www.patreon.com/feross" - }, - { - "type": "consulting", - "url": "https://feross.org/support" - } - ], - "license": "MIT" - }, - "node_modules/safer-buffer": { - "version": "2.1.2", - "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", - "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", - "license": "MIT" - }, - "node_modules/sanitize-html": { - "version": "2.17.2", - "resolved": "https://registry.npmjs.org/sanitize-html/-/sanitize-html-2.17.2.tgz", - "integrity": "sha512-EnffJUl46VE9uvZ0XeWzObHLurClLlT12gsOk1cHyP2Ol1P0BnBnsXmShlBmWVJM+dKieQI68R0tsPY5m/B+Jg==", - "license": "MIT", - "dependencies": { - "deepmerge": "^4.2.2", - "escape-string-regexp": "^4.0.0", - "htmlparser2": "^10.1.0", - "is-plain-object": "^5.0.0", - "parse-srcset": "^1.0.2", - "postcss": "^8.3.11" - } - }, - "node_modules/send": { - "version": "0.19.2", - "resolved": "https://registry.npmjs.org/send/-/send-0.19.2.tgz", - "integrity": "sha512-VMbMxbDeehAxpOtWJXlcUS5E8iXh6QmN+BkRX1GARS3wRaXEEgzCcB10gTQazO42tpNIya8xIyNx8fll1OFPrg==", - "license": "MIT", - "dependencies": { - "debug": "2.6.9", - "depd": "2.0.0", - "destroy": "1.2.0", - "encodeurl": "~2.0.0", - "escape-html": "~1.0.3", - "etag": "~1.8.1", - "fresh": "~0.5.2", - "http-errors": "~2.0.1", - "mime": "1.6.0", - "ms": "2.1.3", - "on-finished": "~2.4.1", - "range-parser": "~1.2.1", - "statuses": "~2.0.2" - }, - "engines": { - "node": ">= 0.8.0" - } - }, - "node_modules/send/node_modules/ms": { - "version": "2.1.3", - "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", - "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", - "license": "MIT" - }, - "node_modules/serve-static": { - "version": "1.16.3", - "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.16.3.tgz", - "integrity": "sha512-x0RTqQel6g5SY7Lg6ZreMmsOzncHFU7nhnRWkKgWuMTu5NN0DR5oruckMqRvacAN9d5w6ARnRBXl9xhDCgfMeA==", - "license": "MIT", - "dependencies": { - "encodeurl": "~2.0.0", - "escape-html": "~1.0.3", - "parseurl": "~1.3.3", - "send": "~0.19.1" - }, - "engines": { - "node": ">= 0.8.0" - } - }, - "node_modules/setprototypeof": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz", - "integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==", - "license": "ISC" - }, - "node_modules/side-channel": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.0.tgz", - "integrity": "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw==", - "license": "MIT", - "dependencies": { - "es-errors": "^1.3.0", - "object-inspect": "^1.13.3", - "side-channel-list": "^1.0.0", - "side-channel-map": "^1.0.1", - "side-channel-weakmap": "^1.0.2" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/side-channel-list": { - "version": "1.0.0", - "resolved": "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.0.tgz", - "integrity": "sha512-FCLHtRD/gnpCiCHEiJLOwdmFP+wzCmDEkc9y7NsYxeF4u7Btsn1ZuwgwJGxImImHicJArLP4R0yX4c2KCrMrTA==", - "license": "MIT", - "dependencies": { - "es-errors": "^1.3.0", - "object-inspect": "^1.13.3" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/side-channel-map": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/side-channel-map/-/side-channel-map-1.0.1.tgz", - "integrity": "sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA==", - "license": "MIT", - "dependencies": { - "call-bound": "^1.0.2", - "es-errors": "^1.3.0", - "get-intrinsic": "^1.2.5", - "object-inspect": "^1.13.3" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/side-channel-weakmap": { - "version": "1.0.2", - "resolved": "https://registry.npmjs.org/side-channel-weakmap/-/side-channel-weakmap-1.0.2.tgz", - "integrity": "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A==", - "license": "MIT", - "dependencies": { - "call-bound": "^1.0.2", - "es-errors": "^1.3.0", - "get-intrinsic": "^1.2.5", - "object-inspect": "^1.13.3", - "side-channel-map": "^1.0.1" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, "node_modules/siginfo": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz", @@ -3436,6 +2092,7 @@ "version": "1.2.1", "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", + "dev": true, "license": "BSD-3-Clause", "engines": { "node": ">=0.10.0" @@ -3480,15 +2137,6 @@ "dev": true, "license": "MIT" }, - "node_modules/statuses": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz", - "integrity": "sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, "node_modules/std-env": { "version": "4.1.0", "resolved": "https://registry.npmjs.org/std-env/-/std-env-4.1.0.tgz", @@ -3496,19 +2144,6 @@ "dev": true, "license": "MIT" }, - "node_modules/sync-fetch": { - "version": "0.4.5", - "resolved": "https://registry.npmjs.org/sync-fetch/-/sync-fetch-0.4.5.tgz", - "integrity": "sha512-esiWJ7ixSKGpd9DJPBTC4ckChqdOjIwJfYhVHkcQ2Gnm41323p1TRmEI+esTQ9ppD+b5opps2OTEGTCGX5kF+g==", - "license": "MIT", - "dependencies": { - "buffer": "^5.7.1", - "node-fetch": "^2.6.1" - }, - "engines": { - "node": ">=14" - } - }, "node_modules/tinybench": { "version": "2.9.0", "resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz", @@ -3584,33 +2219,6 @@ "node": ">=14.0.0" } }, - "node_modules/to-regex-range": { - "version": "5.0.1", - "resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz", - "integrity": "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==", - "license": "MIT", - "dependencies": { - "is-number": "^7.0.0" - }, - "engines": { - "node": ">=8.0" - } - }, - "node_modules/toidentifier": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz", - "integrity": "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==", - "license": "MIT", - "engines": { - "node": ">=0.6" - } - }, - "node_modules/tr46": { - "version": "0.0.3", - "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz", - "integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==", - "license": "MIT" - }, "node_modules/trough": { "version": "2.2.0", "resolved": "https://registry.npmjs.org/trough/-/trough-2.2.0.tgz", @@ -3649,19 +2257,6 @@ "fsevents": "~2.3.3" } }, - "node_modules/type-is": { - "version": "1.6.18", - "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz", - "integrity": "sha512-TkRKr9sUTxEH8MdfuCSP7VizJyzRNMjj2J2do2Jr3Kym598JVdEksuzPQCnlFPW4ky9Q+iA+ma9BGm06XQBy8g==", - "license": "MIT", - "dependencies": { - "media-typer": "0.3.0", - "mime-types": "~2.1.24" - }, - "engines": { - "node": ">= 0.6" - } - }, "node_modules/typescript": { "version": "5.9.3", "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", @@ -3807,33 +2402,6 @@ "url": "https://opencollective.com/unified" } }, - "node_modules/unpipe": { - "version": "1.0.0", - "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz", - "integrity": "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/utils-merge": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz", - "integrity": "sha512-pMZTvIkT1d+TFGvDOqodOclx0QWkkgi6Tdoa8gC8ffGAAqz9pzPTZWAybbsHHoED/ztMtkv/VoYTYyShUn81hA==", - "license": "MIT", - "engines": { - "node": ">= 0.4.0" - } - }, - "node_modules/vary": { - "version": "1.1.2", - "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz", - "integrity": "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, "node_modules/vfile": { "version": "5.3.7", "resolved": "https://registry.npmjs.org/vfile/-/vfile-5.3.7.tgz", @@ -4058,22 +2626,6 @@ "url": "https://github.com/sponsors/jonschlinkert" } }, - "node_modules/webidl-conversions": { - "version": "3.0.1", - "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz", - "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==", - "license": "BSD-2-Clause" - }, - "node_modules/whatwg-url": { - "version": "5.0.0", - "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz", - "integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==", - "license": "MIT", - "dependencies": { - "tr46": "~0.0.3", - "webidl-conversions": "^3.0.0" - } - }, "node_modules/why-is-node-running": { "version": "2.3.0", "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz", @@ -4091,25 +2643,19 @@ "node": ">=8" } }, - "node_modules/ws": { - "version": "8.20.0", - "resolved": "https://registry.npmjs.org/ws/-/ws-8.20.0.tgz", - "integrity": "sha512-sAt8BhgNbzCtgGbt2OxmpuryO63ZoDk/sqaB/znQm94T4fCEsy/yV+7CdC1kJhOU9lboAEU7R3kquuycDoibVA==", - "license": "MIT", - "engines": { - "node": ">=10.0.0" + "node_modules/yaml": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz", + "integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==", + "license": "ISC", + "bin": { + "yaml": "bin.mjs" }, - "peerDependencies": { - "bufferutil": "^4.0.1", - "utf-8-validate": ">=5.0.2" + "engines": { + "node": ">= 14.6" }, - "peerDependenciesMeta": { - "bufferutil": { - "optional": true - }, - "utf-8-validate": { - "optional": true - } + "funding": { + "url": "https://github.com/sponsors/eemeli" } }, "node_modules/zwitch": { diff --git a/package.json b/package.json index 1512bad..f7acd8f 100644 --- a/package.json +++ b/package.json @@ -1,37 +1,28 @@ { "name": "mystra", - "version": "0.1.0", - "description": "MyST renderer for ASTRA analysis specifications", + "version": "0.2.0", + "description": "A MyST plugin that imports/cites ASTRA analysis components into Markdown reports", "type": "module", - "bin": { - "mystra": "./dist/cli.js" - }, "main": "./dist/index.js", + "exports": { + ".": "./dist/index.js" + }, + "files": [ + "dist" + ], "scripts": { "build": "tsc", - "dev": "tsx src/cli.ts", - "start": "node dist/cli.js", "test": "vitest" }, "dependencies": { - "chokidar": "^3.6.0", - "citation-js-utils": "^1.2.8", - "commander": "^12.1.0", - "cors": "^2.8.5", - "express": "^4.21.0", - "js-yaml": "^4.1.0", + "@astra-spec/sdk": "^0.0.3", "myst-parser": "^1.7.1", "myst-spec": "^0.0.5", - "papaparse": "^5.4.0", - "ws": "^8.18.0" + "papaparse": "^5.4.0" }, "devDependencies": { - "@types/cors": "^2.8.0", - "@types/express": "^4.17.0", - "@types/js-yaml": "^4.0.0", "@types/node": "^20.0.0", "@types/papaparse": "^5.3.0", - "@types/ws": "^8.5.0", "tsx": "^4.0.0", "typescript": "^5.5.0", "vitest": "4.1.6" diff --git a/src/cli.ts b/src/cli.ts deleted file mode 100644 index e3644f6..0000000 --- a/src/cli.ts +++ /dev/null @@ -1,75 +0,0 @@ -#!/usr/bin/env node - -/** - * MySTRA CLI — Live ASTRA document rendering via MyST. - */ - -import { resolve } from 'node:path'; -import { Command } from 'commander'; -import { createContentServer } from './server/index.js'; -import { launchTheme } from './theme/launcher.js'; -import type { ChildProcess } from 'node:child_process'; - -const program = new Command(); - -program - .name('mystra') - .description('Live ASTRA document rendering via MyST') - .version('0.1.0') - .argument('[project-dir]', 'Path to ASTRA project directory', '.') - .option('-p, --port ', 'Theme server port', '3000') - .option('--content-port ', 'Content server port', '3100') - .option('-u, --universe ', 'Specific universe to view') - .option('--no-theme', 'Start content server only') - .action(async (projectDirArg: string, opts: any) => { - const projectDir = resolve(projectDirArg); - const themePort = parseInt(opts.port, 10); - const contentPort = parseInt(opts.contentPort, 10); - const universeName: string | undefined = opts.universe; - const useTheme: boolean = opts.theme !== false; - - console.log(`[mystra] Starting MySTRA for ${projectDir}`); - - // Start content server (DOI resolution runs in the background inside) - const server = createContentServer({ - projectDir, - contentPort, - universeName, - }); - - await server.start(); - - // Launch theme server - let themeProcess: ChildProcess | null = null; - if (useTheme) { - themeProcess = await launchTheme({ themePort, contentPort, projectDir }); - if (themeProcess) { - console.log( - `\n MySTRA is running:\n` + - ` Document: http://localhost:${themePort}\n` + - ` Content: http://localhost:${contentPort}\n`, - ); - } - } else { - console.log( - `\n MySTRA content server running:\n` + - ` Content: http://localhost:${contentPort}\n` + - ` Config: http://localhost:${contentPort}/config.json\n`, - ); - } - - // Graceful shutdown - const shutdown = () => { - console.log('\n[mystra] Shutting down...'); - if (themeProcess) { - themeProcess.kill(); - } - server.close(); - process.exit(0); - }; - - process.on('SIGINT', shutdown); - process.on('SIGTERM', shutdown); - }); - -program.parse(); diff --git a/src/doi/cache.ts b/src/doi/cache.ts deleted file mode 100644 index d2cbbf8..0000000 --- a/src/doi/cache.ts +++ /dev/null @@ -1,45 +0,0 @@ -/** - * Disk cache for DOI metadata (CSL-JSON). - * Stored in .mystra-cache/doi/ with DOI as filename. - */ - -import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'node:fs'; -import { join } from 'node:path'; - -export class DOICache { - private cacheDir: string; - - constructor(cacheDir: string) { - this.cacheDir = cacheDir; - } - - private keyToFilename(doi: string): string { - return doi.replace(/\//g, '_') + '.json'; - } - - private ensureDir(): void { - if (!existsSync(this.cacheDir)) { - mkdirSync(this.cacheDir, { recursive: true }); - } - } - - has(doi: string): boolean { - return existsSync(join(this.cacheDir, this.keyToFilename(doi))); - } - - get(doi: string): any | null { - const filePath = join(this.cacheDir, this.keyToFilename(doi)); - if (!existsSync(filePath)) return null; - try { - return JSON.parse(readFileSync(filePath, 'utf-8')); - } catch { - return null; - } - } - - set(doi: string, data: any): void { - this.ensureDir(); - const filePath = join(this.cacheDir, this.keyToFilename(doi)); - writeFileSync(filePath, JSON.stringify(data, null, 2)); - } -} diff --git a/src/doi/fetcher.ts b/src/doi/fetcher.ts deleted file mode 100644 index 1d165ee..0000000 --- a/src/doi/fetcher.ts +++ /dev/null @@ -1,32 +0,0 @@ -/** - * Fetches citation metadata from doi.org as CSL-JSON. - */ - -export async function fetchDOI(doi: string): Promise { - const url = `https://doi.org/${doi}`; - - try { - const controller = new AbortController(); - const timeout = setTimeout(() => controller.abort(), 10_000); - - const response = await fetch(url, { - headers: { - Accept: 'application/vnd.citationstyles.csl+json', - }, - redirect: 'follow', - signal: controller.signal, - }); - - clearTimeout(timeout); - - if (!response.ok) { - console.warn(`[mystra] DOI fetch failed for ${doi}: ${response.status}`); - return null; - } - - return await response.json(); - } catch (err) { - console.warn(`[mystra] DOI fetch error for ${doi}:`, (err as Error).message); - return null; - } -} diff --git a/src/doi/resolver.ts b/src/doi/resolver.ts deleted file mode 100644 index d1ded9e..0000000 --- a/src/doi/resolver.ts +++ /dev/null @@ -1,177 +0,0 @@ -/** - * Orchestrates DOI resolution: collect, fetch, cache, render citations. - */ - -import { DOICache } from './cache.js'; -import { fetchDOI } from './fetcher.js'; -import type { References, CitationData } from '../types/content-server.js'; -import type { PaperInsightSummary } from '../types/papers.js'; - -export interface DOIMetadata { - label: string; - authors: string; - authorShort: string; - year: string; - title: string; - journal: string; - doi: string; - version?: number; - cache_key?: string; - pdf_url?: string; - insights?: PaperInsightSummary[]; -} - -/** - * Resolve all DOIs and build both the references object (for bibliography) - * and a metadata map (for inline citation formatting). - */ -export async function resolveAllDOIs( - dois: string[], - cacheDir: string, -): Promise<{ references: References; metadata: Map }> { - const metadata = new Map(); - if (dois.length === 0) return { references: {}, metadata }; - - const cache = new DOICache(cacheDir); - const uniqueDOIs = [...new Set(dois)]; - - // Fetch missing DOIs with concurrency limit - const concurrency = 5; - const toFetch = uniqueDOIs.filter((doi) => !cache.has(doi)); - - if (toFetch.length > 0) { - console.log(`[mystra] Fetching ${toFetch.length} DOI(s)...`); - } - - for (let i = 0; i < toFetch.length; i += concurrency) { - const batch = toFetch.slice(i, i + concurrency); - const results = await Promise.all( - batch.map(async (doi) => { - const data = await fetchDOI(doi); - if (data) { - cache.set(doi, data); - } - return { doi, data }; - }), - ); - - for (const { doi, data } of results) { - if (!data) { - console.warn(`[mystra] Could not resolve DOI: ${doi}`); - } - } - } - - // Build references and metadata - const order: string[] = []; - const data: Record = {}; - const seenLabels = new Set(); - - for (let i = 0; i < uniqueDOIs.length; i++) { - const doi = uniqueDOIs[i]; - const csl = cache.get(doi); - let label = generateLabel(csl, doi); - - // Deduplicate labels - if (seenLabels.has(label)) { - label = `${label}_${i}`; - } - seenLabels.add(label); - - const meta = extractMetadata(csl, doi, label); - metadata.set(doi, meta); - - order.push(label); - data[label] = { - label, - enumerator: String(i + 1), - doi, - html: formatCitationHTML(meta), - }; - } - - return { - references: { cite: { order, data } }, - metadata, - }; -} - -/** - * Get citation metadata from cache synchronously (for inline rendering). - * Returns null if the DOI hasn't been resolved yet. - */ -export function getCachedMetadata(doi: string, cacheDir: string): DOIMetadata | null { - const cache = new DOICache(cacheDir); - const csl = cache.get(doi); - if (!csl) return null; - const label = generateLabel(csl, doi); - return extractMetadata(csl, doi, label); -} - -function extractMetadata(csl: any | null, doi: string, label: string): DOIMetadata { - if (!csl) { - return { - label, - authors: '', - authorShort: '', - year: '', - title: '', - journal: '', - doi, - }; - } - - const authorList = csl.author ?? []; - - // Handle both family/given and literal (collaboration) author formats - function authorName(a: any): string { - if (a.family) return `${a.family}, ${a.given?.[0] ?? ''}.`; - if (a.literal) return a.literal; - return ''; - } - function authorFamily(a: any): string { - return a.family ?? a.literal ?? 'Unknown'; - } - - const authors = authorList.map(authorName).join(', '); - - let authorShort: string; - if (authorList.length === 0) { - authorShort = 'Unknown'; - } else if (authorList.length === 1) { - authorShort = authorFamily(authorList[0]); - } else if (authorList.length === 2) { - authorShort = `${authorFamily(authorList[0])} & ${authorFamily(authorList[1])}`; - } else { - authorShort = `${authorFamily(authorList[0])} et al.`; - } - - const year = String(csl.issued?.['date-parts']?.[0]?.[0] ?? ''); - const title = csl.title ?? ''; - const journal = csl['container-title'] ?? ''; - - return { label, authors, authorShort, year, title, journal, doi }; -} - -function generateLabel(csl: any | null, doi: string): string { - if (!csl) { - return doi.split('/').pop()?.replace(/[^a-zA-Z0-9]/g, '_') ?? doi; - } - const first = csl.author?.[0]; - const firstAuthor = first?.family ?? first?.literal ?? 'Unknown'; - const year = csl.issued?.['date-parts']?.[0]?.[0] ?? ''; - return `${firstAuthor}_${year}`.replace(/\s+/g, '_'); -} - -function formatCitationHTML(meta: DOIMetadata): string { - if (!meta.authors) { - return `${meta.doi}`; - } - - let html = meta.authors; - if (meta.year) html += ` (${meta.year}).`; - if (meta.title) html += ` ${meta.title}.`; - if (meta.journal) html += ` ${meta.journal}.`; - - return html; -} diff --git a/src/index.ts b/src/index.ts index 6bd3bf1..bdb22c9 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,16 +1,926 @@ /** - * MySTRA — Live ASTRA document rendering via MyST. + * MySTRA — the package entry point and the MyST plugin itself. * - * Library entry point for programmatic use. + * The **default export is the plugin** (reference this package from `myst.yml`'s + * `project.plugins`); named exports at the bottom expose the loader + resolved + * store for programmatic use. + * + * The author writes a normal MyST Markdown report and pulls in ASTRA components + * by id; this plugin reads `astra.yaml` at build time and emits standard MyST + * AST, running on the stock `myst` CLI and themes: + * + * Block "import" (directives): + * :::{astra:decision} covariance_source + * ::: + * :::{astra:output} bao_fit_plot + * ::: + * :::{astra:finding} bao_detected_post_recon + * ::: + * :::{astra:prior-insight} recon_sharpens_bao_peak + * ::: + * :::{astra:inputs} + * ::: # full inputs registry table (root scope) + * :::{astra:outputs} clustering + * ::: # outputs table for the clustering sub-analysis + * :::{astra:subanalysis} reconstruction + * ::: # nav card to the sub-analysis page + * + * Inline "cite" (roles): + * {astra:decision}`covariance_source` + * {astra:output}`hubble_diagram_plot` + * {astra:finding}`subpercent_alpha_iso_precision` + * {astra:prior-insight}`recon_sharpens_bao_peak` + * + * Scoping: a component path is `` (root analysis) or `.` + * (sub-analysis), e.g. `reconstruction.algorithm`. Sub-analysis paths can + * nest (`a.b.id`). Table directives take a bare scope path (`reconstruction`) + * or nothing (root). + * + * The plugin reads the ASTRA project once (cached) and renders each component + * via the per-component helpers in `./transform/`. + * + * The project root defaults to `process.cwd()` (run `myst start` from the + * project dir). Override with `ASTRA_PROJECT_ROOT`; pick a universe with + * `ASTRA_UNIVERSE` (defaults to the first in `universes/`). + */ + +import { basename, join, relative, sep } from 'node:path'; +import { loadASTRASource, resolveArtifact, type ArtifactResolver } from './loader.js'; +import type { Analysis, Input, Insight, Output, Universe } from '@astra-spec/sdk'; +import { + makeProseParser, + resolveNarrativeAnchors, + firstParagraphText, +} from './transform/narrative-parser.js'; +import type { + AnalysisScope, + PriorInsightScope, + ProseParser, +} from './transform/narrative-parser.js'; +import { + admonition, + admonitionTitle, + card, + code, + crossReference, + details, + emphasis, + heading, + inlineCode, + makeTabItem, + paragraph, + strong, + summary, + text, +} from './transform/ast-helpers.js'; +import { renderDecision, isDecisionRendered } from './transform/render-methods.js'; +import { renderFinding } from './transform/render-findings.js'; +import { renderOneOutput, renderInsightEvidence } from './transform/render-evidence.js'; +import { renderInputsTable, renderOutputsTable } from './transform/render-data-sources.js'; +import { parseTableData } from './transform/parse-table-data.js'; +import { resolveOutputs } from './transform/resolve-output.js'; +import { buildResolvedStore } from './transform/resolved-store.js'; + +// ── Project loading + cache ───────────────────────────────────────────── + +function projectRoot(): string { + return process.env['ASTRA_PROJECT_ROOT'] || process.cwd(); +} + +function universeName(): string | undefined { + return process.env['ASTRA_UNIVERSE'] || undefined; +} + +type Source = ReturnType; + +const projectCache = new Map(); + +function getSource(root: string, universe?: string): Source { + const key = `${root}::${universe ?? ''}`; + let src = projectCache.get(key); + if (!src) { + src = loadASTRASource(root, universe); + projectCache.set(key, src); + } + return src; +} + +// ── Scope resolution ──────────────────────────────────────────────────── + +interface Scope { + root: string; + analysis: Analysis; + universe: Universe; + /** Lazily resolves an output id → artifact path within this scope. */ + results: ArtifactResolver; + prose: ProseParser; + /** Local prior_insights merged with all ancestor scopes (option-tab refs). */ + priorInsights: Record; + outputsById: Map; + slug: string; + tabItem: ReturnType; + priorInsightScopes: PriorInsightScope[]; + analysisScopes: AnalysisScope[]; +} + +/** + * Walk from the root analysis into `analysisPath`: descend the analyses + * tree, narrow the universe to each sub-analysis's selections, and + * accumulate the prior-insight / analysis scope stacks the prose parser + * needs for cross-scope anchor resolution. + */ +function resolveScope( + root: string, + universe: string | undefined, + analysisPath: string[], +): Scope { + const source = getSource(root, universe); + let analysis = source.analysis; + let activeUniverse = source.universe; + const priorInsightScopes: PriorInsightScope[] = []; + const analysisScopes: AnalysisScope[] = []; + const slugParts: string[] = []; + // The scope's results root: the project dir, extended by each descended + // sub-analysis's `path:` (relative to its parent, so nesting composes). An + // output's artifact then lives at `/results///`. + let resultsBase = root; + + for (const seg of analysisPath) { + const child = analysis.analyses?.[seg]; + if (!child) { + throw new Error( + `unknown sub-analysis "${seg}" (path: ${analysisPath.join('.') || ''})`, + ); + } + const parentSlug = slugParts.length ? slugParts.join('/') : 'index'; + const localPI = analysis.prior_insights ?? {}; + if (Object.keys(localPI).length > 0) { + priorInsightScopes.push({ slug: parentSlug, priorInsights: localPI }); + } + analysisScopes.push({ slug: parentSlug, analysis }); + + const subNode = activeUniverse.analyses?.[seg]; + activeUniverse = { + id: activeUniverse.id, + description: activeUniverse.description, + decisions: subNode?.decisions ?? {}, + analyses: subNode?.analyses, + }; + if (child.path) resultsBase = join(resultsBase, child.path.replace(/^\.\//, '')); + analysis = child; + slugParts.push(seg); + } + + const slug = slugParts.length ? slugParts.join('/') : 'index'; + const universeId = source.universe.id; + const results: ArtifactResolver = (id) => resolveArtifact(resultsBase, universeId, id); + const prose = makeProseParser({ + analysis, + slug, + priorInsightScopes, + analysisScopes, + results, + }); + const priorInsights = Object.assign( + {}, + ...priorInsightScopes.map((s) => s.priorInsights), + analysis.prior_insights ?? {}, + ); + // Resolved view, keyed by declared id: aliased outputs (`from:`) inherit + // type/description/inputs/decisions/recipe from their source, so the figure/ + // table directive, the provenance disclosure, and inline cards all see the + // real artifact rather than a bare pointer. + const outputsById = new Map( + resolveOutputs(analysis).map(({ resolved }) => [resolved.id, resolved] as const), + ); + + return { + root, + analysis, + universe: activeUniverse, + results, + prose, + priorInsights, + outputsById, + slug, + tabItem: makeTabItem(), + priorInsightScopes, + analysisScopes, + }; +} + +/** Absolute result path → posix project-relative URL for MyST's asset copy. */ +function projectRelative(root: string, absPath: string): string { + return relative(root, absPath).split(sep).join('/'); +} + +function resultUrl(root: string): (absPath: string) => string { + return (absPath) => projectRelative(root, absPath); +} + +/** + * Rewrite `/static/` image URLs (the content-server scheme that + * MySTRA's shared evidence renderer emits) into project-relative result + * paths so MyST's asset pipeline can copy them. Applied to directive + * output as a final pass; covers figures embedded as finding evidence. + */ +function rewriteStaticImages(nodes: any[], scope: Scope): any[] { + const walk = (n: any): void => { + if (!n || typeof n !== 'object') return; + if (n.type === 'image' && typeof n.url === 'string' && n.url.startsWith('/static/')) { + const stem = n.url.slice('/static/'.length).replace(/\.[^.]+$/, ''); + const abs = scope.results(stem); + if (abs) n.url = projectRelative(scope.root, abs); + } + if (Array.isArray(n.children)) n.children.forEach(walk); + }; + nodes.forEach(walk); + return nodes; +} + +// ── Helpers ─────────────────────────────────────────────────────────────── + +function errorNode(message: string): any { + return { + type: 'admonition', + kind: 'error', + children: [ + { type: 'admonitionTitle', children: [{ type: 'text', value: 'ASTRA plugin' }] }, + { type: 'paragraph', children: [{ type: 'text', value: message }] }, + ], + }; +} + +/** Split a component path into [analysisPath, componentId]. */ +function splitPath(arg: unknown): { analysisPath: string[]; id: string | null } { + const parts = String(arg ?? '') + .trim() + .split('.') + .filter(Boolean); + const id = parts.pop() ?? null; + return { analysisPath: parts, id }; +} + +// ── Recognition markers ───────────────────────────────────────────────────── +// +// Every placed ASTRA block carries a stable `astra-` class (+ optional +// `--`) on the node that bears its `-` identifier. The class +// is harmless to book-theme but lets a rich theme select the element +// (`.astra-output`, `[identifier^="output-"]`) and join it to the resolved +// store by id (STRATEGY-A-REFACTOR.md §5). + +/** Add a semantic class to a node, idempotently (space-joined). */ +function addClass(node: any, cls: string): void { + if (!node || typeof node !== 'object') return; + const have = typeof node.class === 'string' ? node.class.split(/\s+/).filter(Boolean) : []; + if (!have.includes(cls)) have.push(cls); + node.class = have.join(' '); +} + +/** + * Tag the carrier node of a rendered component (the one bearing + * `-`, else the first node) with `astra-` and, when given, + * `astra---`. Returns the same node array for chaining. + */ +function tagComponent( + nodes: any[], + kind: string, + idPrefix: string, + id: string, + subtype?: string, +): any[] { + const ident = `${idPrefix}-${id}`; + const carrier = nodes.find((n) => n?.identifier === ident) ?? nodes[0]; + if (carrier) { + addClass(carrier, `astra-${kind}`); + if (subtype) addClass(carrier, `astra-${kind}--${subtype}`); + } + return nodes; +} + +// ── Block directives ("import") ───────────────────────────────────────────── + +/** Directive that resolves a `.` path and renders one component. */ +function componentDirective( + name: string, + render: (id: string, scope: Scope, options: Record) => any[], + options?: Record, +) { + return { + name: `astra:${name}`, + doc: `Import the ASTRA ${name} as a rich block.`, + arg: { type: String, required: true, doc: 'Component path: or .' }, + ...(options ? { options } : {}), + run(data: any): any[] { + const { analysisPath, id } = splitPath(data?.arg); + if (!id) return [errorNode(`astra:${name} requires an id`)]; + try { + const scope = resolveScope(projectRoot(), universeName(), analysisPath); + return rewriteStaticImages(render(id, scope, data?.options ?? {}), scope); + } catch (err) { + return [errorNode(`astra:${name} "${data?.arg}": ${(err as Error).message}`)]; + } + }, + }; +} + +/** + * A collapsed "ASTRA provenance" disclosure for an output: its id + type, the + * upstream products it was derived from, the decisions that parameterise it, + * and the recipe command. Emitted as a sibling after the figure/table so the + * embedded output reads as a first-class, traceable analysis product. + */ +function outputProvenance(output: Output, id: string): any { + const inner: any[] = [ + summary([strong([text('ASTRA provenance')]), text(` — ${id} · ${output.type ?? 'output'}`)]), + ]; + const inputs = output.inputs ?? []; + if (inputs.length > 0) { + const shown = inputs.slice(0, 6).join(', ') + (inputs.length > 6 ? ', …' : ''); + inner.push( + paragraph([ + strong([text('Derived from: ')]), + text(`${inputs.length} upstream product${inputs.length === 1 ? '' : 's'} — `), + inlineCode(shown), + ]), + ); + } + const decisions = output.decisions ?? []; + if (decisions.length > 0) { + const parts: any[] = [strong([text('Decisions: ')])]; + decisions.forEach((d, i) => { + if (i > 0) parts.push(text(', ')); + parts.push(crossReference(`decision-${d}`, [text(d)])); + }); + inner.push(paragraph(parts)); + } + if (output.recipe?.command) { + inner.push(code('bash', output.recipe.command)); + } + return details(inner, false); +} + +/** Directive whose whole arg is a scope path (no trailing component). */ +function tableDirective(name: string, render: (scope: Scope) => any[]) { + return { + name: `astra:${name}`, + doc: `Render the ASTRA ${name} table for an analysis scope (default: root).`, + arg: { type: String, required: false, doc: 'Sub-analysis scope, e.g. clustering' }, + run(data: any): any[] { + const analysisPath = String(data?.arg ?? '') + .trim() + .split('.') + .filter(Boolean); + try { + const scope = resolveScope(projectRoot(), universeName(), analysisPath); + return render(scope); + } catch (err) { + return [errorNode(`astra:${name} "${data?.arg ?? ''}": ${(err as Error).message}`)]; + } + }, + }; +} + +const decisionDirective = componentDirective('decision', (id, scope) => { + const decision = scope.analysis.decisions?.[id]; + if (!decision) throw new Error(`no decision "${id}" in this scope`); + if (!isDecisionRendered(decision, scope.universe)) { + throw new Error( + `decision "${id}" is a bare from-reference or its \`when\` is unmet under universe "${scope.universe.id}"`, + ); + } + return tagComponent( + renderDecision( + id, + decision, + scope.priorInsights, + scope.universe, + scope.prose, + scope.tabItem, + ), + 'decision', + 'decision', + id, + ); +}); + +const outputDirective = componentDirective('output', (id, scope) => { + const output = scope.outputsById.get(id); + if (!output) throw new Error(`no output "${id}" in this scope`); + const figure = renderOneOutput(output, id, scope.results, scope.prose, { + resultUrl: resultUrl(scope.root), + }); + // Rich representation: the rendered artifact + a collapsed ASTRA provenance + // disclosure (id/type, upstream products, decisions, recipe). The carrier + // (figure/table) is tagged `astra-output[ --]` for theme recognition. + return tagComponent([...figure, outputProvenance(output, id)], 'output', 'output', id, output.type); +}); + +const findingDirective = componentDirective( + 'finding', + (id, scope, options) => { + const findings = scope.analysis.findings ?? {}; + const finding = findings[id]; + if (!finding) throw new Error(`no finding "${id}" in this scope`); + const index = Object.keys(findings).indexOf(id) + 1; + // `:compact:` renders just the claim heading + notes + scope (no evidence + // figures) — used for the back-matter hover/click targets so the inline + // hover overlay stays tight and figures aren't duplicated. + if (options?.compact) { + const nodes: any[] = [ + heading(3, [text(`${index}. `), ...scope.prose.inline(finding.claim)], `finding-${id}`), + ]; + if (finding.notes) nodes.push(...scope.prose.blocks(finding.notes)); + if (finding.scope) nodes.push(paragraph([emphasis([text(`Scope: ${finding.scope}`)])])); + return tagComponent(nodes, 'finding', 'finding', id); + } + return tagComponent( + renderFinding( + finding, + index, + id, + scope.results, + scope.outputsById, + scope.prose, + ), + 'finding', + 'finding', + id, + ); + }, + { compact: { type: Boolean, doc: 'Render claim + notes + scope only (no evidence figures).' } }, +); + +const priorInsightDirective = componentDirective('prior-insight', (id, scope) => { + const insight = scope.analysis.prior_insights?.[id] ?? scope.priorInsights[id]; + if (!insight) throw new Error(`no prior_insight "${id}" in this scope`); + // MySTRA's renderPriorInsight emits a `container[kind=prior-insight]`, + // which the stock theme rejects ("no valid content besides caption"). + // For stock rendering we wrap the same content (claim + evidence) in a + // `seealso` admonition — a node every MyST theme renders cleanly — and + // carry the `prior_insight-` identifier so cross-references resolve. + const titleBits = ['Prior insight']; + if (insight.label) titleBits.push(insight.label); + else if (insight.scope) titleBits.push(insight.scope); + const body = [ + paragraph(scope.prose.inline(insight.claim)), + ...renderInsightEvidence(insight), + ]; + const node: any = admonition('seealso', [admonitionTitle([text(titleBits.join(' — '))]), ...body], { + class: 'astra-prior-insight', + }); + node.identifier = `prior_insight-${id}`; + node.label = node.identifier; + return [node]; +}); + +const inputsDirective = tableDirective('inputs', (scope) => { + const inputs = scope.analysis.inputs ?? []; + if (inputs.length === 0) return [errorNode('no inputs in this scope')]; + // Inputs are only carried by this table (no rich input block), so the + // `input-` row identifiers stay as the canonical anchor targets. + const table = renderInputsTable(inputs, scope.prose); + addClass(table, 'astra-inputs'); + return [table]; +}); + +const outputsDirective = tableDirective('outputs', (scope) => { + const outputs = scope.analysis.outputs ?? []; + if (outputs.length === 0) return [errorNode('no outputs in this scope')]; + const table = renderOutputsTable(outputs, scope.prose); + // Strip row identifiers: the canonical `output-` carrier is the rich + // `:::{astra:output}` block. Leaving them here would collide when the + // report both lists an output in the registry and embeds it as a figure. + for (const row of table.children ?? []) { + delete row.identifier; + delete row.label; + } + addClass(table, 'astra-outputs'); + return [table]; +}); + +const subAnalysisDirective = { + name: 'astra:subanalysis', + doc: 'Render a navigation card linking to a sub-analysis page.', + arg: { type: String, required: true, doc: 'Sub-analysis path, e.g. reconstruction' }, + run(data: any): any[] { + const { analysisPath, id } = splitPath(data?.arg); + if (!id) return [errorNode('astra:subanalysis requires a sub-analysis id')]; + try { + const scope = resolveScope(projectRoot(), universeName(), analysisPath); + const sub = scope.analysis.analyses?.[id]; + if (!sub) throw new Error(`no sub-analysis "${id}" in this scope`); + const title = sub.name ?? id; + const url = '/' + [...analysisPath, id].join('/'); + const summary = firstParagraphText(sub.narrative?.summary); + const children = summary ? [paragraph([text(summary)])] : []; + const node: any = card(title, children, url); + node.identifier = `analysis-${id}`; + node.label = node.identifier; + addClass(node, 'astra-subanalysis'); + return [node]; + } catch (err) { + return [errorNode(`astra:subanalysis "${data?.arg}": ${(err as Error).message}`)]; + } + }, +}; + +// ── Inline reference tokens with hover preview cards (Tier 1/2) ── +// +// Each inline ASTRA reference renders as a small token — a kind glyph + the +// element's label — carrying a self-contained hover card built from +// `astra.yaml`. The card is plain inline `span` nodes (which the stock theme +// renders) revealed on hover by custom CSS (`prototype/custom.css`): no theme +// fork, no graph views, just a focused preview of the referenced element. + +type CiteKind = 'decision' | 'output' | 'finding' | 'prior_insight' | 'analysis'; + +const KIND_NAME: Record = { + decision: 'Decision', + finding: 'Finding', + prior_insight: 'Prior insight', + analysis: 'Sub-analysis', + output: 'Output', + value: 'Value', +}; + +function span(cls: string, children: any[]): any { + return { type: 'span', class: cls, children }; +} +function tspan(cls: string, value: string): any { + return span(cls, [text(value)]); +} +function clip(s: string | undefined, n = 220): string { + if (!s) return ''; + const t = s.replace(/\s+/g, ' ').trim(); + return t.length > n ? `${t.slice(0, n - 1)}…` : t; +} +/** snake_case id → readable words, for the inline label when nothing better. */ +function humanize(id: string): string { + return id.replace(/_/g, ' '); +} + +// Neutral by design: tokens and cards carry ONLY semantic classes +// (`astra-ref`/`astra-card` + `--` / `--` modifiers) and text. +// No glyphs, colours, or inline styles are baked into the AST — all appearance +// is left to CSS keyed on these classes, so any theme can restyle the overlays. + +/** A hover preview card: eyebrow + title + optional labelled body lines. */ +function refCard( + kinds: string[], + kindName: string, + id: string, + title: string, + lines: Array<{ cls: string; text: string }>, +): any { + const cls = ['astra-card', ...kinds.map((k) => `astra-card--${k}`)].join(' '); + const kids: any[] = [ + tspan('astra-card__eyebrow', `${kindName} · ${id}`), + tspan('astra-card__title', title), + ]; + for (const l of lines) if (l.text) kids.push(tspan(l.cls, l.text)); + const node: any = span(cls, kids); + // Hidden by default so a bare viewer (plugin only, no theme CSS) shows just + // the clean label — never the card content inline. A theme/stylesheet reveals + // it on hover (overriding this with `display:… !important`). `style` must be + // an object — the React renderer rejects a string. + node.style = { display: 'none' }; + return node; +} + +/** An inline token: label + hover card; kind/subtype carried as classes. */ +function refToken(kinds: string[], label: string, card: any): any { + const cls = ['astra-ref', ...kinds.map((k) => `astra-ref--${k}`)].join(' '); + return span(cls, [tspan('astra-ref__label', label), card]); +} + +/** Build the inline token + preview card for one cited element. */ +function buildCite(kind: CiteKind, id: string, scope: Scope, display?: string | null): any { + if (kind === 'decision') { + const dec = scope.analysis.decisions?.[id]; + if (!dec) return tspan('astra-ref astra-ref--decision', display ?? humanize(id)); + const sel = scope.universe.decisions?.[id] ?? dec.default; + const selLabel = sel ? dec.options?.[sel]?.label ?? sel : null; + const label = display ?? dec.label ?? humanize(id); + const card = refCard(['decision'], KIND_NAME['decision'], id, label, [ + { cls: 'astra-card__pick', text: selLabel ? `Selected: ${selLabel}` : '' }, + { cls: 'astra-card__body', text: clip(dec.rationale) }, + ]); + return refToken(['decision'], label, card); + } + if (kind === 'finding') { + const f = scope.analysis.findings?.[id]; + if (!f) return tspan('astra-ref astra-ref--finding', display ?? humanize(id)); + const label = display ?? f.label ?? humanize(id); + const card = refCard(['finding'], KIND_NAME['finding'], id, clip(f.claim, 160), [ + { cls: 'astra-card__body', text: clip(f.notes, 200) }, + { cls: 'astra-card__meta', text: f.scope ? `Scope: ${f.scope}` : '' }, + ]); + return refToken(['finding'], label, card); + } + if (kind === 'prior_insight') { + const ins = scope.analysis.prior_insights?.[id] ?? scope.priorInsights[id]; + if (!ins) return tspan('astra-ref astra-ref--prior_insight', display ?? humanize(id)); + const label = display ?? ins.label ?? humanize(id); + const ev = (ins.evidence ?? []).find((e) => e.doi && e.quote?.exact); + // Citation hint is the bare DOI; MyST owns author–year resolution. + const card = refCard(['prior_insight'], KIND_NAME['prior_insight'], id, label, [ + { cls: 'astra-card__body', text: clip(ins.claim, 200) }, + { cls: 'astra-card__quote', text: ev?.quote?.exact ? `“${ev.quote.exact}”` : '' }, + { cls: 'astra-card__meta', text: ev?.doi ?? '' }, + ]); + return refToken(['prior_insight'], label, card); + } + if (kind === 'analysis') { + const sub = scope.analysis.analyses?.[id]; + if (!sub) return tspan('astra-ref astra-ref--analysis', display ?? humanize(id)); + const label = display ?? sub.name ?? humanize(id); + const counts = [ + Object.keys(sub.decisions ?? {}).length + ? `${Object.keys(sub.decisions ?? {}).length} decisions` + : '', + (sub.outputs ?? []).length ? `${(sub.outputs ?? []).length} outputs` : '', + ] + .filter(Boolean) + .join(' · '); + const card = refCard(['analysis'], KIND_NAME['analysis'], id, label, [ + { cls: 'astra-card__body', text: clip(firstParagraphText(sub.narrative?.summary), 200) }, + { cls: 'astra-card__meta', text: counts }, + ]); + return refToken(['analysis'], label, card); + } + // output — `subtype` (figure/table/metric/…) is a second modifier class so a + // theme can give each output type its own glyph/treatment. + const o = scope.outputsById.get(id); + if (!o) return tspan('astra-ref astra-ref--output', display ?? humanize(id)); + const subtype = o.type ?? 'output'; + const label = display ?? o.label ?? humanize(id); + const card = refCard(['output', subtype], KIND_NAME['output'], id, label, [ + { cls: 'astra-card__body', text: clip(o.description, 200) }, + { cls: 'astra-card__meta', text: `${o.type ?? 'output'} product` }, + ]); + return refToken(['output', subtype], label, card); +} + +/** Inline citation → glyph token + hover preview card. */ +function citeRole(name: string, kind: CiteKind) { + return { + name: `astra:${name}`, + doc: `Inline reference to an ASTRA ${name}, with a hover preview card.`, + body: { + type: String, + required: true, + doc: 'Path: or ., optionally `|display text` for the inline label', + }, + run(data: any): any[] { + // Optional `|display text` overrides the inline label (the card still + // shows the element's own label/claim). + const [pathPart, ...rest] = String(data?.body ?? '').split('|'); + const display = rest.join('|').trim() || null; + const { analysisPath, id } = splitPath(pathPart); + if (!id) return [text(String(data?.body ?? ''))]; + try { + const scope = resolveScope(projectRoot(), universeName(), analysisPath); + return [buildCite(kind, id, scope, display)]; + } catch { + return [tspan(`astra-ref astra-ref--${kind}`, display ?? humanize(id))]; + } + }, + }; +} + +// ── Value interpolation role ──────────────────────────────────────────────── + +/** Format a numeric string to `sig` significant figures, trimming zeros. */ +function fmtNum(raw: string, sig: number): string { + const x = Number(raw); + if (!isFinite(x)) return String(raw); + // Round to `sig` figures, then let Number→String drop trailing zeros and + // normalise the form (e.g. 200000 not 2.000e+5, 0.0696 not 0.06960). + return String(Number(x.toPrecision(sig))); +} + +function valueError(msg: string): any { + return { type: 'inlineCode', value: `⟨value: ${msg}⟩` }; +} + +/** + * `{astra:value}` — interpolate a real number from a materialised result + * product, so no measured value is ever hard-typed into the prose. + * + * Body grammar (whitespace-separated): + * col= [= ...] [pm] [sig=N] + * + * - `` output id, optionally scoped (`clustering.xi_…`). + * - `col=` the column to read (table outputs). + * - `=` row filters, e.g. `tracer=lrg3_elg1 recon=Post`. + * - `pm` also render `± _std` when that column exists. + * - `sig=N` significant figures (default 4). + * + * e.g. ``{astra:value}`bao_distance_table tracer=lrg3_elg1 col=DV_over_rd pm` `` + * reads `results//bao_distance_table/…csv` and renders `19.88 ± 0.17`. */ +const valueRole = { + name: 'astra:value', + doc: 'Interpolate a numeric cell from a table result product (no hard-typed numbers).', + body: { type: String, required: true, doc: ' col= [= ...] [pm] [sig=N]' }, + run(data: any): any[] { + const tokens = String(data?.body ?? '').trim().split(/\s+/).filter(Boolean); + const path = tokens.shift(); + if (!path) return [valueError('missing output path')]; + const opts: Record = {}; + for (const t of tokens) { + const i = t.indexOf('='); + if (i < 0) opts[t] = true; + else opts[t.slice(0, i)] = t.slice(i + 1); + } + try { + const { analysisPath, id } = splitPath(path); + if (!id) return [valueError(`missing output id in "${path}"`)]; + const scope = resolveScope(projectRoot(), universeName(), analysisPath); + const abs = scope.results(id); + if (!abs) return [valueError(`no result file for "${path}"`)]; + const tbl = parseTableData(abs); + if (!tbl) return [valueError(`"${id}" is not tabular`)]; + const col = typeof opts['col'] === 'string' ? (opts['col'] as string) : null; + if (!col) return [valueError(`missing col= for "${id}"`)]; + const ci = tbl.headers.indexOf(col); + if (ci < 0) return [valueError(`no column "${col}" in "${id}"`)]; + const reserved = new Set(['col', 'pm', 'sig', 'err']); + const filters = Object.entries(opts).filter(([k]) => !reserved.has(k)); + const row = tbl.rows.find((r) => + filters.every(([k, v]) => { + const ki = tbl.headers.indexOf(k); + return ki >= 0 && String(r[ki]).toLowerCase() === String(v).toLowerCase(); + }), + ); + if (!row) { + const desc = filters.map(([k, v]) => `${k}=${v as string}`).join(', ') || '(no filter)'; + return [valueError(`no row [${desc}] in "${id}"`)]; + } + const sig = typeof opts['sig'] === 'string' ? parseInt(opts['sig'] as string, 10) : 4; + let out = fmtNum(row[ci], sig); + // Uncertainty: explicit `err=`, else `pm` uses the `_std` + // convention (matches the distance table; the α table needs `err=`). + const errCol = + typeof opts['err'] === 'string' ? (opts['err'] as string) : opts['pm'] ? `${col}_std` : null; + if (errCol) { + const ei = tbl.headers.indexOf(errCol); + if (ei >= 0 && row[ei] != null && row[ei] !== '' && row[ei] !== '-') { + out += ` ± ${fmtNum(row[ei], 2)}`; + } + } + // Render the number as a token with a focused hover card naming its + // source product, column, and row — so the reader sees the value is + // sourced data and exactly where it comes from (no whole-table overlay). + const output = scope.outputsById.get(id); + const subtype = output?.type ?? 'table'; + const filterDesc = filters.map(([k, v]) => `${k}=${v as string}`).join(', '); + const card = refCard(['value', subtype], KIND_NAME['value'], id, out, [ + { cls: 'astra-card__pick', text: `Column ${col}` }, + { cls: 'astra-card__body', text: filterDesc ? `Row: ${filterDesc}` : '' }, + { + cls: 'astra-card__meta', + text: `${output?.type ?? 'table'} product${output?.label ? ` · ${output.label}` : ''}`, + }, + ]); + return [refToken(['value', subtype], out, card)]; + } catch (err) { + return [valueError((err as Error).message)]; + } + }, +}; + +// ── Transform: ASTRA anchor grammar in author prose ────────────────────────── + +/** + * The ASTRA scope a page maps to, or `null` for non-ASTRA pages (e.g. an + * `about.md`). Scope is derived from the file's basename: `index` → root, + * `` → the `` sub-analysis. (For deeper nesting, declare scope + * explicitly via paths in roles/directives.) + */ +function scopeForFile(vfile: any): Scope | null { + const base = basename(vfile?.path ?? '', '.md'); + const analysisPath = base && base !== 'index' ? [base] : []; + try { + return resolveScope(projectRoot(), universeName(), analysisPath); + } catch { + return null; + } +} + +/** + * Rewrite ASTRA tree-path anchor links (`[text](#decisions.x)`, + * `#outputs.y`, `#analyses.sub.outputs.z`, …) that appear in the *author's* + * prose into `crossReference` nodes (same page) or sub-page links — reusing + * MySTRA's `resolveNarrativeAnchors`. Directives already resolve anchors in + * the prose they render; this covers anchors the author writes directly. + * Author-written output-image anchors gain a `/static/` url here, so the + * same `rewriteStaticImages` pass the directives use rewrites them to a + * project-relative path MyST can copy. + */ +const anchorTransform = { + name: 'astra-anchor-grammar', + doc: 'Resolve ASTRA #path.to.element anchor links to cross-references.', + stage: 'document', + plugin: () => (tree: any, vfile: any) => { + const scope = scopeForFile(vfile); + if (!scope) return; + const resolved = resolveNarrativeAnchors( + tree.children ?? [], + scope.analysis, + scope.slug, + scope.priorInsightScopes, + scope.results, + scope.analysisScopes, + ); + tree.children = rewriteStaticImages(resolved, scope); + }, +}; + +// ── Transform: emit the resolved ASTRA store for rich themes ───────────────── +// +// The theme cannot read `astra.yaml` (it only sees the build output), so the +// plugin bakes a *resolved* projection of the page's analysis scope — keyed by +// id — onto a hidden carrier node's `data`. A rich theme selects the carrier +// (`.astra-store`) and joins each placed element's identifier (`output-`, +// `decision-`, …) to its store entry, enabling cards / dependency graphs / +// alternative layouts without re-implementing ASTRA semantics. The carrier is +// an empty `display:none` div, so it is invisible on book-theme. +// See STRATEGY-A-REFACTOR.md §5. + +/** Ancestor input maps (innermost-last) for resolving aliased `from:` inputs. */ +function parentInputMaps(scope: Scope): Map[] { + return scope.analysisScopes.map( + (s) => new Map((s.analysis.inputs ?? []).map((i) => [i.id, i] as const)), + ); +} + +const storeTransform = { + name: 'astra-resolved-store', + doc: 'Emit the resolved ASTRA data store (keyed by id) for rich themes.', + stage: 'document', + plugin: () => (tree: any, vfile: any) => { + const scope = scopeForFile(vfile); + if (!scope) return; + const store = buildResolvedStore( + scope.analysis, + scope.universe, + scope.results, + scope.slug, + resultUrl(scope.root), + parentInputMaps(scope), + ); + const carrier: any = { + type: 'div', + class: 'astra-store', + identifier: 'astra-store', + style: { display: 'none' }, + data: { astra: store }, + children: [], + }; + (tree.children ??= []).push(carrier); + }, +}; -export { astraToMystAST, buildAllPages } from './transform/index.js'; -export type { ASTRASource } from './transform/index.js'; +// ── Plugin export ───────────────────────────────────────────────────────── -export { loadASTRASource } from './loader/index.js'; +const plugin = { + name: 'astra', + directives: [ + decisionDirective, + outputDirective, + findingDirective, + priorInsightDirective, + inputsDirective, + outputsDirective, + subAnalysisDirective, + ], + roles: [ + citeRole('decision', 'decision'), + citeRole('output', 'output'), + citeRole('finding', 'finding'), + citeRole('prior-insight', 'prior_insight'), + citeRole('analysis', 'analysis'), + valueRole, + ], + transforms: [anchorTransform, storeTransform], +}; -export { createContentServer } from './server/index.js'; -export type { ServerOptions, ContentServer } from './server/index.js'; +export default plugin; -export type { PageData, SiteManifest, PageContent } from './types/content-server.js'; -export type { ASTRAAnalysis, ASTRAUniverse, ASTRADecision, ASTRAInsight } from './types/astra.js'; +// ── Library exports (for programmatic use) ────────────────────────────────── +export { loadASTRASource } from './loader.js'; +export type { ASTRASource } from './loader.js'; +export { buildResolvedStore } from './transform/resolved-store.js'; +export type { + ResolvedStore, + SerializedOutput, + SerializedInput, + SerializedDecision, + SerializedFinding, + SerializedInsight, + SerializedSubAnalysis, + SerializedMetric, + SerializedRecipe, +} from './transform/resolved-store.js'; diff --git a/src/loader.ts b/src/loader.ts new file mode 100644 index 0000000..8c59e71 --- /dev/null +++ b/src/loader.ts @@ -0,0 +1,87 @@ +/** + * Load an ASTRA project for one universe, and resolve output artifacts. + * + * Most of the work is the SDK's: `loadYaml` parses, `resolveAnalysisTree` + * inlines `path:` sub-analyses into one tree (preserving each sub's `path:`). + * What stays here is MySTRA-specific: picking a universe and locating result + * files on disk. + */ + +import { dirname, join, parse as parsePath } from 'node:path'; +import { existsSync, readdirSync, statSync } from 'node:fs'; +import { loadYaml, resolveAnalysisTree } from '@astra-spec/sdk'; +import type { Analysis, Universe } from '@astra-spec/sdk'; + +/** A loaded ASTRA project for one universe; `resolveScope` walks it from here. */ +export interface ASTRASource { + analysis: Analysis; + universe: Universe; + projectDir: string; + /** Slug of the loaded node (`index` for the root analysis). */ + slug: string; +} + +/** Resolve an output id to its artifact's absolute path, or `undefined`. */ +export type ArtifactResolver = (outputId: string) => string | undefined; + +export function loadASTRASource(projectDir: string, universeName?: string): ASTRASource { + const astraPath = join(projectDir, 'astra.yaml'); + if (!existsSync(astraPath)) throw new Error(`No astra.yaml found in ${projectDir}`); + const analysis = resolveAnalysisTree(loadYaml(astraPath), projectDir) as unknown as Analysis; + return { analysis, universe: loadUniverse(projectDir, universeName), projectDir, slug: 'index' }; +} + +/** + * Load one universe from `universes/.yaml` (the file stem is the universe + * id, per the lightcone convention), or the first file when no name is given, + * or a synthetic empty universe when there are none. + */ +function loadUniverse(projectDir: string, name?: string): Universe { + const dir = join(projectDir, 'universes'); + const file = name + ? `${name}.yaml` + : existsSync(dir) + ? readdirSync(dir).filter((f) => /\.ya?ml$/.test(f)).sort()[0] + : undefined; + if (!file || !existsSync(join(dir, file))) { + if (name) throw new Error(`Universe "${name}" not found in ${dir}`); + return { id: 'default', decisions: {} }; + } + return loadYaml(join(dir, file)) as unknown as Universe; +} + +/** + * Locate an output's artifact file — deterministically, on demand. + * + * astra-spec leaves the on-disk path to the runner; lightcone-cli fixes the + * output *directory* as `[/]results///`, so it is + * computed (never scanned). The recipe chooses the file *name*, so we read that + * one directory, preferring `.`, else the first regular file (dotfiles, + * incl. `.lightcone-manifest.json`, skipped). Absent directory → not produced. + */ +export function resolveArtifact( + base: string, + universeId: string, + outputId: string, +): string | undefined { + const dir = join(base, 'results', universeId, outputId); + let entries: string[]; + try { + entries = readdirSync(dir); + } catch { + return undefined; + } + const files = entries + .filter((f) => !f.startsWith('.') && safeIsFile(join(dir, f))) + .sort(); + if (files.length === 0) return undefined; + return join(dir, files.find((f) => parsePath(f).name === outputId) ?? files[0]); +} + +function safeIsFile(path: string): boolean { + try { + return statSync(path).isFile(); + } catch { + return false; + } +} diff --git a/src/loader/index.ts b/src/loader/index.ts deleted file mode 100644 index b4dd69f..0000000 --- a/src/loader/index.ts +++ /dev/null @@ -1,36 +0,0 @@ -/** - * Orchestrates loading of all ASTRA source data. - */ - -import { join } from 'node:path'; -import { existsSync } from 'node:fs'; -import { loadAnalysis } from './yaml-loader.js'; -import { loadUniverses, selectUniverse } from './universe-loader.js'; -import { scanResults } from './result-scanner.js'; -import type { ASTRASource } from '../transform/index.js'; - -export function loadASTRASource( - projectDir: string, - universeName?: string, -): ASTRASource { - const astraPath = join(projectDir, 'astra.yaml'); - if (!existsSync(astraPath)) { - throw new Error(`No astra.yaml found in ${projectDir}`); - } - - const analysis = loadAnalysis(astraPath); - - const universesDir = join(projectDir, 'universes'); - const universes = loadUniverses(universesDir); - const universe = selectUniverse(universes, universeName); - - const results = scanResults(projectDir, universe.id); - - // Top-level analysis is the index page; sub-analyses get their own - // ASTRASource constructed inside buildAllPages with the correct slug. - return { analysis, universe, results, projectDir, slug: 'index' }; -} - -export { loadAnalysis } from './yaml-loader.js'; -export { loadUniverses, selectUniverse } from './universe-loader.js'; -export { scanResults } from './result-scanner.js'; diff --git a/src/loader/result-scanner.ts b/src/loader/result-scanner.ts deleted file mode 100644 index 4cd00be..0000000 --- a/src/loader/result-scanner.ts +++ /dev/null @@ -1,99 +0,0 @@ -/** - * Scans the results directory(ies) for produced output artifacts. - * - * Top-level outputs live at `results//.` (the project - * root). Sub-analysis outputs live at `analyses//results//...`, - * mirroring the convention `lightcone-ui-liam/packages/core/src/bundle.ts` - * uses (`subOutputPath` / `rootOutputPath`). - * - * The walker descends `analyses/*` recursively so sub-of-sub results are - * picked up too. All matches are merged into a single `output_id -> - * absolute path` map. Output IDs are unique within a single analysis - * scope; collisions across scopes are last-write-wins (the practical - * collision risk is low for current reproductions and does not block - * any existing parity work). - */ - -import { readdirSync, existsSync, statSync } from 'node:fs'; -import { join, parse as parsePath } from 'node:path'; - -/** - * Scan a single `results//` directory and merge files into the - * passed-in map. Files in nested directories are skipped — Liam's pipeline - * uses a flat `/` directory per scope and we follow the same - * shape here. Hidden files / dotfiles are skipped. - */ -function mergeResultsDir( - resultsDir: string, - results: Map, -): void { - if (!existsSync(resultsDir)) return; - try { - const files = readdirSync(resultsDir); - for (const file of files) { - const parsed = parsePath(file); - if (parsed.name.startsWith('.')) continue; - const absPath = join(resultsDir, file); - try { - if (statSync(absPath).isDirectory()) continue; - } catch { - continue; - } - results.set(parsed.name, absPath); - } - } catch (err) { - console.warn( - `[mystra] Could not read results directory "${resultsDir}": ${ - err instanceof Error ? err.message : String(err) - }`, - ); - } -} - -/** - * Recursively walk `/analyses/*` and scan each sub-analysis's own - * `results//` directory. The directory layout convention follows - * `lightcone-ui-liam`'s bundle pipeline: - * - * /results//... ← root outputs - * /analyses//results//... ← sub outputs - * /analyses//analyses//results//... ← deeper - */ -function walkAnalysesDir( - scopeDir: string, - universeId: string, - results: Map, -): void { - const analysesDir = join(scopeDir, 'analyses'); - if (!existsSync(analysesDir)) return; - let entries: string[]; - try { - entries = readdirSync(analysesDir); - } catch { - return; - } - for (const entry of entries) { - const subDir = join(analysesDir, entry); - try { - if (!statSync(subDir).isDirectory()) continue; - } catch { - continue; - } - mergeResultsDir(join(subDir, 'results', universeId), results); - walkAnalysesDir(subDir, universeId, results); - } -} - -/** - * Scan the project's results directories (root + every nested - * sub-analysis) and return a map of output_id -> absolute file path. - */ -export function scanResults( - projectDir: string, - universeId: string, -): Map { - const results = new Map(); - mergeResultsDir(join(projectDir, 'results', universeId), results); - walkAnalysesDir(projectDir, universeId, results); - return results; -} diff --git a/src/loader/universe-loader.ts b/src/loader/universe-loader.ts deleted file mode 100644 index 2cffb62..0000000 --- a/src/loader/universe-loader.ts +++ /dev/null @@ -1,50 +0,0 @@ -/** - * Loads universe YAML files from the universes/ directory. - */ - -import { readFileSync, readdirSync, existsSync } from 'node:fs'; -import { join } from 'node:path'; -import yaml from 'js-yaml'; -import type { ASTRAUniverse } from '../types/astra.js'; - -/** - * Load all universe files from a directory. - */ -export function loadUniverses(universesDir: string): ASTRAUniverse[] { - if (!existsSync(universesDir)) return []; - - const files = readdirSync(universesDir) - .filter((f) => f.endsWith('.yaml') || f.endsWith('.yml')) - .sort(); - - return files.map((file) => { - const content = readFileSync(join(universesDir, file), 'utf-8'); - return yaml.load(content) as ASTRAUniverse; - }); -} - -/** - * Select the active universe by name, or default to the first one. - * If no universes exist, returns a synthetic empty universe. - */ -export function selectUniverse( - universes: ASTRAUniverse[], - name?: string, -): ASTRAUniverse { - if (name) { - const found = universes.find((u) => u.id === name); - if (!found) { - throw new Error( - `Universe "${name}" not found. Available: ${universes.map((u) => u.id).join(', ')}`, - ); - } - return found; - } - - if (universes.length > 0) { - return universes[0]; - } - - // Synthetic empty universe when no universe files exist - return { id: 'default', decisions: {} }; -} diff --git a/src/loader/yaml-loader.ts b/src/loader/yaml-loader.ts deleted file mode 100644 index 44fba6c..0000000 --- a/src/loader/yaml-loader.ts +++ /dev/null @@ -1,39 +0,0 @@ -/** - * Loads and parses astra.yaml files. - */ - -import { readFileSync } from 'node:fs'; -import { resolve, dirname, join } from 'node:path'; -import yaml from 'js-yaml'; -import type { ASTRAAnalysis } from '../types/astra.js'; - -/** - * Load an astra.yaml file, recursively resolving sub-analyses with `path` fields. - */ -export function loadAnalysis(filePath: string): ASTRAAnalysis { - const content = readFileSync(filePath, 'utf-8'); - const data = yaml.load(content) as ASTRAAnalysis; - - // Ensure dictionaries default to empty objects - if (!data.decisions) data.decisions = {}; - if (!data.prior_insights) data.prior_insights = {}; - if (!data.findings) data.findings = {}; - - // Recursively resolve sub-analyses with `path` fields - if (data.analyses) { - const baseDir = dirname(filePath); - for (const [id, sub] of Object.entries(data.analyses)) { - if (sub.path) { - const subPath = resolve(baseDir, sub.path, 'astra.yaml'); - data.analyses[id] = loadAnalysis(subPath); - } else { - // Inline sub-analysis — ensure defaults - if (!sub.decisions) sub.decisions = {}; - if (!sub.prior_insights) sub.prior_insights = {}; - if (!sub.findings) sub.findings = {}; - } - } - } - - return data; -} diff --git a/src/papers/index.ts b/src/papers/index.ts deleted file mode 100644 index bf8ea20..0000000 --- a/src/papers/index.ts +++ /dev/null @@ -1,180 +0,0 @@ -import { existsSync, readdirSync, readFileSync } from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import type { DOIMetadata } from '../doi/resolver.js'; -import type { ASTRAAnalysis } from '../types/astra.js'; -import type { PaperDecisionLink, PaperInsightSummary } from '../types/papers.js'; - -interface CachedPaperMeta { - doi: string; - title: string | null; - authors: string[]; - version: number | null; - cache_key: string; -} - -export function resolvePaperCacheDir(): string { - const env = process.env.ASTRA_PAPER_CACHE_DIR; - if (env) return path.resolve(env); - return path.join(os.homedir(), '.cache', 'astra', 'papers'); -} - -function sanitizeDoi(doi: string, version?: number | null): string { - let safe = doi.replace(/\//g, '_').replace(/:/g, '_'); - safe = safe.replace(/[^\w.\-]/g, '_'); - if (version != null) safe = `${safe}_v${version}`; - return safe; -} - -function cleanText(value: unknown): string | null { - if (typeof value !== 'string') return null; - const cleaned = value.replace(/<[^>]+>/g, '').replace(/\s+/g, ' ').trim(); - return cleaned || null; -} - -function readPaperMeta(cacheDir: string, cacheKey: string): CachedPaperMeta | null { - const dir = path.join(cacheDir, cacheKey); - const pdfPath = path.join(dir, 'paper.pdf'); - const metaPath = path.join(dir, 'meta.json'); - if (!existsSync(pdfPath) || !existsSync(metaPath)) return null; - try { - const raw = JSON.parse(readFileSync(metaPath, 'utf-8')) as { - doi?: string; - title?: unknown; - authors?: unknown; - version?: unknown; - }; - const authors = Array.isArray(raw.authors) - ? raw.authors.map(cleanText).filter((author): author is string => Boolean(author)) - : []; - return { - doi: raw.doi ?? '', - title: cleanText(raw.title), - authors, - version: typeof raw.version === 'number' ? raw.version : null, - cache_key: cacheKey, - }; - } catch { - return null; - } -} - -export function findCachedPaper(cacheDir: string, doi: string): CachedPaperMeta | null { - if (!existsSync(cacheDir)) return null; - const base = sanitizeDoi(doi); - const exact = readPaperMeta(cacheDir, base); - if (exact) return exact; - const prefix = `${base}_v`; - let best: { version: number; meta: CachedPaperMeta } | null = null; - for (const entry of readdirSync(cacheDir)) { - if (!entry.startsWith(prefix)) continue; - const match = entry.match(/_v(\d+)$/); - if (!match) continue; - const version = Number(match[1]); - const meta = readPaperMeta(cacheDir, entry); - if (!meta) continue; - if (!best || version > best.version) best = { version, meta }; - } - return best?.meta ?? null; -} - -function decisionHref(slug: string, decisionId: string): string { - return slug === 'index' - ? `/decisions#decision-${decisionId}` - : `/${slug}/decisions#decision-${decisionId}`; -} - -function decisionKey(slug: string, decisionId: string): string { - return slug === 'index' ? decisionId : `${slug}.${decisionId}`; -} - -function insightKey(slug: string, insightId: string): string { - return slug === 'index' ? insightId : `${slug}:${insightId}`; -} - -function collectDecisionLinksByInsight( - analysis: ASTRAAnalysis, - slug = 'index', - byInsight = new Map(), -): Map { - for (const [decisionId, decision] of Object.entries(analysis.decisions ?? {})) { - if (decision?.from) continue; - const link: PaperDecisionLink = { - key: decisionKey(slug, decisionId), - id: decisionId, - label: decision.label ?? decisionId, - slug, - href: decisionHref(slug, decisionId), - }; - - for (const option of Object.values(decision.options ?? {})) { - for (const insightId of option.insights ?? []) { - const links = byInsight.get(insightId) ?? []; - if (!links.some((existing) => existing.key === link.key)) { - links.push(link); - byInsight.set(insightId, links); - } - } - } - } - - for (const [subId, sub] of Object.entries(analysis.analyses ?? {})) { - const subSlug = slug === 'index' ? subId : `${slug}/${subId}`; - collectDecisionLinksByInsight(sub, subSlug, byInsight); - } - - return byInsight; -} - -function collectInsightsByDoi( - analysis: ASTRAAnalysis, - decisionsByInsight: Map, - slug = 'index', - byDoi = new Map(), -): Map { - for (const [insightId, insight] of Object.entries(analysis.prior_insights ?? {})) { - const evidence = insight.evidence?.[0]; - if (!evidence?.doi) continue; - const summary: PaperInsightSummary = { - id: insightKey(slug, insightId), - claim: insight.claim, - quote: evidence.quote?.exact, - page: evidence.location?.page, - informs: decisionsByInsight.get(insightId) ?? [], - }; - const summaries = byDoi.get(evidence.doi) ?? []; - summaries.push(summary); - byDoi.set(evidence.doi, summaries); - } - - for (const [subId, sub] of Object.entries(analysis.analyses ?? {})) { - const subSlug = slug === 'index' ? subId : `${slug}/${subId}`; - collectInsightsByDoi(sub, decisionsByInsight, subSlug, byDoi); - } - - return byDoi; -} - -export function buildPaperMetadata( - metadata: Map, - analysis: ASTRAAnalysis, - cacheDir = resolvePaperCacheDir(), -): Map { - const decisionsByInsight = collectDecisionLinksByInsight(analysis); - const insightsByDoi = collectInsightsByDoi(analysis, decisionsByInsight); - return new Map( - Array.from(metadata.entries(), ([doi, meta]) => { - const cached = findCachedPaper(cacheDir, doi); - return [ - doi, - { - ...meta, - version: cached?.version ?? undefined, - cache_key: cached?.cache_key ?? undefined, - pdf_url: cached ? `/papers/${encodeURIComponent(cached.cache_key)}/paper.pdf` : undefined, - insights: insightsByDoi.get(doi) ?? [], - }, - ]; - }), - ); -} diff --git a/src/server/index.ts b/src/server/index.ts deleted file mode 100644 index 34f52d3..0000000 --- a/src/server/index.ts +++ /dev/null @@ -1,219 +0,0 @@ -/** - * Content server — Express app with all routes, static serving, - * WebSocket, and file watcher. - */ - -import { createReadStream, stat } from 'node:fs'; -import { createServer as createHTTPServer } from 'node:http'; -import { basename, join, resolve, sep } from 'node:path'; -import express from 'express'; -import cors from 'cors'; -import { configHandler } from './routes/config.js'; -import { contentHandler } from './routes/content.js'; -import { xrefHandler } from './routes/xref.js'; -import { astraHandler, buildASTRADataMap, type ASTRAPageData } from './routes/astra.js'; -import { WebSocketManager } from './websocket.js'; -import { startWatcher } from './watcher.js'; -import { loadASTRASource } from '../loader/index.js'; -import { buildAllPages } from '../transform/index.js'; -import { resolveAllDOIs, type DOIMetadata } from '../doi/resolver.js'; -import { buildPaperMetadata, resolvePaperCacheDir } from '../papers/index.js'; -import type { ASTRAAnalysis } from '../types/astra.js'; -import type { PageData, References } from '../types/content-server.js'; - -export interface ServerOptions { - projectDir: string; - contentPort: number; - universeName?: string; -} - -export interface ContentServer { - start(): Promise; - close(): void; - getPages(): PageData[]; -} - -export function createContentServer(options: ServerOptions): ContentServer { - const { projectDir, contentPort, universeName } = options; - - let pages: PageData[] = []; - let astraDataMap: Map = new Map(); - let references: References = {}; - let doiMetadata: Map = new Map(); - /** - * `output_id -> absolute path` from the recursive scanner. Populated by - * `loadASTRASource` (see `loader/result-scanner.ts`); covers the root - * `results//` dir and every sub-analysis's - * `analyses//results//` dir. Used to resolve `/static/` - * URLs by matching the URL's basename against `basename(absPath)` for any - * id in the map. - */ - let resultsByOutputId: Map = new Map(); - let wsManager: WebSocketManager; - let activeUniverseId = ''; - let doiReloadToken = 0; - - function reload(): ASTRAAnalysis { - const source = loadASTRASource(projectDir, universeName); - activeUniverseId = source.universe.id; - resultsByOutputId = source.results; - pages = buildAllPages( - source.analysis, - source.universe, - source.results, - source.projectDir, - ); - astraDataMap = buildASTRADataMap(source.analysis, source.results); - refreshDOIMetadata(source.analysis); - console.log( - `[mystra] Loaded: ${pages.length} page(s), universe "${source.universe.id}"`, - ); - return source.analysis; - } - - function refreshDOIMetadata(analysis: ASTRAAnalysis) { - const token = ++doiReloadToken; - const allDOIs = [...new Set(pages.flatMap((page) => page.dois))]; - - if (allDOIs.length === 0) { - references = {}; - doiMetadata = new Map(); - return; - } - - const cacheDir = join(projectDir, '.mystra-cache', 'doi'); - const paperCacheDir = resolvePaperCacheDir(); - resolveAllDOIs(allDOIs, cacheDir) - .then((result) => { - if (token !== doiReloadToken) return; - references = result.references; - doiMetadata = buildPaperMetadata(result.metadata, analysis, paperCacheDir); - const count = Object.keys(result.references.cite?.data ?? {}).length; - console.log(`[mystra] Resolved ${count} citation(s)`); - if (wsManager) { - wsManager.broadcast({ type: 'RELOAD' }); - } - }) - .catch((err) => { - if (token !== doiReloadToken) return; - console.warn('[mystra] DOI resolution error:', err); - }); - } - - // Initial load - reload(); - - const siteTitle = pages[0]?.title ?? 'ASTRA Analysis'; - const papersRoot = resolvePaperCacheDir(); - const papersRootAbs = resolve(papersRoot); - - // Express app - const app = express(); - app.use(cors()); - - // API routes - app.get('/config.json', configHandler(() => pages, siteTitle)); - app.get('/content/*.json', contentHandler(() => pages, () => references)); - app.get('/myst.xref.json', xrefHandler(() => pages)); - app.get('/astra/*.json', astraHandler(() => astraDataMap)); - - app.get('/doi-metadata/:doi(*)', (req, res) => { - const doi = req.params['doi']; - const meta = doiMetadata.get(doi); - if (meta) { - res.json(meta); - } else { - res.status(404).json({ error: 'DOI not resolved' }); - } - }); - - app.use('/papers', (req, res) => { - const urlPath = decodeURIComponent((req.url ?? '/').split('?')[0]); - const rel = urlPath.replace(/^\/+/, ''); - const target = resolve(papersRootAbs, rel); - if (!target.startsWith(`${papersRootAbs}${sep}`) && target !== papersRootAbs) { - res.status(403).end('forbidden'); - return; - } - stat(target, (err, fileStat) => { - if (err || !fileStat.isFile()) { - res.status(404).end('not found'); - return; - } - res.status(200); - res.setHeader('Content-Type', 'application/pdf'); - res.setHeader('Cache-Control', 'no-store'); - createReadStream(target).pipe(res); - }); - }); - - // Static file serving for result artifacts. The recursive scanner - // already discovered every result file across the root and every - // sub-analysis (`/results//...` plus - // `/analyses//results//...`); we keep its - // map in `resultsByOutputId` and resolve `/static/` by - // matching against the basenames it indexed. Falls through to the - // root `results//` static dir for legacy callers that - // request files we haven't indexed (e.g. ad-hoc figures). - const staticDir = join(projectDir, 'results', activeUniverseId); - app.use('/static', (req, res, next) => { - const urlPath = decodeURIComponent((req.url ?? '/').split('?')[0]); - const rel = urlPath.replace(/^\/+/, ''); - if (!rel) return next(); - // Match by basename across every scanned result file. - for (const absPath of resultsByOutputId.values()) { - if (basename(absPath) === rel) { - res.status(200); - res.setHeader('Cache-Control', 'no-store'); - createReadStream(absPath).pipe(res); - return; - } - } - return next(); - }); - app.use('/static', express.static(staticDir)); - - // HTTP server - const server = createHTTPServer(app); - - // WebSocket - wsManager = new WebSocketManager(server); - - // File watcher - const watcher = startWatcher(projectDir, () => { - console.log('[mystra] File change detected, reloading...'); - try { - reload(); - wsManager.broadcast({ type: 'RELOAD' }); - } catch (err) { - console.error('[mystra] Reload failed:', err); - wsManager.broadcast({ - type: 'LOG', - message: `Reload error: ${err}`, - }); - } - }); - - return { - start() { - return new Promise((resolveStart) => { - server.listen(contentPort, () => { - console.log( - `[mystra] Content server listening on http://localhost:${contentPort}`, - ); - resolveStart(); - }); - }); - }, - - close() { - watcher.close(); - wsManager.close(); - server.close(); - }, - - getPages() { - return pages; - }, - }; -} diff --git a/src/server/routes/astra.ts b/src/server/routes/astra.ts deleted file mode 100644 index 5894cd5..0000000 --- a/src/server/routes/astra.ts +++ /dev/null @@ -1,340 +0,0 @@ -/** - * /astra/.json — structured ASTRA data for a page slug. - * - * Returns the resolved outputs and inputs for a given analysis, keyed by the - * same slug that the content server uses for /content/.json. Renderers - * that want ASTRA-native gallery views (figure previews, file inventories) hit - * this endpoint instead of parsing the MDAST for structural tables. - * - * Outputs carry a `resolved_path` that resolves to `/static/` — the - * same static mount the content server uses for result artifacts. Inputs carry - * their `source` URL (for data inputs) and `from` path (for aliased inputs). - * - * This fills the emission gap named in the dual-branch parity constitution: - * the MDAST tables that MySTRA emits for outputs/inputs convey prose context - * but not the artifact-native shape (gallery cards, file inventory rows) that - * renderers need to match the vanilla paper-view baseline. - */ - -import { basename } from 'node:path'; -import { readFileSync, existsSync } from 'node:fs'; -import type { RequestHandler } from 'express'; -import type { ASTRAAnalysis, ASTRAInput, ASTRAOutput } from '../../types/astra.js'; -import { resolveOutputs } from '../../transform/resolve-output.js'; -import { parseTableData, type TableData } from '../../transform/parse-table-data.js'; - -// ── Public types ────────────────────────────────────────────────────────────── - -export interface SerializedRecipe { - /** Resolved shell command — recipe.command from astra.yaml. */ - command?: string; - /** Container image / Containerfile path (when declared). */ - container?: string; -} - -/** - * Inlined metric-output value data, parsed from the materialised result file - * at build time so renderers can hero-print value ± uncertainty + unit - * without a second round-trip. Mirrors the shape the React port's - * `MetricGalleryCard` and `lightcone-ui-liam`'s `attachMetricData` consume. - * - * Three accepted source shapes: - * - bare number / string → { value } - * - `[value, uncertainty]` 2-tuple → { value, uncertainty } - * - object with at least `value` → spread as-is (value, uncertainty, - * unit, label, …) - * - * Anything else (or an unreadable file) leaves `metric` absent; the renderer - * falls back to a "no value" placeholder. - */ -export interface SerializedMetric { - value?: number | string; - uncertainty?: number | string; - /** Alias for `uncertainty` accepted by some convention; the renderer reads - * either. */ - error?: number | string; - unit?: string; - /** Plural alias accepted by some convention. */ - units?: string; - label?: string; -} - -export interface SerializedOutput { - id: string; - label?: string; - type?: string; - description?: string; - /** Relative URL served by the content server's /static mount, or undefined - * when no result artifact was found for this output. */ - resolved_path?: string; - /** Recipe command + container. When the output is aliased (`from:`), - * recipe is inherited from the source by `resolveOutputs`. */ - recipe?: SerializedRecipe; - /** Input IDs (in the surrounding analysis scope) this output depends on. */ - inputs?: string[]; - /** Decision IDs (declared on the output) that parameterise this artefact. - * Drives the "Decisions affecting this artefact" section on the per-output - * detail page. Absent / empty → "None — no decision flows into this - * artefact's recipe chain" (matching Liam's vanilla template). */ - decisions?: string[]; - /** Re-export pointer for aliased outputs (`from: child.out_id`). When set, - * type/description/inputs/decisions/recipe are inherited from the source; - * the alias node carries only id/from/when. */ - from?: string; - /** - * Parsed table data for table-type outputs. Populated by MySTRA at build - * time using the same CSV/JSON parser as the narrative evidence renderer. - * - * Absent for non-table outputs, missing result files, unsupported - * extensions (parquet, etc.), or when the row count exceeds MAX_INLINE_ROWS. - * When `truncated` is true the source file has more rows than `rows.length`. - */ - table_data?: TableData; - /** - * Inlined metric value, populated for `type: 'metric'` outputs whose - * result file parses as JSON. Absent for non-metric outputs, missing - * result files, non-JSON extensions, or unparseable content. - */ - metric?: SerializedMetric; -} - -export interface SerializedInput { - id: string; - label?: string; - type?: string; - description?: string; - /** URL for data inputs (external dataset reference). */ - source?: string; - /** Path string for aliased inputs that forward to a parent or sibling output. */ - from?: string; -} - -export interface ASTRAPageData { - outputs: SerializedOutput[]; - inputs: SerializedInput[]; -} - -// ── Route handler ───────────────────────────────────────────────────────────── - -export function astraHandler( - getDataMap: () => Map, -): RequestHandler { - return (req, res) => { - const slug = (req.params as Record)[0] ?? 'index'; - const map = getDataMap(); - const data = map.get(slug); - if (!data) { - res.status(404).json({ error: `No ASTRA data for slug: ${slug}` }); - return; - } - res.json(data); - }; -} - -// ── Data builder ────────────────────────────────────────────────────────────── - -/** - * Walk the analysis tree (same recursion as `buildAllPages`) and produce a - * slug → ASTRAPageData map. Called once per reload; the server stores the - * result and serves it via `astraHandler`. - * - * `parentInputScopes` carries the input maps of every ancestor analysis, - * outermost first. Aliased inputs (`from: `) walk the chain inner → - * outer to inherit description / source / type from the source declaration, - * matching Liam's bundle pipeline which renders the resolved input record - * regardless of which scope owns the alias. - */ -export function buildASTRADataMap( - analysis: ASTRAAnalysis, - results: Map, - basePath = '', - parentInputScopes: Map[] = [], -): Map { - const map = new Map(); - const slug = basePath || 'index'; - - const resolvedOuts = resolveOutputs(analysis); - const outputs: SerializedOutput[] = resolvedOuts.map(({ declared, resolved }) => { - // For table-type outputs with a materialized result file, parse and inline - // the data so the React renderer can display it without a second round-trip. - // Uses the same CSV/JSON parser as the narrative evidence renderer — no - // second reader introduced. - const absPath = results.get(declared.id); - const tableData = - resolved.type === 'table' && absPath ? (parseTableData(absPath) ?? undefined) : undefined; - const metric = resolved.type === 'metric' && absPath ? readMetric(absPath) : undefined; - - return { - id: declared.id, - label: resolved.label, - type: resolved.type, - description: resolved.description, - resolved_path: resolvedPath(declared.id, results), - // Provenance (Liam parity for `#/spec/` per-output detail page): - // recipe — command + declared inputs (the "Recipe" section) - // decisions — IDs that parameterise this output (the "Decisions - // affecting this artefact" section; absent → "None") - // from — alias pointer for re-exported outputs - // Aliased outputs (`from:`) inherit recipe/decisions/inputs from the - // source via resolveOutputs above, so the serialised view is the - // resolved one regardless of whether the declaration site was an alias. - recipe: resolved.recipe - ? { command: resolved.recipe.command, container: resolved.recipe.container } - : undefined, - inputs: resolved.inputs, - decisions: resolved.decisions, - from: declared.from, - table_data: tableData, - metric, - }; - }); - - // Build this scope's own input map — used for output-input resolution - // within this scope, and as a parent scope for nested sub-analyses. - const localInputs: Map = new Map( - (analysis.inputs ?? []).map((i) => [i.id, i] as const), - ); - - const inputs: SerializedInput[] = (analysis.inputs ?? []).map((inp: ASTRAInput) => - serializeInput(inp, parentInputScopes), - ); - - map.set(slug, { outputs, inputs }); - - // Recurse into sub-analyses (mirrors buildAllPages depth-first walk). - // Children inherit our scope at the end of `parentInputScopes` so they - // can resolve `from:` references against ancestors. - const childParentScopes = [...parentInputScopes, localInputs]; - for (const [subId, sub] of Object.entries(analysis.analyses ?? {})) { - const subPath = basePath ? `${basePath}/${subId}` : subId; - for (const [subSlug, subData] of buildASTRADataMap( - sub, - results, - subPath, - childParentScopes, - )) { - map.set(subSlug, subData); - } - } - - return map; -} - -/** - * Resolve an input declaration into a `SerializedInput` for the wire. - * - * Plain inputs (no `from:`): pass-through. - * - * Aliased inputs (`from: `): walk `parentInputScopes` from innermost to - * outermost looking for a matching id. The source's content fields - * (`type`, `label`, `description`, `source`) are inherited; the alias keeps - * its own `id` and `from` pointer so consumers can still distinguish "this - * scope declared it" from "inherited from above". - * - * Mirrors Liam's bundle pipeline behaviour (`subInputPath` / aliased inputs - * inherit from the source declaration). The `from:` value MAY use the - * v0.0.7 path syntax (`../id`, `../../id`, `../scope.out_id`) — this lookup - * accepts either the bare id (current mystra emit shape) or the trailing - * id token after the last `.` or `/`. Sibling-output references - * (`../scope.out_id`) leave the alias untouched; that's an output-input - * cross-link and should still display as "from " in the UI. - */ -function serializeInput( - inp: ASTRAInput, - parentInputScopes: Map[], -): SerializedInput { - const out: SerializedInput = { - id: inp.id, - label: inp.label, - type: inp.type, - description: inp.description, - source: inp.source, - from: inp.from, - }; - if (!inp.from) return out; - // Output-input cross-link (`../scope.out_id`): leave alone — the source - // is an Output, not an Input, and the modal renders the from pointer. - if (inp.from.includes('.')) return out; - const targetId = lastSegment(inp.from); - // Walk inner -> outer so the closest declaration wins on conflict. - for (let i = parentInputScopes.length - 1; i >= 0; i--) { - const scope = parentInputScopes[i]; - const src = scope.get(targetId); - if (!src) continue; - return { - ...out, - type: out.type ?? src.type, - label: out.label ?? src.label, - description: out.description ?? src.description, - source: out.source ?? src.source, - }; - } - return out; -} - -/** Strip leading `../` segments from a `from:` path and return the trailing id. */ -function lastSegment(path: string): string { - const parts = path.split('/'); - return parts[parts.length - 1]; -} - -// ── Helpers ─────────────────────────────────────────────────────────────────── - -function resolvedPath( - outputId: string, - results: Map, -): string | undefined { - const absPath = results.get(outputId); - if (!absPath) return undefined; - return `/static/${basename(absPath)}`; -} - -/** - * Read and parse a metric output's result file. Mirrors the - * `attachMetricData` recipe in `lightcone-ui-liam/packages/core/src/bundle.ts` - * — three accepted source shapes (bare scalar, 2-tuple, object), anything - * else returns undefined so the renderer falls back to a placeholder. - * - * Limited to `.json` files; other extensions don't surface as inline metric - * heroes (matches Liam's pipeline). Read errors are swallowed — silently - * dropping is correct here, the React renderer's empty state is the contract. - */ -function readMetric(absPath: string): SerializedMetric | undefined { - if (!absPath.toLowerCase().endsWith('.json')) return undefined; - if (!existsSync(absPath)) return undefined; - try { - const raw: unknown = JSON.parse(readFileSync(absPath, 'utf-8')); - if (typeof raw === 'number' || typeof raw === 'string') { - return { value: raw }; - } - if (Array.isArray(raw) && raw.length >= 1) { - const [value, uncertainty] = raw; - // Only surface scalar value/uncertainty pairs; arrays of objects - // belong on the table-data path, not the metric hero. - if (typeof value !== 'number' && typeof value !== 'string') return undefined; - const out: SerializedMetric = { value }; - if (typeof uncertainty === 'number' || typeof uncertainty === 'string') { - out.uncertainty = uncertainty; - } - return out; - } - if (raw && typeof raw === 'object' && 'value' in raw) { - // Spread the object — accept any of value/uncertainty/error/unit/units/label. - const obj = raw as Record; - const out: SerializedMetric = {}; - if (typeof obj.value === 'number' || typeof obj.value === 'string') - out.value = obj.value; - if (typeof obj.uncertainty === 'number' || typeof obj.uncertainty === 'string') - out.uncertainty = obj.uncertainty; - if (typeof obj.error === 'number' || typeof obj.error === 'string') - out.error = obj.error; - if (typeof obj.unit === 'string') out.unit = obj.unit; - if (typeof obj.units === 'string') out.units = obj.units; - if (typeof obj.label === 'string') out.label = obj.label; - return Object.keys(out).length > 0 ? out : undefined; - } - return undefined; - } catch { - return undefined; - } -} diff --git a/src/server/routes/config.ts b/src/server/routes/config.ts deleted file mode 100644 index 84dcb27..0000000 --- a/src/server/routes/config.ts +++ /dev/null @@ -1,40 +0,0 @@ -/** - * GET /config.json — Site manifest for the MyST book-theme. - */ - -import type { Request, Response } from 'express'; -import type { SiteManifest, ManifestProjectPage } from '../../types/content-server.js'; -import type { PageData } from '../../types/content-server.js'; - -export function configHandler(getPages: () => PageData[], siteTitle: string) { - return (_req: Request, res: Response) => { - const pages = getPages(); - - // Exclude the index page — the theme renders it via the project title - const manifestPages: ManifestProjectPage[] = pages - .filter((p) => p.slug !== 'index') - .map((p) => ({ - title: p.title, - slug: p.slug, - level: p.level, - description: p.frontmatter.description, - })); - - const manifest: SiteManifest = { - version: 1, - myst: '1.0.0', - id: 'mystra', - title: siteTitle, - projects: [ - { - slug: '', - index: 'index', - title: siteTitle, - pages: manifestPages, - }, - ], - }; - - res.json(manifest); - }; -} diff --git a/src/server/routes/content.ts b/src/server/routes/content.ts deleted file mode 100644 index 97834b0..0000000 --- a/src/server/routes/content.ts +++ /dev/null @@ -1,50 +0,0 @@ -/** - * GET /content/*.json — Page AST + frontmatter. - * - * The wildcard captures the full page slug, including any `/` - * separators that nested sub-analyses introduce (`buildAllPages` - * builds slugs by joining the analysis path with `/`, so a - * grandchild lives at slug `parent/child`). The Express handler - * exposes the captured value as `req.params[0]`. - */ - -import type { Request, Response } from 'express'; -import type { PageContent, PageData, References } from '../../types/content-server.js'; -import { sha256 } from '../../utils/hash.js'; - -export function contentHandler( - getPages: () => PageData[], - getReferences: () => References, -) { - return (req: Request, res: Response) => { - // `req.params[0]` is the wildcard capture from `/content/*.json`. - // It's the full slug regardless of depth, e.g. `index`, - // `reconstruction`, `reconstruction/step1`. - const slug = req.params[0] ?? req.params['slug']; - const pages = getPages(); - const page = pages.find((p) => p.slug === slug); - - if (!page) { - res.status(404).json({ error: `Page "${slug}" not found` }); - return; - } - - const astJson = JSON.stringify(page.ast); - const contentHash = sha256(astJson); - const references = getReferences(); - - const content: PageContent = { - kind: 'Article', - sha256: contentHash, - slug: page.slug, - domain: '', - project: '', - mdast: page.ast, - frontmatter: page.frontmatter, - references, - dependencies: page.dependencies, - }; - - res.json(content); - }; -} diff --git a/src/server/routes/xref.ts b/src/server/routes/xref.ts deleted file mode 100644 index 8a518a3..0000000 --- a/src/server/routes/xref.ts +++ /dev/null @@ -1,22 +0,0 @@ -/** - * GET /myst.xref.json — Cross-reference index. - */ - -import type { Request, Response } from 'express'; -import type { XRefIndex, PageData } from '../../types/content-server.js'; - -export function xrefHandler(getPages: () => PageData[]) { - return (_req: Request, res: Response) => { - const pages = getPages(); - - const references = pages.flatMap((p) => p.identifiers); - - const index: XRefIndex = { - version: '1', - myst: '1.0.0', - references, - }; - - res.json(index); - }; -} diff --git a/src/server/watcher.ts b/src/server/watcher.ts deleted file mode 100644 index 9784fef..0000000 --- a/src/server/watcher.ts +++ /dev/null @@ -1,47 +0,0 @@ -/** - * File watcher for live reload. - */ - -import chokidar from 'chokidar'; -import type { FSWatcher } from 'chokidar'; - -export function startWatcher( - projectDir: string, - onReload: () => void, -): FSWatcher { - const resultExts = ['png', 'jpg', 'jpeg', 'svg', 'csv', 'json', 'md']; - const resultGlobs = resultExts.flatMap((ext) => [ - `${projectDir}/results/**/*.${ext}`, - `${projectDir}/analyses/**/results/**/*.${ext}`, - ]); - - const watcher = chokidar.watch( - [ - `${projectDir}/astra.yaml`, - `${projectDir}/analyses/**/astra.yaml`, - `${projectDir}/universes/*.yaml`, - `${projectDir}/universes/*.yml`, - ...resultGlobs, - ], - { - ignoreInitial: true, - ignored: [ - '**/node_modules/**', - '**/.git/**', - '**/.mystra-cache/**', - '**/.dagster/**', - ], - }, - ); - - let debounceTimer: ReturnType | null = null; - - watcher.on('all', (_event, _path) => { - if (debounceTimer) clearTimeout(debounceTimer); - debounceTimer = setTimeout(() => { - onReload(); - }, 300); - }); - - return watcher; -} diff --git a/src/server/websocket.ts b/src/server/websocket.ts deleted file mode 100644 index 108586a..0000000 --- a/src/server/websocket.ts +++ /dev/null @@ -1,34 +0,0 @@ -/** - * WebSocket manager for live reload notifications. - */ - -import { WebSocketServer, WebSocket } from 'ws'; -import type { Server as HTTPServer } from 'node:http'; - -export class WebSocketManager { - private wss: WebSocketServer; - - constructor(server: HTTPServer) { - this.wss = new WebSocketServer({ server, path: '/socket' }); - } - - broadcast(data: { type: 'RELOAD' | 'LOG'; message?: string }): void { - const message = JSON.stringify(data); - for (const client of this.wss.clients) { - if (client.readyState === WebSocket.OPEN) { - client.send(message); - } - } - } - - getConnectionCount(): number { - return this.wss.clients.size; - } - - close(): void { - for (const client of this.wss.clients) { - client.close(); - } - this.wss.close(); - } -} diff --git a/src/theme/launcher.ts b/src/theme/launcher.ts deleted file mode 100644 index 3bc09b2..0000000 --- a/src/theme/launcher.ts +++ /dev/null @@ -1,159 +0,0 @@ -/** - * Spawns the MyST book-theme as a child process. - * - * The theme is a Remix Express app that reads CONTENT_CDN_PORT to know - * where to fetch page JSON from. We either find an already-downloaded - * template or download it via `mystmd`. - */ - -import { spawn, execSync, type ChildProcess } from 'node:child_process'; -import { existsSync } from 'node:fs'; -import { join, resolve } from 'node:path'; - -export interface ThemeLaunchOptions { - themePort: number; - contentPort: number; - projectDir: string; -} - -/** - * Launch the MyST book-theme directly from its template directory. - */ -export async function launchTheme(options: ThemeLaunchOptions): Promise { - const { themePort, contentPort, projectDir } = options; - - const themeDir = await ensureThemeInstalled(projectDir); - if (!themeDir) { - console.warn( - '[mystra] Could not set up theme. Running in content-server-only mode.', - ); - console.warn( - `[mystra] Content API available at http://localhost:${contentPort}`, - ); - return null; - } - - // Install node_modules if missing - const nodeModulesDir = join(themeDir, 'node_modules'); - if (!existsSync(nodeModulesDir)) { - console.log('[mystra] Installing theme dependencies...'); - try { - execSync('npm install --production', { cwd: themeDir, stdio: 'pipe' }); - } catch (err) { - console.error('[mystra] Failed to install theme dependencies:', (err as Error).message); - return null; - } - } - - // Launch the theme server - const child = spawn('node', ['server.js'], { - cwd: themeDir, - env: { - ...process.env, - HOST: 'localhost', - PORT: String(themePort), - CONTENT_CDN_PORT: String(contentPort), - MODE: 'app', - NODE_ENV: 'production', - }, - stdio: ['ignore', 'pipe', 'pipe'], - }); - - child.stdout?.on('data', (data: Buffer) => { - const line = data.toString().trim(); - if (line) console.log(`[theme] ${line}`); - }); - - child.stderr?.on('data', (data: Buffer) => { - const line = data.toString().trim(); - if (line) console.error(`[theme] ${line}`); - }); - - child.on('error', (err) => { - console.error('[mystra] Theme process error:', err.message); - }); - - child.on('exit', (code) => { - if (code !== null && code !== 0) { - console.error(`[mystra] Theme process exited with code ${code}`); - } - }); - - return child; -} - -/** - * Find or download the book-theme template. - * - * Search order: - * 1. _build/templates/site/myst/book-theme/ (previously downloaded by mystmd) - * 2. .mystra-cache/theme/ (our own cache) - * 3. Download via mystmd if available - */ -async function ensureThemeInstalled(projectDir: string): Promise { - // Check common locations for the template - const candidates = [ - join(projectDir, '_build', 'templates', 'site', 'myst', 'book-theme'), - join(process.cwd(), '_build', 'templates', 'site', 'myst', 'book-theme'), - join(projectDir, '.mystra-cache', 'theme'), - ]; - - for (const dir of candidates) { - if (existsSync(join(dir, 'server.js')) && existsSync(join(dir, 'build', 'index.js'))) { - console.log(`[mystra] Using theme at ${dir}`); - return dir; - } - } - - // Try to download via mystmd - console.log('[mystra] Theme not found locally. Downloading via mystmd...'); - const cacheDir = join(projectDir, '.mystra-cache', 'theme'); - - try { - // Use mystmd to fetch the template - execSync( - `npx -y mystmd templates list site --json 2>/dev/null || true`, - { stdio: 'pipe' }, - ); - - // Direct download approach: use mystmd's template fetch - execSync( - `npx -y mystmd templates fetch site/myst/book-theme --path "${cacheDir}" 2>&1`, - { stdio: 'pipe', timeout: 60_000 }, - ); - - if (existsSync(join(cacheDir, 'server.js'))) { - return cacheDir; - } - } catch { - // mystmd not available or fetch failed - } - - // Manual fallback: try npm pack - try { - console.log('[mystra] Trying npm-based theme installation...'); - const { mkdirSync } = await import('node:fs'); - mkdirSync(cacheDir, { recursive: true }); - - execSync( - 'npm init -y && npm install @myst-theme/book@latest 2>&1', - { cwd: cacheDir, stdio: 'pipe', timeout: 120_000 }, - ); - - // The npm package might have the template files - const pkgDir = join(cacheDir, 'node_modules', '@myst-theme', 'book'); - if (existsSync(join(pkgDir, 'server.js'))) { - return pkgDir; - } - } catch { - // npm approach also failed - } - - console.error( - '[mystra] Could not download theme.\n' + - ' Option 1: Run `mystmd start` once in any MyST project to download the theme, then retry.\n' + - ' Option 2: Install mystmd globally: npm install -g mystmd', - ); - - return null; -} diff --git a/src/transform/index.ts b/src/transform/index.ts deleted file mode 100644 index 71ac5c7..0000000 --- a/src/transform/index.ts +++ /dev/null @@ -1,347 +0,0 @@ -/** - * Main ASTRA → MyST AST transform. - * - * astraToMystAST() produces the full page AST for one analysis node. - * buildAllPages() handles recursive sub-analysis page generation. - */ - -import type { ASTRAAnalysis, ASTRAInsight, ASTRAUniverse, ASTRAUniverseNode } from '../types/astra.js'; -import type { PageData, PageFrontmatter, XRefEntry } from '../types/content-server.js'; -import { join } from 'node:path'; -import { blockBreak, makeTabItem } from './ast-helpers.js'; -import { renderNarrativeChunks } from './render-narrative.js'; -import { renderUniverseBanner } from './render-universe-banner.js'; -import { renderFindings } from './render-findings.js'; -import { renderPriorInsights } from './render-prior-insights.js'; -import { renderMethodsSections, isDecisionRendered } from './render-methods.js'; -import { renderInputsTable, renderOutputsTable } from './render-data-sources.js'; -import { renderOutputProvenance } from './render-output-provenance.js'; -import { renderOutputRecipes, hasRecipe } from './render-output-recipe.js'; -import { resolveOutputs } from './resolve-output.js'; -import { renderSubAnalysisCards } from './render-sub-analyses.js'; -import { makeProseParser, firstParagraphText } from './narrative-parser.js'; -import type { AnalysisScope, PriorInsightScope } from './narrative-parser.js'; - -export interface ASTRASource { - analysis: ASTRAAnalysis; - universe: ASTRAUniverse; - results: Map; - projectDir: string; - /** Slug of the host page; needed to build sub-analysis links from - * narrative anchors (e.g. `#analyses.feature_extraction`). */ - slug: string; - /** Ancestor prior_insights keyed by the page slug that owns their - * rendered `prior_insight-` carrier. */ - priorInsightScopes?: PriorInsightScope[]; - /** Ancestor analyses keyed by page slug for cross-scope output - * image embeds such as `#../outputs.`. */ - analysisScopes?: AnalysisScope[]; -} - -export function astraToMystAST(source: ASTRASource): { type: 'root'; children: any[] } { - const { analysis, universe, results, projectDir, slug } = source; - const decisions = analysis.decisions ?? {}; - const priorInsights = analysis.prior_insights ?? {}; - const priorInsightLookup = mergePriorInsights(source.priorInsightScopes ?? [], priorInsights); - const findings = analysis.findings ?? {}; - const inputs = analysis.inputs ?? []; - const outputs = analysis.outputs ?? []; - - // The prose parser is bound once to `(analysis, slug)` and threaded - // into every render-* helper that touches Markdown. This makes the - // narrative anchor grammar `[t](#path.to.element)` work in every - // prose surface, not just narrative sections — anchors in - // rationales, descriptions, claims, captions, and criterion claims - // all resolve to crossReferences against the host analysis. - const proseContext = { - analysis, - slug, - priorInsightScopes: source.priorInsightScopes, - analysisScopes: source.analysisScopes, - results, - }; - const prose = makeProseParser(proseContext); - - // Per-pass tabItem factory — each transform invocation gets its - // own counter so two consecutive transforms produce identical - // `key`s for downstream AST diffing. - const tabItem = makeTabItem(); - - // DOI cache dir threaded through every renderer that emits a cite - // node. Originates from `projectDir`; absent when no project dir - // is available (the cite node falls back to a plain DOI link). - const doiCacheDir = projectDir ? join(projectDir, '.mystra-cache', 'doi') : null; - - // Outputs as an id→Output map so artifact-evidence rendering can - // dispatch on `output.type` (figure/table/metric/data/report) in - // O(1). Builds once per page; broken evidence references emit a - // console.warn at render time when the id isn't in this map. - const outputsById = new Map(outputs.map((o) => [o.id, o] as const)); - - // Page layout: emit a flat sequence of named, addressable blocks - // — narrative chunks AND structural elements at the same level. - // No top-level section h2s, no narrative wrapper. mdast position - // is the spec-declared default; downstream renderers (paper, - // dashboard, DAG) compose layouts however they like by looking - // up `identifier` attributes. - const narrativeChunks = renderNarrativeChunks(analysis, slug, proseContext); - const children: any[] = [ - // Block break separating frontmatter from content - blockBreak('{"class": ""}'), - - // Narrative chunks: each section is an addressable block - // identified by `narrative-
`. Spec-declared order - // (summary → findings → methods → inputs → outputs). - ...narrativeChunks.flatMap((c) => c.mdast), - - // Universe banner — orientation for which decision selections - // are active in this rendering. Includes universe.description - // as prose so any anchor links inside it resolve too. - renderUniverseBanner(universe, decisions, prose), - - // Structural elements as a flat sequence of addressable blocks. - ...renderFindings(findings, results, outputsById, prose, doiCacheDir), - ...renderPriorInsights(priorInsights, prose, doiCacheDir), - ...renderMethodsSections(decisions, priorInsightLookup, universe, prose, tabItem, doiCacheDir), - ...(inputs.length > 0 ? [renderInputsTable(inputs, prose)] : []), - ...(outputs.length > 0 ? [renderOutputsTable(outputs, prose)] : []), - // Per-Output provenance blocks (Output.inputs / Output.decisions - // from astra-spec v0.0.7 PR #19). Sits adjacent to the outputs - // registry table; one container per Output with non-empty - // provenance, addressable as `output--provenance`. The - // resolved view is what renderers see — `from:` chains are - // walked here so consumers (lightcone-ui, vellum, …) never read - // `astra.yaml` directly to recover provenance. - ...renderOutputProvenance(analysis), - // Per-Output recipe carriers (the *how*: command, container, - // resources). One `kind: 'output-recipe'` container per Output - // with a non-empty resolved recipe. Display direction is - // renderer-side: structured `data` slot lets consumers - // pattern-match; fallback children are a `details` block - // collapsed by default. Closes the Recipe coverage hole — no - // consumer needs to read `astra.yaml` to recover the recipe. - ...renderOutputRecipes(analysis), - ...(analysis.analyses && Object.keys(analysis.analyses).length > 0 - ? renderSubAnalysisCards(analysis.analyses, slug, { - priorInsightScopes: nextPriorInsightScopes( - source.priorInsightScopes ?? [], - slug, - priorInsights, - ), - analysisScopes: nextAnalysisScopes(source.analysisScopes ?? [], slug, analysis), - results, - }) - : []), - ]; - - return { type: 'root', children }; -} - -/** - * Recursively build pages for an analysis and all sub-analyses. - */ -export function buildAllPages( - analysis: ASTRAAnalysis, - universe: ASTRAUniverse, - results: Map, - projectDir: string, - basePath = '', - level = 1, - priorInsightScopes: PriorInsightScope[] = [], - analysisScopes: AnalysisScope[] = [], -): PageData[] { - const pages: PageData[] = []; - const slug = basePath || 'index'; - - // Build page for this analysis node. astraToMystAST derives the - // DOI cache dir from `projectDir` and threads it into every - // renderer that emits a cite node — no module-global state. - const ast = astraToMystAST({ - analysis, - universe, - results, - projectDir, - slug, - priorInsightScopes, - analysisScopes, - }); - - // PageFrontmatter.description feeds OpenGraph/SEO/list previews. ASTRA - // v0.0.6 dropped the free-form `description` slot in favour of a - // structured narrative; the summary section is the closest analogue - // (single-paragraph orientation for the analysis), so we surface its - // first paragraph as plain text. No renderer-imposed `subtitle` — - // astra-spec defines no analysis-level subtitle field; pinning one - // here would assert content type in metadata. - const frontmatter: PageFrontmatter = { - title: analysis.name ?? slug, - authors: (analysis.authors ?? []).map((name) => ({ name })), - tags: analysis.tags, - description: firstParagraphText(analysis.narrative?.summary), - }; - - // Collect identifiers for cross-references - const identifiers = collectIdentifiers(analysis, universe, slug); - - // Collect static file dependencies - const dependencies = collectDependencies(results); - - // Collect DOIs - const dois = collectDOIs(analysis); - - pages.push({ - slug, - title: analysis.name ?? slug, - level, - ast, - frontmatter, - identifiers, - dependencies, - dois, - }); - - // Recurse into sub-analyses - if (analysis.analyses) { - const childPriorInsightScopes = nextPriorInsightScopes( - priorInsightScopes, - slug, - analysis.prior_insights ?? {}, - ); - const childAnalysisScopes = nextAnalysisScopes(analysisScopes, slug, analysis); - for (const [id, sub] of Object.entries(analysis.analyses)) { - const subPath = basePath ? `${basePath}/${id}` : id; - - // Get sub-universe selections - const subUniverseNode: ASTRAUniverseNode | undefined = universe.analyses?.[id]; - const subUniverse: ASTRAUniverse = { - id: universe.id, - description: universe.description, - decisions: subUniverseNode?.decisions ?? {}, - analyses: subUniverseNode?.analyses, - }; - - // Sub-analysis results would be in a nested path - // For now, pass the same results map (scanner handles universe-scoped paths) - pages.push( - ...buildAllPages( - sub, - subUniverse, - results, - projectDir, - subPath, - level + 1, - childPriorInsightScopes, - childAnalysisScopes, - ), - ); - } - } - - return pages; -} - -function mergePriorInsights( - scopes: PriorInsightScope[], - local: Record, -): Record { - return Object.assign({}, ...scopes.map((scope) => scope.priorInsights), local); -} - -function nextPriorInsightScopes( - scopes: PriorInsightScope[], - slug: string, - local: Record, -): PriorInsightScope[] { - return Object.keys(local).length > 0 - ? [...scopes, { slug, priorInsights: local }] - : scopes; -} - -function nextAnalysisScopes( - scopes: AnalysisScope[], - slug: string, - analysis: ASTRAAnalysis, -): AnalysisScope[] { - return [...scopes, { slug, analysis }]; -} - -function collectIdentifiers( - analysis: ASTRAAnalysis, - universe: ASTRAUniverse, - slug: string, -): XRefEntry[] { - const entries: XRefEntry[] = []; - const dataPath = `/content/${slug}.json`; - const url = slug === 'index' ? '/' : `/${slug}`; - - const push = (identifier: string) => - entries.push({ identifier, kind: 'heading', data: dataPath, url, implicit: true }); - - // Narrative-chunk identifiers: each non-empty section is an - // addressable block at `narrative-
`. - for (const section of ['summary', 'findings', 'methods', 'inputs', 'outputs'] as const) { - if (analysis.narrative?.[section]) push(`narrative-${section}`); - } - - // Per-element identifiers (`-`) for every structural - // element. Page-level section identifiers (findings, methods, …) - // are no longer published — those h2 wrappers no longer exist. - // The xref contract: only publish ids with a real carrier in the - // emitted AST. Decisions that aren't rendered (bare `from`-refs - // and `when`-unmet ones) are filtered with the same predicate - // renderMethodsSections uses. - for (const id of Object.keys(analysis.findings ?? {})) push(`finding-${id}`); - for (const id of Object.keys(analysis.prior_insights ?? {})) push(`prior_insight-${id}`); - for (const [id, decision] of Object.entries(analysis.decisions ?? {})) { - if (isDecisionRendered(decision, universe)) push(`decision-${id}`); - } - for (const input of analysis.inputs ?? []) push(`input-${input.id}`); - for (const output of analysis.outputs ?? []) push(`output-${output.id}`); - // Per-Output provenance + recipe carriers. Same predicate as - // their respective render modules (resolved view has the relevant - // content) so the xref contract — only publish ids with a real - // carrier — holds even for aliased outputs whose provenance/recipe - // arrives via `from:`. - for (const r of resolveOutputs(analysis)) { - const inputs = r.resolved.inputs ?? []; - const decisions = r.resolved.decisions ?? []; - if (inputs.length > 0 || decisions.length > 0) { - push(`output-${r.declared.id}-provenance`); - } - if (hasRecipe(r)) { - push(`output-${r.declared.id}-recipe`); - } - } - for (const id of Object.keys(analysis.analyses ?? {})) push(`analysis-${id}`); - - return entries; -} - -function collectDependencies(results: Map): string[] { - const deps: string[] = []; - for (const [outputId, filePath] of results) { - const ext = filePath.split('.').pop(); - if (ext && ['png', 'jpg', 'jpeg', 'svg'].includes(ext)) { - deps.push(`/static/${outputId}.${ext}`); - } - } - return deps; -} - -function collectDOIs(analysis: ASTRAAnalysis): string[] { - const dois = new Set(); - - for (const insight of Object.values(analysis.prior_insights ?? {})) { - for (const ev of insight.evidence) { - if (ev.doi) dois.add(ev.doi); - } - } - - for (const finding of Object.values(analysis.findings ?? {})) { - for (const ev of finding.evidence) { - if (ev.doi) dois.add(ev.doi); - } - } - - return Array.from(dois); -} diff --git a/src/transform/narrative-parser.ts b/src/transform/narrative-parser.ts index ee8ead8..d9d603c 100644 --- a/src/transform/narrative-parser.ts +++ b/src/transform/narrative-parser.ts @@ -20,7 +20,8 @@ import { mystParse } from 'myst-parser'; import { parse as parsePath } from 'node:path'; -import type { ASTRAAnalysis, ASTRAInsight, ASTRAOutput } from '../types/astra.js'; +import type { Analysis, Insight, Output } from '@astra-spec/sdk'; +import type { ArtifactResolver } from '../loader.js'; import { crossReference, link } from './ast-helpers.js'; // ── Parsing ─────────────────────────────────────────────────────── @@ -142,25 +143,25 @@ function extractInline(node: any): any[] { /** * Resolution context carried through every render-* helper that - * touches prose. Created once per page in `astraToMystAST` and - * threaded into the renderers via the `ProseParser` factory. + * touches prose. Created once per scope (by the plugin's `resolveScope`) + * and threaded into the renderers via the `ProseParser` factory. */ export interface ProseContext { - analysis: ASTRAAnalysis; + analysis: Analysis; slug: string; priorInsightScopes?: PriorInsightScope[]; analysisScopes?: AnalysisScope[]; - results?: Map; + results?: ArtifactResolver; } export interface PriorInsightScope { slug: string; - priorInsights: Record; + priorInsights: Record; } export interface AnalysisScope { slug: string; - analysis: ASTRAAnalysis; + analysis: Analysis; } /** @@ -248,7 +249,7 @@ function stripPositions(node: any): any { */ export function resolveAnchorPath( path: string, - analysis: ASTRAAnalysis, + analysis: Analysis, slug: string, priorInsightScopes: PriorInsightScope[] = [], ): { identifier: string } | { url: string } { @@ -412,10 +413,10 @@ function astraPathToFragment(segments: string[]): string { */ export function resolveNarrativeAnchors( nodes: any[], - analysis: ASTRAAnalysis, + analysis: Analysis, slug: string, priorInsightScopes: PriorInsightScope[] = [], - results?: Map, + results?: ArtifactResolver, analysisScopes: AnalysisScope[] = [], ): any[] { return nodes.flatMap((node) => @@ -446,10 +447,10 @@ function flatten(r: any | any[] | null | undefined): any[] { function rewrite( node: any, - analysis: ASTRAAnalysis, + analysis: Analysis, slug: string, priorInsightScopes: PriorInsightScope[], - results: Map | undefined, + results: ArtifactResolver | undefined, analysisScopes: AnalysisScope[], ): any | any[] | null { if (!node || typeof node !== 'object') return node; @@ -500,8 +501,8 @@ function isOutputImageAnchor(url: unknown): url is string { function rewriteOutputImage( node: any, - analysis: ASTRAAnalysis, - results: Map | undefined, + analysis: Analysis, + results: ArtifactResolver | undefined, analysisScopes: AnalysisScope[], ): any | null { if (!results) return node; @@ -522,7 +523,7 @@ function rewriteOutputImage( return null; } - const resultPath = results.get(outputId); + const resultPath = results(outputId); if (!resultPath) { console.warn( `[mystra] Narrative image embed references unproduced output id "${outputId}" — dropping image.`, @@ -536,9 +537,9 @@ function rewriteOutputImage( function resolveOutputTarget( path: string, - analysis: ASTRAAnalysis, + analysis: Analysis, analysisScopes: AnalysisScope[], -): { id: string; output: ASTRAOutput | undefined } | undefined { +): { id: string; output: Output | undefined } | undefined { const ref = path.replace(/^#/, ''); if (ref.startsWith('../')) { @@ -550,9 +551,9 @@ function resolveOutputTarget( } function outputTargetFromSegments( - analysis: ASTRAAnalysis, + analysis: Analysis, segments: string[], -): { id: string; output: ASTRAOutput | undefined } | undefined { +): { id: string; output: Output | undefined } | undefined { const [head, ...rest] = segments; if (head === 'outputs' && rest.length === 1) { diff --git a/src/transform/parse-table-data.ts b/src/transform/parse-table-data.ts index b963df8..8176026 100644 --- a/src/transform/parse-table-data.ts +++ b/src/transform/parse-table-data.ts @@ -4,9 +4,8 @@ * Used by two consumers: * - `render-evidence.ts`: builds MDAST table nodes for narrative * evidence rendering (citations, artifact cross-references). - * - `server/routes/astra.ts`: populates `SerializedOutput.table_data` - * so the React renderer can display inline table data on the per-output - * spec page without constructing MDAST. + * - `resolved-store.ts`: populates `SerializedOutput.table_data` so a rich + * theme can display inline table data without constructing MDAST. * * Keeping the parser here rather than in each consumer prevents a second * CSV/JSON reader from appearing in the system (constitution constraint). diff --git a/src/transform/render-data-sources.ts b/src/transform/render-data-sources.ts index 4bc6c4a..2791a4a 100644 --- a/src/transform/render-data-sources.ts +++ b/src/transform/render-data-sources.ts @@ -8,7 +8,7 @@ * exists whether or not any evidence references the output. */ -import type { ASTRAInput, ASTRAOutput } from '../types/astra.js'; +import type { Input, Output } from '@astra-spec/sdk'; import { table, tableRow, @@ -18,7 +18,7 @@ import { } from './ast-helpers.js'; import type { ProseParser } from './narrative-parser.js'; -export function renderInputsTable(inputs: ASTRAInput[], prose: ProseParser): any { +export function renderInputsTable(inputs: Input[], prose: ProseParser): any { // Caller filters out the empty case so the page doesn't render a // stray "no inputs" sentence without a section heading to anchor it. @@ -68,7 +68,7 @@ export function renderInputsTable(inputs: ASTRAInput[], prose: ProseParser): any * still appears wherever it's structurally referenced (typically * under a finding); the row here is the stable anchor target. */ -export function renderOutputsTable(outputs: ASTRAOutput[], prose: ProseParser): any { +export function renderOutputsTable(outputs: Output[], prose: ProseParser): any { const headerRow = tableRow( [ tableCell([text('Output')], true), diff --git a/src/transform/render-evidence.ts b/src/transform/render-evidence.ts index 1978982..073ec65 100644 --- a/src/transform/render-evidence.ts +++ b/src/transform/render-evidence.ts @@ -17,7 +17,7 @@ * > "quoted text" */ -import type { ASTRAEvidence, ASTRAOutput } from '../types/astra.js'; +import type { Evidence, Output } from '@astra-spec/sdk'; import { paragraph, text, @@ -35,11 +35,9 @@ import { table, tableRow, tableCell, - cite, - citeGroup, } from './ast-helpers.js'; import { parse as parsePath } from 'node:path'; -import { getCachedMetadata } from '../doi/resolver.js'; +import type { ArtifactResolver } from '../loader.js'; import type { ProseParser } from './narrative-parser.js'; import { parseTableData, formatValue } from './parse-table-data.js'; @@ -51,19 +49,18 @@ import { parseTableData, formatValue } from './parse-table-data.js'; * references (artifact id not declared) emit a console.warn * rather than silently rendering nothing. * - * `doiCacheDir` is the on-disk cache used by `formatCiteNode` for - * hover-preview metadata. Threaded through the transform context; - * `null` falls back to a plain DOI link. + * DOI evidence renders as a plain DOI link. Resolving the citation + * (author–year text, a reference list) is delegated to MyST natively; + * MySTRA no longer carries its own DOI resolver/cache. */ export function renderEvidenceBlock( - evidence: ASTRAEvidence, - results: Map, - outputs: Map, + evidence: Evidence, + results: ArtifactResolver, + outputs: Map, prose: ProseParser, - doiCacheDir: string | null, ): any[] { if (evidence.doi) { - return renderLiteratureEvidence(evidence, doiCacheDir); + return renderLiteratureEvidence(evidence); } if (evidence.artifact) { return renderArtifactEvidence(evidence, results, outputs, prose); @@ -72,28 +69,15 @@ export function renderEvidenceBlock( } /** - * Format a DOI citation as a cite node for hover previews. - * Returns a single citeGroup node that the book-theme renders - * with a hover tooltip showing the full citation. - * Falls back to a plain DOI link if not resolved. + * Format a DOI as a plain link to `doi.org`. (Citation resolution — a + * reference list, author–year labels — is MyST's job once a bibliography + * is wired; see SPEC.md §6.) */ -function formatCiteNode(doi: string, doiCacheDir: string | null): any { - const meta = doiCacheDir ? getCachedMetadata(doi, doiCacheDir) : null; - - if (meta && meta.authorShort) { - let authorYear = meta.authorShort; - if (meta.year) authorYear += ` (${meta.year})`; - return citeGroup([cite(meta.label, [text(authorYear)], 'narrative')], 'narrative'); - } - - // Fallback: plain DOI link +function formatCiteNode(doi: string): any { return link(`https://doi.org/${doi}`, [text(doi)]); } -function renderLiteratureEvidence( - evidence: ASTRAEvidence, - doiCacheDir: string | null, -): any[] { +function renderLiteratureEvidence(evidence: Evidence): any[] { const nodes: any[] = []; const doi = evidence.doi!; @@ -106,18 +90,103 @@ function renderLiteratureEvidence( nodes.push(blockquote([ paragraph([text(evidence.quote.exact)]), ])); - nodes.push(paragraph([text('— '), formatCiteNode(doi, doiCacheDir)])); + nodes.push(paragraph([text('— '), formatCiteNode(doi)])); } else { - nodes.push(paragraph([formatCiteNode(doi, doiCacheDir)])); + nodes.push(paragraph([formatCiteNode(doi)])); } return nodes; } +/** + * Render a single Output as a standalone block (not as evidence under a + * finding). Used by the `astra:output` MyST directive: an author imports + * one output by id and gets the figure / table / metric rendering inline + * in their prose. + * + * Differences from `renderArtifactEvidence`: + * - The figure container carries the `output-` identifier so the + * block is the cross-reference anchor (in evidence context the table + * row is the carrier; in directive context the rich block is). + * - The figure image URL is built via the optional `resultUrl` callback + * so callers outside the content server (the plugin) can emit a real + * project-relative path instead of the `/static/` mount. + * Defaults to the `/static/` scheme when no callback is given. + * - There is no Evidence, so metric/data/report render without a quote. + * + * A declared-but-unproduced output renders the same "Pending Output" + * admonition as evidence rendering. + */ +export function renderOneOutput( + output: Output, + artifactId: string, + results: ArtifactResolver, + prose: ProseParser, + opts?: { resultUrl?: (absPath: string, outputId: string) => string }, +): any[] { + const resultPath = results(artifactId); + if (!resultPath) { + return [ + admonition('warning', [ + admonitionTitle([text('Pending Output')]), + paragraph([text(`Output "${artifactId}" has not been produced yet.`)]), + ]), + ]; + } + + const identifier = `output-${artifactId}`; + + switch (output.type) { + case 'figure': { + const ext = parsePath(resultPath).ext.slice(1).toLowerCase(); + const url = opts?.resultUrl + ? opts.resultUrl(resultPath, artifactId) + : `/static/${artifactId}.${ext}`; + const figureLabel = output.label ?? artifactId; + const captionChildren = output.description + ? prose.inline(output.description) + : [text(figureLabel)]; + return [ + container( + 'figure', + [image(url, figureLabel, '100%'), caption([paragraph(captionChildren)])], + identifier, + ), + ]; + } + case 'table': { + // Standalone table output: render as a clean, numbered `container[table]` + // with a caption (not the collapsible `details` used in evidence context). + const data = parseTableData(resultPath); + if (data && data.headers.length > 0 && data.rows.length > 0) { + const tableLabel = output.label ?? artifactId; + const captionChildren = output.description ? prose.inline(output.description) : [text(tableLabel)]; + return [ + container('table', [tableNodeFromData(data), caption([paragraph(captionChildren)])], identifier), + ]; + } + const fallback: any = paragraph([text('Table: '), inlineCode(artifactId)]); + fallback.identifier = identifier; + fallback.label = identifier; + return [fallback]; + } + default: { + // metric / data / report: render inline, then tag the first node with + // the `output-` carrier so cross-references resolve to it. + const nodes = renderInlineArtifact(output, {} as Evidence, artifactId, resultPath); + if (nodes.length > 0 && !nodes[0].identifier) { + nodes[0].identifier = identifier; + nodes[0].label = identifier; + } + return nodes; + } + } +} + function renderArtifactEvidence( - evidence: ASTRAEvidence, - results: Map, - outputs: Map, + evidence: Evidence, + results: ArtifactResolver, + outputs: Map, prose: ProseParser, ): any[] { const nodes: any[] = []; @@ -134,7 +203,7 @@ function renderArtifactEvidence( return nodes; } - const resultPath = results.get(artifactId); + const resultPath = results(artifactId); if (!resultPath) { // Output is declared but the artifact file hasn't been produced // yet. Render a "Pending Output" admonition so the page makes @@ -170,7 +239,7 @@ function renderArtifactEvidence( } function renderFigureArtifact( - output: ASTRAOutput, + output: Output, artifactId: string, resultPath: string, prose: ProseParser, @@ -194,7 +263,7 @@ function renderFigureArtifact( } function renderTableArtifact( - output: ASTRAOutput, + output: Output, artifactId: string, resultPath: string, ): any[] { @@ -208,8 +277,8 @@ function renderTableArtifact( } function renderInlineArtifact( - output: ASTRAOutput, - evidence: ASTRAEvidence, + output: Output, + evidence: Evidence, artifactId: string, resultPath: string, ): any[] { @@ -286,14 +355,20 @@ function renderTableDataAsMDAST( if (data.headers.length === 0 || data.rows.length === 0) { return [paragraph([text(`Empty table: ${artifactId}`)])]; } + // Evidence context keeps the collapsible wrapper; standalone output tables + // (renderOneOutput) use `tableNodeFromData` directly for a clean render. + return [details([summary([text(tableLabel)]), tableNodeFromData(data)], false)]; +} - // First column in nested-object tables is the outer key — render in strong. - // We detect this heuristically: headers[0] === '' (parseTableData sets it). +/** + * Build a plain MyST `table` node from parsed `TableData`. Nested-object + * tables (parseTableData sets `headers[0] === ''`) render the outer key in + * the first column as bold. No wrapper — callers decide whether to place it + * in a `details`, a `container[table]`, etc. + */ +export function tableNodeFromData(data: ReturnType & {}): any { const isNestedObject = data.headers[0] === ''; - const displayHeaders = isNestedObject - ? ['', ...data.headers.slice(1)] - : data.headers; - + const displayHeaders = isNestedObject ? ['', ...data.headers.slice(1)] : data.headers; const headerRow = tableRow( displayHeaders.map((c) => tableCell([text(c)], true)), true, @@ -301,14 +376,11 @@ function renderTableDataAsMDAST( const rows = data.rows.map((row) => tableRow( row.map((cell, i) => - isNestedObject && i === 0 - ? tableCell([strong([text(cell)])]) - : tableCell([text(cell)]), + isNestedObject && i === 0 ? tableCell([strong([text(cell)])]) : tableCell([text(cell)]), ), ), ); - - return [details([summary([text(tableLabel)]), table([headerRow, ...rows])], false)]; + return table([headerRow, ...rows]); } /** @@ -318,32 +390,31 @@ function renderTableDataAsMDAST( * a bare citation / artifact reference, depending on populated fields. * * > "quoted text from paper" - * — Author et al. (Year) ← cite node with hover preview + * — https://doi.org/… ← plain DOI link (MyST resolves citations) * > "another quote" - * — Author2 et al. (Year) + * — https://doi.org/… * - * Used by both render-findings.ts and render-prior-insights.ts; + * Used by render-findings.ts and the plugin's prior-insight directive; * each provides its own carrier heading. Cross-references from * option tabs / decisions / narrative point at those carrier ids, * not at the body returned here. */ export function renderInsightEvidence( - insight: { evidence: ASTRAEvidence[] }, - doiCacheDir: string | null, + insight: { evidence?: Evidence[] }, ): any[] { const nodes: any[] = []; - for (const ev of insight.evidence) { + for (const ev of insight.evidence ?? []) { if (ev.doi) { if (ev.quote) { // Quote → attribution pattern nodes.push(blockquote([ paragraph([text(ev.quote.exact)]), ])); - nodes.push(paragraph([text('— '), formatCiteNode(ev.doi, doiCacheDir)])); + nodes.push(paragraph([text('— '), formatCiteNode(ev.doi)])); } else { // No quote, just cite the source - nodes.push(paragraph([formatCiteNode(ev.doi, doiCacheDir)])); + nodes.push(paragraph([formatCiteNode(ev.doi)])); } } else if (ev.artifact) { if (ev.quote) { diff --git a/src/transform/render-findings.ts b/src/transform/render-findings.ts index 95da6bb..a139229 100644 --- a/src/transform/render-findings.ts +++ b/src/transform/render-findings.ts @@ -12,52 +12,24 @@ * description); the renderer doesn't synthesise them. */ -import type { ASTRAInsight, ASTRAOutput } from '../types/astra.js'; +import type { Insight, Output } from '@astra-spec/sdk'; import { heading, paragraph, text, emphasis, - thematicBreak, } from './ast-helpers.js'; +import type { ArtifactResolver } from '../loader.js'; import type { ProseParser } from './narrative-parser.js'; import { renderEvidenceBlock } from './render-evidence.js'; -export function renderFindings( - findings: Record, - results: Map, - outputs: Map, - prose: ProseParser, - doiCacheDir: string | null, -): any[] { - const findingEntries = Object.entries(findings); - // Empty findings → no output. The page no longer wraps findings in - // a section heading, so an empty placeholder would be a stray - // sentence floating in the middle of the document. - if (findingEntries.length === 0) return []; - - const nodes: any[] = []; - let index = 1; - - for (const [findingId, finding] of findingEntries) { - if (index > 1) { - nodes.push(thematicBreak()); - } - nodes.push(...renderFinding(finding, index, findingId, results, outputs, prose, doiCacheDir)); - index++; - } - - return nodes; -} - -function renderFinding( - finding: ASTRAInsight, +export function renderFinding( + finding: Insight, index: number, findingId: string, - results: Map, - outputs: Map, + results: ArtifactResolver, + outputs: Map, prose: ProseParser, - doiCacheDir: string | null, ): any[] { const nodes: any[] = []; const identifier = `finding-${findingId}`; @@ -89,8 +61,8 @@ function renderFinding( } // Evidence blocks (figures, tables, artifact references) - for (const evidence of finding.evidence) { - nodes.push(...renderEvidenceBlock(evidence, results, outputs, prose, doiCacheDir)); + for (const evidence of finding.evidence ?? []) { + nodes.push(...renderEvidenceBlock(evidence, results, outputs, prose)); } return nodes; diff --git a/src/transform/render-methods.ts b/src/transform/render-methods.ts index bf8e008..59efe4c 100644 --- a/src/transform/render-methods.ts +++ b/src/transform/render-methods.ts @@ -10,7 +10,8 @@ * has no opinion about how decisions are organised. */ -import type { ASTRADecision, ASTRAInsight, ASTRAUniverse } from '../types/astra.js'; +import { isConditionMet } from '@astra-spec/sdk'; +import type { Decision, Insight, Universe } from '@astra-spec/sdk'; import { heading, paragraph, @@ -20,15 +21,14 @@ import { details, summary, tabSet, - thematicBreak, crossReference, } from './ast-helpers.js'; import type { ProseParser } from './narrative-parser.js'; /** - * tabItem factory bound to the current transform pass. Created once - * by astraToMystAST and threaded through to every renderer that - * mints tab keys, so the counter is per-transform, not global. + * tabItem factory bound to the current render pass. Created once per + * scope (by the plugin) and threaded through to every renderer that + * mints tab keys, so the counter is per-pass, not global. */ export type TabItemFn = (title: string, children: any[], selected?: boolean) => any; @@ -45,51 +45,25 @@ export type TabItemFn = (title: string, children: any[], selected?: boolean) => * land on nothing. */ export function isDecisionRendered( - decision: ASTRADecision, - universe: ASTRAUniverse, + decision: Decision, + universe: Universe, ): boolean { if (decision.from) return false; if (!decision.options) return false; - if (!isConditionMet(decision.when, universe)) return false; + if (!isConditionMet(decision.when, universe.decisions ?? {})) return false; return true; } -export function renderMethodsSections( - decisions: Record, - priorInsights: Record, - universe: ASTRAUniverse, - prose: ProseParser, - tabItem: TabItemFn, - doiCacheDir: string | null, -): any[] { - const nodes: any[] = []; - const entries = Object.entries(decisions).filter(([, d]) => - isDecisionRendered(d, universe), - ); - - for (let i = 0; i < entries.length; i++) { - const [id, decision] = entries[i]; - nodes.push(...renderDecision(id, decision, priorInsights, universe, prose, tabItem, doiCacheDir)); - // Thematic break between decisions (not after the last one). - if (i < entries.length - 1) { - nodes.push(thematicBreak()); - } - } - - return nodes; -} - -function renderDecision( +export function renderDecision( id: string, - decision: ASTRADecision, - priorInsights: Record, - universe: ASTRAUniverse, + decision: Decision, + priorInsights: Record, + universe: Universe, prose: ProseParser, tabItem: TabItemFn, - doiCacheDir: string | null, ): any[] { const options = decision.options!; - const selectedOptionId = universe.decisions[id] ?? decision.default; + const selectedOptionId = universe.decisions?.[id] ?? decision.default; const selectedOption = selectedOptionId ? options[selectedOptionId] : undefined; const selectedLabel = selectedOption?.label ?? selectedOptionId ?? '(none)'; const decisionLabel = decision.label ?? id; @@ -114,7 +88,7 @@ function renderDecision( const [optionId, option] = optionEntries[i]; const isSelected = optionId === selectedOptionId; if (isSelected) selectedIndex = i; - tabs.push(renderOptionTab(optionId, option, isSelected, priorInsights, prose, tabItem, doiCacheDir)); + tabs.push(renderOptionTab(optionId, option, isSelected, priorInsights, prose, tabItem)); } // Move selected tab to first position (book-theme defaults to first tab) @@ -157,10 +131,9 @@ function renderOptionTab( excluded_reason?: string; }, isSelected: boolean, - priorInsights: Record, + priorInsights: Record, prose: ProseParser, tabItem: TabItemFn, - doiCacheDir: string | null, ): any { // Tab title with selection marker let marker: string; @@ -223,33 +196,3 @@ function renderOptionTab( return tabItem(title, children, isSelected); } - -/** - * Check if a `when` condition is satisfied by the universe. `when` is - * multivalued in astra-spec — multiple conditions are AND'd together. - */ -function isConditionMet( - when: string[] | undefined, - universe: ASTRAUniverse, -): boolean { - if (when === undefined) return true; - - for (const cond of when) { - const negated = cond.startsWith('~'); - const ref = negated ? cond.slice(1) : cond; - const dotIndex = ref.indexOf('.'); - if (dotIndex === -1) continue; - - const decisionId = ref.slice(0, dotIndex); - const optionId = ref.slice(dotIndex + 1); - const selected = universe.decisions[decisionId]; - - if (negated) { - if (selected === optionId) return false; - } else { - if (selected !== optionId) return false; - } - } - - return true; -} diff --git a/src/transform/render-narrative.ts b/src/transform/render-narrative.ts deleted file mode 100644 index b24a054..0000000 --- a/src/transform/render-narrative.ts +++ /dev/null @@ -1,88 +0,0 @@ -/** - * Renders Analysis.narrative as named, addressable chunks. - * - * Each narrative section (summary, findings, methods, inputs, - * outputs) becomes its own block-level entry with a stable id - * anchor (`narrative-
`). The anchor attaches to the first - * child of the section's parsed mdast — no wrapping container, no - * programmatic section heading. Downstream renderers can compose - * sections however they like (header, sidebar, bottom, …) - * independent of mdast position; the id anchor lets cross- - * references find each chunk regardless of position. - * - * mdast order is the spec-declared order (summary → findings → - * methods → inputs → outputs) as a deterministic default; - * renderers are free to reorder. - * - * Section content is Markdown with anchor links of the form - * `[text](#path.to.element)` per the v0.0.6 narrative grammar; both - * Markdown parsing (via myst-parser) and anchor → crossReference - * resolution live in `narrative-parser.ts`. - */ - -import type { ASTRAAnalysis, ASTRANarrative } from '../types/astra.js'; -import { parseProseBlocks } from './narrative-parser.js'; -import type { ProseContext } from './narrative-parser.js'; -import { paragraph } from './ast-helpers.js'; - -const SECTION_ORDER: (keyof ASTRANarrative)[] = [ - 'summary', - 'findings', - 'methods', - 'inputs', - 'outputs', -]; - -export interface NarrativeChunk { - kind: 'narrative'; - section: keyof ASTRANarrative; - identifier: string; - /** Block-level mdast for this section's content. The first node - * carries the chunk's `identifier` so cross-references land. */ - mdast: any[]; -} - -/** - * Decompose an analysis's narrative into per-section addressable - * chunks. Sections without content are omitted; the resulting array - * preserves the spec-declared section order. - */ -export function renderNarrativeChunks( - analysis: ASTRAAnalysis, - slug: string, - context: ProseContext = { analysis, slug }, -): NarrativeChunk[] { - const narrative = analysis.narrative; - if (!narrative) return []; - const chunks: NarrativeChunk[] = []; - for (const section of SECTION_ORDER) { - const md = narrative[section]; - if (!md) continue; - const identifier = `narrative-${section}`; - const blocks = parseProseBlocks(md, context); - // Attach the chunk identifier to its first node so xrefs to - // `#narrative.
` resolve. If the section parsed empty - // (whitespace-only Markdown), synthesize an empty paragraph - // as the carrier. - const carrier = blocks.length > 0 ? blocks[0] : paragraph([]); - carrier.identifier = identifier; - if (!carrier.label) carrier.label = identifier; - const mdast = blocks.length > 0 ? blocks : [carrier]; - chunks.push({ kind: 'narrative', section, identifier, mdast }); - } - return chunks; -} - -/** - * Render one narrative section (raw Markdown string) into mdast, - * resolving in-scope anchor links to crossReferences. Used by - * sub-analysis cards which only ever surface a single section. - */ -export function renderNarrativeSection( - md: string | undefined, - analysis: ASTRAAnalysis, - slug: string, - context: ProseContext = { analysis, slug }, -): any[] { - return parseProseBlocks(md, context); -} diff --git a/src/transform/render-output-provenance.ts b/src/transform/render-output-provenance.ts deleted file mode 100644 index 427738e..0000000 --- a/src/transform/render-output-provenance.ts +++ /dev/null @@ -1,143 +0,0 @@ -/** - * Renders per-Output provenance as structured mdast carriers. - * - * The unit of provenance, as of astra-spec v0.0.7 (PR #19), is the - * Output: `Output.inputs` lists the upstream artifact IDs the output - * depends on, and `Output.decisions` lists the decision IDs that - * parameterize it. The Recipe is pure *how* — runners invoke the - * `command` with the inputs/decisions surfaced via template - * substitution, but the spec contract lives on the Output. - * - * This module emits one container per Output that has any - * provenance to declare. The container shape mirrors the - * `prior-insight` carrier pattern (kebab-case `kind`, snake_case - * structural identifier, `data.astraKind` discriminator): - * - * { - * type: 'container', - * kind: 'output-provenance', - * identifier: 'output--provenance', - * label: 'output--provenance', - * class: 'astra astra-output-provenance', - * data: { - * astraKind: 'output_provenance', - * outputId: '', - * inputs: [...resolved IDs], - * decisions: [...resolved IDs], - * from: | null, - * }, - * children: [ - * , - * , - * ], - * } - * - * The `data` slot is the contract for renderers that want to show - * provenance; the children are a fallback rendering for renderers - * that just walk the AST without inspecting `data`. Either way, the - * carrier is stable: an Output always has at most one provenance - * block, addressable by `output--provenance`. - * - * `Output.from` is resolved before emission — an aliased Output - * inherits its source's `inputs`/`decisions`. The original `from` - * path is preserved in `data.from` so renderers can still expose - * "this is a re-export of X" if they want. - */ - -import type { ASTRAAnalysis } from '../types/astra.js'; -import { - paragraph, - text, - strong, - crossReference, -} from './ast-helpers.js'; -import { resolveOutputs, type ResolvedOutput } from './resolve-output.js'; - -/** - * Emit a flat sequence of provenance carriers, one per Output with - * non-empty provenance. Outputs with no `inputs:` and no `decisions:` - * (after `from:` resolution) get no block — there's nothing to say. - */ -export function renderOutputProvenance(analysis: ASTRAAnalysis): any[] { - const resolved = resolveOutputs(analysis); - return resolved - .filter(hasProvenance) - .map((r) => renderOne(r)); -} - -function hasProvenance(r: ResolvedOutput): boolean { - const inputs = r.resolved.inputs ?? []; - const decisions = r.resolved.decisions ?? []; - return inputs.length > 0 || decisions.length > 0; -} - -function renderOne(r: ResolvedOutput): any { - const { declared, resolved, fromChain, unresolved } = r; - const outputId = declared.id; - const identifier = `output-${outputId}-provenance`; - const inputIds = resolved.inputs ?? []; - const decisionIds = resolved.decisions ?? []; - - const children: any[] = []; - - // Inline phrasing children: provide a fallback rendering so - // renderers that don't inspect `data` still surface the provenance. - // Each input/decision is emitted as a crossReference so anchor - // navigation works without renderer-side resolution. - if (inputIds.length > 0) { - const refs: any[] = inputIds.map((id) => - crossReference(`input-${id}`, [text(id)]), - ); - children.push( - paragraph([ - strong([text('Inputs: ')]), - ...interleave(refs, () => text(', ')), - ]), - ); - } - if (decisionIds.length > 0) { - const refs: any[] = decisionIds.map((id) => - crossReference(`decision-${id}`, [text(id)]), - ); - children.push( - paragraph([ - strong([text('Decisions: ')]), - ...interleave(refs, () => text(', ')), - ]), - ); - } - - return { - type: 'container', - kind: 'output-provenance', - identifier, - label: identifier, - class: 'astra astra-output-provenance', - data: { - astraKind: 'output_provenance', - outputId, - inputs: inputIds, - decisions: decisionIds, - // `from` chain (if any) and unresolved flag — lets renderers - // distinguish "this is a re-export" from "this is local - // provenance" and surface broken references. - from: fromChain.length > 0 ? fromChain.join('.') : null, - unresolved, - }, - children, - }; -} - -/** - * Interleave a list of nodes with separators. Used for ", "-joining - * crossReference chips inline. Avoids `.flatMap` boilerplate at call - * sites and keeps the separator a thunk so each one is a fresh node. - */ -function interleave(items: T[], separator: () => T): T[] { - const out: T[] = []; - for (let i = 0; i < items.length; i++) { - if (i > 0) out.push(separator()); - out.push(items[i]); - } - return out; -} diff --git a/src/transform/render-output-recipe.ts b/src/transform/render-output-recipe.ts deleted file mode 100644 index daa3d0c..0000000 --- a/src/transform/render-output-recipe.ts +++ /dev/null @@ -1,191 +0,0 @@ -/** - * Renders per-Output recipes as structured mdast carriers. - * - * Recipe is the *how* of an Output: a `command` template, an - * execution `container`, and `resources` requirements. As of - * astra-spec v0.0.7 (PR #19), provenance (inputs/decisions/when) - * lives on the parent Output; Recipe shrinks to `{command, - * container, resources}` — pure execution detail. - * - * Why MySTRA emits Recipe at all - * ────────────────────────────── - * The parent constitution ([[vellum-reader/myst-as-ast-layer-for- - * lightcone-ui]]) says MySTRA owns the entire ASTRA → mdast - * translation: any consumer reading `astra.yaml` directly is a - * leak. Recipe is in the spec, so MySTRA must emit something for - * it — even if downstream renderers choose to hide it. The - * constitution body laid out two coherent renderer-side positions - * ("hide" vs "render as collapsible technical detail"); this - * carrier subsumes both. The structured `data` slot lets renderers - * pattern-match (and decide whether to surface, suppress, or - * re-style); the fallback `children` are a `details` block - * collapsed by default, so renderers that just walk the AST get a - * minimal, dwell-friendly disclosure rather than a wall of recipe - * text. - * - * Carrier shape - * ───────────── - * { - * type: 'container', - * kind: 'output-recipe', - * identifier: 'output--recipe', - * label: 'output--recipe', - * class: 'astra astra-output-recipe', - * data: { - * astraKind: 'output_recipe', - * outputId: '', - * command: string | null, - * container: string | null, - * resources: { cpus?, memory?, disk?, gpus?, time_limit? } | null, - * from: | null, - * unresolved: , - * }, - * children: [ - * // Fallback rendering: a `details` block, collapsed by - * // default, containing labeled metadata + a `code` block - * // for the command. - * ], - * } - * - * `Output.from` is resolved before emission via the same path as - * provenance — an aliased Output inherits the source's recipe. - * The original `from:` path is preserved in `data.from` so - * renderers can still expose "this is a re-export of X". - * - * Predicate: emit only when the resolved recipe has at least one - * populated field (command, container, or any resources entry). - * An aliased Output whose source has no recipe gets no carrier — - * mirrors the provenance carrier's "no phantom blocks" contract. - */ - -import type { - ASTRAAnalysis, - ASTRARecipe, - ASTRAResources, -} from '../types/astra.js'; -import { - paragraph, - text, - strong, - inlineCode, - code, - details, - summary, -} from './ast-helpers.js'; -import { resolveOutputs, type ResolvedOutput } from './resolve-output.js'; - -/** - * Emit a flat sequence of recipe carriers, one per Output with a - * non-empty resolved recipe. Outputs with no recipe (after `from:` - * resolution) get no block. - */ -export function renderOutputRecipes(analysis: ASTRAAnalysis): any[] { - const resolved = resolveOutputs(analysis); - return resolved.filter(hasRecipe).map((r) => renderOne(r)); -} - -/** - * True when at least one recipe field is populated. Resources is - * "populated" if it has any non-empty entry; an empty `resources: - * {}` block doesn't earn a carrier. - */ -export function hasRecipe(r: ResolvedOutput): boolean { - const recipe = r.resolved.recipe; - if (!recipe) return false; - if (recipe.command || recipe.container) return true; - return hasResources(recipe.resources); -} - -function hasResources(resources?: ASTRAResources): boolean { - if (!resources) return false; - return ( - resources.cpus != null || - resources.memory != null || - resources.disk != null || - resources.gpus != null || - resources.time_limit != null - ); -} - -function renderOne(r: ResolvedOutput): any { - const { declared, resolved, fromChain, unresolved } = r; - const recipe = resolved.recipe!; - const outputId = declared.id; - const identifier = `output-${outputId}-recipe`; - - return { - type: 'container', - kind: 'output-recipe', - identifier, - label: identifier, - class: 'astra astra-output-recipe', - data: { - astraKind: 'output_recipe', - outputId, - command: recipe.command ?? null, - container: recipe.container ?? null, - resources: hasResources(recipe.resources) ? { ...recipe.resources } : null, - from: fromChain.length > 0 ? fromChain.join('.') : null, - unresolved, - }, - children: fallbackChildren(recipe), - }; -} - -/** - * Fallback rendering: a `details` block (collapsed by default) - * with a "Recipe" summary, the command in a code block, and - * labeled paragraphs for container and resources. Renderers that - * pattern-match on `data` can ignore these; renderers that walk - * the AST get a minimal disclosure. - * - * The block is collapsed by default. That's the "render as - * collapsible technical detail" position from the constitution — - * recipes are visible to the technically curious, tucked away by - * default. Renderers that prefer the "hide" position can suppress - * `kind: 'output-recipe'` carriers; renderers that prefer to - * render their own way can read `data` and emit whatever they - * want. Both positions are reachable from this single emission. - */ -function fallbackChildren(recipe: ASTRARecipe): any[] { - const inner: any[] = [summary([text('Recipe')])]; - - if (recipe.command) { - inner.push(code('bash', recipe.command)); - } - if (recipe.container) { - inner.push( - paragraph([ - strong([text('Container: ')]), - inlineCode(recipe.container), - ]), - ); - } - if (hasResources(recipe.resources)) { - inner.push(paragraph(resourceLine(recipe.resources!))); - } - - return [details(inner, /* open */ false)]; -} - -/** - * Render the resources block as a "key: value, key: value" - * paragraph. Dense by design — recipes are technical detail; a - * single line of compact metadata reads better than a sub-list. - */ -function resourceLine(resources: ASTRAResources): any[] { - const parts: Array<{ label: string; value: string }> = []; - if (resources.cpus != null) parts.push({ label: 'cpus', value: String(resources.cpus) }); - if (resources.memory) parts.push({ label: 'memory', value: resources.memory }); - if (resources.disk) parts.push({ label: 'disk', value: resources.disk }); - if (resources.gpus != null) parts.push({ label: 'gpus', value: String(resources.gpus) }); - if (resources.time_limit) parts.push({ label: 'time_limit', value: resources.time_limit }); - - const out: any[] = [strong([text('Resources: ')])]; - parts.forEach((p, i) => { - if (i > 0) out.push(text(', ')); - out.push(text(p.label + ': ')); - out.push(inlineCode(p.value)); - }); - return out; -} diff --git a/src/transform/render-prior-insights.ts b/src/transform/render-prior-insights.ts deleted file mode 100644 index f35c163..0000000 --- a/src/transform/render-prior-insights.ts +++ /dev/null @@ -1,68 +0,0 @@ -/** - * Renders prior_insights as minimal addressable carriers. - * - * Each prior_insight emits as a single `container` node with kind - * `prior-insight`, identifier `prior_insight-`, structured - * `data` for downstream renderers, and children = [claim paragraph, - * …evidence body]. No heading, no thematic-break separators, no - * `'Scope:'` label paragraph — surfacing prior_insights as a visible - * "section" is a renderer's call (sidebar, hover, hidden, h3, …), - * not the transform's. The transform's job is to publish a stable - * carrier so option-tab crossReferences and narrative anchors - * resolve to *something* on the page. - * - * Asymmetric with findings on purpose: findings are paper-headline - * material by convention, so they keep their h3 + paragraph shape. - * Prior insights are typically supporting context; how to display - * them is downstream's call. - * - * Why `kind: 'prior-insight'` and not `'prior_insight'`: MyST AST - * `container.kind` is conventionally a kebab-case CSS-class-like - * identifier (`figure`, `seealso`, `tip`); the underscore form - * survives in the structural identifier (`prior_insight-`), - * matching the rest of the v0.0.6 anchor grammar. The two - * conventions live next to each other on the same node. - */ - -import type { ASTRAInsight } from '../types/astra.js'; -import { paragraph } from './ast-helpers.js'; -import { renderInsightEvidence } from './render-evidence.js'; -import type { ProseParser } from './narrative-parser.js'; - -export function renderPriorInsights( - priorInsights: Record, - prose: ProseParser, - doiCacheDir: string | null, -): any[] { - return Object.entries(priorInsights).map(([id, insight]) => - renderPriorInsight(id, insight, prose, doiCacheDir), - ); -} - -function renderPriorInsight( - insightId: string, - insight: ASTRAInsight, - prose: ProseParser, - doiCacheDir: string | null, -): any { - const identifier = `prior_insight-${insightId}`; - return { - type: 'container', - kind: 'prior-insight', - identifier, - label: identifier, - class: 'astra astra-prior-insight', - data: { - astraKind: 'prior_insight', - id: insightId, - label: insight.label ?? null, - scope: insight.scope ?? null, - tags: insight.tags ?? null, - derived: insight.derived ?? false, - }, - children: [ - paragraph(prose.inline(insight.claim)), - ...renderInsightEvidence(insight, doiCacheDir), - ], - }; -} diff --git a/src/transform/render-sub-analyses.ts b/src/transform/render-sub-analyses.ts deleted file mode 100644 index 9a60d1d..0000000 --- a/src/transform/render-sub-analyses.ts +++ /dev/null @@ -1,63 +0,0 @@ -/** - * Renders sub-analysis cards linking to child pages. - * - * Card content is the sub-analysis's narrative summary — author - * prose. The renderer doesn't synthesise stat strings ("N decisions - * · M inputs · K outputs"); that's narrating structural data the - * destination page already exposes. - */ - -import type { ASTRAAnalysis } from '../types/astra.js'; -import { card } from './ast-helpers.js'; -import { renderNarrativeSection } from './render-narrative.js'; -import type { AnalysisScope, PriorInsightScope } from './narrative-parser.js'; - -export interface SubAnalysisCardContext { - priorInsightScopes?: PriorInsightScope[]; - analysisScopes?: AnalysisScope[]; - results?: Map; -} - -export function renderSubAnalysisCards( - analyses: Record, - hostSlug: string, - context: SubAnalysisCardContext = {}, -): any[] { - const nodes: any[] = []; - - for (const [id, sub] of Object.entries(analyses)) { - // Sub-analysis preview comes from its own narrative summary; - // anchors in that summary resolve relative to the sub-analysis, - // not the parent — so use the sub's own slug for resolution. - const subSlug = hostSlug === 'index' ? id : `${hostSlug}/${id}`; - // Recursive page builder lives the sub-analysis at this URL, - // so the card link must agree — `/${id}` was wrong for any - // nested case (parent slug `foo` → sub at `/foo/${id}`, not - // `/${id}`). - const cardUrl = `/${subSlug}`; - - const children: any[] = renderNarrativeSection( - sub.narrative?.summary, - sub, - subSlug, - { - analysis: sub, - slug: subSlug, - priorInsightScopes: context.priorInsightScopes, - analysisScopes: context.analysisScopes, - results: context.results, - }, - ); - - // Card carries `identifier: analysis-` so a parent-page - // anchor link to this card resolves; cross-page narrative refs - // (`#analyses.`) still resolve to the sub-analysis URL via - // the resolver, since pages can also be addressed by route. - const cardNode: any = card(sub.name ?? id, children, cardUrl); - cardNode.identifier = `analysis-${id}`; - cardNode.label = cardNode.identifier; - nodes.push(cardNode); - } - - return nodes; -} diff --git a/src/transform/render-universe-banner.ts b/src/transform/render-universe-banner.ts deleted file mode 100644 index 5a51a0d..0000000 --- a/src/transform/render-universe-banner.ts +++ /dev/null @@ -1,68 +0,0 @@ -/** - * Renders the universe banner showing which analysis path is active. - * Collapsible details with a table of decision → selected option. - */ - -import type { ASTRAUniverse, ASTRADecision } from '../types/astra.js'; -import { - details, - summary, - crossReference, - strong, - text, - table, - tableRow, - tableCell, -} from './ast-helpers.js'; -import type { ProseParser } from './narrative-parser.js'; - -export function renderUniverseBanner( - universe: ASTRAUniverse, - decisions: Record, - prose: ProseParser, -): any { - const rows: any[] = []; - - for (const [decisionId, selectedOptionId] of Object.entries(universe.decisions)) { - const decision = decisions[decisionId]; - if (!decision?.options) continue; - - const decisionLabel = decision.label ?? decisionId; - const option = decision.options[selectedOptionId]; - const optionLabel = option?.label ?? selectedOptionId; - - rows.push( - tableRow([ - tableCell([crossReference(`decision-${decisionId}`, [strong([text(decisionLabel)])])]), - tableCell([text(optionLabel)]), - ]), - ); - } - - const headerRow = tableRow( - [ - tableCell([text('Decision')], true), - tableCell([text('Selected')], true), - ], - true, - ); - - // Universe.description gets the same anchor-resolution treatment - // as other prose fields. Inline-only because the banner's - // collapsible summary expects a single line of phrasing content. - const descNodes = prose.inline(universe.description); - - return details( - [ - summary([ - text('Universe: '), - strong([text(universe.id)]), - ...(descNodes.length > 0 - ? [text(' — '), ...descNodes] - : []), - ]), - ...(rows.length > 0 ? [table([headerRow, ...rows])] : []), - ], - true, - ); -} diff --git a/src/transform/resolve-output.ts b/src/transform/resolve-output.ts index cada254..1fa066e 100644 --- a/src/transform/resolve-output.ts +++ b/src/transform/resolve-output.ts @@ -21,17 +21,17 @@ * the spec themselves. */ -import type { ASTRAAnalysis, ASTRAOutput } from '../types/astra.js'; +import type { Analysis, Output } from '@astra-spec/sdk'; export interface ResolvedOutput { /** The original output as declared in this scope. */ - declared: ASTRAOutput; + declared: Output; /** * The resolved view: type, description, inputs, decisions, recipe * filled in from the source if `from:` chains were walked. When * `declared.from` is unset, this equals `declared`. */ - resolved: ASTRAOutput; + resolved: Output; /** * The dot-separated chain that was walked, e.g. * `['preprocessing', 'features']` for `from: preprocessing.features`. @@ -56,8 +56,8 @@ export interface ResolvedOutput { * Returns the original output if no `from:` is set. */ export function resolveOutput( - output: ASTRAOutput, - scope: ASTRAAnalysis, + output: Output, + scope: Analysis, ): ResolvedOutput { if (!output.from) { return { declared: output, resolved: output, fromChain: [], unresolved: false }; @@ -73,7 +73,7 @@ export function resolveOutput( // violations and shouldn't be surfaced. Return an empty resolved // view so consumers see "no provenance" instead of inheriting // from a half-broken declaration. - const empty: ASTRAOutput = { + const empty: Output = { id: output.id, from: output.from, when: output.when, @@ -104,7 +104,7 @@ export function resolveOutput( // Merge: declared keeps its id/from/when (and label, if explicit); // everything else is inherited. - const merged: ASTRAOutput = { + const merged: Output = { id: output.id, from: output.from, when: output.when, @@ -132,11 +132,11 @@ export function resolveOutput( */ function walkOutputPath( parts: string[], - scope: ASTRAAnalysis, -): { output: ASTRAOutput; parent: ASTRAAnalysis } | null { + scope: Analysis, +): { output: Output; parent: Analysis } | null { if (parts.length < 2) return null; - let current: ASTRAAnalysis = scope; + let current: Analysis = scope; // All segments but the last name nested sub-analyses. for (let i = 0; i < parts.length - 1; i++) { const segId = parts[i]; @@ -155,6 +155,6 @@ function walkOutputPath( * Resolve every Output in an analysis — convenience for renderers * that want the resolved view across the registry. */ -export function resolveOutputs(analysis: ASTRAAnalysis): ResolvedOutput[] { +export function resolveOutputs(analysis: Analysis): ResolvedOutput[] { return (analysis.outputs ?? []).map((o) => resolveOutput(o, analysis)); } diff --git a/src/transform/resolved-store.ts b/src/transform/resolved-store.ts new file mode 100644 index 0000000..0f913aa --- /dev/null +++ b/src/transform/resolved-store.ts @@ -0,0 +1,338 @@ +/** + * The resolved ASTRA data store (for rich themes). + * + * Strategy A keeps `astra.yaml` as the single data source, but the theme cannot + * read it (it only sees the build output). So the plugin bakes a *resolved* + * projection of the analysis into the build — keyed by id — and a rich theme + * (e.g. `lightcone-astra`) joins `node identifier → store entry` to render + * cards, dependency graphs, or alternative layouts without re-implementing any + * ASTRA semantics. See STRATEGY-A-REFACTOR.md §5. + * + * This is the salvaged core of the old `/astra/.json` server route + * (`resolveOutputs`, `table_data`, `readMetric`, input aliasing) — but emitted + * as a build artifact, not served live, and with project-relative result URLs + * (MyST's asset pipeline copies them) rather than the old `/static` mount. + * + * The store is built once per page scope and carried on a hidden node's `data` + * (see `astra-plugin.ts`); it is resolved, not raw YAML, so the theme never + * touches `astra.yaml`. + */ + +import { existsSync, readFileSync } from 'node:fs'; +import type { + Analysis, + Decision, + Input, + Insight, + Universe, +} from '@astra-spec/sdk'; +import type { ArtifactResolver } from '../loader.js'; +import { resolveOutputs } from './resolve-output.js'; +import { isDecisionRendered } from './render-methods.js'; +import { parseTableData, type TableData } from './parse-table-data.js'; + +// ── Serialized shapes ─────────────────────────────────────────────────────── + +export interface SerializedRecipe { + command?: string; + container?: string; +} + +/** Inlined metric value (scalar / 2-tuple / object), parsed at build time. */ +export interface SerializedMetric { + value?: number | string; + uncertainty?: number | string; + error?: number | string; + unit?: string; + units?: string; + label?: string; +} + +export interface SerializedOutput { + id: string; + label?: string; + type?: string; + description?: string; + /** Project-relative URL of the result artifact (MyST copies it), if found. */ + resolved_path?: string; + recipe?: SerializedRecipe; + /** Upstream input ids this output depends on (resolved through `from:`). */ + inputs?: string[]; + /** Decision ids that parameterise this artefact. */ + decisions?: string[]; + /** Alias pointer for re-exported outputs (`from: child.out_id`). */ + from?: string; + /** Parsed rows for table outputs (same parser as the evidence renderer). */ + table_data?: TableData; + /** Inlined value for metric outputs whose result file parses as JSON. */ + metric?: SerializedMetric; +} + +export interface SerializedInput { + id: string; + label?: string; + type?: string; + description?: string; + source?: string; + from?: string; +} + +export interface SerializedDecision { + id: string; + label?: string; + rationale?: string; + /** The option id selected under the active universe (or the default). */ + selected?: string; + /** All option ids → their labels. */ + options: Record; +} + +export interface SerializedFinding { + id: string; + label?: string; + claim?: string; + notes?: string; + scope?: string; +} + +export interface SerializedInsight { + id: string; + label?: string; + scope?: string; + claim?: string; + /** First evidence DOI, when present (the theme can resolve the citation). */ + doi?: string; + /** First exact-quote evidence, when present. */ + quote?: string; +} + +export interface SerializedSubAnalysis { + id: string; + name?: string; + summary?: string; + /** Page URL for the sub-analysis (e.g. `/reconstruction`). */ + url: string; + decisions: number; + outputs: number; +} + +/** + * The resolved model for one analysis scope, keyed by id. A theme recognizes a + * placed node by its `identifier` (`output-`, `decision-`, …) + its + * `astra-*` class and looks the data up here. + */ +export interface ResolvedStore { + analysis: { id?: string; name?: string; slug: string }; + outputs: Record; + inputs: Record; + decisions: Record; + findings: Record; + prior_insights: Record; + subanalyses: Record; +} + +// ── Builder ─────────────────────────────────────────────────────────────── + +/** + * Build the resolved store for one analysis scope. + * + * @param analysis the scope's analysis node (already narrowed to the page) + * @param universe the active (scope-narrowed) universe selections + * @param results resolves an output id → its artifact path in this scope + * @param slug the page slug (`index` for root) + * @param resultUrl absolute result path → project-relative URL + * @param parentInputs ancestor input maps (innermost-last) for `from:` aliases + */ +export function buildResolvedStore( + analysis: Analysis, + universe: Universe, + results: ArtifactResolver, + slug: string, + resultUrl: (absPath: string) => string, + parentInputs: Map[] = [], +): ResolvedStore { + const outputs: Record = {}; + for (const { declared, resolved } of resolveOutputs(analysis)) { + const absPath = results(declared.id); + outputs[declared.id] = { + id: declared.id, + label: resolved.label, + type: resolved.type, + description: resolved.description, + resolved_path: absPath ? resultUrl(absPath) : undefined, + recipe: resolved.recipe + ? { command: resolved.recipe.command, container: resolved.recipe.container } + : undefined, + inputs: resolved.inputs, + decisions: resolved.decisions, + from: declared.from, + table_data: + resolved.type === 'table' && absPath ? (parseTableData(absPath) ?? undefined) : undefined, + metric: resolved.type === 'metric' && absPath ? readMetric(absPath) : undefined, + }; + } + + const inputs: Record = {}; + for (const inp of analysis.inputs ?? []) { + inputs[inp.id] = serializeInput(inp, parentInputs); + } + + // Only decisions with a real carrier on the page (same predicate the + // directive uses): bare `from`-references and `when`-unmet decisions have + // no node to join to and don't apply under this universe. + const decisions: Record = {}; + for (const [id, dec] of Object.entries(analysis.decisions ?? {})) { + if (isDecisionRendered(dec, universe)) decisions[id] = serializeDecision(id, dec, universe); + } + + const findings: Record = {}; + for (const [id, f] of Object.entries(analysis.findings ?? {})) { + findings[id] = { + id, + label: f.label, + claim: f.claim, + notes: f.notes, + scope: f.scope, + }; + } + + const prior_insights: Record = {}; + for (const [id, ins] of Object.entries(analysis.prior_insights ?? {})) { + prior_insights[id] = serializeInsight(id, ins); + } + + const subanalyses: Record = {}; + const base = slug === 'index' ? '' : slug; + for (const [id, sub] of Object.entries(analysis.analyses ?? {})) { + subanalyses[id] = { + id, + name: sub.name, + summary: firstParagraph(sub.narrative?.summary), + url: '/' + (base ? `${base}/${id}` : id), + decisions: Object.keys(sub.decisions ?? {}).length, + outputs: (sub.outputs ?? []).length, + }; + } + + return { + analysis: { id: analysis.id, name: analysis.name, slug }, + outputs, + inputs, + decisions, + findings, + prior_insights, + subanalyses, + }; +} + +// ── Per-element serializers ───────────────────────────────────────────────── + +function serializeDecision( + id: string, + dec: Decision, + universe: Universe, +): SerializedDecision { + const options: Record = {}; + for (const [optId, opt] of Object.entries(dec.options ?? {})) { + options[optId] = opt.label; + } + return { + id, + label: dec.label, + rationale: dec.rationale, + selected: universe.decisions?.[id] ?? dec.default, + options, + }; +} + +function serializeInsight(id: string, ins: Insight): SerializedInsight { + const evidence = ins.evidence ?? []; + return { + id, + label: ins.label, + scope: ins.scope, + claim: ins.claim, + doi: evidence.find((e) => e.doi)?.doi, + quote: evidence.find((e) => e.quote?.exact)?.quote?.exact, + }; +} + +/** + * Resolve an input declaration for the store. Aliased inputs (`from: `) + * inherit content fields from the matching ancestor declaration (innermost + * wins). Output-input cross-links (`from: scope.out_id`) are left as-is. + */ +function serializeInput(inp: Input, parentInputs: Map[]): SerializedInput { + const out: SerializedInput = { + id: inp.id, + label: inp.label, + type: inp.type, + description: inp.description, + source: inp.source, + from: inp.from, + }; + if (!inp.from) return out; + if (inp.from.includes('.')) return out; + const targetId = inp.from.split('/').pop() ?? inp.from; + for (let i = parentInputs.length - 1; i >= 0; i--) { + const src = parentInputs[i].get(targetId); + if (!src) continue; + return { + ...out, + type: out.type ?? src.type, + label: out.label ?? src.label, + description: out.description ?? src.description, + source: out.source ?? src.source, + }; + } + return out; +} + +// ── Helpers ─────────────────────────────────────────────────────────────── + +/** + * Read and parse a metric output's result file (`.json` only). Accepts a bare + * scalar, a `[value, uncertainty]` tuple, or an object with at least `value`; + * anything else (or a read error) returns undefined. + */ +function readMetric(absPath: string): SerializedMetric | undefined { + if (!absPath.toLowerCase().endsWith('.json')) return undefined; + if (!existsSync(absPath)) return undefined; + try { + const raw: unknown = JSON.parse(readFileSync(absPath, 'utf-8')); + if (typeof raw === 'number' || typeof raw === 'string') { + return { value: raw }; + } + if (Array.isArray(raw) && raw.length >= 1) { + const [value, uncertainty] = raw; + if (typeof value !== 'number' && typeof value !== 'string') return undefined; + const out: SerializedMetric = { value }; + if (typeof uncertainty === 'number' || typeof uncertainty === 'string') { + out.uncertainty = uncertainty; + } + return out; + } + if (raw && typeof raw === 'object' && 'value' in raw) { + const obj = raw as Record; + const out: SerializedMetric = {}; + if (typeof obj.value === 'number' || typeof obj.value === 'string') out.value = obj.value; + if (typeof obj.uncertainty === 'number' || typeof obj.uncertainty === 'string') + out.uncertainty = obj.uncertainty; + if (typeof obj.error === 'number' || typeof obj.error === 'string') out.error = obj.error; + if (typeof obj.unit === 'string') out.unit = obj.unit; + if (typeof obj.units === 'string') out.units = obj.units; + if (typeof obj.label === 'string') out.label = obj.label; + return Object.keys(out).length > 0 ? out : undefined; + } + return undefined; + } catch { + return undefined; + } +} + +/** First paragraph of a narrative chunk as plain text (best-effort). */ +function firstParagraph(md: string | undefined): string | undefined { + if (!md) return undefined; + const para = md.split(/\n\s*\n/)[0]?.replace(/\s+/g, ' ').trim(); + return para || undefined; +} diff --git a/src/types/astra.ts b/src/types/astra.ts deleted file mode 100644 index c5dce0e..0000000 --- a/src/types/astra.ts +++ /dev/null @@ -1,289 +0,0 @@ -/** - * TypeScript interfaces for the ASTRA data model. - * - * Tracks astra-spec v0.0.7 (commit ed13f48) at - * https://w3id.org/ASTRA/. The schemas live at - * `astra-spec/src/astra/schema/*.yaml`; this file is hand-maintained - * to match them, with consumers (transform, server) typed off these - * interfaces. - * - * Field-level fidelity: every slot the schema declares appears here - * (modulo identifier slots that are encoded as map keys — Option, - * Decision, UniverseNode, DecisionSelection). When astra-spec adds a - * slot, MySTRA absorbs it here first so emission code can pattern- - * match against a well-typed surface. - */ - -// ── W3C Web Annotation Selectors ── -// -// The schema declares only the structural attributes; JSON-LD `@type` -// is implicit in the LinkML class_uri (`oa:TextQuoteSelector`, -// `oa:FragmentSelector`) and not a runtime field on parsed YAML. - -export interface TextQuoteSelector { - exact: string; - prefix?: string; - suffix?: string; -} - -export interface FragmentSelector { - value?: string; - page?: number; -} - -// ── Evidence ── - -export interface ASTRAEvidence { - id: string; - - // Source: exactly one of doi or artifact - doi?: string; - /** - * Reference to an output by id. The output's `type` (figure / - * table / metric / data / report) drives how the artifact - * renders; the output's `label` and `description` carry the - * caption-equivalent metadata. There is no separate - * figure/table selector on Evidence — those would conflate the - * 'what kind' concern that already lives on Output. - */ - artifact?: string; - - // Literature-specific - version?: number; - - // Artifact-specific - snapshot?: string; - source_commit?: string; - - // Content selectors - quote?: TextQuoteSelector; - - // Location hint - location?: FragmentSelector; -} - -// ── Insight (shared model for prior_insights and findings) ── - -export interface ASTRAInsight { - id: string; - /** Short human-readable handle for compact rendering; falls back to id. */ - label?: string; - claim: string; - created_at: string; - evidence: ASTRAEvidence[]; - derived?: boolean; - scope?: string; - tags?: string[]; - notes?: string; -} - -// ── Input ── -// -// As of v0.0.7, an aliased Input (one with `from:`) is a pure pointer: -// type/description/source/ref/ref_version/use_outputs are forbidden on -// the alias and inherited from the source. The non-aliased case still -// requires `type` (validator-enforced), but TypeScript can't express -// "required iff `from` absent" without a discriminated union, so the -// surface uses optional fields and consumers defend at usage sites. - -export interface ASTRAInput { - id: string; - /** Short human-readable handle for compact rendering; falls back to id. */ - label?: string; - type?: 'data' | 'analysis'; - description?: string; - - // Data inputs - source?: string; - - // Analysis inputs - ref?: string; - ref_version?: string; - use_outputs?: string[]; - - /** - * Path to the source: `../id` (ancestor input), `../../id` (further - * ancestor), or `../scope.out_id` (sibling sub's output). Reaching - * downward into own children is not allowed — consume those via - * Output re-export instead. When set, the local node is a pure - * pointer; all content fields are inherited from the source. - */ - from?: string; -} - -// ── Recipe & Resources ── -// -// PR #19 (`Make Output the unit of provenance; modernize Recipe -// vocabulary`) restructured Recipe to be pure *how*: provenance -// (`inputs`, `decisions`, `when`) lives on the parent Output. Recipe -// itself shrinks to {command, resources, container}. Resources gained -// `disk` for cluster runners that schedule scratch space. - -export interface ASTRAResources { - /** CPU cores requested. Fractional allowed (CPU shares). */ - cpus?: number; - /** Memory requirement as a string with units (e.g. '16Gi', '8GB'). */ - memory?: string; - /** Disk requirement as a string with units (e.g. '10Gi', '500Mi'). */ - disk?: string; - /** Number of GPUs (>= 1 when set). */ - gpus?: number; - /** Maximum wall time as a duration string (e.g. '2h', '30m'). */ - time_limit?: string; -} - -export interface ASTRARecipe { - /** - * POSIX shell command. The command is a template — runners - * substitute `{inputs.}`, `{inputs}`, `{decisions.}`, and - * `{output}` placeholders before invoking it. Provenance lives on - * the parent `Output` (`Output.inputs`, `Output.decisions`); this - * recipe is pure *how*. - */ - command?: string; - /** - * Container reference. Either an image name (pulled at runtime, e.g. - * `python:3.9`, `ghcr.io/org/img:latest`) or a path to a Containerfile - * (built from source, e.g. `Containerfile`, `containers/Dockerfile`). - * Disambiguation is the runtime's job, not the schema's. - */ - container?: string; - resources?: ASTRAResources; -} - -// ── Output ── -// -// As of v0.0.7 (PR #19) the Output is the unit of provenance: -// `inputs` and `decisions` declare what materializing this artifact -// depends on, and the recipe is pure *how*. Aliased outputs (those -// with `from:`) inherit type/description/inputs/decisions/recipe -// from the source — only `id`, `from`, and `when` are legal on the -// alias node itself. Resolving the alias is the consumer's job; -// MySTRA emits the resolved view to renderers. - -export interface ASTRAOutput { - id: string; - /** Short human-readable handle for compact rendering; falls back to id. */ - label?: string; - type?: 'metric' | 'figure' | 'table' | 'data' | 'report'; - description?: string; - /** - * Path to a descendant Output: `child.out_id` (own child sub- - * analysis's output) or `child.grand.out_id` (deeper). Reaching - * upward is not allowed. When set, this Output is a re-export - * pointer; type/description/inputs/decisions/recipe are inherited - * from the source. - */ - from?: string; - when?: string[]; - /** - * IDs of upstream artifacts this output depends on. Each reference - * resolves to an Input declared on the surrounding analysis or a - * sibling Output. `from:` chains in the surrounding scope are - * walked transparently (an aliased Input is a valid local - * reference). Drives runner cache keys and recipe input - * substitution. - */ - inputs?: string[]; - /** - * Decision IDs (in the surrounding scope) that parameterize this - * output. Declares the output's provenance contract: re-running - * with a different option for any listed decision must be expected - * to produce a different output. Drives per-output cache keys, - * minimal-universe pruning, and decision-value delivery to - * recipes. - */ - decisions?: string[]; - recipe?: ASTRARecipe; -} - -// ── Option ── - -export interface ASTRAOption { - label: string; - description?: string; - insights?: string[]; - incompatible_with?: string[]; - requires?: string[]; - excluded?: boolean; - excluded_reason?: string; -} - -// ── Decision ── - -export interface ASTRADecision { - // Reference to parent decision (mutually exclusive with local definition) - from?: string; - - // Local definition fields - label?: string; - rationale?: string; - tags?: string[]; - when?: string[]; - default?: string; - options?: Record; -} - -// ── Narrative (structured prose for an Analysis) ── - -/** - * Free-form Markdown prose describing an Analysis, organized into five - * optional sections. Each section may contain anchor links of the form - * `[text](#path.to.element)` (tree-path-first, e.g. `#findings.foo` or - * `#analyses.preprocessing.outputs.features`); `astra validate` enforces - * a conditional requirement that a section be present whenever the - * corresponding structured data exists on the Analysis node. - */ -export interface ASTRANarrative { - summary?: string; - findings?: string; - methods?: string; - inputs?: string; - outputs?: string; -} - -// ── Analysis (self-similar, recursive) ── - -export interface ASTRAAnalysis { - $schema?: string; - id?: string; - version?: string; - name?: string; - authors?: string[]; - tags?: string[]; - narrative?: ASTRANarrative; - inputs?: ASTRAInput[]; - outputs?: ASTRAOutput[]; - // decisions / prior_insights / findings are multivalued inlined in - // the spec with no `required: true`, so a stub analysis legitimately - // has none. Optional in TypeScript matches that semantics; render - // helpers already defend with `?? {}` everywhere they read these. - decisions?: Record; - prior_insights?: Record; - findings?: Record; - /** Image name to pull, or path to a Containerfile to build. */ - container?: string; - path?: string; - analyses?: Record; -} - -// ── Universe ── - -export interface ASTRAUniverseNode { - /** - * Name of a universe in the sub-analysis's universes/ directory; - * an alternative to inline `decisions`. Mirrors the spec's - * `UniverseNode.universe` slot (universe.yaml:46). - */ - universe?: string; - decisions: Record; - analyses?: Record; -} - -export interface ASTRAUniverse { - $schema?: string; - id: string; - description?: string; - decisions: Record; - analyses?: Record; -} diff --git a/src/types/content-server.ts b/src/types/content-server.ts deleted file mode 100644 index b6e8a23..0000000 --- a/src/types/content-server.ts +++ /dev/null @@ -1,111 +0,0 @@ -/** - * Content server response types matching the MyST book-theme's expectations. - */ - -import type { Root } from 'myst-spec'; - -// ── Site manifest (GET /config.json) ── - -export interface SiteManifest { - version: number; - myst: string; - id?: string; - title: string; - projects: ManifestProject[]; - nav?: SiteNavItem[]; - actions?: SiteAction[]; -} - -export interface ManifestProject { - slug: string; - index: string; - title: string; - pages: ManifestProjectPage[]; -} - -export interface ManifestProjectPage { - title: string; - slug: string; - level: number; - short_title?: string; - description?: string; -} - -export interface SiteNavItem { - title: string; - url?: string; - internal?: boolean; - children?: SiteNavItem[]; -} - -export interface SiteAction { - title: string; - url: string; - filename?: string; - internal?: boolean; -} - -// ── Page content (GET /content/*.json) ── - -export interface PageContent { - kind: 'Article'; - sha256: string; - slug: string; - domain: string; - project: string; - mdast: Root; - frontmatter: PageFrontmatter; - references: References; - dependencies: string[]; -} - -export interface PageFrontmatter { - title: string; - subtitle?: string; - description?: string; - authors?: { name: string }[]; - tags?: string[]; -} - -export interface References { - cite?: { - order: string[]; - data: Record; - }; -} - -export interface CitationData { - label: string; - enumerator: string; - doi?: string; - html: string; -} - -// ── Cross-reference index (GET /myst.xref.json) ── - -export interface XRefIndex { - version: '1'; - myst?: string; - references: XRefEntry[]; -} - -export interface XRefEntry { - identifier: string; - kind: string; - data: string; - url: string; - implicit?: boolean; -} - -// ── Internal page data used during generation ── - -export interface PageData { - slug: string; - title: string; - level: number; - ast: Root; - frontmatter: PageFrontmatter; - identifiers: XRefEntry[]; - dependencies: string[]; - dois: string[]; -} diff --git a/src/types/papers.ts b/src/types/papers.ts deleted file mode 100644 index 72186e4..0000000 --- a/src/types/papers.ts +++ /dev/null @@ -1,15 +0,0 @@ -export interface PaperDecisionLink { - key: string; - id: string; - label: string; - slug: string; - href: string; -} - -export interface PaperInsightSummary { - id: string; - claim: string; - quote?: string; - page?: number; - informs: PaperDecisionLink[]; -} diff --git a/src/utils/hash.ts b/src/utils/hash.ts deleted file mode 100644 index 10bbede..0000000 --- a/src/utils/hash.ts +++ /dev/null @@ -1,5 +0,0 @@ -import { createHash } from 'node:crypto'; - -export function sha256(content: string): string { - return createHash('sha256').update(content).digest('hex'); -} diff --git a/tests/fixtures/schema-v0.0.7/README.md b/tests/fixtures/schema-v0.0.7/README.md deleted file mode 100644 index a54cb71..0000000 --- a/tests/fixtures/schema-v0.0.7/README.md +++ /dev/null @@ -1,23 +0,0 @@ -# Vendored ASTRA schema — v0.0.7 - -These YAML files are a frozen copy of `astra-spec/src/astra/schema/` -at version 0.0.7 (commit `ed13f48`). They're the input fixture for -`tests/schema-coverage.test.ts`, which asserts that -`src/types/astra.ts` covers every slot in every class. - -## Discipline - -Every astra-spec release: - -1. Update the vendored copies here from - `astra-spec/src/astra/schema/*.yaml`. -2. Run `npm test`. The coverage test surfaces every slot the TS - types haven't absorbed. -3. Fix the type file, then update `src/types/astra.ts`'s docstring - to declare the new tracked version + commit. - -The coverage test is the mechanical guard that replaces "hand-audit -every release." When the test goes green again, MySTRA is back to -parity. The broader rationale for the guard lives in `SPEC.md` and -in the coverage work merged through -[MySTRA PR #1](https://github.com/LightconeResearch/MySTRA/pull/1). diff --git a/tests/fixtures/schema-v0.0.7/analysis.yaml b/tests/fixtures/schema-v0.0.7/analysis.yaml deleted file mode 100644 index cb003c7..0000000 --- a/tests/fixtures/schema-v0.0.7/analysis.yaml +++ /dev/null @@ -1,654 +0,0 @@ ---- -id: https://w3id.org/ASTRA/analysis -name: analysis -title: ASTRA -description: |- - Agentic Schema for Transparent Research Analysis. - A framework for defining hierarchical scientific analyses with - decision points, evidence-backed insights, and universe specifications. -license: https://creativecommons.org/licenses/by/4.0/ -version: 0.0.7 - -prefixes: - astra: https://w3id.org/ASTRA/ - linkml: https://w3id.org/linkml/ - -default_prefix: astra -default_range: string - -imports: - - linkml:types - - insight - - universe - -# ========================================================================== -# Slots shared across analysis classes -# ========================================================================== - -# -# Identifier pattern: lowercase snake_case, with reserved category -# names excluded so that narrative anchor references like -# `#decisions.` cannot be silently shadowed by an entity named -# after a category. Reserved set: -# inputs, outputs, decisions, findings, prior_insights, analyses, -# options, content, narrative -# Applied to every entity ID: Input, Output, Option, Decision, -# Analysis, Insight, Evidence. -# - -slots: - - from: - description: >- - Reference to a related element via a path expression. When set, - the local node is a pure alias: only `id` and (where applicable) - `when` may be declared alongside `from`; all content fields - (type, description, label, source, options, recipe, etc.) are - inherited from the referenced node. - - Path grammar (uniform across Input, Output, and Decision): - - ../id -- escape one scope upward, then name 'id' - ../../id -- escape two scopes upward, then name 'id' - ../scope.id -- escape upward, then descend into named child - scope.id -- descend into a named child of the current scope - scope.sub.id -- descend through nested children - - Each `from:` may cross one or more scope boundaries. Each class's - `slot_usage` restricts the legal directions for that class: Input - reaches up or up-then-into-sibling, Output reaches into own - children, Decision reaches up only. - - when: - multivalued: true - description: >- - Conditions for when this element is active. - Format: 'decision_id.option_id' or '~decision_id.option_id'. - Multiple conditions are AND'd together. - -# ========================================================================== -# Enumerations -# ========================================================================== - -enums: - - InputType: - description: Type of analysis input - permissible_values: - data: - description: A dataset, file, or external resource - analysis: - description: Outputs from another ASTRA analysis - - OutputType: - description: Type of analysis output - permissible_values: - metric: - description: A metric or measurement - figure: - description: A figure or visualization - table: - description: A table of data - data: - description: A data file or dataset - report: - description: A report or document - -# ========================================================================== -# Classes -# ========================================================================== - -classes: - - # --- Utility --- - - KeyValuePair: - description: A key-value string pair - attributes: - key: - identifier: true - description: The key - value: - required: true - description: The value - - # --- Narrative --- - - Narrative: - description: >- - Structured prose describing an analysis, organized into five - sections: summary, findings, methods, inputs, and outputs. - All sections are schema-optional, but ``astra validate`` - applies a conditional requirement: a section must hold - non-empty prose when the corresponding structured data exists - on the Analysis node. - - - ``findings`` required when Analysis.findings has entries. - - ``methods`` required when Analysis.decisions or - Analysis.analyses has entries. - - ``inputs`` required when Analysis.inputs has entries. - - ``outputs`` required when Analysis.outputs has entries. - - ``summary`` is always optional — no structured counterpart. - - Authors narrate what they declare; stub analyses with only a - summary stay clean. - - Section content is Markdown. Internal references to other - elements of the analysis use anchor links of the form - ``[text](#path.to.element)``. References may appear in any - section — coverage is resolved across the whole narrative, - not per-section — so an author is free to cite a finding from - the summary, or an input from the methods section. - - Anchor grammar is tree-path-first, matching the rest of - ASTRA's reference syntax (the `from:` path grammar with `../` - prefixes for upward escape and `name.subname` for descent). - Sub-analyses are traversed before the category: - - [scaling decision](#decisions.scaling) - [scaling option](#decisions.scaling.options.standard) - [finding](#findings.best_model) - [prior insight](#prior_insights.compute_scaling) - [input](#inputs.iris_data) - [sub-analysis output](#preprocessing.outputs.features) - [sub-analysis decision](#preprocessing.decisions.scaling) - [sub-analysis](#analyses.preprocessing) - - References are interpreted relative to the hosting analysis. - Use '../' prefix to escape to parent scope, as with decision - 'from' (e.g. [see parent](#../decisions.method)). - attributes: - summary: - description: >- - High-level overview of the analysis — its question, scope, - and a brief orientation for readers. - findings: - description: >- - Narrative discussion of the analysis's findings. - Individual findings live under Analysis.findings as - structured Insight objects; this section is the prose - that frames them. - methods: - description: >- - Narrative discussion of the methodology, including - decision points and any sub-analyses. Structured - decisions and nested analyses live under - Analysis.decisions and Analysis.analyses; this section - frames them. - inputs: - description: >- - Narrative discussion of the analysis's inputs. - Individual inputs live under Analysis.inputs as - structured objects; this section frames them. - outputs: - description: >- - Narrative discussion of the expected outputs. - Individual outputs live under Analysis.outputs as - structured objects; this section frames them. - - # --- Execution --- - # - # ASTRA is a specification layer: a recipe describes *what* to run - # and *what it needs*, not how a runner schedules or instruments it. - # Field names follow modern container / cluster conventions so the - # spec reads naturally to anyone familiar with Docker, Kubernetes, - # Slurm, or batch schedulers. - - Resources: - description: >- - Compute resource requirements for a recipe. Values follow - cloud-native conventions (string-with-units for sized - quantities) so cluster executors can consume them directly. - attributes: - cpus: - range: float - minimum_value: 0 - description: >- - CPU cores requested. Fractional values are allowed - (e.g., 0.5) for runners that support CPU shares. - memory: - description: >- - Memory requirement as a string with units - (e.g., '16Gi', '512Mi', '8GB'). - time_limit: - description: >- - Maximum wall time as a duration string - (e.g., '2h', '30m', '1h30m'). - disk: - description: >- - Disk requirement as a string with units - (e.g., '10Gi', '500Mi'). - gpus: - range: integer - minimum_value: 1 - description: Number of GPUs - - Recipe: - description: >- - A build rule that produces an output. A recipe is pure *how*: - a `command` to invoke and the execution context - (`resources`, `container`). - - Recipes do not declare what the output depends on. Provenance - — upstream inputs, decision-driven parameterization, and - activation conditions — is declared on the parent Output - (`inputs`, `decisions`, `when`). Runners surface the resolved - input map and active decision values to the recipe via - `{...}` template substitution (see `command`). - attributes: - command: - description: >- - POSIX shell command to execute (e.g., - 'python src/train.py', 'Rscript analysis.R', - 'julia model.jl'). Any executable invocation is fine. - - The command is a template. Runners substitute these - placeholders before invoking it: - - {inputs.} -- path to the named upstream input - (must be declared in Output.inputs) - {inputs} -- space-separated paths to all declared - inputs, in declaration order - {decisions.} -- active option ID for the named - decision in the current universe - (must be declared in Output.decisions) - {output} -- path the artifact will be written to - - Use {{ and }} to emit literal '{' and '}'. Every - placeholder must resolve to a declared item; the - validator rejects unresolved or undeclared references. - - Static constants belong inline in the command (e.g., - '--max-iter 1000'); there is no separate `params` channel - because varying values are decisions and constants are - just command text. - resources: - range: Resources - inlined: true - description: Compute resource requirements (cpus, memory, time_limit, …) - container: - description: >- - Container image name or path to a Containerfile. - Image names (e.g., 'python:3.9', 'ghcr.io/org/img:latest') are - pulled as pre-built images; file paths (e.g., 'Containerfile', - 'containers/Dockerfile') are built from source. - - # --- Inputs and Outputs --- - - Input: - description: >- - An input to the analysis. Two kinds: data (dataset/file/resource) - or analysis (outputs from another ASTRA analysis). - - Sub-analysis inputs may alias an upstream artifact via `from`, - using the unified path grammar: - - from: ../id -- a parent input - from: ../../id -- a grandparent input - from: ../sibling.out_id -- a sibling sub-analysis's output - - An aliased Input is a pure pointer: only `id` and `from` are - allowed, with all other fields inherited from the source. - slots: - - from - slot_usage: - from: - description: >- - Path to the source: `../id` (ancestor input), `../../id` - (further ancestor), or `../scope.out_id` (sibling sub's - output). Reaching down into own children is not allowed — - consume those via Output re-export instead. - pattern: "^(\\.\\./)+[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)*$" - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Unique identifier for the input - label: - description: >- - Short human-readable name for compact rendering - (margin glyphs, breadcrumbs, card titles). Optional; - tooling falls back to id when absent. - type: - range: InputType - description: >- - Type of input. Required when `from` is unset; forbidden - when `from` is set (inherited from the source). - description: - description: Description of the input - source: - description: URI or path to the data source - ref: - description: Reference to another ASTRA analysis - ref_version: - description: Version of the referenced analysis - use_outputs: - multivalued: true - description: Specific outputs to use from referenced analysis - rules: - # `from_is_pure_alias` — split into one rule per forbidden slot so - # gen-json-schema emits `not: {required: [X]}` per slot. A single - # rule with multiple ABSENT postconditions translates incorrectly - # to `not: {required: [X, Y, ...]}`, which fires only when ALL are - # present. - - title: from_alias_forbids_type - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {type: {value_presence: ABSENT}}} - - title: from_alias_forbids_label - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {label: {value_presence: ABSENT}}} - - title: from_alias_forbids_description - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {description: {value_presence: ABSENT}}} - - title: from_alias_forbids_source - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {source: {value_presence: ABSENT}}} - - title: from_alias_forbids_ref - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {ref: {value_presence: ABSENT}}} - - title: from_alias_forbids_ref_version - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {ref_version: {value_presence: ABSENT}}} - - title: from_alias_forbids_use_outputs - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {use_outputs: {value_presence: ABSENT}}} - - title: type_required_when_not_aliased - description: >- - A non-aliased Input must declare its type. - preconditions: - slot_conditions: - from: - value_presence: ABSENT - postconditions: - slot_conditions: - type: - required: true - - Output: - description: >- - An expected output from the analysis. An Output is either - produced locally (with `inputs`, `decisions`, `recipe`) or - re-exported from a sub-analysis via `from`. - - Re-export grammar: - - from: child.out_id -- own child sub's output - from: child.grandchild.out_id -- descend into nested children - - A re-exported Output is a pure pointer: only `id`, `from`, and - `when` are allowed; type/description/recipe are inherited. - slots: - - from - - when - slot_usage: - from: - description: >- - Path to a descendant Output: `child.out_id` for an own - child sub-analysis's output, or deeper (`child.grand.out_id`) - to descend through nested children. Reaching upward is not - allowed — Outputs flow up only via re-export at each layer. - pattern: "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)+$" - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Unique identifier for the output - label: - description: >- - Short human-readable name for compact rendering - (margin glyphs, breadcrumbs, card titles). Optional; - tooling falls back to id when absent. - type: - range: OutputType - description: >- - Type of output. Required when `from` is unset; forbidden - when `from` is set (inherited from the source). - description: - description: Description of the output - inputs: - multivalued: true - description: >- - IDs of upstream artifacts this output depends on. Each - reference resolves to either an Input declared on the - surrounding analysis (an external dataset/file/analysis) - or a sibling Output (another artifact in scope). Runners - materialize the upstream artifacts before invoking the - recipe and surface the resolved input map to it - (Snakemake-style `{input.x}` substitution, env vars, - sidecar JSON — runner's choice). - - References use plain artifact IDs and resolve through any - `from:` chain in the surrounding analysis scope. An aliased - Input (one with `from:`) is a valid local reference here; - the runner walks the chain to the source. - decisions: - multivalued: true - description: >- - Decision IDs (in the surrounding scope) that parameterize - this output. Declares the output's provenance contract: - re-running with a different option for any listed decision - must be expected to produce a different output. - - Runners use this to (a) compute the per-output cache key, - (b) determine the minimal universe set needed to materialize - the output, and (c) deliver the active option values to the - recipe (via flags, env vars, or a sidecar — runner's choice). - - References use plain decision IDs and resolve through any - `from:` chain in the surrounding analysis scope. - recipe: - range: Recipe - inlined: true - description: How to produce this output (pure *how*; dependencies live on the Output via `inputs`/`decisions`) - rules: - # See note on Input.rules: split per slot so gen-json-schema - # emits one `not: {required: [X]}` per forbidden slot. - - title: from_alias_forbids_type - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {type: {value_presence: ABSENT}}} - - title: from_alias_forbids_label - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {label: {value_presence: ABSENT}}} - - title: from_alias_forbids_description - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {description: {value_presence: ABSENT}}} - - title: from_alias_forbids_inputs - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {inputs: {value_presence: ABSENT}}} - - title: from_alias_forbids_decisions - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {decisions: {value_presence: ABSENT}}} - - title: from_alias_forbids_recipe - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {recipe: {value_presence: ABSENT}}} - - title: type_required_when_not_aliased - description: >- - A non-aliased Output must declare its type. - preconditions: - slot_conditions: - from: - value_presence: ABSENT - postconditions: - slot_conditions: - type: - required: true - - # --- Decisions --- - - Option: - description: An option for a decision point - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Option identifier (the key in the options map) - label: - required: true - description: Human-readable name for the option - description: - description: Detailed description of the option - insights: - multivalued: true - description: Insight IDs supporting this option - incompatible_with: - multivalued: true - description: Decision.option pairs that cannot be selected together - requires: - multivalued: true - description: Decision.option pairs that must also be selected - excluded: - range: boolean - description: Whether this option was considered and rejected - excluded_reason: - description: Why this option was excluded - - Decision: - description: >- - A decision point in the analysis. Either locally defined (with - label and options) or a pure reference to an ancestor decision - via `from`. - - Reference grammar: - - from: ../id -- a parent decision - from: ../../id -- a grandparent decision - - Decisions only flow downward through scopes; sibling-sub or - child references are not legal. An aliased Decision is a pure - pointer: only `id`, `from`, and `when` may be set. - slots: - - from - - when - slot_usage: - from: - description: >- - Path to an ancestor decision: `../id` for a parent decision, - `../../id` for a grandparent, and so on. Reaching laterally - (`../sibling.id`) or downward (`child.id`) is not allowed — - if siblings need a shared decision, lift it to the common - ancestor and have each sub `from:` it. - pattern: "^(\\.\\./)+[a-z][a-z0-9_]*$" - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Decision identifier (the key in the decisions map) - label: - description: Human-readable name for the decision - rationale: - description: Why this decision exists - tags: - multivalued: true - description: Tags for grouping and categorizing - default: - description: Default option ID for baseline universes - options: - range: Option - multivalued: true - inlined: true - description: Map of option IDs to option specifications - rules: - # See note on Input.rules: split per slot so gen-json-schema - # emits one `not: {required: [X]}` per forbidden slot. - - title: from_alias_forbids_label - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {label: {value_presence: ABSENT}}} - - title: from_alias_forbids_options - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {options: {value_presence: ABSENT}}} - - title: from_alias_forbids_default - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {default: {value_presence: ABSENT}}} - - title: from_alias_forbids_rationale - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {rationale: {value_presence: ABSENT}}} - - title: from_alias_forbids_tags - preconditions: {slot_conditions: {from: {value_presence: PRESENT}}} - postconditions: {slot_conditions: {tags: {value_presence: ABSENT}}} - - title: label_and_options_required_when_not_aliased - description: >- - A non-aliased Decision must declare its label and options. - preconditions: - slot_conditions: - from: - value_presence: ABSENT - postconditions: - slot_conditions: - label: - required: true - options: - required: true - - # --- Analysis (self-similar, recursive) --- - - Analysis: - tree_root: true - description: >- - A self-similar analysis specification. Every level has the same - structure: metadata, inputs, outputs, decisions, insights, and - optional sub-analyses. A sub-analysis extracted to its own file - is a valid Analysis on its own. - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Analysis identifier (used as key when nested as a sub-analysis) - version: - pattern: "^\\d+\\.\\d+(\\.\\d+)?$" - description: ASTRA specification version - name: - description: Human-readable name for the analysis - narrative: - range: Narrative - inlined: true - description: >- - Structured prose describing this analysis, split into - five sections (summary, findings, methods, inputs, - outputs). See the Narrative class for section semantics - and the tree-path anchor grammar used for internal - cross-references. - authors: - multivalued: true - description: List of authors - tags: - multivalued: true - description: Tags for categorization - inputs: - range: Input - multivalued: true - inlined_as_list: true - description: Inputs for this analysis - outputs: - range: Output - multivalued: true - inlined_as_list: true - description: Expected outputs from this analysis - decisions: - range: Decision - multivalued: true - inlined: true - description: Decision points in this analysis (keyed by decision ID) - prior_insights: - range: Insight - multivalued: true - inlined: true - description: Prior insights that inform decisions (keyed by insight ID) - findings: - range: Insight - multivalued: true - inlined: true - description: Findings and conclusions from outputs (keyed by insight ID) - container: - description: >- - Default container for recipes in this node. - Image names are pulled; file paths are built from source. - path: - description: >- - Path to a directory containing its own astra.yaml. - Mutually exclusive with inline content fields - (inputs, outputs, decisions, etc.). - analyses: - range: Analysis - multivalued: true - inlined: true - description: Nested sub-analyses (keyed by analysis ID) diff --git a/tests/fixtures/schema-v0.0.7/insight.yaml b/tests/fixtures/schema-v0.0.7/insight.yaml deleted file mode 100644 index 60bda5a..0000000 --- a/tests/fixtures/schema-v0.0.7/insight.yaml +++ /dev/null @@ -1,136 +0,0 @@ ---- -id: https://w3id.org/ASTRA/insight -name: insight -description: |- - Insight and evidence models with W3C Web Annotation-compliant selectors - for referencing content in scientific papers and analysis artifacts. -version: 0.0.7 -license: https://creativecommons.org/licenses/by/4.0/ - -prefixes: - astra: https://w3id.org/ASTRA/ - linkml: https://w3id.org/linkml/ - oa: http://www.w3.org/ns/oa# - -default_prefix: astra -default_range: string - -imports: - - linkml:types - -# ========================================================================== -# W3C Web Annotation Selectors -# ========================================================================== - -classes: - - TextQuoteSelector: - description: >- - W3C TextQuoteSelector for locating text in a document. - The authoritative anchor for verification. - class_uri: oa:TextQuoteSelector - attributes: - exact: - required: true - description: Exact quoted text (1-3 sentences) - prefix: - description: ~20-100 chars before for disambiguation - suffix: - description: ~20-100 chars after for disambiguation - - FragmentSelector: - description: >- - W3C FragmentSelector for PDF locations. - Conforms to RFC 3778/8118 for PDF fragments. - class_uri: oa:FragmentSelector - attributes: - value: - description: Fragment value (e.g., 'page=6') - page: - range: integer - minimum_value: 1 - description: 1-indexed page number - - # ========================================================================== - # Evidence and Insights - # ========================================================================== - - Evidence: - description: >- - Evidence from a source with W3C-compliant selectors. - Can reference literature (by DOI) or analysis artifacts (by output ID). - Exactly one of doi or artifact must be set. - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Evidence identifier - doi: - description: DOI of the source paper (e.g., '10.48550/arXiv.1706.03762') - pattern: "^10\\.\\d{4,}/.*$" - artifact: - description: Output ID referencing a declared output in this analysis - version: - range: integer - minimum_value: 1 - description: Paper version for arXiv papers (version matters for reproducibility) - snapshot: - description: Path to immutable copy of the artifact - source_commit: - description: Git commit that produced the original artifact - quote: - range: TextQuoteSelector - inlined: true - description: Text quote anchor - location: - range: FragmentSelector - inlined: true - description: Location hint (page number for PDFs/reports) - - Insight: - description: >- - A unit of scientific knowledge backed by evidence. - Used for both prior_insights (informing decisions) and - findings (conclusions from the analysis). - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Unique identifier - label: - description: >- - Short human-readable name for compact rendering - (margin glyphs, breadcrumbs, card titles). Optional; - tooling falls back to id when absent. - claim: - required: true - description: What we learned (1-2 sentences) - created_at: - range: datetime - required: true - description: Creation timestamp (ISO 8601) - evidence: - range: Evidence - multivalued: true - inlined_as_list: true - required: true - description: Supporting evidence (papers or analysis artifacts) - derived: - range: boolean - description: True if synthesized/inferred from multiple sources - scope: - description: Applicability conditions - tags: - multivalued: true - description: Categorization tags - notes: - description: Reasoning notes - - InsightCollection: - description: Collection of insights, usable standalone or embedded in an analysis - attributes: - insights: - range: Insight - multivalued: true - inlined: true - description: Map of insight IDs to insights diff --git a/tests/fixtures/schema-v0.0.7/universe.yaml b/tests/fixtures/schema-v0.0.7/universe.yaml deleted file mode 100644 index a16366f..0000000 --- a/tests/fixtures/schema-v0.0.7/universe.yaml +++ /dev/null @@ -1,80 +0,0 @@ ---- -id: https://w3id.org/ASTRA/universe -name: universe -description: |- - Universe specification models. A universe is a complete set of - decisions across the entire analysis tree. -version: 0.0.7 -license: https://creativecommons.org/licenses/by/4.0/ - -prefixes: - astra: https://w3id.org/ASTRA/ - linkml: https://w3id.org/linkml/ - -default_prefix: astra -default_range: string - -imports: - - linkml:types - -# ========================================================================== -# Classes -# ========================================================================== - -classes: - - DecisionSelection: - description: A mapping from a decision ID to the selected option ID - attributes: - decision_id: - identifier: true - description: ID of the decision - option_id: - required: true - description: ID of the selected option - - UniverseNode: - description: >- - A universe node mirroring the analysis tree structure. - Represents decision selections at a specific sub-analysis node. - attributes: - id: - identifier: true - pattern: "^(?!(inputs|outputs|decisions|findings|prior_insights|analyses|options|content|narrative)$)[a-z][a-z0-9_]*$" - description: Node identifier (the sub-analysis key) - universe: - description: >- - Name of a universe in the sub-analysis's universes/ directory. - Alternative to inline decisions. - decisions: - range: DecisionSelection - multivalued: true - inlined: true - description: Decision selections (decision_id to option_id) - analyses: - range: UniverseNode - multivalued: true - inlined: true - description: Sub-analysis universe selections - - Universe: - description: >- - A universe specification - a complete set of decisions - across the entire analysis tree. - attributes: - id: - identifier: true - pattern: "^[a-z][a-z0-9_-]*$" - description: Unique identifier for the universe - description: - description: What this universe represents - decisions: - range: DecisionSelection - multivalued: true - inlined: true - description: Root-level decision selections - analyses: - range: UniverseNode - multivalued: true - inlined: true - description: Sub-analysis universe selections diff --git a/tests/narrative-parser.test.ts b/tests/narrative-parser.test.ts index 4397823..a2a9b64 100644 --- a/tests/narrative-parser.test.ts +++ b/tests/narrative-parser.test.ts @@ -4,7 +4,7 @@ */ import { describe, it, expect, vi } from 'vitest'; -import type { ASTRAAnalysis } from '../src/types/astra.js'; +import type { Analysis } from '@astra-spec/sdk'; import { parseProseBlocks, parseProseInline, @@ -14,7 +14,7 @@ import { /** Minimal Analysis fixture with one finding, one decision, one * sub-analysis — enough to exercise every resolution branch. */ -function fixtureAnalysis(): ASTRAAnalysis { +function fixtureAnalysis(): Analysis { return { name: 'Test', decisions: { @@ -246,7 +246,7 @@ describe('resolveAnchorPath', () => { }); it('resolves #prior_insights. to a prior_insight- identifier', () => { - const aWithPrior: ASTRAAnalysis = { + const aWithPrior: Analysis = { ...a, prior_insights: { compute_scaling: { @@ -376,7 +376,7 @@ describe('resolveAnchorPath', () => { }); it('resolves #narrative.
to the narrative chunk identifier', () => { - const withNarrative: ASTRAAnalysis = { + const withNarrative: Analysis = { ...a, narrative: { findings: 'Some findings prose.', summary: 'Hi.' }, }; @@ -389,7 +389,7 @@ describe('resolveAnchorPath', () => { }); it('falls back when #narrative.
targets an empty section', () => { - const onlySummary: ASTRAAnalysis = { ...a, narrative: { summary: 'Hi.' } }; + const onlySummary: Analysis = { ...a, narrative: { summary: 'Hi.' } }; expect( resolveAnchorPath('#narrative.findings', onlySummary, 'index'), ).toEqual({ url: '#narrative.findings' }); @@ -437,7 +437,7 @@ describe('resolveNarrativeAnchors', () => { const resolved = parseProseBlocks('![Accuracy](#outputs.accuracy_plot)', { analysis: a, slug: 'index', - results: new Map([['accuracy_plot', '/tmp/accuracy_plot.PNG']]), + results: (id) => (id === 'accuracy_plot' ? '/tmp/accuracy_plot.PNG' : undefined), }); const images = collectNodes(resolved, 'image'); expect(images).toHaveLength(1); @@ -451,7 +451,7 @@ describe('resolveNarrativeAnchors', () => { { analysis: a, slug: 'index', - results: new Map([['accuracy_plot', '/tmp/accuracy_plot.svg']]), + results: (id) => (id === 'accuracy_plot' ? '/tmp/accuracy_plot.svg' : undefined), }, ); const images = collectNodes(resolved, 'image'); @@ -465,7 +465,7 @@ describe('resolveNarrativeAnchors', () => { const resolved = parseProseBlocks('![Table](#outputs.results_table)', { analysis: a, slug: 'index', - results: new Map([['results_table', '/tmp/results_table.csv']]), + results: (id) => (id === 'results_table' ? '/tmp/results_table.csv' : undefined), }); expect(collectNodes(resolved, 'image')).toHaveLength(0); diff --git a/tests/page-shape.test.ts b/tests/page-shape.test.ts deleted file mode 100644 index 5e250e0..0000000 --- a/tests/page-shape.test.ts +++ /dev/null @@ -1,1833 +0,0 @@ -/** - * Page-shape tests: ensure the top-level transform emits a flat - * sequence of addressable blocks with no programmatic section - * headings (Findings/Methods/Data Sources/Verification/Sub-Analyses). - * The narrative drives the linear story; structural elements come - * out as bare blocks. - */ - -import { describe, it, expect } from 'vitest'; -import { mkdtempSync, writeFileSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { astraToMystAST, buildAllPages } from '../src/transform/index.js'; -import type { ASTRAAnalysis, ASTRAUniverse } from '../src/types/astra.js'; - -function emptyUniverse(): ASTRAUniverse { - return { id: 'baseline', decisions: {} }; -} - -function fixture(): ASTRAAnalysis { - return { - name: 'Test Analysis', - narrative: { - summary: 'A summary paragraph.', - methods: 'Some methodology prose with [scaling](#decisions.scaling) link.', - }, - decisions: { - scaling: { - label: 'Feature Scaling', - rationale: 'Why this matters.', - options: { standard: { label: 'Standard' } }, - }, - }, - prior_insights: {}, - findings: { - best_model: { - id: 'best_model', - claim: 'SVM wins', - created_at: '2024-01-01', - evidence: [], - }, - }, - inputs: [{ id: 'iris_data', type: 'data', description: 'Iris dataset' }], - outputs: [{ id: 'accuracy', type: 'metric' }], - }; -} - -describe('astraToMystAST page shape', () => { - it('emits no programmatic h2 section headings', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const sectionHeadings = ast.children - .filter((n: any) => n.type === 'heading' && n.depth === 2) - .map((n: any) => n.identifier); - // None of the legacy section identifiers should appear at the - // top level of the page. - expect(sectionHeadings).not.toContain('findings'); - expect(sectionHeadings).not.toContain('methods'); - expect(sectionHeadings).not.toContain('data-sources'); - expect(sectionHeadings).not.toContain('verification'); - expect(sectionHeadings).not.toContain('sub-analyses'); - }); - - it('renders all narrative sections in declaration order', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - // Narrative chunks come before structural elements; the - // summary block should appear before any decision/finding - // heading. - const firstNarrativeIdx = ast.children.findIndex( - (n: any) => n.identifier?.startsWith('narrative-'), - ); - const firstFindingIdx = ast.children.findIndex( - (n: any) => - n.type === 'heading' && n.identifier?.startsWith('finding-'), - ); - expect(firstNarrativeIdx).toBeGreaterThan(-1); - expect(firstFindingIdx).toBeGreaterThan(firstNarrativeIdx); - }); - - it('emits each narrative section as an addressable block', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const narrativeIds = ast.children - .map((n: any) => n.identifier) - .filter((id: string | undefined) => id?.startsWith('narrative-')); - // Fixture has summary + methods only; both should appear, no - // others, in declaration order. - expect(narrativeIds).toEqual(['narrative-summary', 'narrative-methods']); - }); - - it('does not wrap narrative sections in a container or heading', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - // No `container` node with kind narrative-* — chunks live as - // bare paragraphs/headings carrying the identifier directly. - const narrativeContainers = ast.children.filter( - (n: any) => - n.type === 'container' && (n.kind ?? '').startsWith('narrative-'), - ); - expect(narrativeContainers).toHaveLength(0); - }); - - it('skips empty-state placeholders', () => { - const empty: ASTRAAnalysis = { - name: 'Empty', - decisions: {}, - prior_insights: {}, - findings: {}, - }; - const ast = astraToMystAST({ - analysis: empty, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - // No "No findings recorded" / "No inputs declared" / etc. text. - const flat = JSON.stringify(ast); - expect(flat).not.toContain('No findings recorded'); - expect(flat).not.toContain('No inputs declared'); - expect(flat).not.toContain('No success criteria defined'); - }); - - it('does not infer finding↔decision relations from tag overlap', () => { - // Tag-overlap-as-link was the same shape as the deleted - // TAG_TO_SECTION ontology — implicit relational inference baked - // into the renderer. Tags survive on the heading's mdast `data` - // slot for consumers; the renderer no longer synthesises - // crossReferences from overlap. The "depends on:" glue and - // Methodology admonition wrapper are likewise gone. - const a: ASTRAAnalysis = { - ...fixture(), - decisions: { - scaling: { - label: 'Feature Scaling', - tags: ['preprocessing'], - options: { standard: { label: 'Standard' } }, - }, - }, - findings: { - best_model: { - id: 'best_model', - claim: 'SVM wins', - created_at: '2024-01-01', - tags: ['preprocessing'], - evidence: [], - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const flat = JSON.stringify(ast); - expect(flat).not.toContain('This finding depends on'); - expect(flat).not.toContain('"Methodology"'); - - function findAll(predicate: (n: any) => boolean): any[] { - const out: any[] = []; - const stack: any[] = [...ast.children]; - while (stack.length) { - const n = stack.pop(); - if (predicate(n)) out.push(n); - if (Array.isArray(n.children)) stack.push(...n.children); - } - return out; - } - expect(findAll((n) => n.type === 'admonition' && n.kind === 'seealso')).toHaveLength(0); - - // No tag-overlap-derived crossReference. The finding's heading - // is the only thing rendered for the finding; the decision - // heading still exists separately. The author wires explicit - // relations through narrative anchors, not tag overlap. - const findingHeading = findAll( - (n) => n.type === 'heading' && n.identifier === 'finding-best_model', - )[0]; - expect(findingHeading).toBeTruthy(); - expect(findingHeading.data?.tags).toEqual(['preprocessing']); - - // Walk only finding-block siblings between finding heading and - // the next h3 / decision heading: there should be no crossRef - // whose identifier starts with `decision-` (the previous - // tag-overlap output). - const idx = ast.children.indexOf(findingHeading); - expect(idx).toBeGreaterThanOrEqual(0); - const findingBlock: any[] = []; - for (let i = idx; i < ast.children.length; i++) { - const n = ast.children[i]; - if (i > idx && n.type === 'heading') break; - findingBlock.push(n); - } - function collectXRefs(stack: any[]): any[] { - const out: any[] = []; - const queue = [...stack]; - while (queue.length) { - const n = queue.pop(); - if (n?.type === 'crossReference') out.push(n); - if (Array.isArray(n?.children)) queue.push(...n.children); - } - return out; - } - const xrefsInBlock = collectXRefs(findingBlock); - expect(xrefsInBlock.every((x) => !x.identifier?.startsWith('decision-'))).toBe(true); - }); - - it('does not emit a renderer-imposed methods intro paragraph', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - // The legacy intro ("The following sections detail each - // methodological decision…") claimed alternative options - // could be explored via tabs — no longer true under the flat - // addressable-elements layout. - const flat = JSON.stringify(ast); - expect(flat).not.toContain('The following sections detail'); - expect(flat).not.toContain('alternative options can be explored'); - }); - - it('resolves narrative anchors against the host analysis', () => { - // The methods section in the fixture contains a link to - // `#decisions.scaling`; that should be a crossReference in the - // emitted AST. - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const xrefs: any[] = []; - function walk(n: any) { - if (n.type === 'crossReference') xrefs.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(xrefs.some((x) => x.identifier === 'decision-scaling')).toBe(true); - }); -}); - -describe('tab key stability (per-transform counter)', () => { - it('two consecutive transforms produce identical tabItem keys', () => { - // Module-global tabKeyCounter would mint different keys each - // call — downstream consumers diffing AST JSON saw spurious - // changes. Per-transform closure-scoped counter fixes that. - const a: ASTRAAnalysis = { - name: 'WithTabs', - decisions: { - scaling: { - label: 'Scaling', - options: { a: { label: 'A' }, b: { label: 'B' } }, - }, - normalization: { - label: 'Normalization', - options: { x: { label: 'X' }, y: { label: 'Y' } }, - }, - }, - prior_insights: {}, - findings: {}, - }; - const args = { - analysis: a, - universe: { id: 'u', decisions: {} } as ASTRAUniverse, - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }; - const ast1 = astraToMystAST(args); - const ast2 = astraToMystAST(args); - - function collectKeys(root: any): string[] { - const out: string[] = []; - const stack: any[] = [...root.children]; - while (stack.length) { - const n = stack.pop(); - if (n.type === 'tabItem' && n.key) out.push(n.key); - if (Array.isArray(n.children)) stack.push(...n.children); - } - return out.sort(); - } - - expect(collectKeys(ast1)).toEqual(collectKeys(ast2)); - // Sanity: tabs were actually emitted. - expect(collectKeys(ast1).length).toBeGreaterThan(0); - }); -}); - -describe('xref index (collectIdentifiers)', () => { - // collectIdentifiers' contract: every published id has a real - // carrier in the rendered AST. These tests pin that contract for - // the cases that previously broke it. - - it('does not publish methods-* tag-section ids (decisions render flat)', () => { - // Tag-as-structure ontology was deleted; renderMethodsSections - // emits flat per-decision blocks with no h3 group headings. - const a: ASTRAAnalysis = { - name: 'Tagged', - decisions: { - scaling: { - label: 'Scaling', - tags: ['reddening', 'extinction'], - options: { a: { label: 'A' } }, - }, - }, - prior_insights: {}, - findings: {}, - }; - const pages = buildAllPages(a, { id: 'u', decisions: {} }, new Map(), '/tmp'); - const ids = pages[0].identifiers.map((e) => e.identifier); - // Old emitter would have published `reddening-extinction` and/or - // tag-derived slugs — none of those should appear. - expect(ids.every((id) => !id.startsWith('reddening'))).toBe(true); - expect(ids).not.toContain('reddening-extinction'); - expect(ids).not.toContain('reddening'); - }); - - it('publishes decision- only for rendered decisions (skips bare from-refs)', () => { - const a: ASTRAAnalysis = { - name: 'Mixed', - decisions: { - local: { label: 'Local', options: { a: { label: 'A' } } }, - inherited: { from: 'parent.local' }, - }, - prior_insights: {}, - findings: {}, - }; - const pages = buildAllPages(a, { id: 'u', decisions: {} }, new Map(), '/tmp'); - const ids = pages[0].identifiers.map((e) => e.identifier); - expect(ids).toContain('decision-local'); - expect(ids).not.toContain('decision-inherited'); - }); - - it('publishes decision- only for rendered decisions (skips when-unmet)', () => { - // Bug D: previously `collectIdentifiers` published every - // declared decision, but `renderDecision` dropped ones whose - // `when` predicate wasn't satisfied — anchors landed on nothing. - const a: ASTRAAnalysis = { - name: 'Conditional', - decisions: { - always: { label: 'Always', options: { a: { label: 'A' } } }, - only_if_x: { - label: 'Conditional', - when: ['always.b'], - options: { a: { label: 'A' } }, - }, - }, - prior_insights: {}, - findings: {}, - }; - // Universe selects always.a → the `when: always.b` predicate is unmet. - const universe: ASTRAUniverse = { id: 'u', decisions: { always: 'a' } }; - const pages = buildAllPages(a, universe, new Map(), '/tmp'); - const ids = pages[0].identifiers.map((e) => e.identifier); - expect(ids).toContain('decision-always'); - expect(ids).not.toContain('decision-only_if_x'); - }); - - it('does not publish verification-* ids (success_criteria removed)', () => { - // success_criteria was a MySTRA-private extension carried over - // from earlier internal work; v0.0.6 doesn't define it. Ensure - // even an analysis carrying the field at runtime (extra - // properties tolerated) produces no verification-* xrefs. - const a: ASTRAAnalysis = { - name: 'WithStaleField', - decisions: {}, - prior_insights: {}, - findings: {}, - ...({ success_criteria: [{ claim: 'x', output: 'foo' }] } as any), - }; - const pages = buildAllPages(a, { id: 'u', decisions: {} }, new Map(), '/tmp'); - const ids = pages[0].identifiers.map((e) => e.identifier); - expect(ids.every((id) => !id.startsWith('verification-'))).toBe(true); - }); - - it('publishes decision- for when-met conditional decisions', () => { - const a: ASTRAAnalysis = { - name: 'Conditional', - decisions: { - always: { label: 'Always', options: { a: { label: 'A' } } }, - only_if_x: { - label: 'Conditional', - when: ['always.a'], - options: { a: { label: 'A' } }, - }, - }, - prior_insights: {}, - findings: {}, - }; - const universe: ASTRAUniverse = { id: 'u', decisions: { always: 'a' } }; - const pages = buildAllPages(a, universe, new Map(), '/tmp'); - const ids = pages[0].identifiers.map((e) => e.identifier); - expect(ids).toContain('decision-only_if_x'); - }); -}); - -describe('structural-element identifiers (end-to-end)', () => { - it('emits a finding- heading for each finding', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - function find(predicate: (n: any) => boolean): any | undefined { - const stack: any[] = [...ast.children]; - while (stack.length) { - const n = stack.pop(); - if (predicate(n)) return n; - if (Array.isArray(n.children)) stack.push(...n.children); - } - } - expect(find((n) => n.type === 'heading' && n.identifier === 'finding-best_model')).toBeTruthy(); - }); - - it('carries decision.tags as data.tags on the decision heading', () => { - // Tags are no longer used as renderer-imposed grouping - // structure; they survive on the heading's mdast `data` slot - // for downstream consumers that want to compose grouping. - const a: ASTRAAnalysis = { - ...fixture(), - decisions: { - scaling: { - label: 'Feature Scaling', - tags: ['reddening', 'extinction'], - options: { standard: { label: 'Standard' } }, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - function find(predicate: (n: any) => boolean): any | undefined { - const stack: any[] = [...ast.children]; - while (stack.length) { - const n = stack.pop(); - if (predicate(n)) return n; - if (Array.isArray(n.children)) stack.push(...n.children); - } - } - const decisionHeading = find((n) => n.type === 'heading' && n.identifier === 'decision-scaling'); - expect(decisionHeading).toBeTruthy(); - expect(decisionHeading.data?.tags).toEqual(['reddening', 'extinction']); - }); - - it('does not emit any h3 tag-group section heading', () => { - // tag-sections.ts is gone; "Reddening & Extinction" / - // "TRGB Detection Algorithm" / "General" / "Other" headings - // must not appear anywhere on the page. - const a: ASTRAAnalysis = { - ...fixture(), - decisions: { - scaling: { - label: 'Feature Scaling', - tags: ['reddening'], - options: { standard: { label: 'Standard' } }, - }, - untagged: { - label: 'Untagged', - options: { standard: { label: 'Standard' } }, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const flat = JSON.stringify(ast); - expect(flat).not.toContain('Reddening & Extinction'); - expect(flat).not.toContain('"General"'); - expect(flat).not.toContain('"Other"'); - // No h3 with a tag-derived id either. - function findAll(predicate: (n: any) => boolean): any[] { - const out: any[] = []; - const stack: any[] = [...ast.children]; - while (stack.length) { - const n = stack.pop(); - if (predicate(n)) out.push(n); - if (Array.isArray(n.children)) stack.push(...n.children); - } - return out; - } - const h3s = findAll((n) => n.type === 'heading' && n.depth === 3); - expect(h3s.every((h) => !h.identifier?.startsWith('reddening'))).toBe(true); - }); - - it('emits a decision- heading for each decision', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - function find(predicate: (n: any) => boolean): any | undefined { - const stack: any[] = [...ast.children]; - while (stack.length) { - const n = stack.pop(); - if (predicate(n)) return n; - if (Array.isArray(n.children)) stack.push(...n.children); - } - } - expect(find((n) => n.type === 'heading' && n.identifier === 'decision-scaling')).toBeTruthy(); - }); - - it('attaches an input- identifier to each input table row', () => { - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const ids: string[] = []; - function walk(n: any) { - if (n.type === 'tableRow' && n.identifier) ids.push(n.identifier); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(ids).toContain('input-iris_data'); - }); - - it('attaches an output- identifier to each output table row (no evidence)', () => { - // Bug A: every declared output must have a carrier, even with - // no evidence pointing at it from a finding. - const ast = astraToMystAST({ - analysis: fixture(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const ids: string[] = []; - function walk(n: any) { - if (n.type === 'tableRow' && n.identifier) ids.push(n.identifier); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(ids).toContain('output-accuracy'); - }); - - it('emits output- carrier for non-image artifacts (CSV, JSON, plain)', () => { - // Bug A: previously only image artifact evidence carried - // `output-`; JSON/CSV/plain artifacts (and outputs without - // evidence) had no carrier and broke xrefs. - const a: ASTRAAnalysis = { - ...fixture(), - outputs: [ - { id: 'metrics_json', type: 'data' }, - { id: 'sample_csv', type: 'table' }, - { id: 'plain_blob', type: 'data' }, - { id: 'no_evidence', type: 'metric' }, - ], - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const ids: string[] = []; - function walk(n: any) { - if (n.type === 'tableRow' && n.identifier) ids.push(n.identifier); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(ids).toContain('output-metrics_json'); - expect(ids).toContain('output-sample_csv'); - expect(ids).toContain('output-plain_blob'); - expect(ids).toContain('output-no_evidence'); - }); - - it('end-to-end: narrative anchor #inputs. resolves to a crossReference on the input identifier', () => { - const a: ASTRAAnalysis = { - ...fixture(), - narrative: { - methods: 'Use the [iris dataset](#inputs.iris_data) directly.', - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const xrefs: any[] = []; - function walk(n: any) { - if (n.type === 'crossReference') xrefs.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(xrefs.some((x) => x.identifier === 'input-iris_data')).toBe(true); - }); - - it('sub-analysis card omits the renderer-synthesised stats string', () => { - // "N decisions · M inputs · K outputs" was renderer-imposed - // narration of structural data the destination page already - // exposes. The card now contains only the sub-analysis's own - // narrative summary (author prose). - const a: ASTRAAnalysis = { - ...fixture(), - analyses: { - preprocessing: { - name: 'Pre', - decisions: { x: { label: 'X', options: { a: { label: 'A' } } } }, - prior_insights: {}, - findings: {}, - inputs: [{ id: 'foo', type: 'data' }], - outputs: [{ id: 'bar', type: 'metric' }], - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const flat = JSON.stringify(ast); - expect(flat).not.toContain('decisions ·'); - expect(flat).not.toContain('inputs ·'); - expect(flat).not.toContain('outputs"'); - }); - - it('does not pin renderer-imposed subtitle in page frontmatter', () => { - // "ASTRA Analysis" subtitle was the renderer asserting content - // type in metadata. astra-spec defines no analysis-level - // subtitle slot; pages don't carry one unless the data does. - const pages = buildAllPages( - fixture(), - { id: 'u', decisions: {} }, - new Map(), - '/tmp', - ); - expect(pages[0].frontmatter.subtitle).toBeUndefined(); - }); - - it('sub-analysis card URL respects the host slug for nested pages', () => { - // Bug C: parent slug `foo` → sub `bar` lives at `/foo/bar`, - // not `/bar`. The card URL must match the recursive page - // builder's path. - const a: ASTRAAnalysis = { - ...fixture(), - analyses: { - preprocessing: { - name: 'Pre', - decisions: {}, - prior_insights: {}, - findings: {}, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'foo/bar', - }); - const cards: any[] = []; - function walk(n: any) { - if (n.type === 'card') cards.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(cards).toHaveLength(1); - expect(cards[0].url).toBe('/foo/bar/preprocessing'); - }); - - it('sub-analysis card URL on the index slug omits the host segment', () => { - const a: ASTRAAnalysis = { - ...fixture(), - analyses: { - preprocessing: { - name: 'Pre', - decisions: {}, - prior_insights: {}, - findings: {}, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const cards: any[] = []; - function walk(n: any) { - if (n.type === 'card') cards.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(cards).toHaveLength(1); - expect(cards[0].url).toBe('/preprocessing'); - }); - - it('end-to-end: anchor in Option.description resolves into the page output', () => { - // Option descriptions are non-narrative prose. With the - // resolution context threaded through every render-* helper, - // the narrative grammar works here too. - const a: ASTRAAnalysis = { - ...fixture(), - decisions: { - scaling: { - label: 'Feature Scaling', - options: { - standard: { - label: 'Standard', - description: 'Scales features; supports the [SVM finding](#findings.best_model).', - }, - }, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const xrefs: any[] = []; - function walk(n: any) { - if (n.type === 'crossReference') xrefs.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(xrefs.some((x) => x.identifier === 'finding-best_model')).toBe(true); - }); - - it('end-to-end: anchor in Decision.rationale resolves into the page output', () => { - const a: ASTRAAnalysis = { - ...fixture(), - decisions: { - scaling: { - label: 'Feature Scaling', - rationale: 'Driven by the [iris dataset](#inputs.iris_data) characteristics.', - options: { standard: { label: 'Standard' } }, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const xrefs: any[] = []; - function walk(n: any) { - if (n.type === 'crossReference') xrefs.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(xrefs.some((x) => x.identifier === 'input-iris_data')).toBe(true); - }); - - it('end-to-end: anchor + markdown in Option.excluded_reason render and resolve', () => { - const a: ASTRAAnalysis = { - ...fixture(), - decisions: { - scaling: { - label: 'Feature Scaling', - options: { - minmax: { - label: 'MinMax', - excluded: true, - excluded_reason: 'Conflicts with **SVM**; see [the finding](#findings.best_model).', - }, - }, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const xrefs: any[] = []; - const strongs: any[] = []; - function walk(n: any) { - if (n.type === 'crossReference') xrefs.push(n); - if (n.type === 'strong') strongs.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(xrefs.some((x) => x.identifier === 'finding-best_model')).toBe(true); - expect(strongs.length).toBeGreaterThan(0); - // Renderer-glued "Excluded:" prefix is gone. - const flat = JSON.stringify(ast); - expect(flat).not.toContain('Excluded:'); - }); - - it('end-to-end: anchor + markdown in figure-output description render and resolve', () => { - // Figure rendering is driven by Output.type === 'figure'; the - // caption-equivalent metadata lives on Output.description (it - // parses with the narrative anchor grammar like every other - // prose surface). There is no Evidence.figure selector — the - // 'what kind' concern lives on Output, not Evidence. - const a: ASTRAAnalysis = { - ...fixture(), - outputs: [ - { - id: 'best_fit_plot', - type: 'figure', - label: 'Fig. 3', - description: 'Performance versus the [iris baseline](#inputs.iris_data).', - }, - ], - findings: { - best_model: { - id: 'best_model', - claim: 'SVM wins', - created_at: '2024-01-01', - evidence: [ - { id: 'ev1', artifact: 'best_fit_plot' }, - ], - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map([['best_fit_plot', '/tmp/best_fit_plot.png']]), - projectDir: '/tmp', - slug: 'index', - }); - const xrefs: any[] = []; - function walk(n: any) { - if (n.type === 'crossReference') xrefs.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(xrefs.some((x) => x.identifier === 'input-iris_data')).toBe(true); - const flat = JSON.stringify(ast); - expect(flat).not.toContain('[iris baseline](#inputs.iris_data)'); - expect(flat).toContain('Performance versus the'); - }); -}); - -describe('artifact evidence dispatches on Output.type', () => { - // Drop-in spec alignment: figure / table rendering is driven by - // Output.type, not by an Evidence selector. label / description - // on Output carry the caption-equivalent metadata. - - function withOutput(o: { id: string; type: 'figure' | 'table' | 'metric' | 'data' | 'report'; label?: string; description?: string }): ASTRAAnalysis { - return { - name: 'WithOutput', - decisions: {}, - prior_insights: {}, - findings: { - f1: { - id: 'f1', - claim: 'Result', - created_at: '2024-01-01', - evidence: [{ id: 'ev1', artifact: o.id }], - }, - }, - outputs: [o], - }; - } - - it('Output.type=figure renders an image+caption container', () => { - const a = withOutput({ - id: 'plot', - type: 'figure', - label: 'Plot', - description: 'A figure caption.', - }); - const ast = astraToMystAST({ - analysis: a, - universe: { id: 'u', decisions: {} }, - results: new Map([['plot', '/tmp/plot.png']]), - projectDir: '/tmp', - slug: 'index', - }); - const figures: any[] = []; - function walk(n: any) { - if (n.type === 'container' && n.kind === 'figure') figures.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(figures).toHaveLength(1); - const flat = JSON.stringify(figures[0]); - expect(flat).toContain('A figure caption.'); - expect(flat).toContain('/static/plot.png'); - }); - - it('Output.type=table renders a JSON file as a collapsible table', () => { - const tmpDir = mkdtempSync(join(tmpdir(), 'mystra-test-')); - const file = join(tmpDir, 'metrics.json'); - writeFileSync(file, JSON.stringify({ accuracy: 0.95, precision: 0.92 })); - const a = withOutput({ id: 'metrics', type: 'table', label: 'Metrics' }); - const ast = astraToMystAST({ - analysis: a, - universe: { id: 'u', decisions: {} }, - results: new Map([['metrics', file]]), - projectDir: tmpDir, - slug: 'index', - }); - const detailsNodes: any[] = []; - function walk(n: any) { - if (n.type === 'details') detailsNodes.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(detailsNodes.length).toBeGreaterThan(0); - const flat = JSON.stringify(detailsNodes); - expect(flat).toContain('accuracy'); - expect(flat).toContain('Metrics'); - }); - - it('broken evidence.artifact reference (output id not declared) emits console.warn', () => { - const a: ASTRAAnalysis = { - name: 'Broken', - decisions: {}, - prior_insights: {}, - findings: { - f1: { - id: 'f1', - claim: 'Whatever', - created_at: '2024-01-01', - evidence: [{ id: 'ev1', artifact: 'nonexistent' }], - }, - }, - outputs: [], - }; - const warns: string[] = []; - const orig = console.warn; - console.warn = (msg: any) => { warns.push(String(msg)); }; - try { - astraToMystAST({ - analysis: a, - universe: { id: 'u', decisions: {} }, - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - } finally { - console.warn = orig; - } - expect(warns.some((w) => w.includes('nonexistent'))).toBe(true); - }); - - it('declared output but unproduced artifact still renders a Pending Output admonition', () => { - const a = withOutput({ id: 'pending_plot', type: 'figure' }); - const ast = astraToMystAST({ - analysis: a, - universe: { id: 'u', decisions: {} }, - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const flat = JSON.stringify(ast); - expect(flat).toContain('Pending Output'); - expect(flat).toContain('pending_plot'); - }); -}); - -describe('prior_insights as minimal addressable carriers', () => { - // The xref contract just requires every published id to have a - // rendered carrier. Whether prior_insights surface as visible - // sections, sidebars, hovers, or hide entirely is downstream's - // call. Carrier shape: a single `container` with kind - // `prior-insight`, identifier `prior_insight-`, and - // structured `data` — no heading, no thematic-break separators, - // no 'Scope:' label paragraph. - - function withPriors(): ASTRAAnalysis { - return { - name: 'WithPriors', - decisions: { - scaling: { - label: 'Feature Scaling', - options: { - standard: { - label: 'Standard', - insights: ['scaling_helps'], - }, - minmax: { - label: 'MinMax', - insights: ['ghost_insight'], - }, - }, - }, - }, - prior_insights: { - scaling_helps: { - id: 'scaling_helps', - label: 'Scaling helps', - claim: 'Standardization improves SVM convergence.', - created_at: '2024-01-01', - scope: 'feature_engineering', - tags: ['preprocessing', 'svm'], - evidence: [], - }, - unreferenced_prior: { - id: 'unreferenced_prior', - claim: 'Unrelated background knowledge.', - created_at: '2024-01-01', - evidence: [], - }, - }, - findings: {}, - }; - } - - it('emits a `prior-insight` container carrier for every declared prior_insight', () => { - const ast = astraToMystAST({ - analysis: withPriors(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carriers: any[] = []; - function walk(n: any) { - if ( - n.type === 'container' && - n.kind === 'prior-insight' && - n.identifier?.startsWith('prior_insight-') - ) { - carriers.push(n); - } - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - const ids = carriers.map((c) => c.identifier).sort(); - expect(ids).toEqual(['prior_insight-scaling_helps', 'prior_insight-unreferenced_prior']); - }); - - it('carrier holds no heading, no thematic-break separators, no `Scope:` paragraph', () => { - const ast = astraToMystAST({ - analysis: withPriors(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carriers: any[] = []; - function walk(n: any) { - if (n.type === 'container' && n.kind === 'prior-insight') carriers.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - // No heading inside any prior_insight carrier. - for (const c of carriers) { - const flat = JSON.stringify(c); - expect(flat).not.toContain('"type":"heading"'); - expect(flat).not.toContain('"type":"thematicBreak"'); - // The `Scope: …` rendered prefix is gone — scope rides on - // `data.scope` for renderers that want to surface it. - expect(flat).not.toContain('Scope:'); - } - // No thematic-break sibling between the carriers either — - // separators are a layout opinion the transform shouldn't make. - const topLevel = ast.children.filter( - (n: any) => n.type === 'container' && n.kind === 'prior-insight', - ); - const idx0 = ast.children.indexOf(topLevel[0]); - const idx1 = ast.children.indexOf(topLevel[1]); - for (let i = idx0 + 1; i < idx1; i++) { - expect(ast.children[i].type).not.toBe('thematicBreak'); - } - }); - - it('carrier carries structured `data` (astraKind, id, label, scope, tags, derived)', () => { - const ast = astraToMystAST({ - analysis: withPriors(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carriers = new Map(); - function walk(n: any) { - if (n.type === 'container' && n.kind === 'prior-insight') { - carriers.set(n.identifier, n); - } - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - - const scaled = carriers.get('prior_insight-scaling_helps'); - expect(scaled).toBeTruthy(); - expect(scaled.class).toBe('astra astra-prior-insight'); - expect(scaled.data).toEqual({ - astraKind: 'prior_insight', - id: 'scaling_helps', - label: 'Scaling helps', - scope: 'feature_engineering', - tags: ['preprocessing', 'svm'], - derived: false, - }); - - const unref = carriers.get('prior_insight-unreferenced_prior'); - expect(unref).toBeTruthy(); - // Optional fields collapse to `null` (not `undefined`) so the - // shape survives a JSON round-trip without keys disappearing. - expect(unref.data.label).toBeNull(); - expect(unref.data.scope).toBeNull(); - expect(unref.data.tags).toBeNull(); - expect(unref.data.derived).toBe(false); - }); - - it('carrier children are [claim paragraph, …evidence body]', () => { - const ast = astraToMystAST({ - analysis: withPriors(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - let scaled: any; - function walk(n: any) { - if ( - n.type === 'container' && - n.kind === 'prior-insight' && - n.identifier === 'prior_insight-scaling_helps' - ) { - scaled = n; - } - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(scaled).toBeTruthy(); - // First child is a paragraph wrapping the claim's inline phrasing. - expect(scaled.children[0].type).toBe('paragraph'); - const claimText = JSON.stringify(scaled.children[0]); - expect(claimText).toContain('Standardization improves SVM convergence'); - // Evidence body is empty for this fixture (evidence: []), so - // children length is exactly 1. Keeps the carrier minimal. - expect(scaled.children).toHaveLength(1); - }); - - it('an unreferenced prior_insight has a rendered carrier (xref contract)', () => { - const a = withPriors(); - const pages = buildAllPages(a, { id: 'u', decisions: {} }, new Map(), '/tmp'); - const ids = pages[0].identifiers.map((e) => e.identifier); - expect(ids).toContain('prior_insight-unreferenced_prior'); - // Sanity: the carrier truly exists in the AST, not just in the index. - const ast = pages[0].ast; - const found = JSON.stringify(ast).includes('"prior_insight-unreferenced_prior"'); - expect(found).toBe(true); - }); - - it('option.insights renders as crossReference, not inline expansion', () => { - const ast = astraToMystAST({ - analysis: withPriors(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const xrefs: any[] = []; - function walk(n: any) { - if (n.type === 'crossReference') xrefs.push(n); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(xrefs.some((x) => x.identifier === 'prior_insight-scaling_helps')).toBe(true); - - // Only one container carrier per insight (no inline expansion - // duplicating the identifier inside the option tab). - const carriers: any[] = []; - function walkCarrier(n: any) { - if ( - n.type === 'container' && - n.kind === 'prior-insight' && - n.identifier === 'prior_insight-scaling_helps' - ) { - carriers.push(n); - } - for (const c of n.children ?? []) walkCarrier(c); - } - for (const n of ast.children) walkCarrier(n); - expect(carriers).toHaveLength(1); - }); - - it('broken option.insights reference emits a console.warn (no silent drop)', () => { - // The `ghost_insight` ref on the minmax option points at no - // declared prior_insight — log a visible warning and skip the - // crossReference rather than silently dropping. - const warns: string[] = []; - const orig = console.warn; - console.warn = (msg: any) => { warns.push(String(msg)); }; - try { - astraToMystAST({ - analysis: withPriors(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - } finally { - console.warn = orig; - } - expect(warns.some((w) => w.includes('ghost_insight'))).toBe(true); - }); - - it('option-tab crossReference resolves to the carrier id (container, not heading)', () => { - // End-to-end: a click on the supporting-insight ref should land - // on the prior_insight container rendered elsewhere on the page. - const ast = astraToMystAST({ - analysis: withPriors(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrierIds = new Set(); - const refs = new Set(); - function walk(n: any) { - if ( - n.type === 'container' && - n.kind === 'prior-insight' && - n.identifier - ) { - carrierIds.add(n.identifier); - } - if (n.type === 'crossReference') refs.add(n.identifier); - for (const c of n.children ?? []) walk(c); - } - for (const n of ast.children) walk(n); - expect(carrierIds.has('prior_insight-scaling_helps')).toBe(true); - expect(refs.has('prior_insight-scaling_helps')).toBe(true); - }); -}); - -// ────────────────────────────────────────────────────────────────── -// Phase B (complete-astra-coverage): per-Output provenance carriers. -// -// PR #19 (astra-spec v0.0.7) made the Output the unit of provenance: -// `Output.inputs` and `Output.decisions` declare what materializing -// the artifact depends on, and the Recipe is pure *how*. MySTRA -// emits one container per Output with non-empty provenance (after -// `from:` resolution) so downstream renderers (lightcone-ui, vellum) -// pattern-match on emitted mdast instead of reading `astra.yaml` -// directly. -// ────────────────────────────────────────────────────────────────── - -describe('Output provenance carriers', () => { - function withProvenance(): ASTRAAnalysis { - return { - name: 'Provenance fixture', - decisions: { - scaling: { - label: 'Feature Scaling', - options: { standard: { label: 'Standard' } }, - }, - }, - prior_insights: {}, - findings: {}, - inputs: [{ id: 'iris_data', type: 'data', description: 'Iris dataset' }], - outputs: [ - { - id: 'accuracy', - type: 'metric', - description: 'Held-out accuracy', - inputs: ['iris_data'], - decisions: ['scaling'], - }, - { - id: 'plain_metric', - type: 'metric', - description: 'No provenance to declare', - }, - ], - }; - } - - function findContainer(ast: any, kind: string, identifier: string): any | null { - for (const n of ast.children) { - if (n.type === 'container' && n.kind === kind && n.identifier === identifier) { - return n; - } - } - return null; - } - - it('emits a `output-provenance` container per Output with non-empty inputs/decisions', () => { - const ast = astraToMystAST({ - analysis: withProvenance(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const containers = ast.children.filter( - (n: any) => n.type === 'container' && n.kind === 'output-provenance', - ); - expect(containers).toHaveLength(1); - expect((containers[0] as any).identifier).toBe('output-accuracy-provenance'); - }); - - it('does not emit a carrier for Outputs with empty provenance', () => { - const ast = astraToMystAST({ - analysis: withProvenance(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - expect(findContainer(ast, 'output-provenance', 'output-plain_metric-provenance')).toBeNull(); - }); - - it('carrier carries structured `data` (astraKind, outputId, inputs, decisions, from)', () => { - const ast = astraToMystAST({ - analysis: withProvenance(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrier = findContainer(ast, 'output-provenance', 'output-accuracy-provenance'); - expect(carrier).toBeTruthy(); - expect(carrier.data).toMatchObject({ - astraKind: 'output_provenance', - outputId: 'accuracy', - inputs: ['iris_data'], - decisions: ['scaling'], - from: null, - unresolved: false, - }); - // Class is the conventional ASTRA marker for this kind. - expect(carrier.class).toContain('astra-output-provenance'); - }); - - it('emits inline crossReferences for each input and decision (fallback rendering)', () => { - const ast = astraToMystAST({ - analysis: withProvenance(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrier = findContainer(ast, 'output-provenance', 'output-accuracy-provenance'); - const refs: string[] = []; - function walk(n: any) { - if (n.type === 'crossReference') refs.push(n.identifier); - for (const c of n.children ?? []) walk(c); - } - for (const c of carrier.children ?? []) walk(c); - expect(refs).toContain('input-iris_data'); - expect(refs).toContain('decision-scaling'); - }); - - it('publishes the provenance identifier in the xref index', () => { - const tmp = mkdtempSync(join(tmpdir(), 'mystra-prov-')); - writeFileSync( - join(tmp, 'astra.yaml'), - 'name: x\ninputs: []\noutputs: []\ndecisions: {}\n', // unused; we use buildAllPages directly - ); - const pages = buildAllPages( - withProvenance(), - emptyUniverse(), - new Map(), - tmp, - ); - const ids = pages[0].identifiers.map((e) => e.identifier); - expect(ids).toContain('output-accuracy'); - expect(ids).toContain('output-accuracy-provenance'); - // No carrier was emitted for plain_metric — its identifier - // doesn't appear either, matching the xref contract. - expect(ids).not.toContain('output-plain_metric-provenance'); - }); - - it('sits adjacent to (after) the outputs registry table', () => { - const ast = astraToMystAST({ - analysis: withProvenance(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - // Find the outputs table — it carries one tableRow with - // identifier `output-accuracy`. - const outputsTableIdx = ast.children.findIndex((n: any) => { - if (n.type !== 'table') return false; - return (n.children ?? []).some( - (row: any) => row.identifier === 'output-accuracy', - ); - }); - const provenanceIdx = ast.children.findIndex( - (n: any) => - n.type === 'container' && - n.identifier === 'output-accuracy-provenance', - ); - expect(outputsTableIdx).toBeGreaterThan(-1); - expect(provenanceIdx).toBeGreaterThan(outputsTableIdx); - }); -}); - -describe('Output alias resolution', () => { - function withAlias(): ASTRAAnalysis { - return { - name: 'Alias fixture', - decisions: { - scaling: { label: 'Scaling', options: { standard: { label: 'Standard' } } }, - }, - prior_insights: {}, - findings: {}, - inputs: [{ id: 'iris_data', type: 'data' }], - // Top-level output re-exports a sub-analysis output. - outputs: [{ id: 'features', from: 'preprocessing.features' }], - analyses: { - preprocessing: { - name: 'Preprocessing', - decisions: {}, - prior_insights: {}, - findings: {}, - outputs: [ - { - id: 'features', - type: 'data', - description: 'Scaled features', - inputs: ['iris_data'], - decisions: ['scaling'], - }, - ], - }, - }, - }; - } - - it('resolves `from:` so an aliased Output inherits inputs/decisions', () => { - const ast = astraToMystAST({ - analysis: withAlias(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrier = ast.children.find( - (n: any) => - n.type === 'container' && - n.kind === 'output-provenance' && - n.identifier === 'output-features-provenance', - ); - expect(carrier).toBeTruthy(); - expect((carrier as any).data).toMatchObject({ - astraKind: 'output_provenance', - outputId: 'features', - inputs: ['iris_data'], - decisions: ['scaling'], - from: 'preprocessing.features', - unresolved: false, - }); - }); - - it('walks multi-segment paths through nested analyses', () => { - const a: ASTRAAnalysis = { - name: 'Two-level alias', - decisions: {}, - prior_insights: {}, - findings: {}, - outputs: [{ id: 'final', from: 'outer.inner.leaf' }], - analyses: { - outer: { - name: 'Outer', - decisions: {}, - prior_insights: {}, - findings: {}, - analyses: { - inner: { - name: 'Inner', - decisions: {}, - prior_insights: {}, - findings: {}, - outputs: [ - { - id: 'leaf', - type: 'data', - inputs: ['raw'], - decisions: [], - }, - ], - }, - }, - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrier = ast.children.find( - (n: any) => - n.type === 'container' && - n.identifier === 'output-final-provenance', - ); - expect(carrier).toBeTruthy(); - expect((carrier as any).data.inputs).toEqual(['raw']); - expect((carrier as any).data.from).toBe('outer.inner.leaf'); - }); - - it('flags unresolved when `from:` points nowhere', () => { - const a: ASTRAAnalysis = { - name: 'Broken alias', - decisions: {}, - prior_insights: {}, - findings: {}, - // Note: emit-predicate skips outputs with empty resolved - // provenance, so a broken alias produces no carrier — the - // `unresolved` flag is observable through the resolveOutput - // unit, not the rendered AST. - outputs: [ - { - id: 'placeholder', - type: 'data', - inputs: ['some_input'], - from: 'ghost.nope', - }, - ], - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - // The Output's `from:` is set, so per the spec the local - // `inputs`/`decisions` are forbidden — but in practice an - // upstream validator catches that. The emission falls back to - // declared `inputs` (empty after resolution failure) so no - // carrier is emitted. The contract: a broken alias never - // produces phantom provenance. - const carriers = ast.children.filter( - (n: any) => n.type === 'container' && n.kind === 'output-provenance', - ); - expect(carriers).toHaveLength(0); - }); -}); - -// ────────────────────────────────────────────────────────────────── -// Phase C (complete-astra-coverage): per-Output recipe carriers. -// -// Recipe is the *how* of an Output (command, container, resources). -// MySTRA emits one `kind: 'output-recipe'` container per Output with -// a non-empty resolved recipe; structured `data` slot is the renderer -// contract, fallback children are a collapsed `details` block. -// Closes the Recipe coverage hole — no consumer needs to read -// `astra.yaml` to surface (or hide) recipes. -// ────────────────────────────────────────────────────────────────── - -describe('Output recipe carriers', () => { - function withRecipe(): ASTRAAnalysis { - return { - name: 'Recipe fixture', - decisions: {}, - prior_insights: {}, - findings: {}, - inputs: [{ id: 'iris_data', type: 'data' }], - outputs: [ - { - id: 'accuracy', - type: 'metric', - description: 'Held-out accuracy', - inputs: ['iris_data'], - recipe: { - command: 'python src/train.py {inputs} > {output}', - container: 'python:3.11-slim', - resources: { cpus: 4, memory: '8Gi', time_limit: '30m' }, - }, - }, - // No recipe — should produce no carrier. - { id: 'plain_metric', type: 'metric' }, - ], - }; - } - - function findContainer(ast: any, kind: string, identifier: string): any | null { - for (const n of ast.children) { - if (n.type === 'container' && n.kind === kind && n.identifier === identifier) { - return n; - } - } - return null; - } - - it('emits a `output-recipe` container per Output with a non-empty recipe', () => { - const ast = astraToMystAST({ - analysis: withRecipe(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const containers = ast.children.filter( - (n: any) => n.type === 'container' && n.kind === 'output-recipe', - ); - expect(containers).toHaveLength(1); - expect((containers[0] as any).identifier).toBe('output-accuracy-recipe'); - }); - - it('does not emit a carrier for Outputs with no recipe', () => { - const ast = astraToMystAST({ - analysis: withRecipe(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - expect(findContainer(ast, 'output-recipe', 'output-plain_metric-recipe')).toBeNull(); - }); - - it('does not emit a carrier when recipe is present but every field is empty', () => { - const a: ASTRAAnalysis = { - name: 'Empty recipe', - decisions: {}, - prior_insights: {}, - findings: {}, - outputs: [ - // recipe object exists but no command/container/resources; - // emission predicate should skip it. - { id: 'noop', type: 'data', recipe: { resources: {} } }, - ], - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carriers = ast.children.filter( - (n: any) => n.type === 'container' && n.kind === 'output-recipe', - ); - expect(carriers).toHaveLength(0); - }); - - it('carrier carries structured `data` (astraKind, outputId, command, container, resources, from)', () => { - const ast = astraToMystAST({ - analysis: withRecipe(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrier = findContainer(ast, 'output-recipe', 'output-accuracy-recipe'); - expect(carrier).toBeTruthy(); - expect(carrier.data).toMatchObject({ - astraKind: 'output_recipe', - outputId: 'accuracy', - command: 'python src/train.py {inputs} > {output}', - container: 'python:3.11-slim', - resources: { cpus: 4, memory: '8Gi', time_limit: '30m' }, - from: null, - unresolved: false, - }); - expect(carrier.class).toContain('astra-output-recipe'); - }); - - it('fallback children are a single `details` block collapsed by default', () => { - const ast = astraToMystAST({ - analysis: withRecipe(), - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrier = findContainer(ast, 'output-recipe', 'output-accuracy-recipe'); - expect(carrier.children).toHaveLength(1); - const det = carrier.children[0]; - expect(det.type).toBe('details'); - expect(det.open).toBe(false); - // First inner node is the summary "Recipe". - expect(det.children[0].type).toBe('summary'); - // Command renders as a fenced bash code block somewhere inside. - const codeNode = det.children.find((c: any) => c.type === 'code'); - expect(codeNode).toBeTruthy(); - expect(codeNode.lang).toBe('bash'); - expect(codeNode.value).toContain('python src/train.py'); - }); - - it('publishes the recipe identifier in the xref index', () => { - const tmp = mkdtempSync(join(tmpdir(), 'mystra-recipe-')); - writeFileSync(join(tmp, 'astra.yaml'), 'name: x\n'); - const pages = buildAllPages( - withRecipe(), - emptyUniverse(), - new Map(), - tmp, - ); - const ids = pages[0].identifiers.map((e) => e.identifier); - expect(ids).toContain('output-accuracy-recipe'); - expect(ids).not.toContain('output-plain_metric-recipe'); - }); - - it('aliased Output inherits its source recipe', () => { - const a: ASTRAAnalysis = { - name: 'Alias inherits recipe', - decisions: {}, - prior_insights: {}, - findings: {}, - // Top-level re-export of a sub-analysis output. Local node is - // a pure pointer; recipe inherits from the source. - outputs: [{ id: 'features', from: 'preprocessing.features' }], - analyses: { - preprocessing: { - name: 'Preprocessing', - decisions: {}, - prior_insights: {}, - findings: {}, - outputs: [ - { - id: 'features', - type: 'data', - recipe: { - command: 'python preprocess.py', - container: 'python:3.11', - }, - }, - ], - }, - }, - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const carrier = ast.children.find( - (n: any) => - n.type === 'container' && - n.kind === 'output-recipe' && - n.identifier === 'output-features-recipe', - ); - expect(carrier).toBeTruthy(); - expect((carrier as any).data).toMatchObject({ - astraKind: 'output_recipe', - outputId: 'features', - command: 'python preprocess.py', - container: 'python:3.11', - from: 'preprocessing.features', - unresolved: false, - }); - }); - - it('sits after the outputs registry table and after provenance carriers', () => { - const a: ASTRAAnalysis = { - name: 'Order check', - decisions: { - scaling: { label: 'Scaling', options: { standard: { label: 'Standard' } } }, - }, - prior_insights: {}, - findings: {}, - inputs: [{ id: 'raw', type: 'data' }], - outputs: [ - { - id: 'accuracy', - type: 'metric', - inputs: ['raw'], - decisions: ['scaling'], - recipe: { command: 'python score.py' }, - }, - ], - }; - const ast = astraToMystAST({ - analysis: a, - universe: emptyUniverse(), - results: new Map(), - projectDir: '/tmp', - slug: 'index', - }); - const tableIdx = ast.children.findIndex((n: any) => { - if (n.type !== 'table') return false; - return (n.children ?? []).some( - (row: any) => row.identifier === 'output-accuracy', - ); - }); - const provIdx = ast.children.findIndex( - (n: any) => - n.type === 'container' && - n.identifier === 'output-accuracy-provenance', - ); - const recipeIdx = ast.children.findIndex( - (n: any) => - n.type === 'container' && - n.identifier === 'output-accuracy-recipe', - ); - expect(tableIdx).toBeGreaterThan(-1); - expect(provIdx).toBeGreaterThan(tableIdx); - expect(recipeIdx).toBeGreaterThan(provIdx); - }); -}); diff --git a/tests/plugin-emission.test.ts b/tests/plugin-emission.test.ts new file mode 100644 index 0000000..689e30d --- /dev/null +++ b/tests/plugin-emission.test.ts @@ -0,0 +1,248 @@ +/** + * Plugin emission tests (Strategy A). + * + * Drives the ASTRA MyST plugin's directives / roles / transforms against the + * real DESI DR1 BAO project in `prototype/` and asserts the emitted mdast: + * stock node types, recognition markers (`astra-*` classes), stable + * identifiers, project-relative image urls, live value interpolation, scoped + * sub-analysis resolution, and the resolved-store shape. + */ + +import { describe, it, expect, beforeAll } from 'vitest'; +import { fileURLToPath } from 'node:url'; +import { dirname, join } from 'node:path'; +import plugin from '../src/index.js'; + +const PROJECT_ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', 'prototype'); + +beforeAll(() => { + process.env.ASTRA_PROJECT_ROOT = PROJECT_ROOT; + delete process.env.ASTRA_UNIVERSE; // default to first universe (baseline) +}); + +// ── mdast traversal helpers ────────────────────────────────────────────── + +type Node = Record; + +function walk(nodes: Node[] | Node, visit: (n: Node) => void): void { + const arr = Array.isArray(nodes) ? nodes : [nodes]; + for (const n of arr) { + if (!n || typeof n !== 'object') continue; + visit(n); + if (Array.isArray(n.children)) walk(n.children, visit); + } +} + +function findFirst(nodes: Node[], pred: (n: Node) => boolean): Node | undefined { + let found: Node | undefined; + walk(nodes, (n) => { + if (!found && pred(n)) found = n; + }); + return found; +} + +function hasClass(n: Node | undefined, cls: string): boolean { + return typeof n?.class === 'string' && n.class.split(/\s+/).includes(cls); +} + +function byIdentifier(nodes: Node[], id: string): Node | undefined { + return findFirst(nodes, (n) => n.identifier === id); +} + +function textOf(nodes: Node[] | Node): string { + let out = ''; + walk(nodes, (n) => { + if (n.type === 'text' && typeof n.value === 'string') out += n.value; + }); + return out; +} + +// ── Plugin handle lookups ──────────────────────────────────────────────── + +function directive(name: string) { + const d = plugin.directives.find((x: any) => x.name === `astra:${name}`); + if (!d) throw new Error(`no directive astra:${name}`); + return d; +} +function role(name: string) { + const r = plugin.roles.find((x: any) => x.name === `astra:${name}`); + if (!r) throw new Error(`no role astra:${name}`); + return r; +} +function runDirective(name: string, arg?: string, options: Record = {}): Node[] { + return (directive(name) as any).run({ arg, options }) as Node[]; +} +function runRole(name: string, body: string): Node[] { + return (role(name) as any).run({ body }) as Node[]; +} + +// ── Block directives ────────────────────────────────────────────────────── + +describe('block directives', () => { + it('decision → tabSet carrier with astra-decision class + identifier', () => { + const nodes = runDirective('decision', 'covariance_source'); + const carrier = byIdentifier(nodes, 'decision-covariance_source'); + expect(carrier).toBeDefined(); + expect(hasClass(carrier, 'astra-decision')).toBe(true); + expect(findFirst(nodes, (n) => n.type === 'tabSet')).toBeDefined(); + // never the legacy /static scheme + expect(JSON.stringify(nodes)).not.toContain('/static/'); + }); + + it('figure output → container[figure] with project-relative image url + markers', () => { + const nodes = runDirective('output', 'bao_fit_plot'); + const carrier = byIdentifier(nodes, 'output-bao_fit_plot'); + expect(carrier?.type).toBe('container'); + expect(carrier?.kind).toBe('figure'); + expect(hasClass(carrier, 'astra-output')).toBe(true); + expect(hasClass(carrier, 'astra-output--figure')).toBe(true); + const image = findFirst(nodes, (n) => n.type === 'image'); + expect(image?.url).toBe('results/baseline/bao_fit_plot/bao_fit_plot.png'); + expect(image?.url.startsWith('/static/')).toBe(false); + // provenance disclosure emitted alongside + expect(findFirst(nodes, (n) => n.type === 'details')).toBeDefined(); + }); + + it('table output → container[table] tagged astra-output--table', () => { + const nodes = runDirective('output', 'bao_distance_table'); + const carrier = byIdentifier(nodes, 'output-bao_distance_table'); + expect(carrier?.type).toBe('container'); + expect(carrier?.kind).toBe('table'); + expect(hasClass(carrier, 'astra-output--table')).toBe(true); + expect(findFirst(nodes, (n) => n.type === 'table')).toBeDefined(); + }); + + it('finding → astra-finding carrier with identifier', () => { + const nodes = runDirective('finding', 'bao_detected_post_recon'); + const carrier = byIdentifier(nodes, 'finding-bao_detected_post_recon'); + expect(carrier).toBeDefined(); + expect(hasClass(carrier, 'astra-finding')).toBe(true); + }); + + it('finding :compact: → claim heading + scope, no evidence figure', () => { + const nodes = runDirective('finding', 'bao_detected_post_recon', { compact: true }); + expect(byIdentifier(nodes, 'finding-bao_detected_post_recon')).toBeDefined(); + expect(findFirst(nodes, (n) => n.type === 'image')).toBeUndefined(); + }); + + it('prior-insight → seealso admonition with astra-prior-insight class', () => { + const nodes = runDirective('prior-insight', 'combined_systematic_budget'); + const adm = findFirst(nodes, (n) => n.type === 'admonition'); + expect(adm?.kind).toBe('seealso'); + expect(hasClass(adm, 'astra-prior-insight')).toBe(true); + expect(adm?.identifier).toBe('prior_insight-combined_systematic_budget'); + }); + + it('subanalysis → card linking to the sub-page, tagged astra-subanalysis', () => { + const nodes = runDirective('subanalysis', 'reconstruction'); + const carrier = byIdentifier(nodes, 'analysis-reconstruction'); + expect(carrier?.type).toBe('card'); + expect(hasClass(carrier, 'astra-subanalysis')).toBe(true); + expect(carrier?.url).toBe('/reconstruction'); + expect(carrier?.title).toBeTruthy(); + }); + + it('inputs / outputs tables carry their registry classes', () => { + const inputs = runDirective('inputs'); + expect(hasClass(inputs[0], 'astra-inputs')).toBe(true); + const outputs = runDirective('outputs'); + expect(hasClass(outputs[0], 'astra-outputs')).toBe(true); + }); +}); + +// ── Scoped sub-analysis resolution ────────────────────────────────────────── + +describe('sub-analysis scope', () => { + it('resolves a scoped figure output (clustering.xi_multipoles_plot)', () => { + const nodes = runDirective('output', 'clustering.xi_multipoles_plot'); + const carrier = byIdentifier(nodes, 'output-xi_multipoles_plot'); + expect(carrier?.kind).toBe('figure'); + expect(hasClass(carrier, 'astra-output')).toBe(true); + }); + + it('resolves a scoped decision (reconstruction.algorithm)', () => { + const nodes = runDirective('decision', 'reconstruction.algorithm'); + expect(byIdentifier(nodes, 'decision-algorithm')).toBeDefined(); + }); +}); + +// ── Inline roles ──────────────────────────────────────────────────────────── + +describe('inline roles', () => { + it('cite role → neutral astra-ref token with a hidden preview card', () => { + const [token] = runRole('decision', 'covariance_source'); + expect(hasClass(token, 'astra-ref')).toBe(true); + expect(hasClass(token, 'astra-ref--decision')).toBe(true); + const card = findFirst([token], (n) => hasClass(n, 'astra-card')); + expect(card).toBeDefined(); + // hidden by default so a bare viewer never spills the card inline + expect(card?.style).toEqual({ display: 'none' }); + expect(hasClass(card, 'astra-card--decision')).toBe(true); + }); + + it('cite role honours a |display override for the inline label', () => { + const [token] = runRole('prior-insight', 'combined_systematic_budget|the budget'); + const label = findFirst([token], (n) => hasClass(n, 'astra-ref__label')); + expect(textOf([label!])).toBe('the budget'); + }); + + it('value role interpolates a real cell with ± uncertainty', () => { + const [token] = runRole('value', 'bao_distance_table tracer=lrg3_elg1 col=DV_over_rd pm'); + const label = findFirst([token], (n) => hasClass(n, 'astra-ref__label')); + expect(textOf([label!])).toBe('19.88 ± 0.17'); + }); + + it('value role formats to significant figures without ±', () => { + const [token] = runRole('value', 'bao_alpha_values tracer=elg1 recon=Pre col=alpha1_std'); + const label = findFirst([token], (n) => hasClass(n, 'astra-ref__label')); + expect(textOf([label!])).toBe('0.0696'); + }); + + it('value role surfaces a clear error for a missing column', () => { + const [node] = runRole('value', 'bao_distance_table col=not_a_column'); + expect(node.type).toBe('inlineCode'); + expect(node.value).toContain('value'); + }); +}); + +// ── Resolved store transform ───────────────────────────────────────────────── + +describe('resolved-store transform', () => { + function runStore(path: string): Record { + const storeTransform = plugin.transforms.find((t: any) => t.name === 'astra-resolved-store'); + const tree: Node = { type: 'root', children: [] }; + (storeTransform as any).plugin()(tree, { path }); + const carrier = tree.children.find((n: any) => n.class === 'astra-store'); + expect(carrier).toBeDefined(); + return carrier!.data.astra; + } + + it('emits a hidden carrier with the resolved model keyed by id (root scope)', () => { + const store = runStore('index.md'); + const carrierStyle = { display: 'none' }; + // figure output: project-relative path, no /static + const fig = store.outputs['bao_fit_plot']; + expect(fig.type).toBe('figure'); + expect(fig.resolved_path).toBe('results/baseline/bao_fit_plot/bao_fit_plot.png'); + // table output: inlined parsed rows + const tbl = store.outputs['bao_distance_table']; + expect(tbl.type).toBe('table'); + expect(tbl.table_data?.headers).toContain('DV_over_rd'); + // decision: selected option resolved under the active universe + expect(store.decisions['covariance_source'].selected).toBeTruthy(); + // finding + sub-analysis presence + expect(store.findings['bao_detected_post_recon']).toBeDefined(); + expect(store.subanalyses['reconstruction'].url).toBe('/reconstruction'); + // hidden carrier is invisible on book-theme + const storeTransform = plugin.transforms.find((t: any) => t.name === 'astra-resolved-store'); + const tree: Node = { type: 'root', children: [] }; + (storeTransform as any).plugin()(tree, { path: 'index.md' }); + expect(tree.children[0].style).toEqual(carrierStyle); + }); + + it('scopes the store to a sub-analysis page', () => { + const store = runStore('clustering.md'); + expect(store.analysis.slug).toBe('clustering'); + expect(store.outputs['xi_multipoles_plot']).toBeDefined(); + }); +}); diff --git a/tests/schema-coverage.test.ts b/tests/schema-coverage.test.ts deleted file mode 100644 index 878bf80..0000000 --- a/tests/schema-coverage.test.ts +++ /dev/null @@ -1,201 +0,0 @@ -/** - * Schema-coverage guard. - * - * Walks the vendored astra-spec schemas (`tests/fixtures/schema-v0.0.7/`) - * and asserts that `src/types/astra.ts` declares an interface field for - * every slot of every class. This is the mechanical replacement for - * hand-auditing every astra-spec release: when astra-spec adds a slot - * MySTRA hasn't absorbed, this test fails loudly. - * - * The mapping from LinkML class → TS interface lives in - * `CLASS_TO_INTERFACE`. When astra-spec adds a new top-level class, - * extend that map (and add the interface in `astra.ts`). Slots - * intentionally encoded elsewhere (e.g., the `id` slot on `Option`, - * `Decision`, `UniverseNode`, `DecisionSelection` becomes the - * `Record` key in the parent surface) are listed in - * `KEY_AS_PARENT_RECORD_KEY` and excluded from the coverage check. - * - * See `tests/fixtures/schema-v0.0.7/README.md` for the bump - * discipline. - */ - -import { describe, it, expect } from 'vitest'; -import { readFileSync, readdirSync } from 'node:fs'; -import { join, dirname } from 'node:path'; -import { fileURLToPath } from 'node:url'; -import yaml from 'js-yaml'; - -const __dirname = dirname(fileURLToPath(import.meta.url)); -const SCHEMA_DIR = join(__dirname, 'fixtures', 'schema-v0.0.7'); -const TYPES_FILE = join(__dirname, '..', 'src', 'types', 'astra.ts'); - -// LinkML class → TypeScript interface name. -// -// `null` means the class is intentionally encoded into a parent -// surface (typically as a Record key) and has no standalone TS -// interface — see KEY_AS_PARENT_RECORD_KEY for the absorbed slots. -const CLASS_TO_INTERFACE: Record = { - // analysis.yaml - KeyValuePair: null, // not currently used by transforms; passthrough metadata - Narrative: 'ASTRANarrative', - Resources: 'ASTRAResources', - Recipe: 'ASTRARecipe', - Input: 'ASTRAInput', - Output: 'ASTRAOutput', - Option: 'ASTRAOption', - Decision: 'ASTRADecision', - Analysis: 'ASTRAAnalysis', - // insight.yaml - TextQuoteSelector: 'TextQuoteSelector', - FragmentSelector: 'FragmentSelector', - Evidence: 'ASTRAEvidence', - Insight: 'ASTRAInsight', - InsightCollection: null, // wrapper; consumers use Record directly - // universe.yaml - DecisionSelection: null, // collapses to Record on parent - UniverseNode: 'ASTRAUniverseNode', - Universe: 'ASTRAUniverse', -}; - -// Slots whose value lives as the *key* of a Record<…> on the parent -// interface (LinkML `identifier: true` slots inlined into a parent -// map). The coverage check skips these — they're present in the TS -// surface, just not as a named field. -const KEY_AS_PARENT_RECORD_KEY = new Set([ - 'Option.id', - 'Decision.id', // when keyed by id in Analysis.decisions - 'UniverseNode.id', - 'DecisionSelection.decision_id', -]); - -interface LinkMLSchema { - classes?: Record; - slots?: Record; -} - -interface LinkMLClass { - attributes?: Record; - slots?: string[]; -} - -interface LinkMLSlot { - // Currently unused by the coverage check; declared for clarity. - description?: string; -} - -function loadSchema(filename: string): LinkMLSchema { - const text = readFileSync(join(SCHEMA_DIR, filename), 'utf-8'); - return yaml.load(text) as LinkMLSchema; -} - -function loadAllSchemas(): { sharedSlots: Set; classes: Record } { - const files = readdirSync(SCHEMA_DIR).filter((f) => f.endsWith('.yaml')); - const classes: Record = {}; - const sharedSlots = new Set(); - for (const f of files) { - const schema = loadSchema(f); - for (const slotName of Object.keys(schema.slots ?? {})) sharedSlots.add(slotName); - for (const [name, def] of Object.entries(schema.classes ?? {})) { - classes[name] = def; - } - } - return { sharedSlots, classes }; -} - -/** - * Extract the body of a TypeScript interface as plain text. Used to - * grep for slot names without parsing TS — keeps the test - * dependency-free. This is intentionally low-fidelity: it counts a - * slot as "covered" if its name appears anywhere in the interface - * body, which catches comments, JSDoc, and renamed fields. False - * positives are preferable to false negatives here — the test is a - * smoke alarm, not a static analyzer. - */ -function extractInterfaceBody(source: string, name: string): string | null { - // Match: `export interface NAME {…}` (single-line or multi-line). - // Word-boundary anchor stops `ASTRAUniverse` from matching - // `ASTRAUniverseNode` (or any other `${name}…` prefix collision). - // The brace counter handles nested braces in JSDoc / generics. - const re = new RegExp(`export interface ${name}\\b`); - const match = re.exec(source); - if (!match) return null; - const start = match.index; - const open = source.indexOf('{', start); - if (open === -1) return null; - let depth = 0; - for (let i = open; i < source.length; i++) { - if (source[i] === '{') depth++; - else if (source[i] === '}') { - depth--; - if (depth === 0) return source.slice(open + 1, i); - } - } - return null; -} - -function classSlots(klass: LinkMLClass, sharedSlots: Set): string[] { - const all = new Set(); - for (const name of Object.keys(klass.attributes ?? {})) all.add(name); - for (const name of klass.slots ?? []) { - if (sharedSlots.has(name)) all.add(name); - } - return [...all]; -} - -describe('astra-spec coverage in src/types/astra.ts', () => { - const { sharedSlots, classes } = loadAllSchemas(); - const typesSource = readFileSync(TYPES_FILE, 'utf-8'); - - // Sanity: every class in CLASS_TO_INTERFACE actually exists in the - // vendored schema (catch typos in the mapping). - it('CLASS_TO_INTERFACE covers every class declared in the schema', () => { - const declaredClasses = Object.keys(classes).sort(); - const mappedClasses = Object.keys(CLASS_TO_INTERFACE).sort(); - expect(mappedClasses).toEqual(declaredClasses); - }); - - // For each class with a TS interface, every slot must be referenced. - for (const [className, interfaceName] of Object.entries(CLASS_TO_INTERFACE)) { - if (!interfaceName) continue; - - it(`${className} → ${interfaceName}: every slot is referenced`, () => { - const klass = classes[className]; - const slots = classSlots(klass, sharedSlots); - const body = extractInterfaceBody(typesSource, interfaceName); - expect(body, `interface ${interfaceName} not found in src/types/astra.ts`).toBeTruthy(); - - const missing: string[] = []; - for (const slot of slots) { - if (KEY_AS_PARENT_RECORD_KEY.has(`${className}.${slot}`)) continue; - // Identifier slots that aren't in KEY_AS_PARENT_RECORD_KEY - // are still expected as a TS field (e.g. Input.id, Output.id, - // Analysis.id, Universe.id, Evidence.id, Insight.id). - if (!body!.includes(slot)) missing.push(slot); - } - expect(missing).toEqual([]); - }); - } - - // Whole-class coverage: a new top-level class shouldn't slip past - // by being missing from the mapping. This is the one place where a - // missing entry surfaces as a clear error. - it('every class in the schema has either an interface or an explicit `null`', () => { - for (const className of Object.keys(classes)) { - expect( - Object.prototype.hasOwnProperty.call(CLASS_TO_INTERFACE, className), - `class ${className} not in CLASS_TO_INTERFACE — add an interface or mark null`, - ).toBe(true); - } - }); -}); - -describe('schema version banner in src/types/astra.ts', () => { - it('declares the tracked version + commit (so the discipline is auditable)', () => { - const text = readFileSync(TYPES_FILE, 'utf-8'); - // Loose match — just enforce that *some* "Tracks astra-spec - // vX.Y.Z (commit …)" line exists. Bumping the version is part - // of the bump discipline; we don't lock this test to a specific - // version because then the test has to change every release. - expect(text).toMatch(/Tracks\s+astra-spec\s+v\d+\.\d+\.\d+\s*\(commit\s+`?[a-f0-9]+`?\)/); - }); -}); diff --git a/tests/server-routes.test.ts b/tests/server-routes.test.ts deleted file mode 100644 index 3035f91..0000000 --- a/tests/server-routes.test.ts +++ /dev/null @@ -1,92 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it } from 'vitest'; -import { createServer as createNetServer } from 'node:net'; -import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs'; -import { dirname, join } from 'node:path'; -import os from 'node:os'; -import { createContentServer, type ContentServer } from '../src/server/index.js'; - -function write(path: string, content: string) { - mkdirSync(dirname(path), { recursive: true }); - writeFileSync(path, content); -} - -function getFreePort(): Promise { - return new Promise((resolve, reject) => { - const server = createNetServer(); - server.once('error', reject); - server.listen(0, '127.0.0.1', () => { - const address = server.address(); - if (!address || typeof address === 'string') { - server.close(); - reject(new Error('Failed to allocate an ephemeral port')); - return; - } - server.close((err) => { - if (err) reject(err); - else resolve(address.port); - }); - }); - }); -} - -describe('content server routes', () => { - let projectDir = ''; - let port = 0; - let server: ContentServer | null = null; - - beforeEach(async () => { - projectDir = mkdtempSync(join(os.tmpdir(), 'mystra-server-')); - port = await getFreePort(); - - write( - join(projectDir, 'astra.yaml'), - `name: Root analysis -narrative: - summary: Root summary -analyses: - child: - name: Child analysis - narrative: - summary: Child summary - outputs: - - id: child_plot - type: figure - description: Child plot -`, - ); - write(join(projectDir, 'universes', 'baseline.yaml'), `id: baseline\ndecisions: {}\n`); - write( - join(projectDir, 'analyses', 'child', 'results', 'baseline', 'child_plot.png'), - 'child-image', - ); - - server = createContentServer({ - projectDir, - contentPort: port, - universeName: 'baseline', - }); - await server.start(); - }); - - afterEach(() => { - server?.close(); - server = null; - if (projectDir) rmSync(projectDir, { recursive: true, force: true }); - }); - - it('serves nested sub-analysis artifacts through /static and the /astra sidecar', async () => { - const astraRes = await fetch(`http://127.0.0.1:${port}/astra/child.json`); - expect(astraRes.status).toBe(200); - const astra = (await astraRes.json()) as { - outputs: Array<{ id: string; resolved_path?: string }>; - }; - expect(astra.outputs).toEqual([ - expect.objectContaining({ id: 'child_plot', resolved_path: '/static/child_plot.png' }), - ]); - - const staticRes = await fetch(`http://127.0.0.1:${port}/static/child_plot.png`); - expect(staticRes.status).toBe(200); - const bytes = Buffer.from(await staticRes.arrayBuffer()).toString('utf-8'); - expect(bytes).toBe('child-image'); - }); -}); From 3b99a5b66c1c2f659e8bc703bf2b0902e53b126f Mon Sep 17 00:00:00 2001 From: EiffL Date: Mon, 25 May 2026 21:37:12 +0200 Subject: [PATCH 02/17] =?UTF-8?q?refactor:=20rename=20narrative-parser=20?= =?UTF-8?q?=E2=86=92=20prose;=20drop=20dead=20#narrative=20anchor=20branch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The module isn't about the `narrative:` field (Strategy A ignores it — the author writes that prose in the Markdown page). It's the engine for the Markdown embedded in components (rationale, claim, notes, descriptions, captions) + the ASTRA anchor resolver. Renamed to `prose.ts` to match, and repointed every importer. Also removed the `#narrative.
` case in resolveAnchorPath: it minted `narrative-
` identifiers that have no carrier and otherwise fell through to the plain-link default. Dropped its two now-obsolete tests. The `narrative.summary` blurb on sub-analysis cards is kept. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 2 +- STRATEGY-A-REFACTOR.md | 2 +- src/index.ts | 4 +- .../{narrative-parser.ts => prose.ts} | 41 +++++++------------ src/transform/render-data-sources.ts | 2 +- src/transform/render-evidence.ts | 2 +- src/transform/render-findings.ts | 2 +- src/transform/render-methods.ts | 2 +- ...narrative-parser.test.ts => prose.test.ts} | 25 ++--------- 9 files changed, 25 insertions(+), 57 deletions(-) rename src/transform/{narrative-parser.ts => prose.ts} (93%) rename tests/{narrative-parser.test.ts => prose.test.ts} (94%) diff --git a/README.md b/README.md index 80d6420..284e7e5 100644 --- a/README.md +++ b/README.md @@ -146,7 +146,7 @@ src/ ├── loader.ts Load a project for one universe (via the SDK) + resolve result files └── transform/ Per-component renderers used by the plugin ├── ast-helpers.ts Pure AST node constructors - ├── narrative-parser.ts myst-parser wrapper + anchor-grammar resolver + ├── prose.ts Parse component Markdown (myst-parser) + resolve ASTRA anchors ├── parse-table-data.ts CSV/JSON table parser ├── resolve-output.ts Resolves `from:` output/alias chains ├── resolved-store.ts Builds the resolved data store for rich themes diff --git a/STRATEGY-A-REFACTOR.md b/STRATEGY-A-REFACTOR.md index c8738b2..45781a1 100644 --- a/STRATEGY-A-REFACTOR.md +++ b/STRATEGY-A-REFACTOR.md @@ -127,7 +127,7 @@ The lower layers are reused; the server/orchestration layer is removed. and the `schema-coverage` guard test are all gone — the SDK is the single source of truth, pinned in `package.json`. - `src/transform/` per-component renderers actually used by the plugin: - `ast-helpers`, `narrative-parser` (prose + anchor grammar), `parse-table-data`, + `ast-helpers`, `prose` (component-prose parsing + anchor grammar), `parse-table-data`, `resolve-output`, `render-methods` (renderDecision), `render-findings` (renderFinding), `render-evidence` (renderOneOutput / evidence / tables), `render-data-sources` (inputs/outputs tables). diff --git a/src/index.ts b/src/index.ts index bdb22c9..5adb839 100644 --- a/src/index.ts +++ b/src/index.ts @@ -51,12 +51,12 @@ import { makeProseParser, resolveNarrativeAnchors, firstParagraphText, -} from './transform/narrative-parser.js'; +} from './transform/prose.js'; import type { AnalysisScope, PriorInsightScope, ProseParser, -} from './transform/narrative-parser.js'; +} from './transform/prose.js'; import { admonition, admonitionTitle, diff --git a/src/transform/narrative-parser.ts b/src/transform/prose.ts similarity index 93% rename from src/transform/narrative-parser.ts rename to src/transform/prose.ts index d9d603c..7474d4e 100644 --- a/src/transform/narrative-parser.ts +++ b/src/transform/prose.ts @@ -1,21 +1,18 @@ /** - * MyST Markdown parsing for ASTRA prose fields, plus the v0.0.6 - * narrative anchor resolver. + * The prose engine: parse the Markdown embedded in ASTRA *components*, and + * resolve ASTRA anchor links within it. * - * All Markdown content (narrative sections, Insight.claim, Decision - * .rationale, Option/Input/Output.description, captions, finding - * notes, …) flows through `myst-parser` so MySTRA stays MyST-native; - * the bespoke inline parser was retired. Output is `mdast` — the - * same node shape MyST themes consume directly. + * Every Markdown field on a component — `Insight.claim`, `Decision.rationale`, + * `Option/Input/Output.description`, captions, finding notes — flows through + * `myst-parser`, so MySTRA stays MyST-native and emits the same `mdast` themes + * consume. (This is *not* about the `narrative:` field, which Strategy A leaves + * to the author's Markdown page.) * - * Anchor links of the form `[text](#path.to.element)` use the ASTRA - * tree-path grammar described in the Narrative class (astra-spec - * v0.0.6, src/astra/schema/analysis.yaml). They are emitted by - * myst-parser as ordinary `link` nodes; `resolveNarrativeAnchors` - * walks the tree post-parse and rewrites in-scope anchors into MyST - * `crossReference` nodes pointing at the corresponding ASTRA - * element. Anchors that escape the host scope (`../` parent - * traversal) fall back to plain link nodes with the original URL. + * Anchor links `[text](#path.to.element)` use the ASTRA tree-path grammar; they + * arrive from myst-parser as ordinary `link` nodes, and `resolveNarrativeAnchors` + * rewrites in-scope ones into MyST `crossReference` nodes pointing at the + * matching element. Anchors that escape the host scope (`../` parent traversal) + * fall back to plain links with the original URL. */ import { mystParse } from 'myst-parser'; @@ -315,18 +312,8 @@ export function resolveAnchorPath( return rest.length === 1 && (analysis.outputs ?? []).some((o) => o.id === rest[0]) ? { identifier: `output-${rest[0]}` } : { url: `#${ref}` }; - // Narrative chunks: `#narrative.
` resolves to the - // chunk identifier published by render-narrative. - case 'narrative': - if ( - rest.length === 1 && - ['summary', 'findings', 'methods', 'inputs', 'outputs'].includes(rest[0]) && - analysis.narrative && - (analysis.narrative as any)[rest[0]] - ) { - return { identifier: `narrative-${rest[0]}` }; - } - return { url: `#${ref}` }; + // (`#narrative.
` is not resolved: Strategy A renders no narrative + // sections — the author writes that prose in the Markdown page itself.) default: return { url: `#${ref}` }; } diff --git a/src/transform/render-data-sources.ts b/src/transform/render-data-sources.ts index 2791a4a..14f3119 100644 --- a/src/transform/render-data-sources.ts +++ b/src/transform/render-data-sources.ts @@ -16,7 +16,7 @@ import { text, strong, } from './ast-helpers.js'; -import type { ProseParser } from './narrative-parser.js'; +import type { ProseParser } from './prose.js'; export function renderInputsTable(inputs: Input[], prose: ProseParser): any { // Caller filters out the empty case so the page doesn't render a diff --git a/src/transform/render-evidence.ts b/src/transform/render-evidence.ts index 073ec65..13703e3 100644 --- a/src/transform/render-evidence.ts +++ b/src/transform/render-evidence.ts @@ -38,7 +38,7 @@ import { } from './ast-helpers.js'; import { parse as parsePath } from 'node:path'; import type { ArtifactResolver } from '../loader.js'; -import type { ProseParser } from './narrative-parser.js'; +import type { ProseParser } from './prose.js'; import { parseTableData, formatValue } from './parse-table-data.js'; /** diff --git a/src/transform/render-findings.ts b/src/transform/render-findings.ts index a139229..f903790 100644 --- a/src/transform/render-findings.ts +++ b/src/transform/render-findings.ts @@ -20,7 +20,7 @@ import { emphasis, } from './ast-helpers.js'; import type { ArtifactResolver } from '../loader.js'; -import type { ProseParser } from './narrative-parser.js'; +import type { ProseParser } from './prose.js'; import { renderEvidenceBlock } from './render-evidence.js'; export function renderFinding( diff --git a/src/transform/render-methods.ts b/src/transform/render-methods.ts index 59efe4c..9d65213 100644 --- a/src/transform/render-methods.ts +++ b/src/transform/render-methods.ts @@ -23,7 +23,7 @@ import { tabSet, crossReference, } from './ast-helpers.js'; -import type { ProseParser } from './narrative-parser.js'; +import type { ProseParser } from './prose.js'; /** * tabItem factory bound to the current render pass. Created once per diff --git a/tests/narrative-parser.test.ts b/tests/prose.test.ts similarity index 94% rename from tests/narrative-parser.test.ts rename to tests/prose.test.ts index a2a9b64..49daf50 100644 --- a/tests/narrative-parser.test.ts +++ b/tests/prose.test.ts @@ -1,6 +1,6 @@ /** - * Tests for narrative-parser: Markdown → mdast and anchor → crossRef - * resolution per the v0.0.6 narrative grammar. + * Tests for the prose parser: Markdown → mdast and anchor → crossRef + * resolution per the ASTRA anchor grammar. */ import { describe, it, expect, vi } from 'vitest'; @@ -10,7 +10,7 @@ import { parseProseInline, resolveNarrativeAnchors, resolveAnchorPath, -} from '../src/transform/narrative-parser.js'; +} from '../src/transform/prose.js'; /** Minimal Analysis fixture with one finding, one decision, one * sub-analysis — enough to exercise every resolution branch. */ @@ -375,25 +375,6 @@ describe('resolveAnchorPath', () => { ).toEqual({ url: '/preprocessing#output-features' }); }); - it('resolves #narrative.
to the narrative chunk identifier', () => { - const withNarrative: Analysis = { - ...a, - narrative: { findings: 'Some findings prose.', summary: 'Hi.' }, - }; - expect( - resolveAnchorPath('#narrative.findings', withNarrative, 'index'), - ).toEqual({ identifier: 'narrative-findings' }); - expect( - resolveAnchorPath('#narrative.summary', withNarrative, 'index'), - ).toEqual({ identifier: 'narrative-summary' }); - }); - - it('falls back when #narrative.
targets an empty section', () => { - const onlySummary: Analysis = { ...a, narrative: { summary: 'Hi.' } }; - expect( - resolveAnchorPath('#narrative.findings', onlySummary, 'index'), - ).toEqual({ url: '#narrative.findings' }); - }); }); describe('resolveNarrativeAnchors', () => { From 62c3baa7b2c7a5256120c27c6c1ae0616e76f34f Mon Sep 17 00:00:00 2001 From: EiffL Date: Mon, 25 May 2026 21:37:12 +0200 Subject: [PATCH 03/17] docs: add followup.md (validation, citations, live-reload, deep-nesting scope) Captures the deferred work: SDK-based input validation, MyST-native citations/bibliography, astra.yaml live-reload, and deepening the page-scope derivation beyond one level. Each with problem / approach / code refs / risk. Co-Authored-By: Claude Opus 4.7 (1M context) --- followup.md | 206 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 followup.md diff --git a/followup.md b/followup.md new file mode 100644 index 0000000..cc2f34e --- /dev/null +++ b/followup.md @@ -0,0 +1,206 @@ +# MySTRA — follow-up work + +Deferred items from the Strategy-A refactor. Each is independent; none blocks the +current build (`book-theme` baseline works, tests green). Ordered roughly by +value-to-effort. + +--- + +## 1. Validate ASTRA input via `@astra-spec/sdk` + +### Problem +MySTRA assumes `astra.yaml` is well-formed. A malformed spec (missing required +field, dangling `from:` reference, unknown decision in a `when:` clause, a +narrative anchor pointing at a non-existent element) currently fails late and +opaquely — a directive throws and renders a red "ASTRA plugin" admonition, or a +cross-reference silently doesn't resolve, with no pointer to the real cause. + +### Why it matters +The SDK already ships the exact validators MySTRA would otherwise hand-roll, so +this is *additive reuse* — it surfaces bad input early with a precise message +instead of a downstream symptom. It's the natural next step after adopting the +SDK's types + helpers. + +### Proposed approach +Run the SDK validators once per project load (in `loadASTRASource`, +`src/loader.ts`) and report results through MyST's warning channel rather than +throwing. Three tiers, increasing cost: + +- **Semantic (sync, no network) — do first.** `validateAnalysis(data, {basePath})` + returns `SemanticError[]` (dangling `from:`, bad `when:` refs, alias rules, + …). Cheap and offline; wire it unconditionally and log each error. +- **Narrative (sync, no network).** `validateNarrativeAnchors` / + `checkNarrativeCoverage` (→ `NarrativeWarning[]`) / `validateNarrativeSections`. + Especially useful because MySTRA's whole value proposition is anchored prose; + this catches `[t](#decisions.typo)` before it renders as an unresolved link. +- **JSON Schema (async + network).** `validateAnalysisData/File` and + `validateUniverseData/File` are `Promise` and fetch the schema from + `astra-spec.org` on first use unless given `opts.schema`. Two frictions: + `loadASTRASource` is sync, and the network fetch is undesirable at build time. + Mitigate by (a) bundling/pinning the schema and passing it via `opts.schema` + (see `loadAstraSchema` / `setAstraSchema`), and (b) running schema validation + in an opt-in path (env flag or a small offline CLI), not on every build. + +### Relevant APIs & code +- SDK: `validateAnalysis`, `validateUniverse`, `validateNarrativeAnchors`, + `checkNarrativeCoverage`, `validateNarrativeSections`, `SemanticError`, + `NarrativeWarning`, `validateAnalysisData`, `loadAstraSchema`, `setAstraSchema`. +- Hook point: `loadASTRASource` in `src/loader.ts` (already loads the dict the + validators want; `validateAnalysis` accepts the parsed object). + +### Effort & risk +Low for the sync validators (a few lines + a reporting helper). Medium for +schema validation because of the sync/async + network constraints — keep it +opt-in. Risk: the SDK is `v0.0.x`; validator output format may change. Treat +results as advisory (warn, don't hard-fail the build) so a validator quirk can't +break rendering. + +--- + +## 2. Citations / bibliography via MyST + +### Problem +When the DOI subsystem was removed, evidence DOIs began rendering as plain +`https://doi.org/…` links (`formatCiteNode` in `src/transform/render-evidence.ts`). +There is no reference list, and inline citations show a bare DOI instead of an +author–year label. The prototype previously emitted `cite` nodes that produced +`Could not link citation` warnings because no bibliography backed them. + +### Why it matters +A scientific report needs real citations: author–year inline text and a linked +reference section. MyST resolves these natively *if* the project has a +bibliography — so the fix is to feed MyST what it expects rather than re-build a +DOI resolver. + +### Proposed approach +Lean entirely on MyST's citation machinery: + +1. **Collect** every evidence DOI across the loaded analysis tree (walk + `findings` + `prior_insights` → `evidence[].doi`). +2. **Emit a project bibliography** MyST can resolve — generate a + `references.bib` (or inject `project.bibliography` / a citation frontmatter + block) so MyST fetches DOI metadata and builds the reference list. Decide + during implementation whether the plugin writes a `.bib` file as a build side + effect or whether MyST's own DOI resolution can be triggered directly from + `cite` nodes carrying the DOI. +3. **Emit `cite` nodes** (not plain links) from `formatCiteNode` / + `renderInsightEvidence`, keyed by the DOI, so they link into the reference + list and render author–year text. +4. **Reference list** — let the theme/MyST render it; optionally place it via a + directive so the author controls *where* it appears (consistent with the + "author owns composition" principle). + +### Relevant APIs & code +- `src/transform/render-evidence.ts`: `formatCiteNode` (currently `link(...)`), + `renderLiteratureEvidence`, `renderInsightEvidence`. +- MyST: `references.bib` / `project.bibliography`, native DOI resolution, the + `cite`/`citeGroup` mdast nodes (already constructible via `ast-helpers`). +- The SDK does **not** do bibliography generation — this is MyST-side. + +### Effort & risk +Medium. The mechanics of how MyST best ingests DOIs (a generated `.bib` vs. +inline resolution) need a short spike against the stock pipeline. Risk: build-time +network fetches for DOI metadata (cache them); offline builds should degrade to +the current plain-link behaviour rather than fail. + +--- + +## 3. `astra.yaml` live-reload + +### Problem +`myst start` watches Markdown (and its own assets), **not** `astra.yaml` or the +files under `results/`. The plugin also caches the parsed project per +`(root, universe)` in a module-level map (`getSource` / `projectCache` in +`src/index.ts`). So editing the spec or regenerating a result does nothing until +the server is restarted (or a watched `.md` is touched), and even a rebuild would +hit the stale cache. + +### Why it matters +The authoring loop for the *data* (decisions, outputs, results) is currently +"edit → restart". For an agent or human iterating on `astra.yaml` alongside the +narrative, that's the main rough edge in the dev experience. + +### Proposed approach +Two independent pieces — cache freshness and rebuild triggering: + +- **Cache invalidation (necessary, easy).** Key `projectCache` on `astra.yaml`'s + mtime (and re-check on each `getSource`), or drop the cache and re-read per + build pass. Re-reading is simplest and almost certainly fast enough — the parse + is cheap and a build pass touches it a bounded number of times; measure before + optimising. This alone makes a manual rebuild pick up spec changes. +- **Rebuild triggering (the harder half).** `myst start` must be told to rebuild + when `astra.yaml` / `results/**` change. Options, simplest first: + 1. **Document "restart to pick up data changes"** (status quo; acceptable + short-term). + 2. A tiny **sidecar watcher** (chokidar) that `touch`es a watched `.md` (e.g. + the page whose scope changed) when `astra.yaml`/results change, nudging + MyST to rebuild. Crude but external to MyST. + 3. Investigate whether MyST exposes a **plugin/watch hook** to register extra + watched paths; if so, register `astra.yaml` and the resolved result dirs. + +### Relevant APIs & code +- `src/index.ts`: `projectCache`, `getSource(root, universe)` — the cache to + invalidate. +- Result paths to watch come from the deterministic convention in + `src/loader.ts` (`resolveArtifact`): `[/]results//`. + +### Effort & risk +Cache invalidation: trivial. Rebuild triggering: low–medium depending on which +option — start by removing the cache (or mtime-keying it) so a manual rebuild is +correct, then decide if the watcher/hook is worth it. Risk: a naive watcher can +cause rebuild loops; scope it to the specific files and debounce. + +--- + +## 4. Deep-nesting page scope + +### Problem +The two document transforms (`anchorTransform`, `storeTransform`) derive a +page's ASTRA scope from its **file basename only** — `scopeForFile(vfile)` in +`src/index.ts` maps `index` → root and `` → the `` sub-analysis, and +nothing else. So a page for a sub-analysis nested deeper than one level resolves +to the wrong scope or throws; the `try/catch` swallows the error, leaving that +page with **no `astra-store` carrier** (rich-theme joins silently get nothing) +and any author-written `[t](#…)` anchors unresolved. + +Note this only affects the *per-page transforms*. The block directives/roles +already accept arbitrary-depth paths (`a.b.id`), so an author can still place +deep components explicitly; it's the implicit page-scope inference that's +shallow. + +### Why it matters +Today every real reproduction (and the prototype) uses a flat, single-level page +layout (`reconstruction.md`, `clustering.md`), so this is latent — which is +exactly why tests/builds stay green. It becomes a real bug the first time a TOC +nests a sub-sub-analysis as its own page. + +### Proposed approach +Give `scopeForFile` a real page-file → analysis-path mapping. Options (pick the +one that fits how deep TOCs will actually be authored): + +- **Directory structure mirrors nesting** — derive the path from the page's + location relative to the project (`analyses/reconstruction/features.md` → + `['reconstruction','features']`). Natural if pages live under `analyses/…`, but + the current flat convention (`reconstruction.md` at root) wouldn't compose. +- **Dotted filename** — `reconstruction.features.md` → `['reconstruction', + 'features']`. Zero new config, composes to any depth, keeps flat files. +- **Explicit frontmatter** — `astra_scope: reconstruction.features` in the page. + Most explicit / least magic; decouples filename from scope. + +Whichever is chosen, it's a one-spot change: `scopeForFile` is already factored +out and shared by both transforms, so deepening the derivation fixes both at +once. Until then, document the single-level limitation (already noted in +`SPEC.md`). + +### Relevant APIs & code +- `src/index.ts`: `scopeForFile(vfile)` (the basename derivation), consumed by + `anchorTransform` and `storeTransform`; `resolveScope` already handles + arbitrary-depth `analysisPath` arrays, so only the *derivation* of that array + needs to change. + +### Effort & risk +Low–medium; the implementation is small once the convention is chosen — the real +decision is which page-file↔tree mapping to commit to. Risk: picking a +convention that later conflicts with how authors organise multi-page sites, so +prefer the explicit (frontmatter) or composable (dotted) option over inferring +from directory layout. From dc76d887100d8ffd08c88349ccfe132d74099e66 Mon Sep 17 00:00:00 2001 From: EiffL Date: Mon, 25 May 2026 22:35:28 +0200 Subject: [PATCH 04/17] docs: drop stale custom.css references after the prototype went CSS-free The prototype no longer wires a custom.css (it demonstrates the plugin-only baseline), so README/SPEC/plan no longer describe it as the "seed/preview" stylesheet; the rich styling is the lightcone-astra theme's job. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 5 ++--- SPEC.md | 4 ++-- STRATEGY-A-REFACTOR.md | 4 ++-- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 284e7e5..286c098 100644 --- a/README.md +++ b/README.md @@ -64,9 +64,8 @@ site: cards, and "powerful patterns" (e.g. a product-dependency graph) are appearance keyed on the `astra-*` classes the plugin emits, driven from the resolved store. The only change is the `template:` line. (The theme is a - separate deliverable; until it ships, `book-theme` is the baseline. The - prototype's `custom.css` is a reference stylesheet that previews the rich mode - on book-theme via `site.options.style`.) + separate deliverable; until it ships, `book-theme` is the baseline — the + `prototype/` runs plugin-only, no stylesheet.) ## Authoring vocabulary diff --git a/SPEC.md b/SPEC.md index 2ab050e..650d014 100644 --- a/SPEC.md +++ b/SPEC.md @@ -232,5 +232,5 @@ and an output is resolved only when actually rendered. - **Citation bibliography.** Emit a generated `references.bib` / project bibliography so MyST links the reference list. - **Packaging & the theme.** Publish the plugin (working name - `@lightcone/astra-myst`) and build the `lightcone-astra` theme (its stylesheet - seeded by `prototype/custom.css`). + `@lightcone/astra-myst`) and build the `lightcone-astra` theme that styles the + `astra-*` classes (glyphs, per-kind colour, hover cards from the store). diff --git a/STRATEGY-A-REFACTOR.md b/STRATEGY-A-REFACTOR.md index 45781a1..0397690 100644 --- a/STRATEGY-A-REFACTOR.md +++ b/STRATEGY-A-REFACTOR.md @@ -253,8 +253,8 @@ the `astra-*` classes/identifiers; reveals the inline preview cards (built from the store, not from hidden AST spans); renders the rich figure/decision/insight treatments and any author-placed patterns. Start as a **light** theme (base theme + bundled stylesheet) since block content already renders; graduate to custom -React renderers for true popovers/graphs. The prototype's `custom.css` is the -seed of its stylesheet. Until it exists, book-theme is the (clean) baseline. +React renderers for true popovers/graphs. Until it exists, book-theme is the +(clean) baseline — the prototype runs plugin-only, no stylesheet. ## 10. Suggested phasing From af2b999af3a28255e98b0fd33e2bbe7944e337c7 Mon Sep 17 00:00:00 2001 From: EiffL Date: Mon, 25 May 2026 22:35:28 +0200 Subject: [PATCH 05/17] docs(followup): how MyST renders reference popovers + theme extensibility MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Append an appendix: the engine-resolves/theme-renders model; citations as a key→references.cite.data join (cite.tsx) vs cross-references as an identifier→mdast-node resolve (crossReference.tsx); that a theme can add a cite-like mechanism via mergeRenderers + a context provider (in bounds, since it renders baked output). Corrects §2 — DOIs already resolve to citations natively — and records the insight-preview plan (hidden same-page xref targets for the baseline; a store-driven renderer for lightcone-astra). Sources cited. Co-Authored-By: Claude Opus 4.7 (1M context) --- followup.md | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/followup.md b/followup.md index cc2f34e..afbab95 100644 --- a/followup.md +++ b/followup.md @@ -204,3 +204,86 @@ decision is which page-file↔tree mapping to commit to. Risk: picking a convention that later conflicts with how authors organise multi-page sites, so prefer the explicit (frontmatter) or composable (dotted) option over inferring from directory layout. + +--- + +## Appendix: how MyST renders reference popovers (and what it means for insights) + +Findings from reading the build output + the `myst-to-react` source. Relevant to +§2 (citations) and to the future `lightcone-astra` theme. + +### The model: engine resolves, theme renders client-side + +MyST is engine + theme. The engine (`mystmd`) *resolves* references at build time +and writes static JSON (`content/.json`, `myst.xref.json`, …); the theme +(the book-theme React app) *renders* the popovers **client-side**. The server — +`myst start` or any static host — only serves those JSON files; no popover logic +is server-side. + +There are **two distinct popover mechanisms**, sharing only the generic +`HoverPopover` shell: + +**Citations — a key→table join, embedded per page.** The engine recognises a DOI +(even a bare `https://doi.org/…` link — see below), fetches its metadata, rewrites +the inline node to a `cite` node carrying a `label` (the DOI key) + author–year +text, and bakes the full reference into the page's `references.cite.data[label]` +(including a pre-rendered `html` string). `cite.tsx` then does +`useReferences()?.cite?.data[label]` and renders `}>` (the html via `dangerouslySetInnerHTML`). No fetch, no node +lookup — a local key join. + +**Cross-references — an identifier→node resolve.** `crossReference.tsx` resolves +the target *node* by identifier: for a remote page it fetches that page's mdast +(`createExternalUrl({url, remoteBaseUrl, dataUrl, baseurl})` + SWR); for a local +ref it uses `references?.article` (the current page's mdast). It then +`selectMdastNodes(tree, identifier, 3)` and renders the located node via `` inside `HoverPopover`. It never touches `references.cite`. + +### Correction to §2: DOIs already resolve to citations + +This revises the framing in §2 above. Because the engine auto-converts +`doi.org` links into `cite` nodes, the prototype build already contains 39 `cite` +nodes (author–year inline) and a populated `references.cite` table — so inline +citations **and their hover popovers already work** (given network at build to +fetch DOI metadata; offline it falls back to a link). The remaining work in §2 is +narrower than stated: mainly rendering a **reference list** (a `{bibliography}` +directive / placement) and offline-cache behaviour — not "wire up citations from +scratch." (The prototype README's "citations are plain DOI links" note is stale +and should be corrected.) + +### A theme can add a cite-like mechanism — and it's in bounds + +`cite.tsx` is **not** engine magic; it's a theme-layer component built from public +extension points, so a new theme can replicate the pattern: + +- **Renderers are an open, node-type-keyed map.** `DEFAULT_RENDERERS = + mergeRenderers([ … CITE_RENDERERS, CROSS_REFERENCE_RENDERERS, … ])`; a theme + uses `mergeRenderers` to add a renderer for a new node type or override an + existing one, then passes the map to ``. +- **Its data is a plain React context.** `article.tsx` defines `ArticleContext` / + `ArticleProvider({references})` / `useReferences()`. A theme can define its own + provider + hook for an arbitrary data table. + +The boundary a theme must respect: it renders the engine's **build output**; it +must not read source (`astra.yaml`) or invent content. Since the plugin already +bakes the **resolved store** into the build, a theme reading that store and +rendering popovers is doing exactly what `cite.tsx` does with `references` — fully +in bounds. + +### Implication for insight previews + +- **Baseline (book-theme, no theme):** emit the referenced insights as hidden, + **same-page** `crossReference` targets. `selectMdastNodes` searches the mdast + *tree* (`references.article`), not the live DOM — so a `display:none` target is + still found and rendered → citation-quality popover, no fetch, no visible + appendix. (This is the "hidden-targets" approach.) +- **Rich (`lightcone-astra`):** a dedicated, store-driven renderer is the 1:1 + `cite.tsx` analog — `AstraStoreProvider` + `useAstraStore()` + a custom + `NodeRenderer` (keyed on an astra node type/class) + `HoverPopover`, looking the + element up in the resolved store by id. Same shape as citations, richer card. + +### Sources +- `myst-to-react`: [`cite.tsx`](https://github.com/jupyter-book/myst-theme/blob/main/packages/myst-to-react/src/cite.tsx), + [`crossReference.tsx`](https://github.com/jupyter-book/myst-theme/blob/main/packages/myst-to-react/src/crossReference.tsx), + [`index.tsx`](https://github.com/jupyter-book/myst-theme/blob/main/packages/myst-to-react/src/index.tsx) +- `providers`: [`article.tsx`](https://github.com/jupyter-book/myst-theme/blob/main/packages/providers/src/article.tsx) From e5adc5938b0ce8cb8cdc5bcfd94c7231d945a082 Mon Sep 17 00:00:00 2001 From: EiffL Date: Sat, 30 May 2026 09:38:34 +0200 Subject: [PATCH 06/17] =?UTF-8?q?docs(followup):=20refocus=20=E2=80=94=20a?= =?UTF-8?q?dd=20=C2=A71=20hidden=20insight=20targets;=20drop=20=C2=A72=20c?= =?UTF-8?q?itations?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lead with the concrete next piece (hidden same-page crossReference targets for prior_insights, so the option-tab hovers keep working without the 32-block "Analysis record" appendix in index.md). Cross-references the Appendix for the display:none-works-because-mdast-not-DOM mechanism. Drop §2 citations entirely: the appendix already shows DOIs auto-resolve to cite nodes + references.cite in book-theme, so the residue (a reference-list directive, offline behaviour) is too narrow to live as its own follow-up — now a single sentence inside the citation explanation. Trim the rest. Co-Authored-By: Claude Opus 4.7 (1M context) --- followup.md | 379 +++++++++++++++++++++------------------------------- 1 file changed, 156 insertions(+), 223 deletions(-) diff --git a/followup.md b/followup.md index afbab95..231ac1d 100644 --- a/followup.md +++ b/followup.md @@ -1,154 +1,121 @@ # MySTRA — follow-up work -Deferred items from the Strategy-A refactor. Each is independent; none blocks the -current build (`book-theme` baseline works, tests green). Ordered roughly by -value-to-effort. +Deferred items from the Strategy-A refactor. Each is independent; none blocks +the current build (book-theme baseline works, tests green). Ordered roughly by +readiness. --- -## 1. Validate ASTRA input via `@astra-spec/sdk` +## 1. Hidden insight-resolution targets (next concrete piece) ### Problem -MySTRA assumes `astra.yaml` is well-formed. A malformed spec (missing required -field, dangling `from:` reference, unknown decision in a `when:` clause, a -narrative anchor pointing at a non-existent element) currently fails late and -opaquely — a directive throws and renders a red "ASTRA plugin" admonition, or a -cross-reference silently doesn't resolve, with no pointer to the real cause. - -### Why it matters -The SDK already ships the exact validators MySTRA would otherwise hand-roll, so -this is *additive reuse* — it surfaces bad input early with a precise message -instead of a downstream symptom. It's the natural next step after adopting the -SDK's types + helpers. - -### Proposed approach -Run the SDK validators once per project load (in `loadASTRASource`, -`src/loader.ts`) and report results through MyST's warning channel rather than -throwing. Three tiers, increasing cost: - -- **Semantic (sync, no network) — do first.** `validateAnalysis(data, {basePath})` - returns `SemanticError[]` (dangling `from:`, bad `when:` refs, alias rules, - …). Cheap and offline; wire it unconditionally and log each error. -- **Narrative (sync, no network).** `validateNarrativeAnchors` / - `checkNarrativeCoverage` (→ `NarrativeWarning[]`) / `validateNarrativeSections`. - Especially useful because MySTRA's whole value proposition is anchored prose; - this catches `[t](#decisions.typo)` before it renders as an unresolved link. -- **JSON Schema (async + network).** `validateAnalysisData/File` and - `validateUniverseData/File` are `Promise` and fetch the schema from - `astra-spec.org` on first use unless given `opts.schema`. Two frictions: - `loadASTRASource` is sync, and the network fetch is undesirable at build time. - Mitigate by (a) bundling/pinning the schema and passing it via `opts.schema` - (see `loadAstraSchema` / `setAstraSchema`), and (b) running schema validation - in an opt-in path (env flag or a small offline CLI), not on every build. +Decision option tabs (`render-methods.ts`) emit `crossReference`s to +`prior_insight-` so each option lists *"Supporting insights: …"* with a +hover popover on book-theme. The popover only resolves when a +`prior_insight-` **target node exists in the page mdast** — so today the +prototype's `index.md` explicitly places **32 `:::{astra:prior-insight}` blocks** +as an "Analysis record" appendix purely to make the option hovers work. The +appendix is heavy and duplicates content the prose already implies. + +### Approach +A new document-stage transform in `src/index.ts`, alongside `anchorTransform` +and `storeTransform`: + +1. Collect every `prior_insight-` identifier referenced by `crossReference` + nodes on the page (the option tabs the decision directive emitted). +2. Subtract ids the author **did** place explicitly via + `:::{astra:prior-insight}` (dedupe — duplicate identifiers are an error). +3. For each remaining id, render a compact carrier (claim · scope · quote · + citation), reusing the prior-insight render path already in + `render-evidence.ts` / the `priorInsightDirective`. +4. Append all carriers inside a hidden `