diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..db0da81 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,59 @@ +name: Documentation + +on: + push: + branches: [main] + paths: + - "docs/**" + - ".github/workflows/docs.yml" + pull_request: + paths: + - "docs/**" + - ".github/workflows/docs.yml" + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: pages-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + build: + runs-on: ubuntu-latest + defaults: + run: + working-directory: docs + steps: + - uses: actions/checkout@v4 + - uses: voidzero-dev/setup-vp@v1 + with: + working-directory: docs + node-version: "24" + cache: true + run-install: false + - run: vp install --frozen-lockfile + - run: vp check + - run: vp run docs:build + env: + VITEPRESS_BASE: /pyops/ + - uses: actions/upload-pages-artifact@v4 + if: github.event_name != 'pull_request' + with: + path: docs/.vitepress/dist + + deploy: + if: github.event_name != 'pull_request' + needs: build + runs-on: ubuntu-latest + permissions: + pages: write + id-token: write + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - name: Deploy GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 diff --git a/AGENTS.md b/AGENTS.md index a8f6270..30a351d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,8 +15,8 @@ This is a single repo with three cooperating parts: all work happens. It has its own `AGENTS.md`/`CLAUDE.md` (Vite+ toolchain notes). `app/src-tauri/` is the Tauri **desktop shell** that wraps this server in a native window and packages it into a self-contained bundle (vendored Node + bundled - resources) with self-update — see [`docs/desktop.md`](docs/desktop.md). -- **`mod/`** — the Factorio mod (`pyops`, Factorio 2.0): in-game panel, UDP link to + resources) with self-update — see [`docs/development/desktop.md`](docs/development/desktop.md). +- **`mod/`** — the Factorio mod (`pyops`, Factorio 2.1): in-game panel, UDP link to the app, data-dump trigger, Helmod-style production-block view, and the request-combinator planner. Pure Lua, no build step. - **`scripts/`** — dev-only helpers (currently `tunnel-dev`, to expose the dev @@ -38,15 +38,16 @@ dev, a per-OS user-data dir for a packaged build, overridable via `PYOPS_DATA_DI - `factory-solve.server.ts`, `cost-analysis.server.ts`, `effects.ts`, `additives.ts` — planning logic. - `agent.ts`, `agent-tools.server.ts` — the AI assistant (AI SDK v6 + OpenRouter, read-only tools + propose-then-apply writes: draft-a-block, draft-a-plan, - revise-a-block's rate). + and revise a block's rate or recipe set). - `bridge/` — UDP bridge to the mod. `protocol.ts` defines the wire contract; **`PROTOCOL_VERSION` must stay in lockstep with the same constant in `mod/control.lua`** (each side warns on mismatch). -- `solver/` — pure-TS sparse **linear system** per "block" (`block.ts`, - `linalg.ts`). A pinned output goal + `Match` items become equations; - `export`/`import` items are free boundary flows. Recipes and splits are - user-chosen, so there is no LP/optimizer. Handles Py's cyclic recipe chains and - reports fractional building counts. +- `solver/` — the HiGHS-backed linear-program model for each block (`lp.ts`), + constraint diagnosis (`diagnose.ts`), and composed sub-blocks (`subblock.ts`). + Goals, goods made inside the block, and recipe pins become constraints; other + goods remain free boundary flows. Recipes and splits are user-chosen rather than + globally optimized. The solver handles Py's cyclic chains and reports fractional + building counts. - `db/` — Drizzle ORM over `better-sqlite3`. `schema.ts` is the source of truth; `import-factorio.ts` loads the dump; `synthesize.ts` builds pass-2 synthetic recipes (mining/boiling/burning/spoiling/planting/rocket-launch, temperature @@ -90,10 +91,50 @@ build` packages a bundle (run `src-tauri/vendor-node.sh` first). Releases are automated by **release-please** (one product version) — **don't hand-edit the version** in `version.txt` / `app/package.json` / `Cargo.toml` / `tauri.conf.json` / `mod/info.json`; it bumps them all in lockstep from conventional commits. See - [`docs/desktop.md`](docs/desktop.md). + [`docs/development/desktop.md`](docs/development/desktop.md). If setup/runtime/package-manager behavior looks wrong, run `vp env doctor`. +## Documentation site (`docs/`) + +`docs/` is a separate Vite+ package that builds the public VitePress site. The +top-level sections are user-facing; subsystem internals live under +`docs/development/`. Keeping this package separate prevents VitePress's Vite +dependency from entering the desktop app's dependency graph. + +Run documentation commands from inside `docs/`: + +- `vp install` — install the pinned documentation toolchain. +- `vp check` — format, lint, and typecheck documentation sources and config. +- `vp run docs:dev` — start the local documentation server. +- `vp run docs:build` — build the static site and validate internal links. +- `vp run docs:preview` — serve the production build locally. + +The GitHub Pages build sets `VITEPRESS_BASE=/pyops/`; local development uses `/`. +The app and docs have independent lockfiles, so run the relevant package's +install/check commands after changing it. + +Documentation is a practical manual, not a marketing site: + +- Organize user pages around tasks and questions. Lead with the outcome, use the + exact visible UI labels, and state what the user should see after each workflow. +- Assume readers know Factorio production basics, but explain PyOps-specific + concepts and planner terminology when first introduced. +- Prefer current dark-mode screenshots from the real app for UI workflows. Use + localized names and representative project data; never expose API keys, local + paths, personal information, or other secrets in an image. +- Use generated raster illustrations selectively for confusing mental models or + workflows that the UI cannot show. Do not use generated images as fake UI, and + prefer Mermaid/SVG for simple deterministic diagrams. +- Keep screenshots purposeful and maintainable: capture stable states after the + prose is correct rather than illustrating every click. +- Keep published documentation evergreen. Describe the product and architecture + as they exist; do not include issue or PR numbers, commit references, + implementation chronology, migration anecdotes, conversational asides, or + notes about what older versions called something. Put history in GitHub and + the changelog. Mention an upgrade or compatibility distinction only when the + reader must act on it. + ## The Factorio mod (`mod/`) Pure Lua, edited in place — no build step. Key files: `control.lua` (panel + @@ -112,7 +153,7 @@ bridge + live-state sync), `summary.lua` (production-block view), `combinator.lu `.github/workflows/mod-test.yml` (manual — needs a Factorio binary). See `mod/tests/README.md`. Game-API logic (entities/blueprints/GUI) is still hands-on. -- Never assume Py mechanics or Factorio 2.0 API behavior — read the data dump or +- Never assume Py mechanics or Factorio 2.1 API behavior — read the data dump or the relevant Lua and cite the exact value/line. ## Conventions @@ -140,7 +181,7 @@ bridge + live-state sync), `summary.lua` (production-block view), `combinator.lu dynamic imports of server code trip import protection too. The only sanctioned dynamic imports: heavyweight optional deps (`sharp`), Tauri plugins in client code, and the bridge server↔handlers cycle guard. -- **UI work follows the design system** — [`docs/design.md`](docs/design.md): theme +- **UI work follows the design system** — [`docs/development/design.md`](docs/development/design.md): theme tokens (never raw palette colors), the `components/ui` primitives (never hand-rolled buttons/inputs/badges), square corners, `PageHeader`/`EmptyState`/ `Skeleton`, and loading/empty/error states on every async surface. @@ -157,9 +198,12 @@ bridge + live-state sync), `summary.lua` (production-block view), `combinator.lu change adds/renames/removes a user-facing surface (a page, tab, setting, install flow, env var, CLI command) or alters how a subsystem works, update the docs in the same pass: - - User-facing behavior, setup, config → [`README.md`](README.md). - - How a subsystem works → the matching file in [`docs/`](docs/) (architecture, - data-pipeline, solver, bridge, ai-assistant, design). + - User-facing behavior, setup, config → the matching guide in [`docs/`](docs/). + - Product pitch, download path, and contributor entry points → + [`README.md`](README.md). + - How a subsystem works → the matching file in + [`docs/development/`](docs/development/) (architecture, data-pipeline, solver, + bridge, ai-assistant, design). - A structural or workflow change (new top-level dir, build/verify step, convention) → this `AGENTS.md`. diff --git a/README.md b/README.md index 08bdc91..4830126 100644 --- a/README.md +++ b/README.md @@ -2,124 +2,100 @@ PyOps logo -A web-based factory planner and in-game ops assistant for **Factorio**, built for -the **Pyanodons (Py)** overhaul — like [YAFC](https://github.com/Yafc-CE/yafc-ce), -but in the browser, with deep in-game integration and an AI-assisted planner. It -runs locally alongside your Factorio install and reads recipe data straight from -the game; Py-specific views (like TURD) appear only when that data is present, but -it loads whatever mod set you sync. - -**Just want to run it?** PyOps ships as a self-updating **desktop app** for Linux, -macOS, and Windows — no toolchain needed. Grab a build from the -[Releases](https://github.com/ApocDev/pyops/releases) page (it still needs Factorio -installed locally to sync recipe data), or [run it from source](#run-it) to hack on -it. Build/release details: [`docs/desktop.md`](docs/desktop.md). - ---- - -## What it does - -- **Design production blocks** — set output goals + rates, pick recipes/machines/ - modules, and PyOps solves the run-rates and building counts for the whole chain - (cyclic loops, fluid temperatures, byproducts, spoilage). Pin counts, route - byproducts, fold chains into sub-blocks, or extract a recipe into its own block. -- **Balance the whole factory** — every block's imports/exports roll into one - ledger (deficits, surpluses, built-vs-required machines), with what-if. Supply - priorities let recovery blocks feed demand before dedicated fallback production. -- **Explore the data** — a searchable catalogue with a recipe explorer (producers/ - consumers ranked and availability-grouped) and a dependency-tree explorer. -- **Track TURD & research** — Py's tech upgrades are first-class; pick a path and - every block re-solves against your research horizon. -- **Plan with AI** — an OpenRouter-backed assistant drafts whole chains, honouring - what you can build now vs. after research, and can read the live factory. -- **Reach into the running game** — a companion mod links over localhost UDP: an - in-game block panel, locate, live sync of research/TURD/machines, and more. -- **Quality of life** — command palette (Ctrl+K), undo (Ctrl+Z), per-block - snapshots, backup/share, tasks & notes, light/dark theme, responsive to phone. - -Each subsystem has its own doc under [`docs/`](#documentation). - ---- +PyOps is a local factory planner and in-game operations assistant for +**Factorio**. It is designed around the **Pyanodons** mods, but works with vanilla +Factorio and other mod packs by synchronizing recipes, technologies, machines, +and icons from your own game. -## Screenshots +**[Read the PyOps documentation →](https://apocdev.github.io/pyops/)** -**Factory ledger** — every block's flows in one balance sheet; deficits rank by % -of demand met. -![Factory ledger — whole-factory balance with deficits, surpluses, and built-vs-required machines](docs/images/factory.png) +Installation, first project, planning workflows, game integration, reference, +and troubleshooting. -**Block editor** — goals in, solved rates and building counts out; toggle recipes/ -blocks off, fold into sub-blocks, or switch to a flow diagram. -![Block editor — the Basic substrate bio-chain, solved with byproducts](docs/images/block-editor.png) +**[Download the latest release →](https://github.com/ApocDev/pyops/releases)** -**AI assistant** — drafts a whole block from a goal, flagging byproducts, spoilage, -and TURD upgrades. -![AI assistant drafting a py science 1 production block](docs/images/assistant.png) +Self-updating desktop builds for Linux, macOS, and Windows. -**Browse** — every item, fluid, and recipe with produced-by / used-in, grouped by -availability and annotated with waste %. -![Browse — Iron plate with its producers and consumers](docs/images/browse.png) +## What PyOps does ---- +- **Design production blocks** — set output goals and rates, choose recipes and + machines, and solve full production chains including cycles, byproducts, + spoilage, and fluid temperatures. +- **Balance the factory** — combine every block's imports and exports in one + ledger, identify shortfalls and surpluses, and test changes with what-if plans. +- **Explore game data** — search items, fluids, recipes, producers, consumers, + and dependency trees from the mod set you actually use. +- **Plan around progression** — model research horizons and, when present, + Pyanodons TURD choices throughout the factory. +- **Connect to Factorio** — use the companion mod for live research, machine, + location, and production-plan integration. +- **Draft with the Assistant** — optionally use an OpenRouter-backed planning + assistant that understands the current project and can propose production + blocks for review. -## Run it +The [planning guide](https://apocdev.github.io/pyops/guide/) explains how these +parts fit into a complete workflow. -```bash -cd app -vp install # install dependencies (Node LTS + pnpm; Vite+ handles the rest) -vp dev # start PyOps at http://localhost:3000 -``` +## Screenshots + +**Factory ledger** — the balance across every production block, including +deficits, surpluses, and machine requirements. -Then open **⚙ Settings › Game data** and run a sync: PyOps launches Factorio -headlessly, reads its recipe data, and loads it into a local database (~1–2 min the -first time). Needs **Factorio 2.1** with the **Pyanodons** suite + -**pypostprocessing**. +![Factory ledger showing whole-factory balance](docs/images/factory.png) -- **Configuration** (env vars, remote access): [`docs/configuration.md`](docs/configuration.md) -- **In-game features** (companion mod, launching the bridge): [`docs/bridge.md`](docs/bridge.md) -- **AI assistant** needs an [OpenRouter](https://openrouter.ai) key (set it in - Settings or `OPENROUTER_API_KEY`). +**Block editor** — goals in, solved rates and building counts out. -The dev server also exposes the PyOps MCP tool surface at -`http://localhost:3000/mcp` (project configs for Codex and Claude Code ship in the -repo). +![Block editor showing a solved production chain](docs/images/block-editor.png) ---- +**Assistant** — project-aware help for investigating and drafting production +plans. -## Documentation +![Assistant drafting a production block](docs/images/assistant.png) -How PyOps works under the hood lives in [`docs/`](docs/): +## Developing PyOps -- [Architecture](docs/architecture.md) — the one-app-plus-mod model and repo layout. -- [Data pipeline](docs/data-pipeline.md) — how the Factorio data sync works. -- [Block solver](docs/solver.md) — the planning math. -- [Factorio bridge](docs/bridge.md) — the in-game link and its setup. -- [AI assistant](docs/ai-assistant.md) — the planning agent. -- [Configuration](docs/configuration.md) — environment variables and remote access. -- [Desktop app](docs/desktop.md) — how the Tauri bundle is built and released. +The repository contains three cooperating parts: -Contributing: `vp check` and `vp test` must be clean; the mod (`mod/`) is pure Lua, -no build step. See [`AGENTS.md`](AGENTS.md) for the full toolchain and conventions. +- `app/` — the TanStack Start application and Tauri desktop shell; +- `mod/` — the Factorio companion mod; +- `docs/` — the VitePress documentation site. ---- +Start the application from source with [Vite+](https://viteplus.dev/): -## Credits & inspiration +```bash +cd app +vp install +vp dev +``` + +Run `vp check` and `vp test` from `app/` before submitting application changes. +The companion mod is pure Lua and has no build step. + +For architecture, subsystem contracts, desktop packaging, and contributor +workflows, read the hosted +[development documentation](https://apocdev.github.io/pyops/development/). +Repository-specific agent conventions remain in [`AGENTS.md`](AGENTS.md). + +To work on the documentation site: + +```bash +cd docs +vp install +vp run docs:dev +``` -- **[YAFC](https://github.com/Yafc-CE/yafc-ce)** — the planner model, cost-analysis - approach, and the "design blocks, balance the factory" shape. -- **[Helmod](https://mods.factorio.com/mod/helmod)** — the in-game production-block - panel is heavily inspired by Helmod's; no Helmod assets are bundled. -- **[Factory Search](https://mods.factorio.com/mod/FactorySearch)** — the "locate in - game" feature relays to its remote interface. -- **[pypostprocessing](https://mods.factorio.com/mod/pypostprocessing)** — makes a - clean, planner-friendly data dump possible. +## Credits and inspiration ---- +- **[YAFC](https://github.com/Yafc-CE/yafc-ce)** — the planner model, + cost-analysis approach, and the design-blocks/balance-factory workflow. +- **[Helmod](https://mods.factorio.com/mod/helmod)** — inspiration for the + in-game production-block panel; no Helmod assets are bundled. +- **[Factory Search](https://mods.factorio.com/mod/FactorySearch)** — the locate + action can relay to its remote interface. +- **[pypostprocessing](https://mods.factorio.com/mod/pypostprocessing)** — + supplies additional planner-oriented metadata for Pyanodons data dumps. ## License -Free software under the **GNU General Public License v3.0** — see [`LICENSE`](LICENSE). -Copyright (C) 2026 ApocDev. You're free to use, study, modify, and share it -(including commercially), but any distributed version or derivative must stay open -under the same GPLv3 terms — matching [YAFC](https://github.com/Yafc-CE/yafc-ce) and -[Helmod](https://mods.factorio.com/mod/helmod). Contributions accepted under the same -license. +PyOps is free software under the **GNU General Public License v3.0**. See +[`LICENSE`](LICENSE). Copyright (C) 2026 ApocDev. diff --git a/app/src/components/context-menu.tsx b/app/src/components/context-menu.tsx index ae0ca31..22fddae 100644 --- a/app/src/components/context-menu.tsx +++ b/app/src/components/context-menu.tsx @@ -9,7 +9,7 @@ import { import { cn } from "#/lib/utils.ts"; /** - * Right-click context menu shell (docs/design.md): a Radix `DropdownMenu` + * Right-click context menu shell (docs/development/design.md): a Radix `DropdownMenu` * anchored at the pointer, so Escape-to-close, focus containment/roving, * `role="menu"` semantics, and click-away come from the primitive rather than a * hand-rolled backdrop (#86). Callers stay declarative — own the open state and diff --git a/app/src/components/empty-state.tsx b/app/src/components/empty-state.tsx index 983e6ee..ca50cb2 100644 --- a/app/src/components/empty-state.tsx +++ b/app/src/components/empty-state.tsx @@ -4,7 +4,7 @@ import type { LucideIcon } from "lucide-react"; import { cn } from "#/lib/utils.ts"; /** - * The one empty-state component (docs/design.md): say what's missing and, via + * The one empty-state component (docs/development/design.md): say what's missing and, via * `action`, how to fill it. Async surfaces must never render blank. */ export function EmptyState({ diff --git a/app/src/components/filter-empty-state.tsx b/app/src/components/filter-empty-state.tsx index e453e06..e7deb6a 100644 --- a/app/src/components/filter-empty-state.tsx +++ b/app/src/components/filter-empty-state.tsx @@ -2,7 +2,7 @@ import { Button } from "#/components/ui/button.tsx"; import { EmptyState } from "#/components/empty-state.tsx"; /** The standard "no matches for X" state of a filtered list (#87): names the - * query and offers to clear it (docs/design.md — a filtered-empty surface says + * query and offers to clear it (docs/development/design.md — a filtered-empty surface says * so and offers the fix). Render it when the unfiltered list is non-empty but * the filtered one is. */ export function FilterEmptyState({ diff --git a/app/src/components/page-header.tsx b/app/src/components/page-header.tsx index 8418a0b..9a00a51 100644 --- a/app/src/components/page-header.tsx +++ b/app/src/components/page-header.tsx @@ -3,12 +3,12 @@ import * as React from "react"; import { cn } from "#/lib/utils.ts"; /** - * The one page-title component (docs/design.md): every route renders exactly one + * The one page-title component (docs/development/design.md): every route renders exactly one * PageHeader so the h1 scale, description style, and action alignment never drift. * Toolbars (filters, sort controls) go in `children`, below the title row. * * The header is **sticky** to the top of its scroll container so the title and the - * toolbar that drives the content below stay reachable on long pages (docs/design.md + * toolbar that drives the content below stay reachable on long pages (docs/development/design.md * "Scroll model"). It sits on a solid `bg-background` above row hover/sticky-subheader * layers (`z-20`), with a bottom rule marking the stuck bar against the scrolling * content behind it. The negative `-mx-4/-mt-4` (paired with `px-4/pt-4`) let the bar diff --git a/app/src/components/query-boundary.tsx b/app/src/components/query-boundary.tsx index 6d65544..5a28484 100644 --- a/app/src/components/query-boundary.tsx +++ b/app/src/components/query-boundary.tsx @@ -13,7 +13,7 @@ type QueryLike = Pick, "data" | "isLoading" | "isError" | " /** * The one loading / error / empty convention for a data-bearing surface - * (docs/design.md "Interaction states"): render `` around a + * (docs/development/design.md "Interaction states"): render `` around a * `useQuery` and it shows a `Skeleton` while loading, a retryable * {@link QueryError} on failure, an `EmptyState` when the data is empty, and * finally `children(data)` — so pages stop hand-rolling the same four branches. diff --git a/app/src/components/query-error.tsx b/app/src/components/query-error.tsx index 251bf53..75287c5 100644 --- a/app/src/components/query-error.tsx +++ b/app/src/components/query-error.tsx @@ -5,7 +5,7 @@ import { Button } from "#/components/ui/button.tsx"; import { Callout } from "#/components/ui/callout.tsx"; /** - * The house inline error surface (docs/design.md "Interaction states"): a + * The house inline error surface (docs/development/design.md "Interaction states"): a * destructive `Callout` that says what failed and offers a retry, so pages stop * hand-rolling `{q.isError &&
}`. Pair * with {@link QueryBoundary} for the loading/empty/error trio, or drop in on its diff --git a/app/src/components/route-error.tsx b/app/src/components/route-error.tsx index 6c715a4..e5b364e 100644 --- a/app/src/components/route-error.tsx +++ b/app/src/components/route-error.tsx @@ -5,7 +5,7 @@ import { Button } from "#/components/ui/button.tsx"; import { EmptyState } from "#/components/empty-state.tsx"; /** - * The router's root `errorComponent` (docs/design.md "Interaction states"): any + * The router's root `errorComponent` (docs/development/design.md "Interaction states"): any * loader/render error thrown under a route lands here instead of a white screen. * "Retry" resets the error boundary and invalidates the router so the failed * loaders and queries re-run. For an in-body query failure use `QueryBoundary` / diff --git a/app/src/components/route-pending.tsx b/app/src/components/route-pending.tsx index d9b8fc8..e5f84e2 100644 --- a/app/src/components/route-pending.tsx +++ b/app/src/components/route-pending.tsx @@ -1,7 +1,7 @@ import { Skeleton } from "#/components/ui/skeleton.tsx"; /** - * The router's default `pendingComponent` (docs/design.md "Interaction states"): + * The router's default `pendingComponent` (docs/development/design.md "Interaction states"): * a neutral skeleton scaffold — a header bar plus a few content rows — shown * while a route with a loader resolves, so navigation never flashes a blank * pane. Pages with their own richer skeletons still render those once mounted. diff --git a/app/src/components/stat-table.tsx b/app/src/components/stat-table.tsx index ed17a83..340124a 100644 --- a/app/src/components/stat-table.tsx +++ b/app/src/components/stat-table.tsx @@ -3,7 +3,7 @@ import { type ReactNode } from "react"; import { cn } from "#/lib/utils.ts"; /** - * The desktop header row of the app's stat-table anatomy (see `docs/design.md`): + * The desktop header row of the app's stat-table anatomy (see `docs/development/design.md`): * a muted `hidden md:flex` line with a `flex-1` lead label over the rows' lead * cells and fixed-width right-aligned labels over their `StatCell` columns. * Each `cols[i].w` must match the `w` the rows pass to the corresponding diff --git a/app/src/components/ui/callout.tsx b/app/src/components/ui/callout.tsx index 16da1cc..0e563ab 100644 --- a/app/src/components/ui/callout.tsx +++ b/app/src/components/ui/callout.tsx @@ -4,7 +4,7 @@ import { AlertTriangle, CheckCircle2, Info, OctagonAlert, type LucideIcon } from import { cn } from "#/lib/utils.ts"; /** - * Block-level status message (docs/design.md): a tinted row with an icon and a + * Block-level status message (docs/development/design.md): a tinted row with an icon and a * short sentence — "game not connected", "already balanced", "infeasible". * `variant="box"` (default) is a standalone bordered panel; `variant="strip"` * is the full-bleed row used inside cards/drawers (borders only top/bottom diff --git a/app/src/components/ui/dialog.tsx b/app/src/components/ui/dialog.tsx index 2f0b3b6..87ef61f 100644 --- a/app/src/components/ui/dialog.tsx +++ b/app/src/components/ui/dialog.tsx @@ -11,7 +11,7 @@ import { cn } from "#/lib/utils.ts"; * the panel docks to the bottom edge as a sheet (full-width, thumb-reachable); * at `md+` it floats centered. Pick by role, not screen size — the responsive * switch is built in: Dialog for confirmations and focused edits, Sheet for - * side rails/drawers, CursorHover for hover detail (docs/design.md). + * side rails/drawers, CursorHover for hover detail (docs/development/design.md). */ function Dialog({ ...props }: React.ComponentProps) { diff --git a/app/src/components/ui/label.tsx b/app/src/components/ui/label.tsx index c15268f..4049c63 100644 --- a/app/src/components/ui/label.tsx +++ b/app/src/components/ui/label.tsx @@ -18,7 +18,7 @@ function Label({ className, ...props }: React.ComponentProps) { return ( diff --git a/app/src/components/ui/tooltip.tsx b/app/src/components/ui/tooltip.tsx index 1fcd40f..2f16923 100644 --- a/app/src/components/ui/tooltip.tsx +++ b/app/src/components/ui/tooltip.tsx @@ -4,7 +4,7 @@ import { Tooltip as TooltipPrimitive } from "radix-ui"; import { cn } from "#/lib/utils.ts"; /** - * Styled, keyboard-accessible tooltip (docs/design.md). Wraps a single + * Styled, keyboard-accessible tooltip (docs/development/design.md). Wraps a single * hoverable/focusable control and floats `content` beside it on hover **or** * keyboard focus, dismissable with Escape — the accessibility native `title` * never gave us, and themed to match the app instead of the OS bubble. diff --git a/app/src/db/import-factorio.test.ts b/app/src/db/import-factorio.test.ts index a5fb86c..3c58f18 100644 --- a/app/src/db/import-factorio.test.ts +++ b/app/src/db/import-factorio.test.ts @@ -49,6 +49,33 @@ describe("importFactorioDump — recipe categories", () => { }); }); +describe("importFactorioDump — TURD master detection", () => { + it("infers current planner masters from their sub-tech gate prerequisites", () => { + const db = runImport({ + technology: { + "dhilmos-upgrade": { prerequisites: ["dhilmos-mk04"] }, + "double-intake": { + prerequisites: ["dhilmos-upgrade", "turd-select-double-intake"], + }, + "turd-select-double-intake": { enabled: false }, + automation: { prerequisites: [] }, + "legacy-turd-master": { is_turd: true }, + }, + }); + const rows = db + .prepare(`SELECT name, is_turd isTurd FROM technologies ORDER BY name`) + .all() as { name: string; isTurd: number }[]; + expect(rows).toEqual([ + { name: "automation", isTurd: 0 }, + { name: "dhilmos-upgrade", isTurd: 1 }, + { name: "double-intake", isTurd: 0 }, + { name: "legacy-turd-master", isTurd: 1 }, + { name: "turd-select-double-intake", isTurd: 0 }, + ]); + db.close(); + }); +}); + // Fixture values are a real slice of the Py dump (data-raw-dump.json): // technology.microfilters grants change-recipe-productivity fawogae-spore +0.15 // and navens-spore +0.4; technology.mining-productivity-1 grants diff --git a/app/src/db/import-factorio.ts b/app/src/db/import-factorio.ts index 144e6c7..93beed2 100644 --- a/app/src/db/import-factorio.ts +++ b/app/src/db/import-factorio.ts @@ -89,6 +89,22 @@ const jsonList = (x: unknown): string | null => { const recipeCategory = (r: Record): string => arr(r.categories)[0] ?? (r.category as string | undefined) ?? "crafting"; +/** Planner dumps represent a TURD choice as a sub-tech gated by both its master + * and a synthetic `turd-select-` technology. Current pypostprocessing no + * longer adds its old nonstandard `is_turd` marker to the master prototype, so + * derive the masters from the stable prerequisite shape instead. */ +function inferredTurdMasters(technologies: Record): Set { + const masters = new Set(); + for (const [name, tech] of Object.entries(technologies)) { + const prerequisites = arr(tech?.prerequisites); + const gate = `turd-select-${name}`; + if (!prerequisites.includes(gate)) continue; + const candidates = prerequisites.filter((prerequisite) => prerequisite !== gate); + if (candidates.length === 1) masters.add(candidates[0]); + } + return masters; +} + const TABLES = [ "recipe_ingredients", "recipe_products", @@ -505,6 +521,7 @@ export function importFactorioDump( "inserter-stack-size-bonus": "inserter", "bulk-inserter-capacity-bonus": "bulk-inserter", }; + const turdMasters = inferredTurdMasters(raw.technology ?? {}); for (const [name, t] of Object.entries(raw.technology ?? {})) { ins.tech.run({ name, @@ -513,7 +530,8 @@ export function importFactorioDump( order: t.order ?? null, unit_count: t.unit?.count ?? null, enabled: t.enabled === false ? 0 : 1, - is_turd: t.is_turd ? 1 : 0, + // Honor the legacy dump marker while supporting current planner dumps. + is_turd: t.is_turd || turdMasters.has(name) ? 1 : 0, }); for (const pre of arr(t.prerequisites)) ins.techPrereq.run(name, pre); const stackAcc: Record = {}; diff --git a/app/src/db/schema.ts b/app/src/db/schema.ts index 3ebdbf2..4283ef6 100644 --- a/app/src/db/schema.ts +++ b/app/src/db/schema.ts @@ -308,9 +308,9 @@ export const technologies = sqliteTable("technologies", { order: text(), unitCount: real("unit_count"), // research cost multiplier (unit.count) enabled: bool("enabled").notNull().default(true), - // Pyanodon TURD: master techs are flagged is_turd; their selectable sub-techs - // (from the dump-time yafc integration) are normal techs whose prerequisites - // are [master, turd-select-] + // Pyanodon TURD: the importer marks master techs from each selectable sub-tech's + // planner prerequisites [master, turd-select-] (and honors the legacy + // dump is_turd marker when present). isTurd: bool("is_turd").notNull().default(false), }); diff --git a/app/src/design-system.test.ts b/app/src/design-system.test.ts index 616002a..9b4a549 100644 --- a/app/src/design-system.test.ts +++ b/app/src/design-system.test.ts @@ -4,7 +4,7 @@ import { join } from "node:path"; import { fileURLToPath } from "node:url"; /** - * Mechanical enforcement of docs/design.md (issue #103): raw palette classes, + * Mechanical enforcement of docs/development/design.md (issue #103): raw palette classes, * stray corner rounding, hex colors, and off-scale text sizes fail the suite * with a file:line list. If a violation is ever intentional, add it to the * matching EXCEPTIONS list with a comment saying why. @@ -42,7 +42,7 @@ function violations(pattern: RegExp, opts?: { tsxOnly?: boolean; inStrings?: boo return out; } -describe("design system (docs/design.md)", () => { +describe("design system (docs/development/design.md)", () => { it("uses semantic tokens, not raw Tailwind palette classes", () => { expect( violations( diff --git a/app/src/styles.css b/app/src/styles.css index 245a10e..d40b4d7 100644 --- a/app/src/styles.css +++ b/app/src/styles.css @@ -33,7 +33,7 @@ --accent-foreground: oklch(0.21 0.006 285.885); --destructive: oklch(0.577 0.245 27.325); --destructive-foreground: oklch(0.577 0.245 27.325); - /* Status hues — the app's semantic colors (see docs/design.md). + /* Status hues — the app's semantic colors (see docs/development/design.md). success = healthy/met/produced/live · warning = attention/consumed/import/behind info = neutral notice/stock/edit affordance · surplus = export/byproduct/positive net */ --success: oklch(0.596 0.145 163.225); @@ -49,7 +49,7 @@ --chart-4: oklch(0.508 0.118 165.612); --chart-5: oklch(0.432 0.095 166.913); /* Square corners app-wide — rounded/rounded-md/… all resolve to 0 through this. - rounded-full (status dots, spinners) is unaffected. See docs/design.md. */ + rounded-full (status dots, spinners) is unaffected. See docs/development/design.md. */ --radius: 0rem; --sidebar: oklch(0.985 0 0); --sidebar-foreground: oklch(0.141 0.005 285.823); diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 0000000..be703ae --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1,2 @@ +.vitepress/cache/ +.vitepress/dist/ diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts new file mode 100644 index 0000000..a62616a --- /dev/null +++ b/docs/.vitepress/config.mts @@ -0,0 +1,99 @@ +import { defineConfig } from "vitepress"; + +const base = process.env.VITEPRESS_BASE ?? "/"; + +export default defineConfig({ + title: "PyOps", + description: "Plan, understand, and operate your Factorio factory with PyOps.", + base, + head: [["link", { rel: "icon", type: "image/svg+xml", href: `${base}logo.svg` }]], + cleanUrls: true, + lastUpdated: true, + markdown: { + image: { lazyLoading: true }, + }, + themeConfig: { + logo: "/logo.svg", + nav: [ + { text: "User guide", link: "/getting-started/" }, + { text: "Development", link: "/development/" }, + { text: "Releases", link: "https://github.com/ApocDev/pyops/releases" }, + ], + sidebar: { + "/development/": [ + { + text: "Development", + items: [ + { text: "Overview", link: "/development/" }, + { text: "Architecture", link: "/development/architecture" }, + { text: "Data pipeline", link: "/development/data-pipeline" }, + { text: "Block solver", link: "/development/solver" }, + { text: "Factorio bridge", link: "/development/bridge" }, + { text: "AI assistant", link: "/development/ai-assistant" }, + { text: "Design system", link: "/development/design" }, + { text: "Desktop app and releases", link: "/development/desktop" }, + { text: "Advanced configuration", link: "/development/configuration" }, + ], + }, + ], + "/": [ + { + text: "Start here", + items: [ + { text: "Documentation home", link: "/" }, + { text: "Getting started", link: "/getting-started/" }, + { text: "Install PyOps", link: "/getting-started/install" }, + { text: "Choose a project", link: "/getting-started/project" }, + { text: "Sync game data", link: "/getting-started/sync-game-data" }, + { text: "Build your first block", link: "/getting-started/first-block" }, + { text: "Read the Factory view", link: "/getting-started/factory" }, + ], + }, + { + text: "Plan your factory", + items: [ + { text: "Planning guide", link: "/guide/" }, + { text: "Planning horizon", link: "/guide/planning-horizon" }, + { text: "Work with blocks", link: "/guide/blocks" }, + { text: "Block boundaries", link: "/guide/block-boundaries" }, + { text: "Balance the plan", link: "/guide/balance" }, + { text: "Explore recipes and dependencies", link: "/guide/explore" }, + { text: "Plan TURD upgrades", link: "/guide/turd" }, + { text: "Back up and share", link: "/guide/backups-and-sharing" }, + { text: "Connect Factorio", link: "/guide/in-game-link" }, + ], + }, + { + text: "Plan and operate", + items: [ + { text: "Use the Assistant", link: "/guide/assistant" }, + { text: "Track tasks and notes", link: "/guide/tasks-and-notes" }, + ], + }, + { + text: "Help and reference", + items: [ + { text: "Troubleshooting", link: "/troubleshooting/" }, + { text: "Keyboard and navigation", link: "/reference/keyboard-and-navigation" }, + { text: "Settings and storage", link: "/reference/settings-and-storage" }, + { text: "Advanced configuration", link: "/reference/advanced-configuration" }, + { text: "Concepts and limitations", link: "/reference/concepts-and-limitations" }, + { text: "Frequently asked questions", link: "/troubleshooting/faq" }, + ], + }, + ], + }, + search: { provider: "local" }, + outline: { level: [2, 3], label: "On this page" }, + docFooter: { prev: "Previous", next: "Next" }, + socialLinks: [{ icon: "github", link: "https://github.com/ApocDev/pyops" }], + editLink: { + pattern: "https://github.com/ApocDev/pyops/edit/main/docs/:path", + text: "Edit this page on GitHub", + }, + footer: { + message: "Released under the GNU GPL v3.0 license.", + copyright: "Copyright 2026 ApocDev", + }, + }, +}); diff --git a/docs/.vitepress/theme/AppScreenshot.vue b/docs/.vitepress/theme/AppScreenshot.vue new file mode 100644 index 0000000..1d38b4b --- /dev/null +++ b/docs/.vitepress/theme/AppScreenshot.vue @@ -0,0 +1,62 @@ + + + diff --git a/docs/.vitepress/theme/PlanningLoop.vue b/docs/.vitepress/theme/PlanningLoop.vue new file mode 100644 index 0000000..39f7d36 --- /dev/null +++ b/docs/.vitepress/theme/PlanningLoop.vue @@ -0,0 +1,24 @@ + diff --git a/docs/.vitepress/theme/index.ts b/docs/.vitepress/theme/index.ts new file mode 100644 index 0000000..faffbb2 --- /dev/null +++ b/docs/.vitepress/theme/index.ts @@ -0,0 +1,11 @@ +import DefaultTheme from "vitepress/theme"; +import type { Theme } from "vitepress"; +import AppScreenshot from "./AppScreenshot.vue"; +import "./styles.css"; + +export default { + extends: DefaultTheme, + enhanceApp({ app }) { + app.component("AppScreenshot", AppScreenshot); + }, +} satisfies Theme; diff --git a/docs/.vitepress/theme/styles.css b/docs/.vitepress/theme/styles.css new file mode 100644 index 0000000..83c5211 --- /dev/null +++ b/docs/.vitepress/theme/styles.css @@ -0,0 +1,299 @@ +:root { + --vp-c-brand-1: #d2842d; + --vp-c-brand-2: #b66e20; + --vp-c-brand-3: #985a18; + --vp-c-brand-soft: rgb(210 132 45 / 14%); + --vp-home-hero-name-color: var(--vp-c-brand-1); + --vp-home-hero-image-background-image: none; + --vp-home-hero-image-filter: none; + --vp-button-brand-border: var(--vp-c-brand-1); + --vp-button-brand-bg: var(--vp-c-brand-1); + --vp-button-brand-hover-border: var(--vp-c-brand-2); + --vp-button-brand-hover-bg: var(--vp-c-brand-2); +} + +.VPButton, +.VPFeature, +.VPMenu, +.VPNavBarSearch .DocSearch-Button, +.VPNavScreenMenuGroup, +.VPLocalSearchBox .shell, +.VPLocalSearchBox .search-bar input, +.vp-doc div[class*="language-"] { + border-radius: 0; +} + +.VPButton { + font-weight: 600; +} + +.VPFeature { + border-color: var(--vp-c-divider); + background: var(--vp-c-bg-soft); + box-shadow: none; +} + +a.VPFeature:hover { + border-color: var(--vp-c-brand-1); +} + +.VPHomeHero .name, +.VPHomeHero .text { + max-width: 760px; +} + +.VPHomeHero .text { + font-size: clamp(2rem, 5vw, 3.5rem); + letter-spacing: -0.04em; +} + +.VPHomeHero .tagline { + max-width: 720px; + font-size: 1rem; + line-height: 1.7; +} + +.VPHomeHero .image-src { + max-width: 180px; + max-height: 180px; +} + +.vp-doc { + line-height: 1.75; +} + +.app-screenshot { + margin: 24px 0; +} + +.app-screenshot__trigger { + position: relative; + display: block; + width: 100%; + padding: 0; + border: 0; + background: transparent; + color: inherit; + cursor: zoom-in; + text-align: left; +} + +.app-screenshot__trigger img { + display: block; + width: 100%; + border: 1px solid var(--vp-c-divider); + background: var(--vp-c-bg-alt); +} + +.app-screenshot__trigger:hover img, +.app-screenshot__trigger:focus-visible img { + border-color: var(--vp-c-brand-1); +} + +.app-screenshot__trigger:focus-visible { + outline: 2px solid var(--vp-c-brand-1); + outline-offset: 3px; +} + +.app-screenshot__expand { + position: absolute; + right: 8px; + bottom: 8px; + display: grid; + width: 28px; + height: 28px; + place-items: center; + border: 1px solid var(--vp-c-divider); + background: var(--vp-c-bg); + color: var(--vp-c-text-2); + font-size: 16px; + line-height: 1; +} + +.app-screenshot__trigger:hover .app-screenshot__expand, +.app-screenshot__trigger:focus-visible .app-screenshot__expand { + border-color: var(--vp-c-brand-1); + color: var(--vp-c-brand-1); +} + +.app-screenshot figcaption { + margin-top: 8px; + color: var(--vp-c-text-2); + font-size: 0.875rem; + line-height: 1.5; +} + +.app-screenshot--compact { + max-width: 720px; +} + +.app-screenshot--compact .app-screenshot__trigger { + width: auto; + max-width: 100%; +} + +.app-screenshot--compact img { + width: auto; + max-width: 100%; +} + +.app-screenshot-lightbox { + width: 100vw; + max-width: none; + height: 100vh; + max-height: none; + margin: 0; + padding: 20px; + border: 0; + background: transparent; + color: var(--vp-c-text-1); +} + +.app-screenshot-lightbox::backdrop { + background: rgb(0 0 0 / 84%); + backdrop-filter: blur(3px); +} + +.app-screenshot-lightbox__panel { + position: relative; + display: flex; + width: 100%; + height: 100%; + flex-direction: column; + align-items: center; + justify-content: center; + gap: 10px; +} + +.app-screenshot-lightbox__panel img { + display: block; + width: auto; + max-width: 100%; + height: auto; + max-height: calc(100vh - 92px); + border: 1px solid var(--vp-c-divider); + background: var(--vp-c-bg-alt); + object-fit: contain; +} + +.app-screenshot-lightbox__panel p { + max-width: 960px; + margin: 0; + color: rgb(255 255 255 / 75%); + font-size: 0.875rem; + line-height: 1.5; + text-align: center; +} + +.app-screenshot-lightbox__close { + position: absolute; + z-index: 1; + top: 0; + right: 0; + display: grid; + width: 36px; + height: 36px; + place-items: center; + border: 1px solid rgb(255 255 255 / 24%); + background: rgb(18 19 22 / 94%); + color: rgb(255 255 255 / 82%); + cursor: pointer; + font: inherit; + font-size: 24px; + line-height: 1; +} + +.app-screenshot-lightbox__close:hover, +.app-screenshot-lightbox__close:focus-visible { + border-color: var(--vp-c-brand-1); + color: var(--vp-c-brand-1); + outline: none; +} + +.planning-loop { + display: grid; + grid-template-columns: repeat(4, minmax(0, 1fr)); + gap: 20px; + margin: 24px 0; + padding: 0; + list-style: none; +} + +.planning-loop li { + position: relative; + display: flex; + min-height: 136px; + flex-direction: column; + gap: 8px; + justify-content: center; + padding: 16px; + border: 1px solid var(--vp-c-divider); + background: var(--vp-c-bg-soft); +} + +.planning-loop li:not(:last-child)::after { + position: absolute; + z-index: 1; + top: 50%; + right: -17px; + color: var(--vp-c-brand-1); + content: "→"; + font-size: 22px; + transform: translateY(-50%); +} + +.planning-loop strong { + color: var(--vp-c-text-1); + font-size: 1rem; +} + +.planning-loop li > span:last-child { + color: var(--vp-c-text-2); + font-size: 0.875rem; + line-height: 1.45; +} + +.planning-loop__number { + display: grid; + width: 28px; + height: 28px; + place-items: center; + background: var(--vp-c-brand-soft); + color: var(--vp-c-brand-1); + font-weight: 700; +} + +@media (max-width: 767px) { + .VPHomeHero .image { + display: none; + } + + .app-screenshot-lightbox { + padding: 8px; + } + + .app-screenshot-lightbox__close { + top: 4px; + right: 4px; + } + + .app-screenshot-lightbox__panel img { + max-height: calc(100vh - 72px); + } + + .planning-loop { + grid-template-columns: 1fr; + gap: 18px; + } + + .planning-loop li { + min-height: auto; + } + + .planning-loop li:not(:last-child)::after { + top: auto; + right: 50%; + bottom: -20px; + transform: translateX(50%) rotate(90deg); + } +} diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index da0022c..0000000 --- a/docs/README.md +++ /dev/null @@ -1,22 +0,0 @@ -# PyOps documentation - -Developer-facing docs: how PyOps is put together and why. For installing and using -PyOps, see the [top-level README](../README.md). - -- [Architecture](architecture.md) — the one-app-plus-mod model, the system diagram, - repository layout, and per-project databases. -- [Data pipeline](data-pipeline.md) — how PyOps dumps Factorio's prototype data and - turns it into a queryable SQLite store + icon atlas. -- [Block solver](solver.md) — the linear-system block solver and the factory-level - what-if LP. -- [Factorio bridge](bridge.md) — the localhost UDP link between the app and the - companion mod, and what flows across it. -- [AI assistant](ai-assistant.md) — the planning agent, its tools, and the MCP - surface. -- [Design system](design.md) — the web app's shared visual/interaction spec: - tokens, type scale, primitives, page anatomy, and required interaction states. -- [Desktop app & releases](desktop.md) — the Tauri shell, packaging into a - self-contained bundle, the release-please pipeline, and self-update. - -See also [`AGENTS.md`](../AGENTS.md) for the contributor/agent working guide -(toolchain commands, conventions, issue tracking). diff --git a/docs/ai-assistant.md b/docs/ai-assistant.md deleted file mode 100644 index 90ece45..0000000 --- a/docs/ai-assistant.md +++ /dev/null @@ -1,501 +0,0 @@ -# AI assistant - -Code: `app/src/server/agent.ts` (config + system prompt + model resolution), -`app/src/server/agent-tools.server.ts` (the tools), `app/src/routes/api.chat.ts` (the -streaming chat route), `app/src/server/assistant-run-store.ts` (active run -replay), and `app/src/routes/mcp.ts` (the MCP surface). - -A planning agent (Vercel AI SDK v6 → [OpenRouter](https://openrouter.ai), default -`~anthropic/claude-sonnet-latest`) drafts whole production chains over the Pyanodons -recipe data. - -## Conversations (persistence) - -Chats are saved **per-project** so you can leave one and resume it. The active -conversation lives in the URL (`/assistant?c=`), so chats are linkable. - -**Run continuity:** the live AI-SDK `Chat` instances live in an app-level store -(`app/src/lib/chat-store.ts`) **outside the React route tree**, one per -conversation. The backend also owns each run: `api.chat.ts` saves submitted -messages immediately, streams through a process-local replay buffer, and saves -the finished assistant message. Because the server keeps reading the stream after -a browser disconnect, runs keep going across in-app navigation and browser -reloads while the app process stays up. Reload recovery uses AI SDK `resume: -true` and `GET /api/chat?stream=`; restarting `vp dev` clears -only the in-memory active-run buffer, not saved conversations. - -In-progress runs are surfaced by the nav count, a pulsing dot next to each -running chat in the sidebar (resynced from the server), and an "Assistant is -working…" line in the chat itself. The Stop button is explicit cancellation: it -posts the latest partial assistant message to `/api/chat`, aborts the server-side -run, then stops the local stream. Route cleanup, tab close, and reload are treated -as disconnects, not cancellation. - -A toolbar sits inside the message input box (below the textarea), holding the -context gauge (see Context compaction), a **model** pill, and a **reasoning** -pill, with the send button on the right. The model pill opens a popover with the -curated list + a custom-id field + make-default / clear; the reasoning pill opens -an Auto / Low / Medium / High menu (levels grey out on models that don't support -reasoning effort). When -`PYOPS_AGENT_MODEL` is unset, each conversation can override the app default with -a free-text OpenRouter model id or one of the curated choices. The override is -stored on the conversation, so branches can diverge by model. If -`PYOPS_AGENT_MODEL` is set, it remains a hard deployment override and the -per-chat picker is read-only. - -Each conversation also stores an optional OpenRouter reasoning effort: model -default, low, medium, or high. The picker is enabled when the resolved model -advertises OpenRouter's `reasoning` parameter — detected live from the -`/api/v1/models` catalogue (`server/openrouter-models.ts`), with the static -`model-capabilities.ts` table as the offline fallback. When set, `api.chat.ts` -sends it as -`providerOptions.openrouter.reasoning.effort` with `exclude: false`; unsupported -or custom models stay on provider/model defaults and receive no reasoning effort -parameter. The transcript renders streamed reasoning parts as collapsed blocks. -Title generation does not display reasoning and asks known reasoning models for -low, excluded reasoning so they have budget to emit the short title without -wasting visible transcript space. - -Message controls in the transcript support editing/resending a user message, -retrying an assistant answer, and branching a new conversation from any message. -Editing an earlier user message replaces it and retries from that point; branching -copies the transcript prefix into a new saved conversation and opens it. - -## Context compaction - -Compaction is anchored to **real token counts**, not a guess. Each completed turn -records OpenRouter's actual usage — the last tool-loop step's input+output (that -request carries the whole conversation, so it's the true context fill; the -aggregate `usage` sums every step's prompt and over-counts) — plus the concrete -model that served it, onto the conversation row (`last_*_tokens`, `last_model_id`). -Context windows come live from OpenRouter's `/api/v1/models` catalogue -(`server/openrouter-models.ts`, cached 6h, `~…-latest` aliases resolved to the -newest concrete model in the family); `lib/model-capabilities.ts` is only the -offline fallback. Before each turn, `api.chat.ts` compares the last real count -against the model's real window and falls back to a chars/4 estimate only when no -turn has completed yet — so we don't summarize (and shed detail) until the context -is genuinely close to full. - -When usage reaches 75% of the window, the oldest prefix is summarized into one -synthetic `system` message and the newest turns stay verbatim. The summary is -generated with the selected model, using low excluded reasoning when the model -supports reasoning effort. If summarization fails or there is no API key, PyOps -falls back to a local extractive summary rather than dropping context. - -The compacted transcript is saved back to the project database and used as the -AI-SDK `originalMessages` for the in-flight response, so browser reloads and the -finished assistant message both keep the compacted form. The summary message also -stores the replaced originals in an ignored `data-compaction` part; the UI renders -that as an "Earlier conversation summarized" block with a nested original-message -viewer. Only the text summary is sent back to the model. - -**Context gauge.** The input toolbar shows a filling ring with the percent of the -context window used (green → amber → red), backed by `conversationTokenStatusFn`. -Clicking it force-compacts the conversation now (`compactConversationFn` → -`compactMessagesForContext(..., { force: true })`); the live client chat swaps to -the returned messages and the stale real-token count is cleared so the gauge -reflects the smaller compacted size until the next turn measures it for real. - -Code: `db/conversations.server.ts` (queries, usage columns + `recordTurnUsage`, and an -idempotent `CREATE TABLE IF NOT EXISTS` / `ALTER TABLE` so existing project dbs -gain the tables/columns without a manual push), `server/conversations.ts` (server -fns incl. `generateTitleFn`, `conversationTokenStatusFn`, `compactConversationFn`, -active-run resync), `server/openrouter-models.ts` (live context windows + reasoning -support), `server/conversation-compaction.ts` (trigger + summarization), -`server/assistant-run-store.ts`, the client store (`compactChat`), and the gauge + -input-bar model/reasoning pills in `routes/assistant.tsx`. Message `parts` are stored as JSON -strings. - -## Tools - -The agent's tools are **read-only** wrappers over the query layer, plus a few -propose-then-apply write actions. They're the minimum surface needed to reason -about "how do I make X": - -- **Fuzzy name resolution** — map a loose item/recipe name to a stable internal - handle. -- **Recipe-candidate ranking** — find and rank the recipes that produce a good. - Each candidate's `machine` names the building a draft would ACTUALLY solve - with: the user's stored category favorite (`q.getFavoriteMachines`, the - building-picker star), else the same low-tier `pickDefaultMachine` fallback - `computeBlock`/`recipeDefaultsFn` use — never just "the fastest" (#130). Its - availability note ("needs ``") is judged against that resolved machine, - since that's what actually gates the draft. `fastestMachine` is surfaced - separately, and only when a tier exists that's **strictly faster** - (`craftingSpeed` greater, not just a different name) than the resolved pick — - a same-speed tier tie must not be misreported as an upgrade — so the - tier ladder stays visible without misattributing the availability gate. When - the research horizon isn't `future`, the favorite/fallback pool itself is - first restricted to machines the horizon already reaches (`availableNow`) — - the same restriction `computeBlock`/`recipeDefaultsFn` apply — so a locked - favorite falls through to the unlocked fallback instead of naming an - unbuildable machine (#130). -- **TURD choices** (`turdChoices`) — the full mutually-exclusive branch set of a - TURD master (looked up by master, recipe, or good): each branch's description, - the recipes it swaps (old→new) or newly **unlocks**, and its always-on modules. - It walks the tech-prerequisite graph, so unlike `availableTurds`/`turdConsistency` - (which key off recipe _replacements_) it also sees branches that grant a brand-new - recipe. `recipeInfo.turd` returns the same full detail for every master touching a - recipe. This is what the agent consults for "what does this TURD give / which - branch is best" — never assume a master has a single choice. -- **Research path** (`researchPath`, `app/src/db/queries.server.ts` `researchPath`/ - `orderTechSteps`/`rankUnlockTechs`) — given a target (a technology, a recipe, or - an item/fluid good — resolved in that priority), returns the **not-yet-researched** - prerequisite closure in **dependency order** (prerequisites first, the tech that - actually unlocks the target last), each step with its own science-pack cost, plus - `totalCost` summed across the WHOLE path. It always reads the REAL - researched-tech state synced from the connected save (or marked manually in - Settings) — independent of the current planning-horizon mode, which governs - recipe _availability_, not what's already done. `alreadyUnlocked` means the - target is already covered: either start-enabled (the static column), OR - already covered by that synced researched-tech state — e.g. a recipe whose - `enabled` column hasn't caught up, or a technology target itself already - researched — checked up front, not only inside the step-ordering walk, so a - target you already have never comes back with a nonsensical zero-step - "route". A recipe/good reachable by more than one tech reports the - cheapest (lowest-tier) route as `targetTech` and the others as `alternateRoutes` - (name only — call the tool again with one of those to expand it). Pyanodons' - `turd-select-*` gate technologies (a TURD branch pick, not a science action — - verified zero cost, zero prerequisites of their own) are excluded from the - step list and instead surfaced in `turdGatesNeeded` when the branch isn't - already selected (`pickable` = master undecided; `blocked` = a different - branch is already chosen, needing a respec) — same non-committal framing as - `availableTurds`/`turdConsistency`. The tool's `limit` (default 40) bounds - the `steps` list to the entries closest to the target, dropping the earliest/ - most-foundational ones first and reporting the drop count in `stepsOmitted` - — `totalCost` always sums the whole path regardless, so the reported total - stays correct even when a deep, mostly-unresearched target's closure runs - into the hundreds of techs. This is the natural companion to - TARGET-mode plans: state the research route ("research `electronics` → - `battery-mk01`, ~40 `py-science-pack-1` total") instead of just naming gating - packs. -- **Factory-wide coherence audit** (`coherenceAudit`, - `app/src/server/coherence-audit.server.ts`, #11) — the cross-block balance in - one call, reusing the Coherence page's wiring query: under-supplied goods - (each with its producer blocks' ids + rates, so the agent can propose - `reviseBlock` resizes), overproduced links, imports no block produces - (with a craftable flag), and dangling byproducts. Each dangling byproduct - carries a **disposal verdict**: `route` (productive consuming recipes exist), - `void` (only a vent/void/incinerate disposal recipe), or `nowhere` - (store/buffer — an open problem). The void classifier is data-driven (a - recipe that consumes only the good and returns at most a fraction of it), so - it matches Py's `*-pyvoid*` venting/sinkhole/incineration families without - name-matching. `byproductSinks` uses the same classifier to list - `voidOptions` separately from real consumers. -- **Factory-wide demand-override simulation** (`whatIf`, - `app/src/server/factory-solve.server.ts`'s `factoryWhatIf`, #127) — the - same LP the What-if page (`routes/whatif.tsx`) uses, exposed as a tool: - given one or more `{good, rate}` overrides, it treats every block as a - fixed-ratio super-recipe and solves for the scale factor each needs, - reporting `blocksToResize` (every block whose rate changes, INCLUDING the - ripple through upstream feeder blocks, each entry's `blockId`+`rate` ready - to pass straight into `reviseBlock`/a `submitPlan` `updates` entry), - `rawsNeeded` (external draw current vs. projected), and `overproduced` - (byproduct surplus the ripple creates, with an `absorb` hint naming an - existing sink block when one exists). Because the underlying solver only - honors an override on a good classified as a `demand` (a primary output - nothing else consumes — see the module's `classify` helper), overriding a - good another block still consumes is a silent no-op in the LP; the tool - detects this (the good is absent from the solved `demands` list) and - surfaces it in `ignoredOverrides` rather than returning a misleadingly - unchanged result. Report-only, like the page it mirrors — it never writes; - the agent still has to call `reviseBlock`/`submitPlan` to apply anything. -- **Additive/commodity classifier** (`app/src/server/additives.ts`) — decides - whether an input should be _imported_ (a cross-cutting commodity like an acid, - gas, or solvent — stop recursing) or _built_ (part of the target's own lineage — - recurse into a sub-chain). The signal is fan-out ubiquity: in Py, commodities sit - at 10s–100s of consumers while private intermediates sit at 1–2, so a simple - threshold plus a short override list classifies the common case. Per-block user - pins override it. -- **Draft-a-block** — assemble the reasoning into a reviewable single-block draft. -- **Multi-goal + keep-in-stock goals** (#38) — `blockDraftInput` (shared by - `submitBlock`'s and `submitPlan`'s blocks) accepts either the legacy single - `target`+`rate` shorthand or a `goals` array of `{ name, rate? , stock?, -window? }`: each goal is EITHER a throughput `rate` OR a keep-in-stock - `stock` amount (+ optional refill `window` seconds, default 600 — - `STOCK_WINDOW_DEFAULT` in `lib/goals.ts`). A stock goal's solver rate is - DERIVED as `stock/window` (`resolveGoals` in `agent-tools.server.ts`) — a - buffer-refill rate, never a fabricated continuous one — which is the correct - primitive for a construction/mall block ("keep 80 vrauks-paddock on hand") - that has no honest per-second production rate. `goals[0]` (or the legacy - `target`) still anchors the block's naming/sizing and the draft's - `target`/`rate`/`targetDisplay` fields, so existing UI/back-compat code that - only reads those keeps working; the draft's new `goals` array is what the - apply path (`routes/assistant.tsx`) actually persists into `BlockData.goals`, - stock/window included. `buildingBillBlockInput` accepts the same `goals` - shape, so a mall block's several stock goals are all reflected in its machine - bill too. `reviseBlockInput` stays rate/recipes-only by design (#38 scoped - down): `rate` re-rates only the block's anchor goal, and `buildBlockUpdate` - preserves the rest of its stored `goals` — including any stock ones — via - `withPrimaryRate` (`lib/goals.ts`) rather than collapsing the block to a - single goal. -- **Revise-a-block** — propose changing an _existing_ block: RAISE/LOWER its - output rate to meet new demand, and/or REPLACE its recipe set (#12 — e.g. - swap to a higher-yield variant), instead of building a duplicate. A recipe - revision re-solves the new set and returns the diff (recipes added/removed) - plus any byproducts the block's current solve doesn't export - (`newByproducts`), so closure damage is visible before the user applies. -- **Draft-a-plan** — assemble several solved block drafts for one request (and, - optionally, resizes of existing blocks), then let the user apply all of them in - one action. When a block is a building/mall-supply block, the system prompt - steers the agent to give it keep-in-stock goals (seeded from `buildingBill`'s - machine `count`) instead of a fabricated rate, and to recurse into each - machine's own intermediates as further blocks/plan entries. -- **Solved building counts, module-filled and shared** — `submitBlock`/ - `reviseBlock`/`submitPlan` no longer discard the machine counts `computeBlock` - already solves: every draft carries a `buildings` field, `{ recipe, machine, -count }[]` (fractional, module/TURD-beacon effects folded in — the same - counts the block editor shows). The two-pass solve that fills each row's - module slots with `computeBlock`'s own suggestions (adopt → re-solve) is - factored into one shared helper, `solveWithModuleFill` — used by BOTH the - block-draft path AND `buildingBill`, so the two tools' counts always AGREE - for the same recipes. Before this fix `buildingBill` ran a bare - `computeBlock({ goals, recipes })` with no module pass, which for Py's - creature/farm buildings (intentionally near-useless unmoduled — vrauks - paddocks, tree farms) inflated its reported count by roughly 10-15× versus - the same block's `submitBlock` draft. **`buildingBill`** is the cross-block - machine BILL for "include the buildings needed to build this": given the - same `{ target, rate, recipes }[]` shape as `submitPlan`'s blocks (or the - `goals` array, see above), it solves each independently with the shared - module fill (a failing block is skipped into `skipped`, not a hard error), - CEILS each block's per-recipe machine count to a whole building, then sums by - machine entity across every block. Each machine entity is mapped to the ITEM - that places it (`q.getItem(entity)`; Factorio's convention is entity name == - item name — there's no separate `place_result` column in the schema, so - `item` comes back `null` with a note if no matching item prototype exists) - and given its top 1–2 producing recipes (`optionsFor`, the same shape - `recipeOptions` uses). Belts/inserters/logistics are deliberately out of - scope — machine items only. The agent is told to call this once a plan's - blocks are chosen, then decide per machine item whether an existing mall - block supplies it, an existing block should be resized (`reviseBlock`/plan - `updates`), or it needs its own new block, sized with a keep-in-stock goal. -- **Factory-wide power rollup** (`factoryPower`, #129) — total electric DEMAND - vs GENERATION across every enabled block, in one call. Demand reuses each - block's cached `electricityW` (the same figure the Factory page's header - totals — no re-solve). Generation reads the `pyops-electricity` pseudo-good's - PRODUCER-end flow (`block_flows` role `primary`/`stock`/`byproduct`, never - `import`) already recorded from each block's last real solve — a block whose - recipes include a `kind: "generating"` recipe (turbine, generator, solar - panel, burner-generator, …) nets a positive export there, so a "power block" - is identified with no new convention. Returns `totalDemandW`/ - `totalGenerationW`/`netW` plus `topConsumers` (top `limit` blocks by draw) - and `generators` (every net-producing block). A block can appear on BOTH - lists (e.g. a reactor block that also draws power for its own auxiliary - machines) — demand and generation are computed independently per block, so - only the two TOTALS should be compared, never a per-block net. Heat is - intentionally NOT rolled up here — it's a short-range (~15 tile), block-local - mechanic (Py hard mode); read a block's `heatW` from its own solve - (`submitBlock`/`reviseBlock`) instead. -- **Belts/inserters for one good** (`logisticsFor`, #126) — the logistics half - `buildingBill` deliberately leaves out: given `{ good, rate }`, for an ITEM it - returns every belt tier UNLOCKED under the research horizon (the same - `unlockedItems` gating `availableMachines`/module auto-fill use — belt/loader/ - inserter entities are themselves crafted items) with its whole belt count and - saturation (how full the built belts run, so "can one yellow belt feed this?" - reads straight off the first row), and every unlocked inserter/loader with the - whole-device count to move the rate through one feed point. Stack sizes - reflect the researched belt/inserter/bulk-inserter bonuses (`tech_stack_bonuses`) - via the same `placedBeltStack`/`inserterHandStack` math the block editor's - per-row logistics readout uses (#21, `lib/logistics.ts`) — evaluated across - every unlocked tier instead of the user's one selected pick (unlike the - editor's manual belt/mover picker, which is intentionally unfiltered). A FLUID - short-circuits to `{ kind: 'fluid', note }` — pipe throughput isn't modelled. - Pair with `buildingBill` for full construction coverage: machines from - `buildingBill`, belts/inserters/loaders from this. -- **Built-vs-required status** (`blockBuildStatus`, #123) — audits blocks that - already exist, from the last synced game state: per block, per recipe, the - machine, the required WHOLE-building count (ceiled from `block_machines`' - cached solved count — the same source `buildings` reports), the built count - from `built_machines`, and the missing delta. Works entirely offline (no - bridge round-trip, no re-solve). While the bridge is connected, build, mine, - death, and recipe changes refresh the snapshot within about one second without - opening the in-game panel; the tool still returns `syncedAt`/`syncedCount` so - the agent can flag stale offline data. Pass a - `blockId` (a `factoryBlocks` id) for one block's full breakdown, even fully - built or disabled (`limit` is ignored in this mode); omit it to list up to - `limit` (1–30, default 10) **enabled** blocks with a shortfall, worst-missing - first, matching every other factory-wide rollup's enabled-only convention - and bounding the same nested `recipes`/`machineFallback` payload that made - an unlimited listing unbounded on a factory with many under-built blocks. - Built counts are force-wide (`built_machines` has - no block association), so two blocks sharing the identical machine+recipe - each compare independently against the same built count. Machines whose - entity type never reports a recipe to the game — boilers, generators, - reactors, offshore-pumps (`mod/control.lua`'s `RECIPE_TYPES`, e.g. a - `generate-heat-*` local heat source) — come back with `built`/`missing` - null on their `recipes` rows and are instead summarized once per machine in - `machineFallback`, mirroring `machineSufficiency`'s existing - recipe-aware/machine-total fallback (`queries.server.ts`) rather than - silently misreporting them as permanently missing. The system prompt steers - the agent here instead of `gameEval`/`gameProduction` for "what's built" - questions, since this tool works even when the bridge is disconnected. -- **Tasks & notes** — `listTasks`/`getTask` read the user's planning to-do tree; - `createTask` files one (with optional checklist steps and entity links); - `updateTask`/`addTaskStep`/`linkTask` edit it. Unlike block drafts, these apply - **directly** (low-stakes, reversible on the Tasks page), so the agent can file a - follow-up after drafting — it's told to do so when the user agrees or asks what's - left, checking `listTasks` first to avoid duplicates. A separate **Enhance** - action on a task (`enrichTaskFn`, not an agent tool) rewrites a rough capture's - title/body into something sharper while preserving the original intent. - `listNotes` (#128) is a **read-only** sibling over the separate `notes` table - (`db/tasks.server.ts`'s `listNotes()`) — a flat, deliberately-dumb scratch - surface (title + freeform body, no steps, no tree) the user writes for - themselves. It returns every note's `{ id, title, body }` in one call (the - table is small — no pagination). Writing/editing notes is out of scope for the - agent: Tasks already cover assistant-initiated follow-ups, and notes stay the - user's own space. -- **Synced production stats** (`productionStats`) — batched actual produced/consumed - per good (items or fluid /s, force-wide) read from the `production_stats` table - (`db/queries.server.ts` `productionStatsFor`/`getProductionStats`/ - `setProductionStats`), which the mod keeps as a full-replace snapshot via - `state.stats` (`server/bridge/handlers/stats.ts`) — pushed periodically while - playing and refreshed on every save-load resync. Works with the game closed, - unlike the live tools below. Because the snapshot is a full replace that drops - near-zero rows before inserting, a good's absence once a sync has landed means - ~0 flow, not "unknown" — the result carries `syncedAt`/`syncedCount` (from - `meta.stats_synced_at`/`stats_synced_count`, the same fields - `productionComparisonFn` already surfaces to the factory ledger UI) so the - agent can tell a real "nothing's flowing" from "never synced". `gameProduction` - (below) stays the LIVE source of truth when the companion mod is connected — - prefer it when the bridge is up; reach for `productionStats` when it isn't, or - to check many goods (e.g. a plan's imports) in one batch. -- **Live game-world (read-only)** — `gameContext`, `gameInspectArea`, - `gameFindEntities`, `gameProduction` query the _running_ factory through the - bridge (app→mod→Factorio), so the agent can ground a task in real evidence - ("what's built here", "is X actually being made"). Bounded and structured; they - return a clear error when the companion mod isn't connected. -- **In-game Lua eval, gated per call** (#15) — the in-app assistant's `gameEval` - does **not** execute: it returns the snippet as a _proposal_, rendered in the - chat as a card (`components/assistant/game-eval-card.tsx`) showing the exact - Lua and its `note`, with **Run in game** / **Dismiss** controls. Only the - user's Run sends `cmd.eval` over the bridge (`bridgeEvalFn` in - `server/bridge/fns.ts`); the result shows inline with a "Share result with - assistant" chip that feeds it back into the chat. This makes per-call consent - real and lets the agent request careful in-game _write_ actions too. The MCP - surface swaps in a direct-executing variant (`gameEvalDirect`, exposed as - `mcpTools.gameEval`) — developer debugging has no chat UI to approve through. - Defense in depth: the mod's `pyops-allow-eval` per-user setting (default on) - refuses every `cmd.eval` when off — including the MCP screenshot capture - below, which rides on eval. - -**Developer/MCP-only tools.** `gameScreenshot`, `gameReloadMods`, `gameShowBlock`, -and `gameCloseSummary` are **not** in the in-app assistant's tool set -(`agentTools`) — only on the MCP surface (`mcpTools`, see below). The in-app chat -can't consume a local PNG path, and ordinary planning shouldn't trigger a mod -reload or drive the in-game summary panel open/closed; these are for an external -agent (e.g. Claude over MCP) debugging the mod/bridge integration directly: - -- `gameScreenshot` captures the game (GUI included) to a PNG path, optionally - auto-cropped to a top-level GUI element (`panel`) or an explicit `crop`/`scale` - — built for designing the in-game panel live (snap, look, tweak) without a - Factorio reload. -- `gameReloadMods` asks the connected mod to call `game.reload_mods()` after a - mod-code edit, for a screenshot → tweak → reload loop. -- `gameShowBlock`/`gameCloseSummary` push a saved block to (or close) the - in-game Helmod-style summary panel, exactly like the web "show in game" - button — for self-testing the mod's UI via screenshots. - -Single-block drafts still use `submitBlock`. `reviseBlock` re-solves an existing -block (looked up by its `factoryBlocks` id) at a new rate and/or with a revised -recipe set and returns an amber **Resize/Revise block #N** card with an **Apply -update** button: rate-only changes apply through `setBlockRateFn`, recipe -revisions through `setBlockRecipesFn` (which swaps the doc's recipe list via -`lib/block-doc.ts` `withRecipeSet` — pruning removed recipes' machine/module/ -pin config — then re-solves and persists). Multi-output requests or requests -for complete supporting production use `submitPlan`, which returns a plan card -with one preview per block, an optional **resize existing blocks** section, and a -**Create N blocks · resize M** action. Creating a plan saves each proposed block -through the normal block save path (solved flows, machine requirements, power, -cache) and applies each resize through `setBlockRateFn`, exactly like a manually -edited block. The agent is told to check each existing block's current -`makes[].rate` and resize rather than duplicate when it's too small. - -Both apply paths persist a draft's FULL `goals` array (#38), not just its -`target`/`rate`: `routes/assistant.tsx`'s single-block **Create block** handler -and the plan card's **Create N blocks** handler both build `BlockData.goals` -from the draft's `goals` (falling back to a synthesized single goal for an -older cached draft that predates the field), so a keep-in-stock goal's -`stock`/`window` survive into the saved block exactly as the schema (`Goal` in -`db/schema.ts`) expects. The draft card itself renders every goal (not just the -anchor) when a block has more than one, showing a stock goal as "keep N -(refill Xm)" instead of a rate. - -Once a card's block exists in the store it can go **straight into the game** -(#14): the draft card's post-create state, the revise card (its block already -exists), and the plan card's created list each render a **Show in game** button -(`components/assistant/show-in-game-button.tsx`) that pushes the block to the -in-game build-sheet panel via `bridgeShowBlockFn` — the same panel whose -building rows hand out the configured blueprint / request-combinator, so a plan -flows from chat to construction. It reports "game not connected" when the -bridge is down. - -Draft, update, and plan cards carry **one-click follow-up chips** (#13) built -from the solved draft data: a **Draft \ @ rate** chip per suggested -sub-block and a **Route \** chip per byproduct -(`components/assistant/follow-up-chips.tsx`). Clicking one sends the matching -request as the next chat message (disabled while a run is in flight), so the -"draft super-alloy @ 3.3/s next" advice is actionable without retyping. - -When the user asks to include building materials or construction coverage, the -agent is expected to call `buildingBill` with the plan's blocks and cover the -MACHINE ITEMS it returns — not silently reinterpret the request down to just raw -recipe ingredients. For each machine item it decides: an existing mall block -already supplies it (import), an existing block should be resized -(`reviseBlock`/plan `updates`), or it needs its own new block; a large bill is -grouped by shared material chains (steel/circuits/gears feeding several machine -types) rather than dropped silently. Raw resources, electricity, and broad -commodities remain imports unless the user specifically asks to produce them -too. The agent is also told not to defer this (or byproduct routing) to a -follow-up question — a requested plan ships complete, in the same turn. When -the request separately asks for belts/inserters/logistics coverage too, the -agent additionally calls `logisticsFor` per relevant good/rate and reports both -halves together — machines from `buildingBill`, belts/inserters/loaders from -`logisticsFor`. - -The same tool bodies back two front doors: the in-app agent (`agentTools`) and -the MCP route (`routes/mcp.ts`), which registers **every** tool in `mcpTools` — -`agentTools` plus the developer-only tools above and the direct-executing -`gameEval` — for external MCP clients over `POST /mcp` (JSON-RPC). This lets an -external agent — e.g. Claude driving the _running game_ via the read-only -game-world tools, screenshotting the mod's UI, or reloading mods after an edit — -exercise and debug the integration directly, not just the in-app assistant. The -handler (`utils/mcp-handler.ts`) is single-shot per request and waits for the -tool's real async result (db / bridge / LLM), so slow tools work. - -The repo ships project-scoped MCP client config for Codex (`.codex/config.toml`) -and Claude Code (`.mcp.json`). Both point at `http://localhost:3000/mcp`, so run -`vp dev` in `app/` before using the external tools. Claude Code marks the project -server pending until the user approves it in an interactive `claude` session. - -## Planning horizon - -Before each chat turn, `api.chat.ts` injects the current planning horizon into the -system prompt: - -- **Now** — plan only with recipes the user can build right now (research - enabled/available within their current science, TURD active or pickable). -- **Future** — any recipe is fair game, but the agent must call out what needs - unlocking (which science packs) or which TURD path to select. - -## Configuration - -The key resolves **env → app-config**. The model resolves -**env → conversation override → app-config → default**: - -- `OPENROUTER_API_KEY` env, **or** set it in **Settings → Assistant** (stored in - `app-config.json`, app-level). Missing everywhere → the assistant returns a 500 - pointing at both. -- `PYOPS_AGENT_MODEL` env, **or** the active conversation's model, **or** the - model field in **Settings → Assistant**, else `DEFAULT_MODEL`. Any OpenRouter - id. Env wins when set; the per-conversation picker is for interactive local use - when env is unset. -- Per-conversation reasoning effort is optional and applies only to OpenRouter - calls for the active chat when the resolved model is in the known reasoning - model list. Leave it on model default for provider/model routing defaults; - choose low/medium/high when a supported reasoning model needs an explicit - effort. - -Resolution lives in `server/app-config.server.ts` (`resolveApiKey` / `resolveModel`). - -The agent runs a bounded tool loop (`MAX_STEPS`, currently 60) — drafting a full Py -chain takes many calls. diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 6b9f2cf..0000000 --- a/docs/architecture.md +++ /dev/null @@ -1,238 +0,0 @@ -# Architecture - -PyOps is **one app** plus a **game mod**. There is no separate backend service — -the [TanStack Start](https://tanstack.com/start) app _is_ the backend: React UI in -the browser, and server functions + Nitro routes running server-side in the same -process. SQLite, the solver, the UDP bridge, the data pipeline, and the AI all -live as server-side modules. - -``` -┌──────────────────────────── app/ (TanStack Start) ────────────────────────────┐ -│ │ -│ React UI (routes/, components/, lib/) │ -│ │ server functions (createServerFn) + Nitro API routes │ -│ ▼ │ -│ server/ ── factorio data · solver · cost LP · AI agent · bridge │ -│ solver/ ── pure-TS linear-system block solver │ -│ db/ ── Drizzle ORM over better-sqlite3 (per-project .db files) │ -│ │ -└──────────────▲────────────────────────────────────────────────▲───────────────┘ - │ localhost UDP (bridge) │ reads - │ │ - ┌───────────┴───────────┐ ┌───────────┴───────────┐ - │ mod/ (Factorio 2.1) │ │ Factorio data dumps │ - │ in-game panel + │ factorio --dump-data │ data-raw-dump.json, │ - │ live-state sync │ ───────────────────────▶ │ locale, icon sprites │ - └───────────────────────┘ └────────────────────────┘ -``` - -The pieces, in their own docs: - -- **[Data pipeline](data-pipeline.md)** — Factorio dump → SQLite + icon atlas. -- **[Block solver](solver.md)** — turning a block of chosen recipes into run-rates - and machine counts, plus the factory-level what-if. -- **[Factorio bridge](bridge.md)** — the UDP link to the companion mod. -- **[AI assistant](ai-assistant.md)** — the planning agent and MCP surface. - -## Repository layout - -``` -pyops/ -├── app/ TanStack Start app — UI + backend (the whole product) -│ ├── src/ -│ │ ├── routes/ file-based routes: UI pages + API routes -│ │ ├── components/ shared React UI (shadcn/Radix + Tailwind v4) -│ │ ├── lib/ icons, recipe cards, modals -│ │ ├── server/ server-only modules (data, solver glue, bridge, AI) -│ │ ├── solver/ pure-TS linear-system block solver + tests -│ │ └── db/ Drizzle schema, import, synthesize, queries -│ ├── src-tauri/ Tauri desktop shell + packaging (see desktop.md) -│ ├── drizzle/ generated SQL migrations, applied in-process -│ ├── icon-data/ generated icon atlas (gitignored) -│ ├── projects/ per-project .db files, each self-named (gitignored) -│ └── app-config.json app-level config: active project + AI key/model (gitignored) -├── mod/ Factorio 2.1 companion mod (Lua, no build step) -│ ├── control.lua panel + UDP bridge + live-state sync -│ ├── summary.lua Helmod-style production-block view -│ ├── combinator.lua in-game request-combinator planner -│ └── data.lua, settings.lua -├── scripts/ dev-only helpers (tunnel-dev) -└── docs/ this documentation -``` - -The app uses **[Vite+](https://viteplus.dev/)** (the `vp` CLI) as its toolchain — -Vite, Rolldown, Vitest, Oxlint, Oxfmt under one wrapper — not the bare -`pnpm dev`/`pnpm build` scripts. See [`AGENTS.md`](../AGENTS.md) for the command -reference. - -**Responsive UI.** The desktop layout degrades to tablet/phone/Steam Deck rather -than assuming a wide screen. The global nav collapses to a hamburger drawer below -the width where its full bar fits (~1400px, so the 1280px Deck uses the drawer); -the fixed left rails (block, browse, assistant, tasks) collapse below `md` via a -shared `SidebarShell` built on a `Sheet` drawer primitive (radix Dialog); and the -dense data tables (factory, whatif) stack into labelled cards on phones via a -shared `StatCell` instead of squeezing fixed columns. Reordering uses dnd-kit so -it works by touch — the recipe rows via a drag grip, and the block sidebar via -whole-row drag (mouse needs a small move, touch a short press-hold, so a tap still -opens and an immediate finger-drag still scrolls). A Playwright harness, -[`app/e2e/responsive.e2e.ts`](../app/e2e/responsive.e2e.ts), screenshots every -route across a desktop/tablet/phone matrix and asserts no route scrolls sideways -at tablet/phone widths. - -## Per-project databases - -Each "project" (usually a different mod list) is its own SQLite file under -`projects/`. The files are the source of truth — there's no registry: each db -self-describes its name/createdAt in its own `meta`, and the active project id -lives in `app-config.json`. The `db` export (`app/src/db/index.server.ts`) is a proxy -that always points at the active connection, so the query layer never changes when -you switch. Schema is provisioned in-process: on first connect (and when creating a -project) the bundled `drizzle/` migrations are applied via drizzle-orm's `migrate()` -(`app/src/server/provision.ts`), so no dev tooling is needed at runtime. A new -project starts empty; you then run a [data sync](data-pipeline.md) to fill it. The -relevant code lives in `app/src/server/projects.ts`, `app/src/server/provision.ts`, -and `app/src/db/index.server.ts`. - -Every writable project connection uses the same SQLite policy before migrations -or application queries run: WAL journal mode, a 5-second busy timeout, -foreign-key enforcement, and `synchronous=NORMAL`. WAL keeps readers moving while -SQLite serializes writers; the timeout absorbs brief overlap with imports and -desktop lifecycle operations. Cache size, temporary storage, memory mapping, and -automatic checkpoint thresholds stay at SQLite's defaults instead of becoming -another application memory/cache policy. Short-lived read-only handles (project -listing, validation, and backup) get the same timeout and integrity setting but -do not try to change the database's persistent journal mode or their irrelevant -write-durability setting. - -Because connections are cached, migrations added while the server is running (the -dev-server case) are **not** picked up — the fix is a restart. The app shell polls a -cheap drift check (`app/src/server/db-migrations.server.ts`: bundled -`drizzle/meta/_journal.json` vs the active db's `__drizzle_migrations` rows) and -shows a "restart the app to apply" banner when any are pending; migrations are never -auto-applied at runtime. - -## Export / import (#82) - -Two surfaces under **Settings › Backup & share**: - -- **Project backup** — the whole project as its `.db` file. `GET /api/backup` - streams an online-backup snapshot of the active db (better-sqlite3's backup API, - so it's consistent while the app runs); `POST /api/backup?name=…` installs an - uploaded db as a **new** project (validated, fresh id, never overwriting; the - bundled migrations upgrade an older backup on first connect). A route handler - because both directions move a whole file (`app/src/routes/api.backup.ts`, - `app/src/server/backup.server.ts`). -- **Block / plan JSON** — shareable, versioned envelopes (`{ pyops: 1, kind: -"block" | "plan", … }`) carrying a block's full editor doc (goals, recipes, - per-recipe picks) — a plan adds sidebar folders. The pure logic (validation, - legacy-doc migration, name-collision suffixing) is `app/src/lib/plan-export.ts`; - the db side is `app/src/server/export.server.ts` (server fns in - `export-fns.ts`). Imports always create **new** blocks (suffixed names, remapped - folders) and re-solve them; references the target's data dump doesn't have are - flagged broken — the same degrade path as mod drift — never rejected. Single - blocks also export from the block editor's toolbar. Snapshots (#85) build on the - same serialization. - -The block editor keeps the solve server-authoritative without solving the same -document twice. Opening a block solves the SQLite-loaded document once; edits -are coalesced for a short idle window, then one `saveBlockFn` request solves and -persists the document and returns that exact solve for the UI. Module auto-fill -hints follow lazily from the solved row rates and never invoke the LP. External -writes (undo and snapshot restore) rehydrate and solve the new persisted -document, while the existing `updatedAt` guard still rejects stale-tab saves. -Save requests are serialized per editor; edits made during an in-flight solve -collapse into one follow-up save of the newest document, so responses cannot -land out of order. - -## Block snapshots (#85) - -Per-block restore points, complementing undo (which unwinds recent edits): -a snapshot freezes a block's full definition — the face (name/icon/enabled) as -columns plus the editor doc as JSON, the **same serialization as the export -envelope's block** — into a `block_snapshots` row. Two kinds: - -- **manual** — the user's named points ("Snapshot now" in the block editor's - history drawer, label optional), kept until deleted. -- **auto** — taken silently before destructive/structural writes: block delete, - snapshot restore, scale-to-demand/assistant resizes, and (throttled to one per - 10-minute editing burst) ordinary saves. Capped at the newest 20 per block, - pruned on write; deduped against the newest snapshot so repeat operations - don't stack identical rows. - -The logic lives in `app/src/server/snapshots.server.ts` (server fns in -`snapshot-fns.ts`); the drawer is `app/src/components/block/snapshot-sheet.tsx`. -**Restore** replaces the block's definition in place (identity — id, folder, -sort order — preserved): it auto-snapshots the current state first, re-solves -through the normal persist machinery, and runs as ONE tracked undo action, so a -restore is both undoable and re-restorable. **Diff** (`app/src/lib/block-diff.ts`, -pure) compares a snapshot against the live editor doc — goals added/removed/ -re-rated, recipes added/removed/toggled, machine/fuel/module/beacon picks, -made marks, pins, spoil plans — rendered in the scale-plan drawer's from → to -language with display names resolved server-side. Snapshot bookkeeping itself is -not a planning edit: `block_snapshots` carries no undo triggers and every -capture runs `{ undo: false }`, and rows deliberately survive block deletion -(a recycle bin; restore-from-deleted UI is future work). - -## Undo (planning edits) - -Multi-level undo for planning edits, built on the canonical -[SQLite trigger pattern](https://www.sqlite.org/undoredo.html): `AFTER -INSERT/UPDATE/DELETE` triggers on the **user-planning tables only** (blocks, -block_groups, module_presets, tasks, task_steps, task_links, notes — never -imported reference data, live-state tables, or caches like -`block_flows`/`block_machines`) write the inverse SQL of every row change into -`undo_log`. The triggers live in a migration (`drizzle/0004_undo_log.sql` — -hand-written there because drizzle can't model triggers) and only fire while a -current-action marker row exists in `undo_current`, so any write that bypasses -the wrapper is simply untracked (fail-soft), and the high-volume system writes -are excluded by construction. - -One undo step = one user action: every mutating server-fn path runs through -`withUndoAction(name, fn)` (`app/src/server/undo-action.server.ts`), which opens -one action id + the marker, runs the mutation, and closes it — so "apply this -plan" pops as a single undo. Tracking is opt-out: system writes to planning -tables (cache re-solves, LLM-computed priorities, undo execution itself) pass -`{ undo: false }`. Undo is linear (strictly top-of-stack), the last 50 actions -are kept (trimmed on write), and the log is per project db. - -`undoLast()` (`app/src/server/undo.server.ts`, exposed as -`undoStatusFn`/`undoLastFn` in `app/src/server/undo.ts`) executes the top -action's inverse statements in one transaction without re-logging, then -re-solves the touched blocks through the normal persist machinery so the -untracked caches stay consistent, and returns the changed block ids so open -editors can rehydrate. If a migration adds a column to a triggered table, that -table's triggers must be regenerated in the same migration — -`app/src/server/undo.test.ts` has a coverage check that fails when a trigger -goes stale. - -On the client every undo trigger — Ctrl+Z (`components/undo-hotkey.tsx`, via -the hotkey layer and deliberately **not** `allowInInputs`, so text fields keep -native undo), the ↶ nav affordance (`components/undo-button.tsx`, tooltip = -top-of-stack name, disabled at depth 0), and the palette's "Undo last action" — -runs through `lib/undo-client.ts`'s `runUndo`: pop the stack, toast the result -(the shared toast primitive: `components/ui/toast.tsx` + `lib/toast-store.ts`), -invalidate the planning query families, and push the reverted doc into any open -block editor via the open-editor registry (`lib/block-editors.ts`) — the editor -registers a `hydrate` callback so its auto-save can't write pre-undo state -back, and an `onDeleted` escape for when the undo reverted the block's -creation. The editor also sends `baseUpdatedAt` (its hydration point) with -every save; a `{ conflict }` rejection (another tab/undo/assistant write got -there first) reloads the fresh doc and toasts instead of clobbering it. Editor -saves name themselves on the stack where the call site knows what changed -(`pendingAction` labels in the block doc store, merged by -`lib/undo-names.ts`); the debounced auto-save otherwise stays generic -(`Edit block "…"`). - -Destructive actions ride the same rails (#83): a small undo-logged delete -(task, subtask, step, note, folder, module preset) fires immediately — no -confirm — and shows a toast whose **Undo** button is just a shortcut into -`runUndo` (`deletedToast`/`undoToast` in `lib/undo-client.ts`). Big or -irreversible deletes go through `ConfirmDialog` -(`components/confirm-dialog.tsx`, on the `ui/alert-dialog.tsx` primitive — -never `window.confirm`): block deletion states the recipe/goal counts being -destroyed (from `listBlocks`) and still gets the undo toast; project removal, -companion-mod removal, and chat deletion aren't in the undo log, so their -dialogs say so and their toasts carry no Undo button. In-editor row removals -(recipe rows, snapshot delete) keep their two-click arm instead — they mutate -the editor's local doc ahead of the debounced auto-save, so a toast-Undo could -pop the wrong action. diff --git a/docs/bridge.md b/docs/bridge.md deleted file mode 100644 index b8f9ca4..0000000 --- a/docs/bridge.md +++ /dev/null @@ -1,180 +0,0 @@ -# Factorio bridge - -Code: `app/src/server/bridge/` (app side) and `mod/control.lua` (game side). - -A localhost UDP socket (`node:dgram`, default port **37657**) talks to the -companion mod. The mod sends JSON request datagrams and polls for replies; the app -dispatches them to handlers (`app/src/server/bridge/handlers/`) and answers on the -same socket. The socket is a process singleton stashed on `globalThis` so Vite HMR -re-evaluating the module reuses the existing bind instead of throwing -`EADDRINUSE`. Transient UDP delivery errors during a game or dev-server reload -clear the remembered peer but do not close the app's listener; the mod's next -heartbeat registers its current source port and restores the link. - -## Status in the UI - -The global nav carries a compact status indicator (`app/src/components/bridge-indicator.tsx`): -a colored dot + label (game linked / no game / mod mismatch / bridge error) with a -tooltip, linking to **Settings › In-game link**. It shares the `["bridgeStatus"]` -query with the fuller **Live bridge** card on that tab, so the two never disagree — -and mounting either is what `ensureBridge()`s the socket, so the listener comes up -on any page. The same tab hosts the companion-mod installer (see below). - -## What flows across it - -- **Live state → app:** researched technologies, TURD selections, placed machines - (keyed by the recipe each crafts — and mining drills by the resource they're on, - as the solver's synthetic `mine-` recipe, so built-vs-required lines up - per ore), and item production stats — pushed on connect and on relevant in-game - events, always as the full authoritative set (no delta merging on the app side). - Machine build/mine/death/recipe changes are debounced into a full refresh within - one second, independent of whether the in-game panel is open; a once-per-minute - reconciliation catches changes made by scripts that do not raise those events. -- **Commands → game:** show a production-block panel in-game (`cmd.show_block`) - and close it again (`cmd.hide_block`), locate producers/consumers/storage - (`cmd.locate`, relayed to the - [Factory Search](https://mods.factorio.com/mod/FactorySearch) mod's remote - interface), and put an app-built blueprint string on the player's cursor - (`cmd.blueprint` — e.g. the sushi planner's set-point combinator; the mod - refuses politely if the cursor is holding something). -- **Sushi-loop tracer (`mod/sushi.lua`):** hover a belt and press ALT+B — the mod - flood-fills the belt loop (belts, undergrounds, splitters; lane balancers are - handled as graphs), prunes feed-on/feed-off spurs so only the circulating core - counts, sets one "read entire belt (hold)" reader per game segment (segments - end at splitters and sideload merges; undergrounds CONTINUE the segment, - buried span included — probed empirically with isolated readers), places the - readers at the junction cluster where segments meet, and connects them with a - minimum spanning tree of short player-legal wires (prototype reach; if a gap - can't be bridged belt-to-belt it prints GPS-linked power-pole suggestions - instead). Skip-and-warn on belts you already wired. The measured loop - (`sushi.trace`: tiles/segments/closed) lands in the app, where the sushi - planner offers it as a one-click loop length. SHIFT+ALT+B removes exactly what - the last trace added. A `pyops-sushi` remote interface exposes trace/untrace - for tooling. Read accuracy: splitter internals aren't circuit-readable (the - read undercounts by the in-splitter transit population) and "entire belt" - zones bleed into adjacent feeder branches — on a circulating loop the error - is a handful of items either way. Verified against exact item censuses on a - scripted torture loop (5 splitters, bypass arms, tier-weaved undergrounds). -- **Task panel:** the in-game panel's **Tasks** tab pulls the project's - tasks with `task.list` (the app replies with the full set — title, status, - priority, body, steps, and links resolved to Factorio sprite paths) and renders - them as a master-detail list (`mod/tasks.lua`, styled with the `pyops_*` kit). - Read-only for now (status/step writes come later); it re-pulls on open, on the - refresh button, and after a capture. -- **New task → app:** the panel's **+ New task** dialog sends a title + - description plus best-effort anchors (surface/position + the entity the player - last hovered) as `task.capture`; the app files the task and replies - `task.captured`, after which the mod re-pulls `task.list`. The anchors become - `entity`/`location` task links. (The web `/tasks` page keeps itself fresh to a - mod-side write via refetch-on-focus + a light visible interval — no push channel - yet.) -- **Read-only inspection (app→mod request/response):** the app pushes a - `cmd.*` (`game_context`, `inspect_area`, `find_entities`, `production`) with a - `request_id`; the mod runs a bounded game query and replies `bridge.result` - echoing that id. `server/bridge/inspect.ts` correlates the reply to the - awaiting caller (with a timeout). These back the assistant's read-only - game-world tools — no whole-map dumps. (This reuses the same app→peer push as - `request.sync`; the mod must be polling — i.e. the bridge enabled.) -- **Developer visual loop:** MCP clients get `gameScreenshot` for GUI-inclusive - screenshots and `gameReloadMods` for a safe mod reload. `gameReloadMods` sends - `cmd.dev.reload_mods`; the mod acknowledges, schedules `game.reload_mods()` for - the next tick, then the app waits for the normal bridge heartbeat/resync before - further inspection. This replaces desktop click automation for normal - `control.lua` / GUI iteration. If the currently loaded mod predates the command, - reload Factorio manually once. - -## Transport requirements - -The mod uses Factorio's `helpers.send_udp` / `recv_udp`, which the engine only -exposes when the game is launched with `--enable-lua-udp `. That `` is -the socket **Factorio binds for itself**, so it must be a _different_ free port -than the app's bridge port (`PYOPS_BRIDGE_PORT`, default `37657`) — two processes -can't bind the same loopback UDP port, and Factorio otherwise fails at startup with -`Opening Lua UDP Socket failed: Binding IPv4 socket failed: Address already in use`. -Use e.g. `--enable-lua-udp 37658` and leave the mod's `pyops-bridge-port` setting at -the app's port; the mod always sends to that app port, and the app replies to -whatever source port Factorio's socket used. The app's **Live bridge** card has a -**Launch Factorio** button that sets this flag automatically (picking a free port -next to the app's), so users normally don't touch it by hand. There's no enable -toggle — the mod runs the bridge automatically whenever the game was launched with -the flag, and disables itself for the session (with an in-panel hint) if the flag is -absent. The loopback-only socket means the game and the app must run on the same -machine. - -## The wire contract - -`app/src/server/bridge/protocol.ts` is the pure envelope layer — request/response -types plus parse/serialize, with no transport and no Node dependencies, so handlers -and tests can use it without touching the socket. - -The contract version (`PROTOCOL_VERSION`) lives in **both** `protocol.ts` and -`mod/control.lua` and must stay in lockstep — bump both sides whenever the message -shapes change. Each side reports its version and warns when the other disagrees. - -## The companion mod - -`mod/` is a normal Factorio 2.1 mod — pure Lua, no build step: - -- `control.lua` — the in-game panel, the UDP bridge, and live-state sync. -- `summary.lua` — the Helmod-style production-block view (`cmd.show_block`), - including the clickable factory cell that puts a configured blueprint on the - cursor. Goods are rendered as locked signal buttons (styled with the blue/yellow - slot styles — blue for products, yellow for ingredients — so the recipe/in/out - split reads at a glance) so the engine's own smart-pipette (Q) grabs the - item/fluid as a filter signal; Q over a factory/beacon cell pipettes the building. - A titlebar toggle shows a Helmod-style belts/inserters readout on each good — the - counts are computed app-side (the `cmd.show_block` payload carries per-good - `belts`/`inserters` plus a top-level `logistics` descriptor with the chosen - belt/mover for the icons), honouring the web Logistics **show-belts/show-inserters** - toggles (so it renders just belts, just inserters, or both), and reusing the web - Logistics math and picks rather than recomputing in Lua. -- `combinator.lua` — the in-game request-combinator planner. -- `data.lua`, `settings.lua` — prototypes and the per-player bridge port setting. - -It's verified hands-on in-game; there's no automated test harness for the Lua side. -After editing anything under `mod/`, reload it in-game. Once a save is already -running a mod version with the developer command above, the MCP `gameReloadMods` -tool can do that reload without window automation. Prototype/data-stage edits and -cases where the mod fails before the bridge starts still need a normal game -restart or manual reload. - -### Installing it - -**Settings › In-game link** installs `mod/` into the Factorio mods folder for you -(`app/src/server/companion-mod.server.ts`, `companion-mod-fns.ts`, -`components/companion-mod-card.tsx`). Two OS-aware methods: a **symlink** -(recommended — the installed mod tracks the repo; a directory junction on Windows, -so no admin/Developer Mode) or a plain **copy**. It detects current install state -(linked / copied / broken / out-of-date) and only ever removes a target it can -prove is ours (a symlink, or a directory whose `info.json` name is `pyops`). The -target is always `/pyops` — the folder name must equal the mod's info.json -name. - -To install by hand instead, link or copy `mod/` into your mods folder as a folder -named `pyops`, from the repo root: - -```bash -# Linux -ln -s "$PWD/mod" ~/.factorio/mods/pyops -# macOS -ln -s "$PWD/mod" ~/"Library/Application Support/factorio/mods/pyops" -``` - -```powershell -# Windows (PowerShell) — directory junction, no admin needed: -New-Item -ItemType Junction -Path "$env:APPDATA\Factorio\mods\pyops" -Target "$PWD\mod" -``` - -Or copy the `mod` folder in and rename the copy to `pyops` (re-copy after updates). - -### Launching the game - -Easiest: **Launch Factorio** in the **Live bridge** card — it starts the game with -`--enable-lua-udp` on a free port (through Steam if that's your copy, so cloud -saves / overlay / achievements keep working). To start it yourself, add -`--enable-lua-udp 37658` (Steam: right-click the game → Properties → Launch -Options). That port is the socket Factorio **binds for itself**, so it must be a -_different_ free port than the app's bridge port (`PYOPS_BRIDGE_PORT`, default -`37657`) — the two can't share one loopback UDP port. Leave the mod's **PyOps -bridge UDP port** setting at the app's port. With PyOps running, the in-game panel -connects automatically — there's no toggle to flip. diff --git a/docs/configuration.md b/docs/configuration.md deleted file mode 100644 index 5556b09..0000000 --- a/docs/configuration.md +++ /dev/null @@ -1,22 +0,0 @@ -# Configuration - -App-level settings live in the **⚙ Settings** UI (active project, the OpenRouter -key/model, the research horizon, the companion mod, backup/share). The environment -variables below are for source runs and overrides — set them in `app/.env.local` -(or the process environment). All are optional. - -| Setting | Default | Purpose | -| --------------------- | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | -| `FACTORIO_BIN` | `~/.local/share/Steam/.../bin/x64/factorio` | Path to the Factorio executable used for data syncs. | -| `FACTORIO_DATA_DIR` | `~/.factorio` | Factorio user data (mods, `script-output`). | -| `OPENROUTER_API_KEY` | — | AI **Assistant** key. Set here _or_ in **Settings → Assistant** (env wins). | -| `PYOPS_AGENT_MODEL` | `~anthropic/claude-sonnet-latest` | Any OpenRouter model id. Set here, per chat, or in **Settings → Assistant** (env wins). | -| `PYOPS_BRIDGE_PORT` | `37657` | UDP port the app's bridge listens on (the mod's send target). Use a _different_ port for `--enable-lua-udp`. | -| `PYOPS_DATA_DIR` | working dir (dev) / per-OS user data dir (packaged) | Where `projects/*.db` and `app-config.json` live. See `app/src/server/paths.server.ts`. | -| `DATABASE_URL` | active project's file (else `projects/default.db`) | Override the local SQLite file directly. | -| `PYOPS_ALLOWED_HOSTS` | tunnel providers' domains | Extra hostnames the dev server accepts (comma-separated, or `true` to allow any) — for custom tunnels. | - -Reaching the dev server remotely (phone, another machine) is handled by -[`scripts/tunnel-dev`](../scripts/tunnel-dev) — it auto-picks cloudflared / ngrok / -tailscale and exposes `:3000`; run `scripts/tunnel-dev --help` for provider and -custom-hostname options. diff --git a/docs/data-pipeline.md b/docs/data-pipeline.md deleted file mode 100644 index 233e033..0000000 --- a/docs/data-pipeline.md +++ /dev/null @@ -1,179 +0,0 @@ -# Data pipeline - -Everything in PyOps starts from the game itself. PyOps runs Factorio headlessly to -dump its fully-resolved prototype data, then imports it into SQLite. This is -orchestrated end-to-end from **Settings › Game data** in the UI -(`app/src/server/dump.server.ts`): - -1. **Write a helper mod** (`pyops-dump`) into your Factorio mods folder and enable - it. It sets `data.data_crawler = "yafc pyops"`, which makes `pypostprocessing` - run its planner integration — TURD sub-techs become real technologies, farm - recipes get representable fluids, etc. A `data-final-fixes.lua` then patches up - anything that integration leaves engine-invalid (1.x-style recipe results, - missing icons, broken TURD unlock effects) and exports the TURD - recipe-replacement map as mod-data. -2. **Dump** via `factorio --dump-data`, `--dump-prototype-locale`, and optionally - `--dump-icon-sprites`. (Icons load the _full_ game/renderer and are slow, so - they're opt-in; data + locale dump in seconds.) -3. **Disable the helper mod again** — it must never be active during normal play. -4. **Import** the dump into SQLite (`app/src/db/import-factorio.ts`), then - synthesize the recipes the engine doesn't model as recipes: mining, boiling, - burning, spoiling, planting (agricultural towers), rocket launches, and - per-temperature fluid variants (`app/src/db/synthesize.ts`). Recipe categories - come from Factorio 2.1's `categories` array, with the singular 2.0 `category` - field retained as an import fallback. -5. **Rebuild the icon atlas** (`buildIconAtlas`, `app/src/server/icon-atlas.ts`): - pack the dumped sprites into content-hash-deduped 4096² sheets + a - `(type, name) → slot` manifest, written to the data dir's `icon-data/`. The app - serves them at `/icons/*` (`app/src/routes/icons.$.ts`), cached `immutable` and - cache-busted by the data fingerprint in the sheet URLs (`?v=…`). -6. **Compute cost analysis** — a YAFC-style LP that assigns each good an intrinsic - cost (`app/src/server/cost-analysis.server.ts`, a port of YAFC's `CostAnalysis.cs`). -7. **Apply mod migrations** — read each enabled mod's `migrations/*.json` and - auto-apply any newly-present prototype renames to saved blocks - (`app/src/server/migrations.ts`; see drift resilience below). -8. **Refresh solve projections** — advance the SQLite-owned solve generation and - re-solve each stale saved block once, after every reference-data writer has - finished. Factory totals therefore never depend on an application cache or on - remembering which in-memory layer to invalidate. - -The enabled mod set is fingerprinted (a hash of mod names) and stamped into the DB, -so the planner knows which version of the game its data reflects. The full mod list -is persisted alongside it (`mod_list` in `meta`): each mod's name, **version**, and -enabled state (`readMods`, `app/src/server/dump.server.ts` — `mod-list.json` carries only -name + enabled, so versions are recovered from the `name_x.y.z.zip` entries in the -mods directory). This records the provenance of the reference data — shown on the -Settings → Game data tab — and gives drift detection and rename capture -a concrete previous state to diff against, not just a hash. - -**Drift detection** (`modDriftFn`, `diffMods`/`redumpNeeded` in `dump.server.ts`) compares -the game's _current_ mod set against that persisted baseline, by name **and** -version, and categorizes the change (added / removed / enabled / disabled / -version-changed). `needsRedump` is true only when the _enabled_ mods or their -versions changed (disabled-mod churn doesn't affect the data). When drift is -detected a **guided modal** (`DriftModal`, opened via the shared `drift-store`) -pops with the categorized changes and an ignore/re-sync choice, then walks the dump -as a step-by-step progress flow (`lib/sync-steps.ts` maps each `SyncPhase` to a -labelled step) ending in a summary that links to the Factory block change-report. It -re-checks on app start, on project switch (a full reload), on bridge reconnect -(Factorio likely restarted), and every couple of hours; an ignored drift leaves a -small "data stale" chip in the nav to re-open the modal, and Settings → Game data -shows the same detail. Reading the mod set is cheap (two small file reads), so -checking often costs little. - -Saved blocks additionally carry a **per-block solve fingerprint** -(`blockReferenceFingerprint`, `app/src/db/queries.server.ts`): a hash over the _current_ -definitions of just the recipes and goal goods that block references, prefixed by -the current `solve_projection_generation` from SQLite `meta`. The generation -advances when imported reference data, research/productivity state, or TURD -selection changes. It makes projection validity a database fact: unchanged bridge -heartbeats do not advance it, and only rows from the current generation are treated -as fresh. The content hash still pinpoints an altered or vanished referenced recipe. - -When a block references a recipe or goal good that no longer exists, `computeBlock` -refuses to solve it — solving the surviving subset would -silently produce wrong rates — and instead returns `broken` with the missing -references. The block's input doc and its last-good cached I/O are preserved -untouched (so re-enabling the mod or re-importing restores it), the block view and -sidebar flag it, and its old generation stamp remains stale rather than blessing -the preserved values as current. - -**Pure renames are auto-applied during the dump** (`migrations.ts`), so this broken -fallback is reserved for references that genuinely changed meaning or disappeared. -Mods ship declarative `migrations/*.json` files (`{ "recipe": [["old","new"], …], -"item"/"fluid"/"entity": … }`) inside their zips; a mod's own Lua runtime can't read -other mods' migration files, but the backend reads them straight from the zips (via -`fflate`) or unpacked folders. Each dump records which migration files it has seen -(`migrations_applied` in `meta`, keyed by `mod/file`); a file newly present since the -last dump is a new rename, applied across every saved block's references — the -`goals`, `recipes`, the recipe-keyed `machines`/`fuels`/`modules`/`beacons` and -their member names, disposition keys, and `iconName`. The **first** run after this -ships only records the baseline and applies nothing (existing blocks already -reference current names), so renames fire only on a genuine future mod update. The -`.lua` migration files are procedural save-state scripts, not renames, and are -skipped. - -## Why a helper mod - -`pypostprocessing` already ships a YAFC/planner integration, but it triggers only -when a crawler marker is present and was written against YAFC's lenient Lua crawler -rather than the real engine validator. The `pyops-dump` helper supplies that marker -and then repairs the fallout so a real `--dump-data` run succeeds: it normalizes -1.x-style `result =` recipes, fills in missing icons and fluid-box volumes, drops -recipes whose result item never got created (and scrubs the now-dangling -`unlock-recipe` tech effects), and rebuilds TURD sub-tech unlock effects that the -integration leaves empty. On Factorio 2.1 it also evicts the Py modules that the -integration intentionally re-runs after resetting their globals, avoiding stale -`require` cache results. It's strictly a dump-time tool — `dump.server.ts` enables -it, runs the dumps, and disables it again in a `finally` block so it never lingers -for normal play. - -## Data model notes - -The schema (`app/src/db/schema.ts`) models Factorio reference data plus PyOps' -own planning state (blocks, groups, TURD selections, cost analysis, meta). A few -deliberate choices: - -- **Quality is not modelled** — Py has none. -- **Fluid temperatures are** — ingredients carry a min/max range, products an exact - temperature; the synthetic-recipe pass generates per-temperature variants. -- **Energy is pseudo-fluids** — `pyops-electricity`, `pyops-heat`, and - `pyops-fluid-fuel` flow through the model as goods (1 unit = 1 MJ). Electricity - and heat are filtered out of normal import/byproduct lists and surfaced - separately as power/heat in watts; fluid fuel is a real matched flow (#115) that - shows as an explicit "Fluid fuel (MJ)" import/export, rendered in power units. - Reactors also persist their prototype's `neighbour_bonus` - (`crafting_machines.neighbour_bonus`, #94) so the solver can scale heat output - for an assumed reactor-farm layout. -- **Fluid fuel is fungible** (#25) — fluids carry no fuel category; an unfiltered - `burns_fluid` energy source (Py: glassworks, smelter, antimony drills, the oil - boiler) accepts _any_ fluid with a `fuel_value`, so those machines draw MJ from - the shared `pyops-fluid-fuel` pool, fed by one synthetic `burn-fluid-` - conversion per fuel-valued fluid (1 unit → its `fuel_value` in MJ; 59 in the Py - dump). A `fluid_box.filter` on the energy source pins the machine to that one - fluid instead (Py oil/gas powerplants). `burns_fluid: false` sources are - **temperature-fed** (#114): they drain their filter fluid for its heat content - (Py uf6 reactors, compost plants, the solar tower). The import derives their - drain via `db/fluid-energy.ts` and stores it on `crafting_machines`: - `fluid_fuel_per_sec` (a fixed units/s — an explicit `fluid_usage_per_tick`, or - the engine's derivation from the source's `maximum_temperature`, e.g. - nuclear-reactor-mk01's 300 kW ÷ ((250° − 0.01°) × 20 J/°) ≈ 60.0024 uf6/s) and - `fluid_fuel_energy_j` (usable J per unit — `scale_fluid_usage` sources like the - compost plants follow the energy draw instead). `crafting_machines` carries - `burns_fluid` + `fluid_fuel_filter`, and burner/fluid `effectivity` is folded - into the stored `energy_usage_w` (fuel draw = energy ÷ effectivity). -- **Logistics prototypes** — `belts`, `loaders`, and `inserters` tables capture the - bits needed to size belt/inserter counts per block row (the **Logistics** display, - issue #21): belt/loader `speed`, and per-inserter `rotation_speed` / - `extension_speed` / pickup+drop vectors / `bulk` / `max_belt_stack_size`. Stacking - research lives in `tech_stack_bonuses` (one row per tech per effect: - `belt-stack-size-bonus`, `inserter-stack-size-bonus`, `bulk-inserter-capacity-bonus`), - summed over the in-effect tech set (`queries.stackBonuses`, following the research - horizon) to derive the current belt placed-stack and inserter hand stack. The - throughput math is a pure module (`app/src/lib/logistics.ts`) — belts are - `speed × 480 × stack`; inserters use the swing model ported from the in-game - `inserter-throughput-lib` (inventory→inventory case). The per-row arithmetic runs - client-side so changing belt/inserter tier is instant (no re-solve). -- **Research productivity** (#92) — `tech_productivity_bonuses` captures the two - flat-productivity tech effects (one row per tech per target): Factorio 2.0 - `change-recipe-productivity` keyed by its target recipe, and - `mining-drill-productivity-bonus` under the sentinel key `''` (applies to every - synthetic mining recipe). `queries.productivityBonuses` sums those over the - in-effect tech set (gated by the research horizon like `stackBonuses`) and - applies them in the solver's effects stage; in NOW mode, a - `research_mining_productivity_bonus` meta value overrides the mining sum with - an exact force bonus, including repeatable levels. The bridge sync writes that - value from the running save, and the Planning horizon control can set it - manually for modless play. The bridge also writes - `research_recipe_productivity_bonuses`, an exact per-recipe map read from - `LuaRecipe.productivity_bonus`; without that map, recipe productivity is derived - from the researched-tech list. Recipes also carry - `maximum_productivity` (the 2.0 productivity cap; NULL = engine default +300% - — Py sets 1e6 on nearly every recipe). -- **Rocket logistics** (issue #22) — `items.weight` (rocket-lift weight) feeds an - optional launches/min readout: `floor(rocket_lift_weight / weight)` per rocket, - then `rate × 60 / capacity`. `rocket_lift_weight` and `default_item_weight` come - from `utility-constants.default` (stashed in `meta` at import). Only ~15% of items - set an explicit `weight`; the rest are runtime-derived from recipes, which Py's - cyclic graph makes impractical to recompute, so unset items fall back to - `default_item_weight` (flagged in the tooltip as an estimate). diff --git a/docs/design.md b/docs/design.md deleted file mode 100644 index c6fb4bc..0000000 --- a/docs/design.md +++ /dev/null @@ -1,231 +0,0 @@ -# Design system - -The shared visual/interaction spec for the web app (issue #17). Every UI change — -new page, new component, touched-up route — follows this document. The base layer -(theme tokens in `app/src/styles.css`, primitives in `app/src/components/ui/`) -enforces most of it by default; this doc is the contract for everything the base -layer can't enforce. - -**The prime directive: don't hand-roll what the system provides.** If you're -writing `className="rounded border px-2 …"` on a `