From 6a3d780928f05c2410df909e2392736aedf5caad Mon Sep 17 00:00:00 2001 From: "personal-stack-agents[bot]" Date: Mon, 8 Jun 2026 11:06:38 +0000 Subject: [PATCH 1/4] docs: shared-extraction handoff dossier for next agent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Complete, self-contained handoff for the shared-code extraction program so a later session can resume without rediscovery: the council brief + consolidated plan + task DAG (council-* — copied out of git-ignored .council/), a synthesis PLAN, the full issue/milestone mirror (ISSUES), the versioning + Pipeline Complete contracts, the environment/credentials/blockers writeup, and an ordered NEXT-STEPS. personal-stack-first; website later. Part of #605. --- .../shared-extraction/CI-PIPELINE-COMPLETE.md | 52 ++ .../ENVIRONMENT-AND-BLOCKERS.md | 68 +++ docs/handoff/shared-extraction/ISSUES.md | 45 ++ docs/handoff/shared-extraction/NEXT-STEPS.md | 48 ++ docs/handoff/shared-extraction/PLAN.md | 35 ++ docs/handoff/shared-extraction/README.md | 33 ++ docs/handoff/shared-extraction/VERSIONING.md | 36 ++ .../shared-extraction/council-brief.md | 132 +++++ .../council-consolidated-plan.md | 129 +++++ .../shared-extraction/council-tasks.json | 486 ++++++++++++++++++ 10 files changed, 1064 insertions(+) create mode 100644 docs/handoff/shared-extraction/CI-PIPELINE-COMPLETE.md create mode 100644 docs/handoff/shared-extraction/ENVIRONMENT-AND-BLOCKERS.md create mode 100644 docs/handoff/shared-extraction/ISSUES.md create mode 100644 docs/handoff/shared-extraction/NEXT-STEPS.md create mode 100644 docs/handoff/shared-extraction/PLAN.md create mode 100644 docs/handoff/shared-extraction/README.md create mode 100644 docs/handoff/shared-extraction/VERSIONING.md create mode 100644 docs/handoff/shared-extraction/council-brief.md create mode 100644 docs/handoff/shared-extraction/council-consolidated-plan.md create mode 100644 docs/handoff/shared-extraction/council-tasks.json diff --git a/docs/handoff/shared-extraction/CI-PIPELINE-COMPLETE.md b/docs/handoff/shared-extraction/CI-PIPELINE-COMPLETE.md new file mode 100644 index 00000000..00402b72 --- /dev/null +++ b/docs/handoff/shared-extraction/CI-PIPELINE-COMPLETE.md @@ -0,0 +1,52 @@ +# The `Pipeline Complete` CI contract + +**Org rule:** every repo has exactly ONE CI pipeline, and it terminates in a +single aggregating job named **`Pipeline Complete`**. That job name is the ONLY +required status check in the common branch ruleset; every other job feeds it via +`needs:`. Adding/renaming an upstream job never touches branch protection — just +keep it in the aggregator's `needs:`. + +personal-stack's active "Main" ruleset **already requires** the `Pipeline +Complete` check (context `Pipeline Complete`, integration_id 15368 = GitHub +Actions). No current workflow emits it, so **no PR can merge** until the +consolidated pipeline exists. This is gated on the workflows-token fix (PR #620; +see ENVIRONMENT-AND-BLOCKERS.md). + +## Canonical aggregator pattern + +```yaml + pipeline-complete: + name: Pipeline Complete + if: always() + needs: [lint, test, build] # every gating job + runs-on: ubuntu-latest + steps: + - name: Verify all gating jobs succeeded + uses: re-actors/alls-green@release/v1 + with: + jobs: ${{ toJSON(needs) }} +``` + +`if: always()` + `alls-green` means a skipped/failed/cancelled gating job fails +the aggregator — a green `Pipeline Complete` truly means everything passed. +Canonical template: `ExtraToast/repo-template/.github/workflows/ci.yml`. + +## Consolidating personal-stack CI (issue #608) + +Today these run separately: +- `fast.yml` (push, non-main) — change-detection, lint-kotlin matrix, lint/typecheck frontend, unit tests. +- `full.yml` (PR) — integration, e2e, architecture, security, contracts. +- `contract-validate.yml`, `migration-guard.yml`, `vault-bootstrap-validate.yml` (PR). +- `system-tests.yml` (workflow_call), `nightly.yml`, `build-and-publish.yml`, `prod-smoke.yml`, `crac-train.yml`. + +Target: one `ci.yml` on `pull_request` (+ push to main) that runs the gating set +(keep the existing change-detection + matrix logic — fold jobs in rather than +rewrite) and ends in a `Pipeline Complete` job depending on all of them. Keep +nightly/build-publish/prod-smoke/crac-train as separate non-gating workflows. +Migrate by reference into reusable workflows from `ExtraToast/github-workflows` +once that repo exists (issue #612) — but the first cut can be in-repo. + +**Note:** workflow files cannot be pushed by the automation token until PR #620 +is merged + the App perms approved + the token re-minted. Until then, author the +workflow under `.github/workflows-pending/` and have a `workflow`-scoped actor +move it (the repo-template README documents the exact `git mv`). diff --git a/docs/handoff/shared-extraction/ENVIRONMENT-AND-BLOCKERS.md b/docs/handoff/shared-extraction/ENVIRONMENT-AND-BLOCKERS.md new file mode 100644 index 00000000..06040c52 --- /dev/null +++ b/docs/handoff/shared-extraction/ENVIRONMENT-AND-BLOCKERS.md @@ -0,0 +1,68 @@ +# Environment, credentials & blockers — READ BEFORE GITHUB WORK + +Hard-won facts about what the automation account can and cannot do here. + +## GitHub identity & capability + +Authenticated as the **`personal-stack-agents[bot]`** GitHub App (via `GH_TOKEN` +/ `gh auth token`; also a minting endpoint, see below). Empirically: + +| Can | Cannot | +|---|---| +| Create issues, milestones, labels | **Edit** issue bodies / **comment** on issues (GraphQL `updateIssue`/`addComment` + REST PATCH all 403) | +| Create & push branches, open PRs | Push `.github/workflows/*` (see token minter) | +| Push non-workflow file contents | Apply/modify rulesets (no `administration`) | +| | Assign issues/PRs to `ExtraToast` (`replaceActorsForAssignable` 403) | + +Implication: keep tracking status in `ISSUES.md` (this dossier), not by editing +GitHub issues. Cross-linking works only via `Part of #605` mentions in bodies at +creation time. + +## The token minter (root cause of the workflows block) + +`assistant-api` `GitHubAppInstallationTokenClient` mints short-lived, single-repo +installation tokens. It **requests a fixed permission set**, so the issued token +carries only that set even if the App is granted more. Originally +`contents`/`pull_requests`/`actions:write` → could not push workflows or edit +issues. + +- **Fix:** PR **#620** widens `REQUESTED_PERMISSIONS` to add `issues:write`, + `workflows:write`, `packages:read` (still no `administration`; `packages` + read-only — CI publishes). Setup doc updated. +- **Activation sequence (operator):** merge #620 → widen the App's repository + permissions on the settings page (Issues, Workflows: Read+write; Packages: + Read) → **approve the updated permissions on each installation** → drop the + cached token (`/tmp/.gh-app-token` in the runner Pod, or recreate the Pod) so + the next mint is wider. Deploy assistant-api for the minter change to take + effect. +- **Exact App permissions** (repository): Contents, Pull requests, Issues, + Workflows, Actions, Commit statuses = **Read & write**; Packages = **Read**; + Metadata = Read (mandatory). **Do NOT grant Administration** (rulesets are + operator-applied) or Packages **write** (release workflow's `GITHUB_TOKEN` + publishes). + +## Push auth gotcha + +The SSH remote uses a **read-only deploy key** — `git push` over SSH fails +("key is read only"). Push over **HTTPS** with the App token: +`git push https://x-access-token:$(gh auth token)@github.com/ExtraToast/.git HEAD:`. +**Scrub the token from the remote afterwards** (`git remote set-url origin `); +don't let it persist in `.git/config` or a tracking ref. + +## Tooling gotchas + +- **Codex workers cannot run gradle** (workspace-write sandbox blocks sockets) — + gradle-gated verification must be done by a non-sandboxed agent or in CI. Plan + fan-out accordingly. +- **Council** is configured all-Codex (`~/.claude/skills/council/council.toml`); + never run it with `claude:opus` — it drains credits fast. Use sonnet/haiku or + codex. +- `.council/runs/**` is **git-ignored**, so the council artifacts are copied into + this dossier (`council-*.md/json`) to preserve them. + +## repo-template state + +PR `ExtraToast/repo-template#1` is open. All standard files pushed; the CI +(`ci.yml`) + release (`release.yml`) workflows are **staged** under +`.github/workflows-pending/` with an activation README, because of the workflows +block above. Activate them (and personal-stack's) once the token is fixed. diff --git a/docs/handoff/shared-extraction/ISSUES.md b/docs/handoff/shared-extraction/ISSUES.md new file mode 100644 index 00000000..816d841f --- /dev/null +++ b/docs/handoff/shared-extraction/ISSUES.md @@ -0,0 +1,45 @@ +# Tracking issues & milestones (mirror) + +Created in `ExtraToast/personal-stack`. Mirrored here because the automation +token can **create** issues but cannot **edit or comment** them, so this file is +the maintainable source for status — update it in PRs, and update the GitHub +issues out-of-band (operator) when possible. + +## Milestones + +| # | Title | Scope | +|---|---|---| +| 2 | M0 Foundations & conventions | repo-template, ruleset, Pipeline Complete CI, tag→release versioning, version catalog + Renovate. Unblocks all. | +| 3 | M1 Shared build tooling | gradle-conventions + github-workflows; personal-stack adopts. | +| 4 | M2 Kotlin/Spring commons | kotlin-spring-commons (modular) + reconcile + adopt. | +| 5 | M3 Vue web commons | vue-web-commons (npm) + adopt. | +| 6 | M4 OpenAPI, agent-kit, Stalwart | openapi tooling + agent-kit + stalwart-provisioner + adopt. | +| 7 | M5 Platform blueprints | homelab-platform-blueprints + adopt. | +| 8 | M6 Deploy schema & authz (design) | deploy-config-schema + authz-model design. | + +## Epic + +- **#605** — Epic: extract shared code into ExtraToast repos + version-pinned deploys. Holds the scope decisions + cross-cutting requirements. All issues below carry `Part of #605`. + +## Tracking issues + +| # | Milestone | Title / objective | +|---|---|---| +| #606 | M0 | **Enable workflows permission end-to-end.** Minter must request `workflows`+`issues`(+`packages:read`); App perms approved; re-mint. **Addressed by PR #620.** | +| #607 | M0 | **repo-template + apply common ruleset to all repos.** Merge repo-template#1; operator applies ruleset (needs admin — operator-owned); confirm `Pipeline Complete` required everywhere. | +| #608 | M0 | **Consolidate personal-stack CI into one `Pipeline Complete` pipeline.** Fold fast/full/contract-validate/migration-guard/vault-bootstrap-validate into one PR-triggered `ci.yml` ending in a `Pipeline Complete` aggregator. Blocked by #606. | +| #609 | M0 | **tag→release + Flux version-pinning, drop Keel `:latest`.** release-please tags vX.Y.Z; version-tagged images; release PR bumps explicit image tags in `platform/cluster/flux/**`; remove `keel.sh/policy: force` + `:latest`. | +| #610 | M0 | **Central version catalog + Renovate.** `gradle/libs.versions.toml` (`dev.extratoast.*`) + pinned npm manifest + Renovate (group ExtraToast artifacts, pin, gate on Pipeline Complete). | +| #611 | M1 | **Extract gradle-conventions + adopt.** Plugins + stable IDs + publish; personal-stack swaps `includeBuild build-logic` for pinned plugin coords; GH Packages creds in pluginManagement + dependencyResolutionManagement. | +| #612 | M1 | **Extract github-workflows + adopt.** Move actions + reusable workflows (deltas → inputs); personal-stack consumes pinned refs; Pipeline Complete built from shared reusable workflows. | +| #613 | M2 | **Extract kotlin-spring-commons + reconcile + adopt.** Modular publish; reconcile website vault + typed CommandBus; account for the two CommandBusConfig copies + 3 direct result-handler call sites; Dockerfiles resolve packages instead of copying libs. | +| #614 | M3 | **Extract vue-web-commons + adopt.** Package properly (publishConfig, peerDeps); UIs swap workspace dep for `@extratoast/vue-web-commons`; update imports, theme CSS, pnpm workspace, Dockerfiles. | +| #615 | M4 | **OpenAPI: openapi-client-gradle + api-contract-checks + adopt.** Split external client gen from contract-drift checks; update export tasks, assistant-ui scripts, package.json, contract-validate. | +| #616 | M4 | **Extract agent-kit + adopt.** Build from platform/agents/kit + council; preserve renderer flags + knowledge-api installer-serving; CI fails on drift of .claude/.codex/.agents/installer resource/runner entrypoint. | +| #617 | M4 | **Extract stalwart-provisioner + adopt.** Move Dockerfile/apply/bootstrap/accounts/plan templates/schema/validate; v2 `passwordRef` schema; keep domain accounts + DNS local. | +| #618 | M5 | **Extract homelab-platform-blueprints + adopt.** Generic module/script subset as flake input; host data/secrets/jobs/render outputs stay local. | +| #619 | M6 | **Design deploy-config-schema + authz-model.** Design-only; run after platform boundaries proven. | + +## Consumer-rewiring conflict nodes (serialize per repo) + +`settings.gradle.kts`, `gradle.properties`, root `package.json`/lockfiles, Dockerfiles, workflows, and service build files are first-class conflict nodes — sequence consumer-rewiring PRs so they don't collide. diff --git a/docs/handoff/shared-extraction/NEXT-STEPS.md b/docs/handoff/shared-extraction/NEXT-STEPS.md new file mode 100644 index 00000000..80788493 --- /dev/null +++ b/docs/handoff/shared-extraction/NEXT-STEPS.md @@ -0,0 +1,48 @@ +# Next steps — resume here + +Ordered. M0 unblocks everything; do it first. Reference the tracking issue in +every PR; keep `ISSUES.md` current (GitHub issues can't be edited by the bot). + +## 0. Clear the workflows-token blocker (issue #606 — PR #620) + +1. Merge **PR #620** (minter requests `workflows`/`issues`/`packages:read`). + - ⚠️ #620 itself can't merge until `Pipeline Complete` exists OR an operator + merges it under the ruleset. Operator may need to merge #620 manually + (chicken-and-egg: the fix that enables CI can't pass the CI gate yet). +2. Operator widens App repository permissions + approves on each installation; + deploy assistant-api; drop the cached runner token (see ENVIRONMENT-AND-BLOCKERS.md). +3. Re-mint the session token; confirm a workflow push succeeds. + +## 1. Activate CI + ruleset (issues #607, #608) + +4. Move `repo-template`'s staged workflows into `.github/workflows/` + (`git mv .github/workflows-pending/*`), push, merge repo-template#1. +5. Build personal-stack's consolidated `ci.yml` ending in `Pipeline Complete` + (CI-PIPELINE-COMPLETE.md). Open PR → #608. Once green, personal-stack PRs can + merge normally. +6. (Operator) Confirm the ruleset's required check resolves on personal-stack + and is applied to the other ExtraToast repos. + +## 2. Versioning foundation (issues #610, #609) + +7. Add `gradle/libs.versions.toml` + pinned npm manifest + Renovate config (#610). +8. Implement tag→release (release-please), version-tagged images, Flux explicit + tag pinning, and remove Keel `:latest`/force on in-house images (#609). + See VERSIONING.md for exact files. + +## 3. Extractions, in dependency order (council DAG: `council-tasks.json`) + +9. **M1:** gradle-conventions (#611) → github-workflows (#612); personal-stack adopts (pinned plugin coords + reusable workflows). +10. **M2:** kotlin-spring-commons (#613) — reconcile divergences first; adopt per service module. +11. **M3:** vue-web-commons (#614). +12. **M4:** openapi tooling (#615), agent-kit (#616), stalwart-provisioner (#617). +13. **M5:** homelab-platform-blueprints (#618). +14. **M6:** design deploy-config-schema + authz-model (#619). + +## Working rules + +- **personal-stack only**; website is a later program. +- PRs: squash-only, linear history, impersonal commit/PR voice, **no Co-Authored-By / no "Claude" attribution** (repo convention in CLAUDE.md), assigned/labelled per CLAUDE.md. +- Each new ExtraToast repo is bootstrapped from `repo-template`; apply the common ruleset (operator) so `Pipeline Complete` gates it. +- Verify gradle changes by actually building (not in a codex sandbox) or via CI. +- Serialize consumer-rewiring PRs around the conflict-node files (ISSUES.md). diff --git a/docs/handoff/shared-extraction/PLAN.md b/docs/handoff/shared-extraction/PLAN.md new file mode 100644 index 00000000..007b47dc --- /dev/null +++ b/docs/handoff/shared-extraction/PLAN.md @@ -0,0 +1,35 @@ +# Extraction plan (synthesis) + +Full detail in `council-consolidated-plan.md`; this is the operator-decided framing. + +## Decisions (operator-approved) + +- **Order:** migrate **personal-stack first**; website later. +- **Distribution:** published artifacts — Maven (`dev.extratoast.*`) + npm (`@extratoast`) on GitHub Packages; reusable GitHub Actions via pinned `uses:` to `ExtraToast/github-workflows`; container images on `ghcr.io/extratoast`. +- **Ownership:** all shared repos under `ExtraToast`; personal-stack is the canonical source. Either codebase may be refactored to adopt the better pattern; pick the stronger implementation per concern. + +## Candidate repositories (classification) + +**Extract now — foundational** +- `gradle-conventions` — Gradle convention plugins as Maven plugin-marker artifacts. ⚠️ resolve version drift first (gradle.properties Kotlin 2.1.20 / Spring 4.0.4 / jOOQ 3.19.18 vs build-logic 2.3.21 / 4.0.6 / 3.21.4). jOOQ/OpenAPI/detekt-path/toolchain opt-in. Risk: plugin resolution precedes build. +- `github-workflows` — reusable composite actions (prepare-ci-host, setup-java-gradle, setup-node-*) + reusable workflows; path/host/image/pkg-manager deltas become inputs; setup-node supports pnpm + Yarn Berry. + +**Extract now — shared code & tooling** +- `kotlin-spring-commons` — modular Maven artifacts: command, events, blocks, web, exceptions, vault, vault-jwt, observability, timing-web, timing-jooq, crac, email, archunit-test. **Largest dedup win.** Reconcile website's diverged `vault` + typed CommandBus first; messaging deferred (hardcodes `personal-stack.events`/queue names); each Spring add-on owns its `AutoConfiguration.imports`. +- `vue-web-commons` — framework-neutral Vue utils/composables/JWT/Faro/form-toast-mutation as `@extratoast/vue-web-commons` (build output + dist + d.ts + peerDeps). No PrimeVue↔Vuetify convergence forced. `useAuth`/`useTheme` need auth-routes/role/CSRF/storage/theme injected. +- `openapi-client-gradle` + `api-contract-checks` — split external client generation (Gradle plugin) from internal contract-drift checks (scripts/CLI). +- `agent-kit` — renderer/templates/manifest + council; preserve `render-agent-kit.py` flags and the knowledge-api installer-serving model. Already installed into multiple repos. +- `stalwart-provisioner` — schema-driven mail provisioning (GHCR image + validation CLI), v2 schema with `passwordRef` union (env/file/Vault-VSO). + +**Extract cautiously — after design** +- `homelab-platform-blueprints` — reusable NixOS/k3s/Flux module subsets + generic bootstrap/validate scripts as a Nix flake input. Keep host data, secrets, app manifests, Nomad/Consul jobs, `fleet.yaml`, render outputs LOCAL. + +**Design-only for now** +- `deploy-config-schema` — JSON-schema-driven deploy/infra config from `fleet.yaml` + `platform/tooling` + render scripts; adapters generate Traefik/Gatus/Keel/Nomad inputs. +- `authz-model` — shared permission vocabulary (roles/claims/permissions schema + generated constants). NO auth services / OAuth clients / redirect URIs / domain evaluators merged. + +**Do NOT extract:** datastore/ORM (Postgres/jOOQ vs MariaDB/JPA), Nomad/Consul prod jobs, raw DNS zone contents, PrimeVue/Tailwind & Vuetify component systems (first release), personal-stack RabbitMQ event constants, project-specific system-tests, OAuth clients/redirect URIs, concrete permission assignments, association-specific roles. + +## Platform differences to RESPECT (keep pluggable, never force-merge) + +Nomad+Consul vs k3s+Flux · Postgres/jOOQ/Flyway vs MariaDB/JPA · PrimeVue+Tailwind vs Vuetify · pnpm vs Yarn Berry · Spring-Modulith-hexagonal+CommandBus vs DDD-layered. Commons must not hard-depend on jOOQ-only or PrimeVue-only types unless split into a `-jooq` / `-primevue` add-on artifact. diff --git a/docs/handoff/shared-extraction/README.md b/docs/handoff/shared-extraction/README.md new file mode 100644 index 00000000..545f4b9b --- /dev/null +++ b/docs/handoff/shared-extraction/README.md @@ -0,0 +1,33 @@ +# Shared-code extraction — handoff dossier + +This directory is a complete handoff for the program that extracts code shared +between **personal-stack** and **website** into standalone **ExtraToast** repos, +and re-bases personal-stack onto version-pinned dependencies + version-pinned +deploys. It exists so another agent/session can resume without re-discovering +anything. **personal-stack is migrated first; website comes later.** + +## Read in this order + +1. `council-brief.md` — the original problem statement + the operator's scope decisions. +2. `council-consolidated-plan.md` — the council's motivated repo list, classification, and ordering (the core research deliverable). +3. `council-tasks.json` — the 24-task / 6-wave extraction DAG (machine-readable). +4. `PLAN.md` — synthesis: repos, distribution, ownership, and how the operator's added requirements layer on top. +5. `ISSUES.md` — the epic + every tracking issue (numbers, objectives, work, deps, milestone) — mirrored here because the automation token cannot edit/comment issues. +6. `VERSIONING.md` — the tag→release + Flux-pinned deploy model and the dependency-pinning model. +7. `CI-PIPELINE-COMPLETE.md` — the one-pipeline-per-repo `Pipeline Complete` contract + how to consolidate personal-stack's CI. +8. `ENVIRONMENT-AND-BLOCKERS.md` — GitHub App permissions, the token minter, push auth, and tooling gotchas. **Read before doing GitHub work.** +9. `NEXT-STEPS.md` — the exact ordered actions to resume. + +## Status at handoff (2026-06-08) + +- ✅ Council research + plan complete (ran on Codex; $0 Claude cost). +- ✅ `ExtraToast/repo-template` built — PR `ExtraToast/repo-template#1` (CI/release workflows **staged** under `.github/workflows-pending/` pending the token fix). +- ✅ personal-stack plan-of-record: milestones **M0–M6** (#2–#8), epic **#605**, tracking issues **#606–#619**. +- ✅ Minter fix PR **#620** (`Closes #606`) — widens minted-token permissions to include `workflows`/`issues`/`packages:read`. +- ⛔ **Blocked until #620 merges + App perms approved + token re-minted:** pushing `.github/workflows/*` (so the `Pipeline Complete` pipeline can't be added yet, and personal-stack's ruleset already requires that check → no PR merges). + +## Org repos (all under ExtraToast, created, README-only except repo-template) + +repo-template · gradle-conventions · github-workflows · kotlin-spring-commons · +vue-web-commons · openapi-client-gradle · api-contract-checks · agent-kit · +stalwart-provisioner · platform-blueprints · deploy-config-schema · authz-model diff --git a/docs/handoff/shared-extraction/VERSIONING.md b/docs/handoff/shared-extraction/VERSIONING.md new file mode 100644 index 00000000..37c2e55c --- /dev/null +++ b/docs/handoff/shared-extraction/VERSIONING.md @@ -0,0 +1,36 @@ +# Versioning & deploy model (operator-approved) + +Replaces "main is the deploy" + Keel `:latest`. Two halves: how shared artifacts +are versioned/consumed, and how personal-stack itself is released/deployed. + +## Shared artifacts (each ExtraToast repo) + +- Released via **release-please**: squash-merged conventional-commit PRs → a release PR → merging it tags `vX.Y.Z`, writes `CHANGELOG.md`, bumps `.release-please-manifest.json`. +- The published-release event publishes the exact version: Maven → GitHub Packages `dev.extratoast.*`; npm → `@extratoast/*`; images → `ghcr.io/extratoast//:X.Y.Z`. +- Pre-1.0 uses minor as the breaking lever (`bump-minor-pre-major`). + +## How personal-stack consumes shared deps (exact pins, no ranges) + +- **Single source of truth:** `gradle/libs.versions.toml` for `dev.extratoast.*`; a pinned npm manifest for `@extratoast/*`. No `^`/`~`. +- **Renovate** opens exact-version bump PRs, grouping ExtraToast artifacts into one "platform bump"; every bump PR must pass `Pipeline Complete`. +- GitHub Actions / reusable workflows pinned to a release tag (digest via Renovate): `uses: ExtraToast/github-workflows/.github/workflows/.yml@vX.Y.Z`. + +## How personal-stack is released & deployed (deploy a specific version) + +Today: build-and-publish tags `:latest` + `:`; **Keel** polls `:latest` every 2 min and force-rolls; Flux reconciles `main`. Target model: + +1. A git tag `vX.Y.Z` (release-please) builds images tagged with the **version** (keep `:` for traceability). No `:latest` for in-house images. +2. A **release PR bumps the EXPLICIT image tags** in `platform/cluster/flux/**` to that version. +3. **Deploying a version = reconciling the commit that pins it.** **Rollback = `git revert`** of the bump. +4. **Remove** `keel.sh/policy: force` + `keel.sh/match-tag` + `:latest` from in-house Deployments (issue #609). (Third-party images like `linuxserver/*`, `mariadb`, `ollama` are out of scope.) + +This makes "which version is live" a reviewable, revertable fact in git, and makes deploying an arbitrary past version a one-line tag change. + +## Files/areas to touch (personal-stack) + +- `.github/workflows/build-and-publish.yml` — add version tag derived from the release tag; stop relying on `:latest` for in-house rollout. +- `platform/cluster/flux/apps/**/deployment*.yaml` + cronjobs/pods — replace `:latest` with the pinned version; drop Keel force annotations on in-house images (grep: `keel.sh/policy`, `ghcr.io/extratoast/personal-stack/*:latest`). +- New: release-please config + `gradle/libs.versions.toml` + Renovate config. +- `docs/` / runbook — document deploy = reconcile tag, rollback = revert. + +Canonical reference for the model lives in `ExtraToast/repo-template/VERSIONING.md`. diff --git a/docs/handoff/shared-extraction/council-brief.md b/docs/handoff/shared-extraction/council-brief.md new file mode 100644 index 00000000..38db97b6 --- /dev/null +++ b/docs/handoff/shared-extraction/council-brief.md @@ -0,0 +1,132 @@ +# Brief: Identify shared code to extract into standalone repositories + +## Context + +Two sibling codebases live side-by-side in this workspace and share a large +amount of structure, tooling, and intent. The goal is to **improve alignment** +between them, **stop re-inventing the wheel**, and let improvements to CI, +testing, setup, infra and tooling be made **once** and shared. + +- **personal-stack** (`ExtraToast/personal-stack`, working dir `.`) — the more + mature codebase. Self-hosted platform for jorisjonkers.dev. Kotlin 2.1 / + Spring Boot 4 / Java 21 hexagonal services, Vue 3 + PrimeVue + Tailwind + frontends, PostgreSQL (jOOQ + Flyway), Valkey, RabbitMQ, Vault. Nomad + + Consul orchestration in prod *and* a newer NixOS + k3s + Flux platform layer. + Rich `libs/kotlin-common`, `libs/vue-common`, `build-logic` convention + plugins, a full `platform/` (nix, cluster/flux, scripts, agents kit + council), + and a deep CI suite (10 workflows, 3 composite actions). +- **website / Blueshell** (`ESA-Blueshell/website`, sibling at + `../website` → absolute `/workspace/website`) — student-association platform. + Spring Boot 4 (Kotlin) + Vue 3 + **Vuetify** frontend, **MariaDB (JPA)**, + **Yarn Berry**, NixOS + **k3s + FluxCD** (Keel image automation, no CI deploy + step). Nearly-empty `libs/kotlin-common` (only `vault/`), `build-logic` + convention plugins, smaller `platform/` (nix, cluster/flux), `services/api` + with generated OpenAPI clients (discord, brevo) and `libs/openapi-specs`. + +Both projects are owned by the same author. **Decisions already made for this +run:** + +1. **Deliverable** = research + a ready-to-execute extraction plan. Do NOT move + any code in this run. Output a motivated list of candidate shared repos plus + a parallel task DAG for later execution. +2. **Distribution** = published artifacts. Kotlin/Spring commons + Gradle + build-logic → **Maven packages on GitHub Packages**; vue-common → **npm + package on GitHub Packages**. Reusable GitHub Actions → their own repo + consumed as versioned `uses:` refs. Infra/script/tooling repos consumed as + appropriate (template repo, CLI install, or pinned action). +3. **Ownership** = all extracted shared repos live under the **`ExtraToast`** + GitHub org. personal-stack is the canonical source; website is refactored to + consume the shared artifacts. Either codebase may be refactored to adopt the + better pattern of the other — pick the stronger implementation per concern. + +## Known overlaps already observed (verify + expand, don't take as exhaustive) + +- **Kotlin/Spring commons** — personal-stack `libs/kotlin-common` is rich + (command bus, domain events, observability/tracing, request timing filters, + RabbitMQ config, Vault secret provider + transit client, web exception + handling + ProblemDetail, ArchUnit hexagonal rules, CRaC, email). website's + `libs/kotlin-common` has only `vault/` and it has **diverged** + (`VaultTransitClient.kt` exists in both but differs; website adds + `VaultTransitJwtEncoder.kt`). Reconcile and extract the superset. +- **Vue commons** — personal-stack `libs/vue-common` (AppShell, form + components, useApi/useAuth/useToast/useTheme composables, Faro observability, + jwt utils). website uses Vuetify with no shared lib. Decide how much is + framework-agnostic vs PrimeVue/Vuetify-specific. +- **Gradle build-logic** — both have detekt/ktlint/kotlin/spring/testing/ + test-logging convention plugins (largely parallel). personal-stack adds + jooq-codegen; website adds openapi-client. Extract a shared convention-plugin + set / version catalog. +- **GitHub Actions** — both have composite actions `prepare-ci-host`, + `setup-java-gradle`, and `setup-node-pnpm`(personal) vs `setup-node-yarn` + (website), plus large reusable workflows. Extract reusable workflows + + actions. +- **Agent / council tooling** — personal-stack `platform/agents/kit` already + *renders and installs* the council, skills, and KB hooks into both repos from + templates (`render-agent-kit.py`, `manifest.yaml`, installer). This is + effectively a product already shipped into both — prime extraction candidate. +- **Stalwart mail provisioning** — `infra/stalwart` is near-identical in both + (Dockerfile, accounts.json, apply.sh, plan.ndjson.tmpl). Extract a shared + mail-provisioning module/tool. +- **DNS zones** — both keep a `infra/dns/*.zone` + management approach. +- **Platform / NixOS + k3s + Flux** — both have `platform/nix` (base, k3s, + roles modules) and `platform/cluster/flux`. personal-stack is far richer + (profiles, image building, more modules, render/restore/bootstrap scripts). + This is the "set up servers + scripts" surface the author wants shared. +- **OpenAPI / API clients / docs** — website has `libs/openapi-specs` + + generated clients (discord, brevo) + openapi generation scripts; personal- + stack has contract validation + generated clients. Shared OpenAPI tooling and + a "general API clients" approach is an explicit goal. +- **JSON-Schema-driven deploy/infra config** — the author explicitly wants a + *simple JSON-schema-based way to manage deployments and infra* shared between + both. This likely does not fully exist yet; assess what each has (Nomad job + templates + render scripts vs Flux manifests + render scripts) and propose the + shared schema + tooling repo. +- **Permission structures** — auth/permission model. personal-stack has a full + OAuth2/OIDC auth-api + Traefik forward-auth; website issues tokens via Spring + Authorization Server. Author wants *common permission structures* shared. +- **System tests / code-quality config** — both have `services/system-tests`, + shared detekt config, prettier/eslint, gitleaks. Assess shared test harness + + shared lint/quality config + CI validation of those tools. + +## Platform differences to RESPECT (do not force-merge) + +- Orchestration: personal-stack still runs **Nomad + Consul** in prod; website + is **k3s + Flux** only. (personal-stack also has a k3s nix layer — note the + convergence but don't assume it.) +- Datastore/ORM: **PostgreSQL + jOOQ + Flyway** vs **MariaDB + JPA**. +- Frontend UI kit: **PrimeVue + Tailwind** vs **Vuetify**. +- Package manager: **pnpm** vs **Yarn Berry**. +- Domain style: **Spring Modulith hexagonal + CommandBus** vs **DDD layered**. + +Shared repos must be designed so these differences stay pluggable (e.g. commons +must not hard-depend on jOOQ-only or PrimeVue-only types unless cleanly +separated into a `-jooq` / `-primevue` add-on artifact). + +## Your task + +1. **Inventory & verify** the genuinely shared (or trivially alignable) surface + across both repos. Read both — personal-stack at `.` and website at + `/workspace/website`. Distinguish: (a) identical/near-identical duplication, + (b) same concern, divergent implementation (pick the better one), + (c) exists in one, wanted in both, (d) aspirational/shared-goal-not-built-yet. +2. **Propose a concrete list of extraction-candidate repositories.** For each: + - proposed repo name (under `ExtraToast/…`) and one-line purpose; + - what moves into it (paths from each repo), and which implementation wins + when they diverge, with justification; + - distribution mechanism (Maven pkg / npm pkg / reusable-actions repo / + template repo / CLI) consistent with the decisions above; + - how each project consumes it and what changes in each consumer; + - **motivation**: concrete duplication removed / alignment gained / why it's + worth a separate repo vs leaving in place; + - effort (S/M/L), risk, and ordering/dependencies between extractions; + - explicit call-out if a concern should NOT be extracted (and why). +3. **Sequence it.** Produce a dependency-ordered DAG of extraction tasks + (foundational/low-risk first: build-logic, actions, kotlin-common; then + vue-common, infra modules, agent-kit; then the aspirational schema-driven + deploy + permissions work last as design tasks). Each task should be + independently executable by a worker later. + +## Definition of done + +`consolidated_plan.md` containing the motivated repo list + the per-repo detail +above, and `tasks.json` expressing the extraction DAG. No code moved. diff --git a/docs/handoff/shared-extraction/council-consolidated-plan.md b/docs/handoff/shared-extraction/council-consolidated-plan.md new file mode 100644 index 00000000..933d4ee3 --- /dev/null +++ b/docs/handoff/shared-extraction/council-consolidated-plan.md @@ -0,0 +1,129 @@ +# Consolidated Extraction Plan + +## Evidence and gates + +personal-stack is the canonical source and has verified local evidence for the key surfaces: root Gradle include-build wiring in settings.gradle.kts, stale version declarations in gradle.properties, build-logic convention plugins, libs/kotlin-common packages, private raw-source libs/vue-common packaging, Dockerfiles that copy build-logic and libs, OpenAPI scripts in package.json and service build files, agent-kit renderer and installer serving, Stalwart provisioning files, and coupled CI workflows. + +The website repo must be verified before any website-mutating work starts. Local ../website access is the hard gate for execution because the brief is about sibling repos and local dirty or generated state matters. GitHub MCP/default-branch reads are acceptable only as preliminary evidence and must be pinned by ref; they do not replace local verification for implementation winners or consumer rewiring. + +## Classification + +Extract now: Gradle conventions, GitHub actions/workflows, Kotlin/Spring commons, Vue web commons, agent kit, Stalwart provisioner, OpenAPI tooling split into client generation and contract checks. + +Extract cautiously after design: homelab platform blueprints. + +Design only for now: schema-driven deploy config and shared authorization model. + +Do not extract as raw shared code: datastore/ORM implementation, Nomad/Consul production jobs, raw DNS zone contents, PrimeVue/Tailwind or Vuetify component systems in the first Vue release, personal-stack RabbitMQ event constants, project-specific system-test suites, OAuth clients, redirect URIs, concrete service-permission assignments, and association-specific roles. + +## Candidate repositories + +### ExtraToast/gradle-conventions +Purpose: Maven-published Gradle convention plugins consumed from GitHub Packages. + +Move: personal-stack build-logic/src/main/kotlin and build-logic/build.gradle.kts patterns; website build-logic equivalents after local verification. personal-stack wins for maturity, but not as-is. The shared repo must publish plugin marker artifacts, define stable plugin IDs, add TestKit coverage, and make Java toolchain, detekt config path/default resource, integration-test model, Jacoco thresholds, jOOQ codegen, and OpenAPI-client generation opt-in. Resolve the stale version story before publishing: gradle.properties currently lists Kotlin 2.1.20, Spring Boot 4.0.4, and jOOQ 3.19.18 while build-logic uses Kotlin 2.3.21, Spring Boot 4.0.6, and jOOQ 3.21.4 with a 3.21.2 runtime comment. + +Distribution: Maven package on GitHub Packages under dev.extratoast. + +Consumption: both repos replace includeBuild build-logic with pluginManagement plugin coordinates, GitHub Packages credentials in pluginManagement.repositories, and GitHub Packages credentials in dependencyResolutionManagement.repositories. Keep a staged fallback until fresh-clone local dev, CI, and Docker builds resolve plugins. + +Motivation: removes parallel Gradle convention plugins and creates one place for testing, lint, Java/Kotlin, Spring, jOOQ, and OpenAPI build behavior. Effort M/L, risk high because plugin resolution happens before the build can run. + +### ExtraToast/github-workflows +Purpose: versioned reusable composite actions and small workflows. + +Move: .github/actions/prepare-ci-host, setup-java-gradle, setup-node-pnpm, website setup-node-yarn equivalents, and reusable workflow fragments only after all path, host, image, script, and package-manager differences become inputs. Do not lift personal-stack system-tests.yml wholesale until .github/scripts, infra/scripts/run-strict-command.sh, DNS setup, GHCR image names, hostnames, permissions, and callers full.yml and nightly.yml are parameterized. + +Distribution: standalone actions repo consumed through pinned uses refs and tags, ideally public or explicitly allowed cross-org. + +Consumption: both repos replace local composite actions gradually. setup-java-gradle must accept GitHub Packages credentials because Gradle will pull published plugins and libraries. setup-node must support pnpm and Yarn Berry. + +Motivation: removes duplicated CI setup and fixes action deltas once. Effort M, risk medium. + +### ExtraToast/kotlin-spring-commons +Purpose: Maven modules for reusable Kotlin/Spring code. + +Move: personal-stack libs/kotlin-common packages archunit, blocks, command, crac, email, event, exception, observability, timing, vault, web, and only generic messaging abstractions. Reconcile website libs/kotlin-common vault files and typed CommandBus locally before choosing APIs. Proposed modules: command, events, blocks, web, exceptions, vault, vault-jwt, observability, timing-web, timing-jooq, crac, email, archunit-test. ArchUnit is test-support only, not runtime core. Messaging is deferred or abstractions-only because RabbitMqConfig hardcodes personal-stack.events, auth.user.registered, and queue names. + +Implementation winners: personal-stack wins for breadth and Spring Boot auto-config maturity; website may win parts of typed CommandBus and Vault JWT after compatibility tests. CommandBus adoption must account for personal-stack's two CommandBusConfig copies and three currently direct result-handler call sites. + +Distribution: Maven packages on GitHub Packages under dev.extratoast. Each Spring add-on owns its own AutoConfiguration.imports metadata. + +Consumption: personal-stack replaces libs:kotlin-common dependencies per service; website replaces its small vault lib with shared vault/vault-jwt and adopts command APIs only after tests. Dockerfiles must stop copying workspace libs/kotlin-common and must be able to resolve packages inside build stages. + +Motivation: largest duplication/alignment win, especially command bus, Vault, web exceptions, timing, observability, and architecture tests. Effort XL, risk high. + +### ExtraToast/vue-web-commons +Purpose: npm package for framework-neutral Vue utilities and optionally the existing personal-stack component exports if personal-stack adopts without prior local component migration. + +Move: libs/vue-common utilities, composables, types, JWT helpers, Faro bootstrap, form/mutation/toast helpers, and tests. The current package is private, exports raw src/index.ts and SFC source, and treats Vue/Pinia/router/Faro as normal dependencies; it needs build output, dist exports, generated declarations, publishConfig, and peerDependencies. useAuth and useTheme are not generic until auth routes, role mapping, CSRF cookie, storage keys, and theme assumptions are injected. + +Distribution: npm package on GitHub Packages under @extratoast. + +Consumption: personal-stack UI packages replace @personal-stack/vue-common workspace dependency with @extratoast/vue-web-commons, update imports, theme CSS import, pnpm lock/workspace, and UI Dockerfiles. Website adoption waits for local evidence and Yarn Berry registry/PnP auth. + +Motivation: shared frontend auth/API/error/observability behavior without forcing PrimeVue/Tailwind or Vuetify convergence. Effort M/L, risk medium. + +### ExtraToast/openapi-client-gradle and ExtraToast/api-contract-checks +Purpose: split external client generation from internal service contract validation. + +Move: website-style external spec/client generation from libs/openapi-specs and openapi-client conventions after local verification into openapi-client-gradle. Move personal-stack assistant-api/knowledge-api export tasks, assistant-ui openapi-typescript contract check pattern, package scripts, contract-banner.mjs behavior, and contract-validate workflow logic into api-contract-checks. + +Distribution: Maven Gradle plugin for client generation; reusable scripts/actions or npm/CLI package for contract checks as appropriate. + +Consumption: personal-stack updates service export tasks, assistant-ui scripts, package.json, pnpm lock, and contract-validate workflow. Website updates external client conventions and generated-output policy. + +Motivation: avoids conflating Java external clients with TS contract drift checks while sharing the repeatable parts. Effort M, risk medium. + +### ExtraToast/agent-kit +Purpose: pinned renderer/installer kit for Claude and Codex parity. + +Move: platform/agents/kit renderer/templates/manifest behavior and platform/agents/council source, but preserve render-agent-kit.py --check, --write, --output, and --doctor. The extraction must include the knowledge-api installer serving model: InstallerController serves ClassPathResource installer/install.sh and substitutes @VERSION@ and @KB_URL@; platform/tooling tests assert manifest entries and /install.sh Flux routes. + +Distribution: pinned release tarball or CLI, with checked-in generated resource or build-time vendoring chosen explicitly. + +Consumption: personal-stack keeps CI that fails if .claude, .codex, .agents, services/knowledge-api installer resource, and agent-runner entrypoint drift from the same pinned kit. Website becomes a pure consumer after local verification. + +Motivation: this is already a product installed into multiple repos; extraction prevents agent tooling drift. Effort M/L, risk medium-high due runtime serving and parity tests. + +### ExtraToast/stalwart-provisioner +Purpose: schema-driven mail account/DKIM/provisioning tool. + +Move: infra/stalwart Dockerfile, apply.sh, bootstrap.sh, accounts.json, plan.ndjson.tmpl, plan.dev.ndjson, dev fixtures, schema.min.json, and validate-plan.py patterns; website equivalents after local verification. Use a v2 schema with passwordRef union supporting env vars, mounted files, and Vault/VSO-backed files. + +Distribution: GHCR image plus validation CLI. + +Consumption: both repos replace local provisioning logic with schema + pinned image/CLI while keeping domain-specific accounts and DNS local. + +Motivation: near-duplicate provisioning with meaningful credential-mode divergence; schema removes repeated shell logic. Effort M, risk medium. + +### ExtraToast/homelab-platform-blueprints +Purpose: reusable NixOS/k3s/Flux blueprints and bootstrap helpers. + +Move: only small reusable module subsets and generic bootstrap/validate scripts after local website verification. Keep personal-stack host data, secrets, app manifests, Nomad/Consul production jobs, platform/inventory/fleet.yaml, platform/scripts/render, and generated Flux outputs local until deploy-config-schema is ready. + +Distribution: Nix flake input plus template/CLI. + +Motivation: shares server setup without force-merging orchestration differences. Effort L, risk high. + +### ExtraToast/deploy-config-schema +Purpose: late design/build of JSON-schema-driven deploy/infra configuration. + +Move/design from personal-stack platform/inventory/fleet.yaml, platform/tooling, platform/scripts/render, and verified website Flux ingress/app samples. Generate Traefik, Gatus, catalog, Keel/image metadata, and later Nomad job inputs through adapters. + +Distribution: schema + CLI + tests. + +Motivation: explicit author goal, but aspirational and coupled. Effort L, risk high. Run after platform boundaries are proven. + +### ExtraToast/authz-model +Purpose: shared schema/generated constants/evaluator contracts for roles, claims, permissions, groups, and host gates. + +Move: no implementation initially. Study personal-stack ServicePermission, Role, AuthVerificationController, migrations, tests, and UI permission editing plus website auth locally. Do not merge auth services, OAuth client registrations, redirect URIs, or domain permission evaluators. + +Distribution: Maven/TypeScript generated constants only after design. + +Motivation: aligns permission vocabulary without baking one site's domain into the other. Effort L, risk high. + +## Ordering + +Hard gates first: local website inventory, package/action access, and container-stage auth. Then publish foundational shared repos: Gradle conventions and GitHub workflows. Then Kotlin/Spring commons, Vue commons, OpenAPI tooling, agent kit, and Stalwart. Consumer rewiring is serialized per repo wherever files overlap: settings.gradle.kts, gradle.properties, root package files, lockfiles, Dockerfiles, workflows, and service build files are first-class conflict nodes. Platform blueprints, deploy schema, and authz model come last. diff --git a/docs/handoff/shared-extraction/council-tasks.json b/docs/handoff/shared-extraction/council-tasks.json new file mode 100644 index 00000000..1999bcf4 --- /dev/null +++ b/docs/handoff/shared-extraction/council-tasks.json @@ -0,0 +1,486 @@ +[ + { + "id": "t00-verify-website-local-inventory", + "title": "Verify local website inventory gate", + "objective": "Establish local sibling access to ../website, inventory the website paths needed for extraction decisions, and write an evidence ledger that marks every website implementation winner as verified or blocked. Do not mutate either codebase beyond the ledger.", + "output_format": "docs/extraction/website-evidence-ledger.md with repo path, claim, evidence path, and status rows.", + "paths": [ + "docs/extraction/website-evidence-ledger.md" + ], + "depends_on": [], + "difficulty": "moderate", + "model": "haiku", + "verify": "test -f docs/extraction/website-evidence-ledger.md && test -d ../website && test -r ../website/settings.gradle.kts && test -r ../website/libs/kotlin-common/src/main/kotlin/net/blueshell/common/vault/VaultTransitClient.kt", + "boundaries": "Read-only for source code. Do not decide website winners from the task brief alone. If ../website is unavailable, leave the ledger explicit about the block and do not start dependent website tasks." + }, + { + "id": "t01-provision-packages-actions-and-container-auth", + "title": "Provision artifact, action, and Docker build auth", + "objective": "Document and implement the repository-side wiring needed before consumers can pull ExtraToast Maven, npm, and reusable-actions artifacts, including Docker build-stage credentials or a public-package decision.", + "output_format": "Updated CI/auth configuration plus docs/extraction/package-access.md explaining local, CI, cross-org, and container-stage access.", + "paths": [ + "docs/extraction/package-access.md", + ".github/workflows/build-and-publish.yml", + ".github/workflows/crac-train.yml", + ".npmrc" + ], + "depends_on": [ + "t00-verify-website-local-inventory" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -f docs/extraction/package-access.md && rg -q 'GitHub Packages' docs/extraction/package-access.md && rg -q 'BuildKit|public package|public packages' docs/extraction/package-access.md && test -f .github/workflows/build-and-publish.yml && test -f .github/workflows/crac-train.yml", + "boundaries": "Do not rewrite service dependencies or remove workspace libs. This task owns package/action access policy and Docker secret plumbing only; consumer import/build rewiring happens later." + }, + { + "id": "t02-create-gradle-conventions-repo", + "title": "Create ExtraToast gradle-conventions", + "objective": "Create the standalone Gradle convention plugin repository with plugin IDs, plugin-marker publication, TestKit tests, configurable Java/detekt/integrationTest/Jacoco behavior, and opt-in jOOQ and OpenAPI plugins.", + "output_format": "A sibling ../gradle-conventions repository ready to publish Maven plugin artifacts to GitHub Packages.", + "paths": [ + "../gradle-conventions/**" + ], + "depends_on": [ + "t00-verify-website-local-inventory" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -f ../gradle-conventions/settings.gradle.kts && test -f ../gradle-conventions/build.gradle.kts && test -d ../gradle-conventions/src/test && cd ../gradle-conventions && ./gradlew test publishToMavenLocal", + "boundaries": "Do not edit personal-stack or website consumers. Do not introduce a version catalog unless the task proves it is needed and documents separate adoption." + }, + { + "id": "t03-create-github-workflows-repo", + "title": "Create ExtraToast github-workflows", + "objective": "Create the reusable actions/workflows repository with parameterized prepare-host, setup-java-gradle, setup-node, and small workflow fragments, including pinned-version consumption guidance and cross-org access requirements.", + "output_format": "A sibling ../github-workflows repository with composite actions, reusable workflow docs, and self-tests where practical.", + "paths": [ + "../github-workflows/**" + ], + "depends_on": [ + "t00-verify-website-local-inventory" + ], + "difficulty": "moderate", + "model": "haiku", + "verify": "test -f ../github-workflows/README.md && test -d ../github-workflows/.github/actions && rg -q 'package-manager' ../github-workflows && rg -q 'uses:' ../github-workflows/README.md", + "boundaries": "Do not move personal-stack system-tests.yml unless all script paths, image names, hostnames, callers, permissions, and secrets are explicit inputs." + }, + { + "id": "t04-adopt-foundation-personal-stack", + "title": "Adopt Gradle conventions and workflow foundation in personal-stack", + "objective": "Wire personal-stack to consume published gradle-conventions and github-workflows artifacts, resolve stale gradle.properties version authority, configure pluginManagement and dependencyResolutionManagement GitHub Packages repositories, and replace local CI setup actions only where inputs preserve behavior.", + "output_format": "A personal-stack commit that updates Gradle settings/build logic and CI action references without touching application source.", + "paths": [ + "settings.gradle.kts", + "gradle.properties", + "build-logic/**", + ".github/actions/**", + ".github/workflows/fast.yml", + ".github/workflows/full.yml", + ".github/workflows/nightly.yml", + ".github/workflows/build-and-publish.yml", + ".github/workflows/crac-train.yml" + ], + "depends_on": [ + "t01-provision-packages-actions-and-container-auth", + "t02-create-gradle-conventions-repo", + "t03-create-github-workflows-repo" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "./gradlew :platform:tooling:test && git diff --check", + "boundaries": "This is the only personal-stack task in this wave allowed to edit root Gradle plugin resolution and shared CI setup files. Do not remove libs/kotlin-common or libs/vue-common yet." + }, + { + "id": "t05-adopt-foundation-website", + "title": "Adopt Gradle conventions and workflow foundation in website", + "objective": "Wire website to consume the shared Gradle conventions and reusable actions after local inventory verifies its current build-logic, Java baseline, package manager, and CI behavior.", + "output_format": "A website commit updating only foundation Gradle and CI setup files.", + "paths": [ + "../website/settings.gradle.kts", + "../website/gradle.properties", + "../website/build-logic/**", + "../website/.github/actions/**", + "../website/.github/workflows/**" + ], + "depends_on": [ + "t01-provision-packages-actions-and-container-auth", + "t02-create-gradle-conventions-repo", + "t03-create-github-workflows-repo" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -d ../website && cd ../website && ./gradlew test && git diff --check", + "boundaries": "Do not touch website service source, frontend packages, generated OpenAPI output, or infra. Keep website-specific Java and OpenAPI-client behavior configurable rather than forcing personal-stack defaults." + }, + { + "id": "t06-create-kotlin-spring-commons-repo", + "title": "Create ExtraToast kotlin-spring-commons", + "objective": "Create the Maven module family for command, events, blocks, web, exceptions, vault, vault-jwt, observability, timing add-ons, crac, email, and archunit-test. Reconcile Vault and CommandBus APIs with compatibility tests and keep RabbitMQ event constants out of shared runtime modules.", + "output_format": "A sibling ../kotlin-spring-commons repository with module coordinates, auto-configuration metadata per add-on, tests, and publishToMavenLocal support.", + "paths": [ + "../kotlin-spring-commons/**" + ], + "depends_on": [ + "t02-create-gradle-conventions-repo" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -f ../kotlin-spring-commons/settings.gradle.kts && rg -q 'archunit-test|archunit' ../kotlin-spring-commons && rg -q 'AutoConfiguration.imports' ../kotlin-spring-commons && cd ../kotlin-spring-commons && ./gradlew test publishToMavenLocal", + "boundaries": "Do not edit consumer repos. ArchUnit must be a test-support artifact. Messaging must be deferred or limited to generic abstractions and property-driven declarations." + }, + { + "id": "t07-create-vue-web-commons-repo", + "title": "Create ExtraToast vue-web-commons", + "objective": "Create the publishable npm package with dist exports, generated declarations, peer dependency policy, publishConfig, tests, and adapter-based auth/theme/API behavior.", + "output_format": "A sibling ../vue-web-commons package ready for npm publication to GitHub Packages.", + "paths": [ + "../vue-web-commons/**" + ], + "depends_on": [ + "t00-verify-website-local-inventory" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -f ../vue-web-commons/package.json && rg -q 'publishConfig' ../vue-web-commons/package.json && rg -q 'peerDependencies' ../vue-web-commons/package.json && cd ../vue-web-commons && npm test", + "boundaries": "Do not assume PrimeVue or Vuetify. Either retain existing personal-stack component exports in the package or require a separate consumer migration before import rewrites." + }, + { + "id": "t08-create-openapi-tooling-repos", + "title": "Create OpenAPI tooling repositories", + "objective": "Create ExtraToast/openapi-client-gradle for external client generation and ExtraToast/api-contract-checks for service contract export and TypeScript drift checks, keeping the two mechanisms separate.", + "output_format": "Sibling ../openapi-client-gradle and ../api-contract-checks repositories with tests and usage docs.", + "paths": [ + "../openapi-client-gradle/**", + "../api-contract-checks/**" + ], + "depends_on": [ + "t02-create-gradle-conventions-repo" + ], + "difficulty": "moderate", + "model": "haiku", + "verify": "test -f ../openapi-client-gradle/settings.gradle.kts && test -f ../api-contract-checks/README.md && cd ../openapi-client-gradle && ./gradlew test publishToMavenLocal && cd ../api-contract-checks && npm test", + "boundaries": "Do not edit package.json, service build files, generated clients, or contract workflows in either consumer. Generated-output policy is documented here and adopted later." + }, + { + "id": "t09-create-agent-kit-repo", + "title": "Create ExtraToast agent-kit", + "objective": "Create the standalone agent-kit repository preserving renderer commands, manifest semantics, Claude/Codex parity, council templates, installer rendering, and pinned-release consumption.", + "output_format": "A sibling ../agent-kit repository with renderer, templates, tests, release packaging docs, and check-mode support.", + "paths": [ + "../agent-kit/**" + ], + "depends_on": [ + "t00-verify-website-local-inventory" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -f ../agent-kit/render-agent-kit.py && ../agent-kit/render-agent-kit.py --check && ../agent-kit/render-agent-kit.py --doctor", + "boundaries": "Do not edit personal-stack generated installer resources or .claude/.codex mirrors here. The consumer task chooses checked-in resource, build-time vendoring, or remote endpoint." + }, + { + "id": "t10-create-stalwart-provisioner-repo", + "title": "Create ExtraToast stalwart-provisioner", + "objective": "Create the Stalwart provisioning image and validation CLI with a v2 account schema supporting env, file, and Vault/VSO-backed password references.", + "output_format": "A sibling ../stalwart-provisioner repository with schema, validator, Dockerfile, tests, and migration docs.", + "paths": [ + "../stalwart-provisioner/**" + ], + "depends_on": [ + "t00-verify-website-local-inventory" + ], + "difficulty": "moderate", + "model": "haiku", + "verify": "test -f ../stalwart-provisioner/schema/schema.json && test -f ../stalwart-provisioner/Dockerfile && cd ../stalwart-provisioner && ./scripts/validate-fixtures.sh", + "boundaries": "Do not change consumer accounts or deployment manifests. Keep raw account data and DNS zones local to each repo." + }, + { + "id": "t11-create-platform-blueprints-repo", + "title": "Create ExtraToast homelab-platform-blueprints", + "objective": "Create the reusable NixOS/k3s/Flux blueprint repository from verified generic modules and bootstrap helpers, without moving host data, app manifests, secrets, or Nomad/Consul jobs.", + "output_format": "A sibling ../homelab-platform-blueprints repository with a Nix flake, reusable modules, templates, and validation docs.", + "paths": [ + "../homelab-platform-blueprints/**" + ], + "depends_on": [ + "t00-verify-website-local-inventory" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -f ../homelab-platform-blueprints/flake.nix && cd ../homelab-platform-blueprints && nix flake check", + "boundaries": "Do not edit platform/scripts/render, platform/inventory/fleet.yaml, generated Flux output, or service-specific manifests in consumers." + }, + { + "id": "t12-adopt-kotlin-commons-personal-stack", + "title": "Adopt kotlin-spring-commons in personal-stack", + "objective": "Replace personal-stack's workspace-local libs:kotlin-common consumption with published kotlin-spring-commons modules service by service, update imports, keep Spring auto-config behavior, and update JVM Dockerfiles to stop copying workspace common code once package resolution works inside build stages.", + "output_format": "A personal-stack commit removing or shrinking libs/kotlin-common only after all consuming services compile and tests pass.", + "paths": [ + "libs/kotlin-common/**", + "settings.gradle.kts", + "services/auth-api/build.gradle.kts", + "services/assistant-api/build.gradle.kts", + "services/knowledge-api/build.gradle.kts", + "services/agent-gateway/build.gradle.kts", + "services/system-tests/build.gradle.kts", + "services/auth-api/Dockerfile", + "services/assistant-api/Dockerfile", + "services/knowledge-api/Dockerfile", + "services/agent-gateway/Dockerfile", + "services/agent-runner/Dockerfile", + "services/auth-api/src/**", + "services/assistant-api/src/**", + "services/knowledge-api/src/**", + "services/agent-gateway/src/**" + ], + "depends_on": [ + "t04-adopt-foundation-personal-stack", + "t06-create-kotlin-spring-commons-repo" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "./gradlew :services:auth-api:test :services:assistant-api:test :services:knowledge-api:test :services:agent-gateway:test && git diff --check", + "boundaries": "Do not edit frontend package files, OpenAPI generated clients, workflow reusable-action refs, agent-kit files, or platform inventory. Keep personal-stack RabbitMQ constants local unless genericized." + }, + { + "id": "t13-adopt-kotlin-and-openapi-website", + "title": "Adopt kotlin commons and OpenAPI tooling in website", + "objective": "After local verification, adopt kotlin-spring-commons and the split OpenAPI tooling in website, reconciling Vault, typed CommandBus, external specs, and generated-output policy without forcing jOOQ/Postgres or personal-stack domain style.", + "output_format": "A website commit updating backend shared libraries, OpenAPI client generation, and generated-output policy.", + "paths": [ + "../website/libs/kotlin-common/**", + "../website/libs/openapi-specs/**", + "../website/settings.gradle.kts", + "../website/services/api/build.gradle.kts", + "../website/services/api/src/**", + "../website/build-logic/**" + ], + "depends_on": [ + "t05-adopt-foundation-website", + "t06-create-kotlin-spring-commons-repo", + "t08-create-openapi-tooling-repos" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -d ../website && cd ../website && ./gradlew test && git diff --check", + "boundaries": "Do not touch website frontend Yarn files or visual components. Do not force jOOQ, Postgres, personal-stack RabbitMQ event names, or personal-stack hexagonal ArchUnit presets onto website." + }, + { + "id": "t14-adopt-vue-commons-personal-stack", + "title": "Adopt vue-web-commons in personal-stack", + "objective": "Publish and consume @extratoast/vue-web-commons in personal-stack UI packages, update package names/imports/theme CSS, pnpm workspace and lockfile, and UI Dockerfiles so installs no longer depend on workspace-local libs/vue-common.", + "output_format": "A personal-stack commit updating the Vue package wiring and all three UI consumers.", + "paths": [ + "libs/vue-common/**", + "package.json", + "pnpm-workspace.yaml", + "pnpm-lock.yaml", + "services/app-ui/package.json", + "services/auth-ui/package.json", + "services/assistant-ui/package.json", + "services/app-ui/Dockerfile", + "services/auth-ui/Dockerfile", + "services/assistant-ui/Dockerfile", + "services/app-ui/src/**", + "services/auth-ui/src/**", + "services/assistant-ui/src/**" + ], + "depends_on": [ + "t04-adopt-foundation-personal-stack", + "t07-create-vue-web-commons-repo" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "pnpm install --frozen-lockfile && pnpm --filter @personal-stack/app-ui typecheck && pnpm --filter @personal-stack/auth-ui typecheck && pnpm --filter @personal-stack/assistant-ui typecheck && git diff --check", + "boundaries": "Do not edit backend Gradle files or OpenAPI contract files except imports owned by UI consumers. If component exports are removed from the shared package, first migrate consumers to local components in this task." + }, + { + "id": "t15-adopt-vue-commons-website", + "title": "Adopt vue-web-commons in website", + "objective": "After local verification, wire website's Yarn Berry frontend to @extratoast/vue-web-commons using .yarnrc.yml scope registry/auth, PnP-compatible package metadata, and only framework-neutral APIs.", + "output_format": "A website commit updating frontend package manager files and framework-neutral imports.", + "paths": [ + "../website/package.json", + "../website/yarn.lock", + "../website/.yarnrc.yml", + "../website/services/**/package.json", + "../website/services/**/src/**" + ], + "depends_on": [ + "t13-adopt-kotlin-and-openapi-website", + "t07-create-vue-web-commons-repo" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -d ../website && cd ../website && yarn install --immutable && yarn test && git diff --check", + "boundaries": "Do not move Vuetify visual components into the shared package. Do not edit website backend Gradle or OpenAPI files in this task." + }, + { + "id": "t16-adopt-openapi-contracts-personal-stack", + "title": "Adopt OpenAPI contract checks in personal-stack", + "objective": "Replace local OpenAPI generation and contract drift checks with the split shared tooling, covering assistant-api, knowledge-api, assistant-ui generated types, root package scripts, and contract-validate workflow.", + "output_format": "A personal-stack commit with shared OpenAPI tooling wired and generated-output policy preserved.", + "paths": [ + "package.json", + "pnpm-lock.yaml", + ".github/workflows/contract-validate.yml", + "services/assistant-api/build.gradle.kts", + "services/assistant-api/CONTRACT.md", + "services/assistant-api/openapi.json", + "services/assistant-api/src/integrationTest/kotlin/com/jorisjonkers/personalstack/assistant/contract/**", + "services/assistant-ui/package.json", + "services/assistant-ui/scripts/contract-banner.mjs", + "services/assistant-ui/src/api/generated.ts", + "services/knowledge-api/build.gradle.kts", + "services/knowledge-api/openapi.json", + "services/knowledge-api/src/integrationTest/kotlin/com/jorisjonkers/personalstack/knowledge/contract/**" + ], + "depends_on": [ + "t12-adopt-kotlin-commons-personal-stack", + "t14-adopt-vue-commons-personal-stack", + "t08-create-openapi-tooling-repos" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "./gradlew :services:assistant-api:exportOpenApiSpec :services:knowledge-api:exportOpenApiSpec && pnpm --filter @personal-stack/assistant-ui contract:check && git diff --check", + "boundaries": "Do not edit unrelated UI features or backend application code. Keep generated files committed if that remains the documented policy." + }, + { + "id": "t17-adopt-agent-kit-personal-stack", + "title": "Adopt external agent-kit in personal-stack", + "objective": "Consume the pinned ExtraToast/agent-kit release while preserving same-branch Claude/Codex parity, generated installer resource behavior, InstallerController token substitution, Flux /install.sh routing, and platform/tooling tests.", + "output_format": "A personal-stack commit updating agent-kit source consumption and parity checks without breaking knowledge-api install.sh serving.", + "paths": [ + "platform/agents/kit/**", + "platform/agents/council/**", + ".agents/**", + ".claude/**", + ".codex/**", + "services/knowledge-api/src/main/resources/installer/install.sh", + "services/knowledge-api/src/main/kotlin/com/jorisjonkers/personalstack/knowledge/installer/InstallerController.kt", + "services/knowledge-api/src/integrationTest/kotlin/com/jorisjonkers/personalstack/knowledge/installer/InstallerControllerIntegrationTest.kt", + "platform/tooling/src/test/kotlin/com/jorisjonkers/personalstack/platform/AgentKitManifestTest.kt", + "platform/tooling/src/test/kotlin/com/jorisjonkers/personalstack/platform/PlatformKnowledgeApiFluxTest.kt" + ], + "depends_on": [ + "t04-adopt-foundation-personal-stack", + "t09-create-agent-kit-repo" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "platform/agents/kit/render-agent-kit.py --check && ./gradlew :platform:tooling:test :services:knowledge-api:integrationTest && git diff --check", + "boundaries": "Do not edit Gradle convention files, Vue packages, or Stalwart files. The task must fail if only Claude or only Codex generated output changes." + }, + { + "id": "t18-adopt-agent-kit-website", + "title": "Adopt external agent-kit in website", + "objective": "Install the pinned agent-kit in website after local verification, generating Claude, Codex, and .agents surfaces together and adding a parity check that uses the same pinned kit version.", + "output_format": "A website commit with generated agent surfaces and parity check wiring.", + "paths": [ + "../website/.agents/**", + "../website/.claude/**", + "../website/.codex/**", + "../website/platform/agents/**", + "../website/.github/workflows/**" + ], + "depends_on": [ + "t05-adopt-foundation-website", + "t09-create-agent-kit-repo" + ], + "difficulty": "moderate", + "model": "haiku", + "verify": "test -d ../website && cd ../website && test -d .agents && test -d .claude && test -d .codex && git diff --check", + "boundaries": "Do not edit website application code, Gradle files, or infra manifests except a narrowly scoped parity workflow if needed." + }, + { + "id": "t19-adopt-stalwart-provisioner-personal-stack", + "title": "Adopt Stalwart provisioner in personal-stack", + "objective": "Replace personal-stack's local Stalwart provisioning scripts/schema with the shared v2 schema, pinned image/CLI, and migrated account references while preserving dev plan validation, bootstrap idempotency, and no-runtime-download behavior.", + "output_format": "A personal-stack commit updating infra/stalwart to consume the shared provisioner.", + "paths": [ + "infra/stalwart/**" + ], + "depends_on": [ + "t10-create-stalwart-provisioner-repo" + ], + "difficulty": "moderate", + "model": "haiku", + "verify": "infra/stalwart/schema/validate-plan.py infra/stalwart/plan.dev.ndjson && git diff --check", + "boundaries": "Do not edit raw DNS zones, unrelated platform manifests, or mail account domain data beyond schema migration." + }, + { + "id": "t20-adopt-stalwart-provisioner-website", + "title": "Adopt Stalwart provisioner in website", + "objective": "After local verification, migrate website Stalwart provisioning to the shared v2 schema and pinned image/CLI, preserving its stronger hostname, DKIM, and Vault-backed credential behavior where verified.", + "output_format": "A website commit updating infra/stalwart only.", + "paths": [ + "../website/infra/stalwart/**" + ], + "depends_on": [ + "t10-create-stalwart-provisioner-repo" + ], + "difficulty": "moderate", + "model": "haiku", + "verify": "test -d ../website/infra/stalwart && cd ../website && git diff --check", + "boundaries": "Do not edit website DNS zones or platform app manifests unless the existing Stalwart deployment reference must be pointed at the pinned image." + }, + { + "id": "t21-adopt-platform-blueprints-personal-stack", + "title": "Adopt platform blueprints in personal-stack", + "objective": "Adopt the reusable platform blueprints only for generic NixOS/k3s/bootstrap pieces, keeping personal-stack host data, Flux app manifests, secrets, Nomad/Consul production jobs, and render outputs local.", + "output_format": "A personal-stack commit with a minimal flake/template/CLI consumption path and tests showing rendered platform output is unchanged where applicable.", + "paths": [ + "platform/nix/**", + "platform/cluster/flux/**", + "platform/inventory/**", + "platform/tooling/**", + "platform/scripts/**" + ], + "depends_on": [ + "t11-create-platform-blueprints-repo", + "t17-adopt-agent-kit-personal-stack", + "t19-adopt-stalwart-provisioner-personal-stack" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "./gradlew :platform:tooling:test && git diff --check", + "boundaries": "Do not force-merge Nomad and k3s. Do not move platform/scripts/render into this repo; that belongs to deploy-config-schema design unless explicitly separated." + }, + { + "id": "t22-adopt-platform-blueprints-website", + "title": "Adopt platform blueprints in website", + "objective": "After local verification, adopt only generic platform blueprints in website while preserving its k3s/Flux-only deployment model, Keel image automation, host data, app manifests, and secrets.", + "output_format": "A website commit with minimal platform blueprint consumption and unchanged rendered output where tests exist.", + "paths": [ + "../website/platform/nix/**", + "../website/platform/cluster/flux/**", + "../website/platform/scripts/**" + ], + "depends_on": [ + "t11-create-platform-blueprints-repo", + "t20-adopt-stalwart-provisioner-website" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -d ../website/platform && cd ../website && git diff --check", + "boundaries": "Do not introduce Nomad/Consul concepts into website. Do not edit website application services." + }, + { + "id": "t23-design-deploy-schema-and-authz-model", + "title": "Design deploy schema and authz model", + "objective": "Produce final design docs and proof-of-concept schemas for ExtraToast/deploy-config-schema and ExtraToast/authz-model, grounded in both repos after prior extraction boundaries are known.", + "output_format": "docs/extraction/deploy-config-schema.md, docs/extraction/authz-model.md, and optional schema drafts without consumer rewiring.", + "paths": [ + "docs/extraction/deploy-config-schema.md", + "docs/extraction/authz-model.md" + ], + "depends_on": [ + "t16-adopt-openapi-contracts-personal-stack", + "t21-adopt-platform-blueprints-personal-stack", + "t22-adopt-platform-blueprints-website" + ], + "difficulty": "hard", + "model": "sonnet", + "verify": "test -f docs/extraction/deploy-config-schema.md && test -f docs/extraction/authz-model.md && rg -q 'Non-goals' docs/extraction/deploy-config-schema.md && rg -q 'Non-goals' docs/extraction/authz-model.md", + "boundaries": "Design only. Do not edit auth services, OAuth clients, redirect URIs, permission evaluators, platform render scripts, generated Flux output, or raw DNS zones." + } +] \ No newline at end of file From 41f138d927f9273118d7f3e7510d8d9322a1d4a1 Mon Sep 17 00:00:00 2001 From: "personal-stack-agents[bot]" Date: Mon, 8 Jun 2026 11:09:58 +0000 Subject: [PATCH 2/4] docs: add repo-template contents + decisions to handoff dossier Makes the dossier self-contained for the operator-added repo-template: a file-by-file enumeration of what it contains, the 'what else to include' decisions, and how every repo bootstraps from it. README read-order updated. Part of #605. --- docs/handoff/shared-extraction/README.md | 1 + .../shared-extraction/REPO-TEMPLATE.md | 53 +++++++++++++++++++ 2 files changed, 54 insertions(+) create mode 100644 docs/handoff/shared-extraction/REPO-TEMPLATE.md diff --git a/docs/handoff/shared-extraction/README.md b/docs/handoff/shared-extraction/README.md index 545f4b9b..680a72cb 100644 --- a/docs/handoff/shared-extraction/README.md +++ b/docs/handoff/shared-extraction/README.md @@ -12,6 +12,7 @@ anything. **personal-stack is migrated first; website comes later.** 2. `council-consolidated-plan.md` — the council's motivated repo list, classification, and ordering (the core research deliverable). 3. `council-tasks.json` — the 24-task / 6-wave extraction DAG (machine-readable). 4. `PLAN.md` — synthesis: repos, distribution, ownership, and how the operator's added requirements layer on top. +4a. `REPO-TEMPLATE.md` — the operator-added `repo-template` repo: full contents + the "what else to include" decisions + how every repo uses it. 5. `ISSUES.md` — the epic + every tracking issue (numbers, objectives, work, deps, milestone) — mirrored here because the automation token cannot edit/comment issues. 6. `VERSIONING.md` — the tag→release + Flux-pinned deploy model and the dependency-pinning model. 7. `CI-PIPELINE-COMPLETE.md` — the one-pipeline-per-repo `Pipeline Complete` contract + how to consolidate personal-stack's CI. diff --git a/docs/handoff/shared-extraction/REPO-TEMPLATE.md b/docs/handoff/shared-extraction/REPO-TEMPLATE.md new file mode 100644 index 00000000..4e890945 --- /dev/null +++ b/docs/handoff/shared-extraction/REPO-TEMPLATE.md @@ -0,0 +1,53 @@ +# repo-template — the operator-added template repository + +`ExtraToast/repo-template` is the repo every ExtraToast repository is +bootstrapped from. It was added at the operator's request ("a template +repository for future repos that has important things such as rulesets, and new +PR and issue templates... consider what else to put into such a template repo"). +Built in PR `ExtraToast/repo-template#1`. This file mirrors its contents and the +decisions so the dossier is self-contained; the repo itself is the source of +truth. + +## What it contains + +| Path | Purpose | +|---|---| +| `.github/workflows/ci.yml` | The canonical single CI pipeline ending in the **`Pipeline Complete`** aggregator job (`if: always()` + `re-actors/alls-green` over `needs`). The only required check. *(staged in `.github/workflows-pending/` until the workflows-token fix; see ENVIRONMENT-AND-BLOCKERS.md)* | +| `.github/workflows/release.yml` | Tag→release via release-please; published-release event publishes the artifact (Maven/npm/image). *(also staged)* | +| `release-please-config.json`, `.release-please-manifest.json` | Versioning state for release-please. | +| `.github/rulesets/main.json` | The common branch ruleset **as code** — faithful to the operator's supplied ruleset: `deletion` + `non_fast_forward` + `required_linear_history`, pull_request (squash-only `allowed_merge_methods`, 0 approvals), and `required_status_checks` = **`Pipeline Complete`** (integration_id 15368). | +| `scripts/apply-ruleset.sh` | Idempotent apply of that ruleset to a repo (create-or-update). **Operator-run** — the agent token has no `administration`. | +| `.github/PULL_REQUEST_TEMPLATE.md` | PR template: What/Why, **Tracking** (`Closes #` / `Part of #`), Verification (incl. `Pipeline Complete` green + tracking issue updated), Versioning impact. Impersonal voice. | +| `.github/ISSUE_TEMPLATE/{bug_report,feature_request,task}.yml` + `config.yml` | Bug / feature / task forms; blank issues disabled; contact link to CONTRIBUTING. | +| `.github/CODEOWNERS` | `* @ExtraToast`. | +| `renovate.json` | Exact-pin (`rangeStrategy: pin`), groups `dev.extratoast.*` / `@extratoast/*` / `ExtraToast/github-workflows` into one platform bump, pins action digests, `minimumReleaseAge`. | +| `.github/dependabot.yml` | GitHub-Actions security-update fallback (Renovate is primary). | +| `SECURITY.md` | Private reporting; gitleaks in CI; secrets from Vault; latest-version-only support. | +| `CONTRIBUTING.md` | Branch/PR flow, squash-only + linear history, the one-pipeline/`Pipeline Complete` rule, versioning, commit/PR voice, tracking discipline. | +| `VERSIONING.md` | Canonical statement of the tag→release + exact-pin consumption + version-pinned deploy model (mirrored in this dossier's VERSIONING.md). | +| `docs/REPO_SETUP.md` | How to bootstrap a new repo from the template + a file-by-file table. | +| `.editorconfig`, `.gitignore`, `.gitleaks.toml`, `LICENSE`, `README.md` | Baseline hygiene + proprietary license matching personal-stack. | + +## "What else to put in it" — decisions made + +Included beyond the operator's explicit asks (rulesets, PR/issue templates): +release-please + manifest (versioning is an org requirement), Renovate (exact +pins), CODEOWNERS, SECURITY, CONTRIBUTING, VERSIONING, REPO_SETUP, gitleaks, +editorconfig, gitignore, LICENSE, and the ruleset-as-code + apply script. + +Deliberately **excluded**: language/stack-specific build files (each repo adds +its own — JVM/Gradle vs npm vs Nix differ), and any `administration`-requiring +automation (rulesets are operator-applied). The `ci.yml` jobs are placeholders +(`lint`/`test`/`build`) to be replaced per repo or wired to reusable workflows +from `ExtraToast/github-workflows` once that exists — but the `pipeline-complete` +aggregator must always remain and list every gating job in `needs:`. + +## How every repo uses it + +1. Create the repo from the template (`gh repo create … --template ExtraToast/repo-template`). +2. Operator runs `scripts/apply-ruleset.sh ` so `Pipeline Complete` is required. +3. Replace placeholder CI jobs with the real ones (keep the aggregator). +4. Set the release artifact + starting version. +5. Enable Renovate; adjust CODEOWNERS/README. + +Tracking: bootstrapping/ruleset rollout is issue **#607**. From 311b66deea2e7d18636ca47ec2f64c354a811691 Mon Sep 17 00:00:00 2001 From: "personal-stack-agents[bot]" Date: Mon, 8 Jun 2026 11:47:44 +0000 Subject: [PATCH 3/4] docs: correct handoff dossier after unblock #620 is merged and the minter now grants workflows/issues, so: workflow pushes work, repo-template#1 is merged, issues are editable with a per-repo token, and personal-stack already merges (full.yml already emits the Pipeline Complete job the ruleset requires). Records the narrowed remaining #608 scope (fold the three separate PR workflows into the aggregator) and that #610 waits on M1 artifacts. Part of #605. --- docs/handoff/shared-extraction/NEXT-STEPS.md | 16 ++++++++++++---- docs/handoff/shared-extraction/README.md | 10 +++++++++- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/docs/handoff/shared-extraction/NEXT-STEPS.md b/docs/handoff/shared-extraction/NEXT-STEPS.md index 80788493..2cee9201 100644 --- a/docs/handoff/shared-extraction/NEXT-STEPS.md +++ b/docs/handoff/shared-extraction/NEXT-STEPS.md @@ -1,11 +1,19 @@ # Next steps — resume here -Ordered. M0 unblocks everything; do it first. Reference the tracking issue in -every PR; keep `ISSUES.md` current (GitHub issues can't be edited by the bot). +> **2026-06-08 update — read README.md "STATUS UPDATE" first.** Step 0 below is +> DONE: #620 is merged, the minter grants workflows/issues, and +> `repo-template#1` is merged. Issues *can* be edited (mint a per-repo token). +> personal-stack already merges (full.yml has the `Pipeline Complete` job). +> Resume at the narrowed #608 (fold the 3 extra PR workflows into the +> aggregator), then #610 (after M1 publishes artifacts) and #609. -## 0. Clear the workflows-token blocker (issue #606 — PR #620) +Ordered. Reference the tracking issue in every PR; keep `ISSUES.md` current and, +where possible, update the GitHub issues too (editing works with a fresh +per-repo token). -1. Merge **PR #620** (minter requests `workflows`/`issues`/`packages:read`). +## 0. Clear the workflows-token blocker (issue #606 — PR #620) — ✅ DONE + +1. ~~Merge **PR #620**~~ (merged; minter now requests `workflows`/`issues`/`packages:read`). - ⚠️ #620 itself can't merge until `Pipeline Complete` exists OR an operator merges it under the ruleset. Operator may need to merge #620 manually (chicken-and-egg: the fix that enables CI can't pass the CI gate yet). diff --git a/docs/handoff/shared-extraction/README.md b/docs/handoff/shared-extraction/README.md index 680a72cb..7e026953 100644 --- a/docs/handoff/shared-extraction/README.md +++ b/docs/handoff/shared-extraction/README.md @@ -25,7 +25,15 @@ anything. **personal-stack is migrated first; website comes later.** - ✅ `ExtraToast/repo-template` built — PR `ExtraToast/repo-template#1` (CI/release workflows **staged** under `.github/workflows-pending/` pending the token fix). - ✅ personal-stack plan-of-record: milestones **M0–M6** (#2–#8), epic **#605**, tracking issues **#606–#619**. - ✅ Minter fix PR **#620** (`Closes #606`) — widens minted-token permissions to include `workflows`/`issues`/`packages:read`. -- ⛔ **Blocked until #620 merges + App perms approved + token re-minted:** pushing `.github/workflows/*` (so the `Pipeline Complete` pipeline can't be added yet, and personal-stack's ruleset already requires that check → no PR merges). +- ⛔ ~~Blocked until #620 merges + App perms approved + token re-minted~~ — **RESOLVED, see update below.** + +## STATUS UPDATE — 2026-06-08, post-unblock (supersedes the two corrections below) + +- ✅ **Workflows unblocked.** PR **#620 is merged** and assistant-api rolled it out; the minter now grants `workflows`/`issues`/`packages:read`. repo-template's workflows are activated and **`ExtraToast/repo-template#1` is merged** — `Pipeline Complete` ran green there. +- ✅ **Issue editing works** with a correctly-scoped fresh token (epic #605 checklist was edited via the API). **Correction:** earlier text saying "the token cannot edit/comment issues" was due to using a stale / wrong-repo-scoped token. **The minter issues SINGLE-REPO tokens** — mint one per target repo: `POST $GITHUB_APP_TOKEN_URL {"repoUrl":"https://github.com/ExtraToast/.git"}`. A token minted for repo A returns 403 on repo B. Prefer minting fresh per repo over the session `GH_TOKEN`. +- ✅ **personal-stack was NOT actually merge-blocked.** **Correction:** `full.yml` already contains a job named `Pipeline Complete` that aggregates its gating jobs, and the ruleset matches on JOB name — so the required check has existed all along (that is how #620 merged). Earlier "no PR merges" text compared workflow names, not job names, and was wrong. +- ▶ **Remaining #608 work** is therefore narrower: `contract-validate`, `migration-guard`, `vault-bootstrap-validate` are SEPARATE PR workflows that do NOT feed `full.yml`'s aggregator, so they don't gate merges. Fold them into the `Pipeline Complete` aggregator (convert to `workflow_call`, call from `full.yml` gated by `detect-changes`, add to the aggregator's `needs`) — a careful path-gating refactor, verify via CI. +- ▶ **#610 is partly premature:** there are no `dev.extratoast.*` / `@extratoast/*` artifacts to pin until M1 publishes them; only the catalog/Renovate scaffolding can be added now. ## Org repos (all under ExtraToast, created, README-only except repo-template) From 0bb9843e09ec3095f6992b4d29360dcad2636bc6 Mon Sep 17 00:00:00 2001 From: "personal-stack-agents[bot]" Date: Mon, 8 Jun 2026 12:46:41 +0000 Subject: [PATCH 4/4] docs: correct versioning/deploy model in handoff dossier MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit personal-stack stays continuously deployed (Flux + Keel auto-roll), not version-pinned. SemVer + Renovate apply to the consumed sub-packages (extracted libs/tooling, reusable workflows, future API/frontend pair repos — M7/#625). Rewrites VERSIONING.md and adds a model-correction note to README. Reflects the #623 revert (#624) and that Keel is kept. Part of #605. --- docs/handoff/shared-extraction/README.md | 4 + docs/handoff/shared-extraction/VERSIONING.md | 94 ++++++++++++-------- 2 files changed, 62 insertions(+), 36 deletions(-) diff --git a/docs/handoff/shared-extraction/README.md b/docs/handoff/shared-extraction/README.md index 7e026953..c0b2a245 100644 --- a/docs/handoff/shared-extraction/README.md +++ b/docs/handoff/shared-extraction/README.md @@ -27,6 +27,10 @@ anything. **personal-stack is migrated first; website comes later.** - ✅ Minter fix PR **#620** (`Closes #606`) — widens minted-token permissions to include `workflows`/`issues`/`packages:read`. - ⛔ ~~Blocked until #620 merges + App perms approved + token re-minted~~ — **RESOLVED, see update below.** +## MODEL CORRECTION — 2026-06-08 (versioning/deploy) — authoritative + +The operator corrected the deploy model: **personal-stack stays continuously deployed (Flux + Keel `:latest` auto-roll) and is NOT version-pinned.** SemVer versioning + Renovate pinning apply to the **sub-packages** it consumes — extracted libs/tooling, reusable workflows, and the future **API/frontend pair repos** (each pair split into its own versioned repo; milestone **M7**, issue **#625**). The earlier "tag→release, Flux-pinned, drop Keel" work (#623) was **reverted in #624**; **Keel is kept**; `renovate.json` is retained. See the rewritten **`VERSIONING.md`** for the authoritative model. M0 = #608 (done) + Renovate (kept); #609 reduces to the revert; #610 catalog adoption waits on M1. + ## STATUS UPDATE — 2026-06-08, post-unblock (supersedes the two corrections below) - ✅ **Workflows unblocked.** PR **#620 is merged** and assistant-api rolled it out; the minter now grants `workflows`/`issues`/`packages:read`. repo-template's workflows are activated and **`ExtraToast/repo-template#1` is merged** — `Pipeline Complete` ran green there. diff --git a/docs/handoff/shared-extraction/VERSIONING.md b/docs/handoff/shared-extraction/VERSIONING.md index 37c2e55c..569e1e51 100644 --- a/docs/handoff/shared-extraction/VERSIONING.md +++ b/docs/handoff/shared-extraction/VERSIONING.md @@ -1,36 +1,58 @@ -# Versioning & deploy model (operator-approved) - -Replaces "main is the deploy" + Keel `:latest`. Two halves: how shared artifacts -are versioned/consumed, and how personal-stack itself is released/deployed. - -## Shared artifacts (each ExtraToast repo) - -- Released via **release-please**: squash-merged conventional-commit PRs → a release PR → merging it tags `vX.Y.Z`, writes `CHANGELOG.md`, bumps `.release-please-manifest.json`. -- The published-release event publishes the exact version: Maven → GitHub Packages `dev.extratoast.*`; npm → `@extratoast/*`; images → `ghcr.io/extratoast//:X.Y.Z`. -- Pre-1.0 uses minor as the breaking lever (`bump-minor-pre-major`). - -## How personal-stack consumes shared deps (exact pins, no ranges) - -- **Single source of truth:** `gradle/libs.versions.toml` for `dev.extratoast.*`; a pinned npm manifest for `@extratoast/*`. No `^`/`~`. -- **Renovate** opens exact-version bump PRs, grouping ExtraToast artifacts into one "platform bump"; every bump PR must pass `Pipeline Complete`. -- GitHub Actions / reusable workflows pinned to a release tag (digest via Renovate): `uses: ExtraToast/github-workflows/.github/workflows/.yml@vX.Y.Z`. - -## How personal-stack is released & deployed (deploy a specific version) - -Today: build-and-publish tags `:latest` + `:`; **Keel** polls `:latest` every 2 min and force-rolls; Flux reconciles `main`. Target model: - -1. A git tag `vX.Y.Z` (release-please) builds images tagged with the **version** (keep `:` for traceability). No `:latest` for in-house images. -2. A **release PR bumps the EXPLICIT image tags** in `platform/cluster/flux/**` to that version. -3. **Deploying a version = reconciling the commit that pins it.** **Rollback = `git revert`** of the bump. -4. **Remove** `keel.sh/policy: force` + `keel.sh/match-tag` + `:latest` from in-house Deployments (issue #609). (Third-party images like `linuxserver/*`, `mariadb`, `ollama` are out of scope.) - -This makes "which version is live" a reviewable, revertable fact in git, and makes deploying an arbitrary past version a one-line tag change. - -## Files/areas to touch (personal-stack) - -- `.github/workflows/build-and-publish.yml` — add version tag derived from the release tag; stop relying on `:latest` for in-house rollout. -- `platform/cluster/flux/apps/**/deployment*.yaml` + cronjobs/pods — replace `:latest` with the pinned version; drop Keel force annotations on in-house images (grep: `keel.sh/policy`, `ghcr.io/extratoast/personal-stack/*:latest`). -- New: release-please config + `gradle/libs.versions.toml` + Renovate config. -- `docs/` / runbook — document deploy = reconcile tag, rollback = revert. - -Canonical reference for the model lives in `ExtraToast/repo-template/VERSIONING.md`. +# Versioning & deploy model (corrected 2026-06-08) + +> **This supersedes the earlier "tag → release, Flux-pinned, drop Keel" model.** +> personal-stack is **continuously deployed** and is NOT version-pinned. SemVer +> versioning applies to the **sub-packages** it consumes. + +## personal-stack (the platform) — continuous auto-deploy, unchanged + +- `main` is the deploy. Flux reconciles manifests from `main`; **Keel** auto-rolls + in-house `:latest` images. **Keep both.** +- personal-stack itself is **not** released, tagged, or version-pinned. No + release-please, no `vX.Y.Z` deploy, no Flux image-tag pinning, no manual + "reconcile a tag" deploy. (The earlier #623 attempt at this was reverted in + #624.) +- Deploys stay **automatic**: merge to `main` → image rebuild → Keel auto-roll. + +## Sub-packages — SemVer-versioned and consumed via Renovate + +The things personal-stack *depends on* are versioned; the platform pins and +upgrades them, and each upgrade flows through the normal auto-deploy. + +What gets versioned: +1. **Extracted shared libraries/tooling** — `gradle-conventions`, + `kotlin-spring-commons` (per-module), `vue-web-commons`, `openapi-client-gradle`, + `api-contract-checks`, `agent-kit`, `stalwart-provisioner`. Each is released + with release-please (`vX.Y.Z`) and published to GitHub Packages + (`dev.extratoast.*` Maven / `@extratoast/*` npm) or GHCR. +2. **Reusable GitHub workflows** — `ExtraToast/github-workflows`, pinned by + release tag + digest. +3. **API/frontend pair repos** (future, milestone **M7**, issue #625) — each + pair split into its own repo, independently versioned, consumed as a pinned + image version. + +How personal-stack consumes them (exact pins, no ranges): +- **Gradle**: versions only in `gradle/libs.versions.toml`. +- **npm**: exact pins in the manifest. +- **Actions/workflows**: pinned `uses: …@vX.Y.Z`. +- **Pair-repo images**: pinned image version in the Flux manifest. +- **Renovate** (`renovate.json`, kept) opens exact-version bump PRs (ExtraToast + artifacts grouped). A bump PR passes `Pipeline Complete`, merges, and the + normal Keel/Flux auto-roll deploys it. **Renovate is the upgrade mechanism; + deploy stays automatic.** + +## Where SemVer/release tooling lives + +Inside each **sub-package repo** (bootstrapped from `repo-template`, which carries +release-please + `ci.yml` + the ruleset). personal-stack only *consumes* — it +carries `renovate.json` and the version catalog, nothing else version-related. + +## Net effect of the correction + +| | Earlier (wrong) | Corrected | +| --- | --- | --- | +| personal-stack deploy | tag → Flux-pinned, manual | **continuous, Flux+Keel auto** | +| Keel | removed | **kept** | +| release-please on personal-stack | yes | **no** (reverted, #624) | +| Versioned units | personal-stack images | **sub-packages** (libs/tooling, workflows, pair repos) | +| Renovate | pin shared deps | **same — kept** |