testing maestro

Author: eluce2

Auto Run Docs/Initiation/2026-03-19-ProofKit-Release-Readiness/RELEASE-READINESS-01.md

@@ -0,0 +1,33 @@
+# Phase 01: AI-Friendly Docs — SEO, Sitemap & llms.txt Improvements
+
+This phase makes the ProofKit documentation site discoverable and consumable by both search engines and AI agents. By the end, the docs will have proper SEO metadata (sitemap, robots.txt, OpenGraph tags), improved llms.txt endpoints, and a working agent-detection middleware that serves markdown to LLM crawlers. This is the highest-priority work because better discoverability directly impacts how well AI assistants can help users build with ProofKit.
+
+## Tasks
+
+- [ ] Add SEO fundamentals to the Fumadocs site (`apps/docs/`):
+  - Create `apps/docs/src/app/sitemap.ts` using Next.js App Router's built-in sitemap generation. Import `source` from `@/lib/source` and iterate `source.getPages()` to produce URLs rooted at `https://proofkit.dev/docs/`. Include the llms.txt routes as well.
+  - Create `apps/docs/src/app/robots.ts` using Next.js App Router's built-in robots generation. Allow all crawlers, point to the sitemap URL, and explicitly allow `/llms.txt`, `/llms-full.txt`, and `/llms/` paths.
+  - Update `generateMetadata` in `apps/docs/src/app/docs/(docs)/[[...slug]]/page.tsx` to include OpenGraph metadata (`og:title`, `og:description`, `og:type: article`, `og:url`) using the page's title, description, and constructed URL. Use Next.js `Metadata` type's built-in `openGraph` field.
+  - Add a root-level `metadata` export in `apps/docs/src/app/layout.tsx` with `metadataBase` set to `https://proofkit.dev` so all relative OG URLs resolve correctly.
+
+- [ ] Improve the llms.txt endpoints for better agent consumption:
+  - In `apps/docs/src/app/llms.txt/route.ts`: fix the `"package"` field in `notify-intent.yml` — it still says `"with-changesets"` (template placeholder) instead of identifying the actual package. Note: this is in `.github/workflows/notify-intent.yml` line 59. Update the payload to dynamically determine the package name from the changed files, or use a static value like `"proofkit"`.
+  - In the same llms.txt route, add a `## Getting Started` section that tells agents: "To scaffold a new ProofKit project, run `pnpm create proofkit` or `npx create-proofkit@latest`. For package-specific usage, see the per-package links below."
+  - Add cache headers (`Cache-Control: public, max-age=3600, s-maxage=86400`) to all three llms.txt route responses so CDN can cache them.
+  - In `apps/docs/src/app/llms/[package]/route.ts`, ensure the response includes a header comment pointing to the main llms.txt for context.
+
+- [ ] Create agent-detection middleware for serving markdown to LLM crawlers:
+  - Create `apps/docs/src/middleware.ts` that detects common AI/LLM user agents (e.g., `GPTBot`, `ChatGPT-User`, `Claude-Web`, `Anthropic`, `CCBot`, `Google-Extended`, `PerplexityBot`, `Bytespider`, and generic patterns like `bot` + `ai` in user-agent).
+  - When an LLM agent is detected requesting a `/docs/` page, rewrite the request to the corresponding `/llms/` endpoint or serve the raw MDX content as plain text. The simplest approach: redirect detected agents to `/llms-full.txt` for root requests or `/llms/{package}` for package-specific requests by parsing the URL slug.
+  - For non-agent requests, pass through normally.
+  - Configure the middleware matcher in the export to only run on `/docs/:path*` routes to avoid affecting API routes or static assets.
+
+- [ ] Audit and improve llms.txt content accuracy:
+  - Read through each package's docs directory (`apps/docs/content/docs/{package}/`) and compare the descriptions in the llms.txt PACKAGES array against the actual content. Ensure package descriptions are accurate and helpful for an AI agent trying to understand what each package does.
+  - Check that `apps/docs/src/app/llms/[package]/route.ts` correctly maps all 7 packages and that the content returned for each is complete (not truncating or missing pages).
+  - Verify the `getLLMText()` function in `apps/docs/src/lib/get-llm-text.ts` properly strips frontmatter and produces clean markdown. Test edge cases: pages with code blocks, tables, links.
+
+- [ ] Build and verify the docs site compiles with all changes:
+  - Run `pnpm --filter @proofkit/docs build` to ensure no build errors from the new sitemap, robots, middleware, and metadata changes.
+  - Run `pnpm run ci` from the repo root to validate lint, typecheck, and tests all pass.
+  - If any issues arise, fix them before completing this task.

Auto Run Docs/Initiation/2026-03-19-ProofKit-Release-Readiness/RELEASE-READINESS-02.md

@@ -0,0 +1,42 @@
+# Phase 02: Doc-Staleness Detection & Cross-Linking
+
+This phase builds the automated doc-staleness detection system that runs during the release process. It also improves cross-linking between docs, SKILL.md files, and CLI help output so that AI agents and users can navigate between related resources. The goal is to ensure documentation never silently drifts from the actual package behavior.
+
+## Tasks
+
+- [ ] Build a doc-staleness detection script that compares package source changes against doc updates:
+  - Create `scripts/check-doc-staleness.ts` (or `.mjs`) at the repo root.
+  - The script should:
+    1. For each package in `packages/*/`, get the most recent commit that touched `src/` files
+    2. Get the most recent commit that touched the corresponding `apps/docs/content/docs/{package}/` directory
+    3. If source was modified more recently than docs, flag it as potentially stale
+    4. Also check if the package's SKILL.md was updated after the last source change
+    5. Output a structured report (JSON or markdown) listing stale packages with: package name, days since last source change, days since last doc update, list of changed source files
+  - Use `child_process.execSync` with `git log` commands — keep it simple, no external dependencies.
+  - Add a `"check:docs"` script to root `package.json` that runs this script.
+  - The script should exit with code 0 (always succeed) but print warnings prominently. It's advisory, not blocking.
+
+- [ ] Integrate doc-staleness check into the release workflow:
+  - In `.github/workflows/release.yml`, add a new job called `doc-review` that runs after the test jobs but before the release job.
+  - The job should: checkout, setup Node, install deps, run `pnpm check:docs`.
+  - Capture the output and post it as a PR comment (using `actions/github-script`) on the changeset version PR if one exists. This way maintainers see the staleness report before merging the release.
+  - Make this job `continue-on-error: true` so it never blocks the release.
+
+- [ ] Fix the `notify-intent.yml` placeholder issue:
+  - In `.github/workflows/notify-intent.yml` line 59, the `"package"` field is `"with-changesets"` (a template placeholder from `intent setup`).
+  - Change it to `"proofkit"` or dynamically determine it from the changed files path (e.g., extract the package name from the first changed file matching `packages/*/`).
+  - Also update the comment block at the top of the file (lines 11-13) to remove the template variable references.
+
+- [ ] Improve cross-linking between docs, skills, and CLI output:
+  - In each package's main doc page (`apps/docs/content/docs/{package}/index.mdx`), add a "For AI Agents" callout or section at the bottom that points to the package's SKILL.md file location and the llms.txt per-package endpoint. Use a Fumadocs `Callout` component if available, or a simple blockquote.
+  - In each SKILL.md file's `sources` frontmatter field, verify the doc URLs point to the correct pages on proofkit.dev. Update any that are wrong or missing.
+  - In the CLI's help output (check `packages/cli/src/` for the main command definitions), ensure the `--help` text mentions `https://proofkit.dev/docs/cli` for full documentation.
+
+- [ ] Write tests for the doc-staleness script:
+  - Create `scripts/__tests__/check-doc-staleness.test.ts` (or co-locate with the script).
+  - Test the core logic: parsing git log output, comparing dates, generating the report structure.
+  - Mock `execSync` to provide controlled git log output rather than depending on actual git history.
+  - Run `pnpm run ci` to confirm everything passes.
+
+- [ ] Run full CI checks:
+  - Run `pnpm run ci` from the repo root to validate lint, typecheck, and tests all pass with the new script, workflow changes, and cross-linking updates.

Auto Run Docs/Initiation/2026-03-19-ProofKit-Release-Readiness/RELEASE-READINESS-03.md

@@ -0,0 +1,37 @@
+# Phase 03: CLI Test Hardening & Agent-Friendly Output
+
+This phase speeds up the `proofkit init` test suite, prepares the test infrastructure for upcoming new templates, and ensures CLI output includes guidance that helps AI agents use ProofKit correctly. The CLI is the primary entry point for new users (many of whom are non-developers working with AI assistants), so its output quality directly impacts the AI-assisted experience.
+
+## Tasks
+
+- [ ] Profile the existing CLI test suite to identify bottlenecks:
+  - Run `pnpm --filter @proofkit/cli test` with timing output (e.g., `time` prefix or Vitest's `--reporter=verbose`) and record how long each test file takes.
+  - Check `packages/cli/vitest.config.ts` — note that `fileParallelism: false` is currently set. Investigate which tests actually need sequential execution (shared state, filesystem conflicts) vs. which could safely run in parallel.
+  - Look at the test-layer.ts and init-fixtures.ts helpers — check if tests share any mutable state that would prevent parallelization.
+  - Document findings as comments in the vitest config file explaining why parallelism is disabled (if truly needed) or enable it if tests are safe to parallelize.
+
+- [ ] Optimize test execution speed:
+  - In test files that create filesystem artifacts (`executor.test.ts`, `integration.test.ts`, `init-scaffold-contract.test.ts`), ensure each test uses a unique temp directory (e.g., via `crypto.randomUUID()` or `Date.now()` suffix) so they don't conflict when run in parallel.
+  - If tests are currently writing to a shared output directory, refactor to use isolated directories per test.
+  - Try enabling `fileParallelism: true` in `vitest.config.ts` after ensuring isolation. If specific test files still need sequential execution, use Vitest's `test.sequential` or split them into a separate config.
+  - Check if the `pnpm build` step in the test script (`"test": "pnpm build && vitest run"`) can be cached or skipped when source hasn't changed. Consider using `turbo` for caching the build step.
+
+- [ ] Prepare test infrastructure for new templates:
+  - Review the existing template test pattern in `init-scaffold-contract.test.ts` to understand how templates are validated (deterministic output checks).
+  - Create a reusable test helper function (e.g., `createTemplateTestSuite(templateName, options)`) in `packages/cli/tests/test-utils.ts` or a new `template-test-helpers.ts` that:
+    1. Sets up an isolated temp directory
+    2. Runs the init scaffold for the given template
+    3. Reads and returns the generated artifacts (package.json, config files, etc.)
+    4. Provides assertion helpers for common checks (has correct dependencies, has correct scripts, config matches expected shape)
+  - Refactor existing template tests to use this helper where it reduces duplication. Don't force it if the existing tests are already clean.
+
+- [ ] Make CLI output more AI-agent-friendly:
+  - In the post-init success message (find in `packages/cli/src/` — likely in the init command handler or executor), add a line pointing to docs: "Full documentation: https://proofkit.dev/docs"
+  - If there's a `--help` flag handler for the main `proofkit` command, ensure it includes the docs URL.
+  - In error messages from the CLI, ensure they include enough context for an AI agent to diagnose the issue (e.g., "Missing required field X. See https://proofkit.dev/docs/cli/reference for options.").
+  - Review the AGENTS.md at the repo root — ensure it mentions the CLI and how to use it for project scaffolding.
+
+- [ ] Run all tests and verify improvements:
+  - Run `pnpm --filter @proofkit/cli test` and compare timing to the baseline recorded in the profiling task.
+  - Run `pnpm run ci` from the repo root to validate everything passes.
+  - If any test failures occur, fix them before completing this task.

Auto Run Docs/Initiation/2026-03-19-ProofKit-Release-Readiness/RELEASE-READINESS-04.md

@@ -0,0 +1,52 @@
+# Phase 04: Release Hardening & Beta Exit Preparation
+
+This phase hardens the CI/CD pipeline, validates the full release process end-to-end, and prepares for exiting beta prerelease mode. By the end, the release pipeline will be validated and ready for the stable release cut. The actual `changeset pre exit` command should be run manually by a maintainer when ready — this phase prepares everything needed for that moment.
+
+## Tasks
+
+- [ ] Audit and fix CI workflow issues:
+  - Review `.github/workflows/release.yml` for any gaps or issues:
+    - Verify all job dependencies are correct (lint → typecheck → test → smoke → release)
+    - Ensure Node version is consistent across jobs (some use 22, release uses 24 — verify this is intentional)
+    - Check that the `changesets/action@v1` step has correct configuration for the beta prerelease mode
+  - Review `.github/workflows/continuous-release.yml` (PR checks):
+    - Verify the `pkg-pr-new publish` step works correctly for preview packages
+    - Ensure all parallel jobs (lint, typecheck, test, build) have consistent Node/pnpm versions
+  - Check that Doppler OIDC authentication is properly configured for smoke tests (`.github/workflows/release.yml` smoke test job). Don't modify secrets, just verify the workflow references are correct.
+
+- [ ] Validate the changeset configuration for stable release:
+  - Read `.changeset/config.json` and verify settings are correct for public release:
+    - `access: "public"` is set
+    - `baseBranch: "main"` is correct
+    - `ignore` list only contains packages that should not be published (`@proofkit/docs`, `@proofkit/typegen-web`)
+    - `updateInternalDependencies: "patch"` is appropriate
+  - Read `.changeset/pre.json` and document the current beta state: which packages have beta versions, how many changesets are queued.
+  - Create a checklist file at `docs/release/beta-exit-checklist.md` with structured markdown (front matter: type: reference, tags: [release, beta, checklist]) documenting:
+    1. Pre-exit steps (run full CI, verify all tests pass, review doc staleness report)
+    2. The exit command: `pnpm changeset pre exit`
+    3. Post-exit steps (run `pnpm changeset version`, review generated changelogs, commit, push, let CI release)
+    4. Rollback plan if something goes wrong
+    5. List of packages and their current beta versions vs expected stable versions
+
+- [ ] Ensure all packages build cleanly for release:
+  - Run `pnpm build` from the repo root and verify all packages compile without errors.
+  - Run `pnpm --filter @proofkit/registry build` separately since it's private — verify it still builds even if it might be removed later.
+  - Check that each package's `package.json` has correct `exports`, `main`, `types`, and `files` fields for npm publishing.
+  - Run `pnpm dlx publint` on each package (or check if it's already part of the build pipeline) to validate package entry points.
+
+- [ ] Run the complete CI pipeline locally to simulate a release:
+  - Run `pnpm run ci` (lint + typecheck + test) from the repo root.
+  - Run `pnpm build` to verify build succeeds.
+  - Run `pnpm changeset status` to see the current changeset queue and verify it reports correctly.
+  - If smoke tests can be run locally (check if Doppler secrets are available), run `pnpm --filter @proofkit/cli test:smoke`. If not, note this as a CI-only gate.
+  - Fix any issues found during this validation.
+
+- [ ] Create a release runbook document:
+  - Create `docs/release/release-runbook.md` with structured markdown front matter (type: reference, tags: [release, process, runbook]) documenting:
+    1. The complete release process from changeset creation to npm publish
+    2. How the CI pipeline validates each step
+    3. How to handle common failure scenarios (failed publish, partial release, reverted changeset)
+    4. How the doc-staleness check (from Phase 02) integrates into the process
+    5. How skill staleness checks (notify-intent → check-skills) fit into the release cycle
+    6. Contact/escalation info if the release pipeline breaks
+  - Use `[[Beta-Exit-Checklist]]` wiki-link to cross-reference the checklist from the previous task.