-
Notifications
You must be signed in to change notification settings - Fork 4
feat(cli): cli overhaul #178
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sarahxsanders
wants to merge
23
commits into
main
Choose a base branch
from
feat/cli-overhaul
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
a952bd4
feat(cli): document cli block schema (wizard CLI overhaul phase 0)
sarahxsanders c76bfb2
feat(cli): parse `cli:` blocks and emit cli-manifest.json
sarahxsanders 0bfcd8e
refactor(cli): rename group/leaf to command/parentCommand
sarahxsanders 53b8520
chore(audit): retire audit-3000 skill
sarahxsanders 4850b99
docs: add CONTRIBUTING.md covering the cli: block schema
sarahxsanders 2538801
feat(cli): naming convention enforcement + JSON Schema for drift prev…
sarahxsanders dbe9848
docs: add CLAUDE.md for agent-facing repo orientation
sarahxsanders 08dda26
feat(cli): add `default` field for family no-leaf behavior
sarahxsanders 89890d9
docs: flag wizard docs/cli.md regen in the cli: block workflow
sarahxsanders c814e36
refactor(cli): revert revenue + migrate to flat commands
sarahxsanders 9b654de
docs: align cli: block docs with new default + flat-vs-family semantics
sarahxsanders 3b8b572
refactor(cli): use full product names — no shorthand
sarahxsanders 5a2d53d
fix(cli): close validation gaps in cli: block handling
sarahxsanders 21687b0
refactor(cli): rename cli block fields to role/recommended
sarahxsanders 5f3aadd
Merge remote-tracking branch 'origin/main' into feat/cli-overhaul
sarahxsanders 8c1d942
feat(cli): self-validate cli-manifest against published schema
sarahxsanders f870f3e
Merge remote-tracking branch 'origin/main' into feat/cli-overhaul
sarahxsanders aa900b4
docs(cli): note that cli-manifest version is shared with the main man…
sarahxsanders 778bd7b
feat(cli): make events the recommended audit leaf
sarahxsanders 01a60f5
refactor(cli): drop context-mill self-validation; validate in the wizard
sarahxsanders 839eda9
docs(cli): drop cli-manifest schema; document flat->family migration
sarahxsanders 282c9a4
feat(cli): publish cliEntries inside skill-menu.json
sarahxsanders 549cfc5
refactor(cli): drop cli-manifest.json emit; cliEntries lives in skill…
sarahxsanders File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| # context-mill | ||
|
|
||
| This repo packages PostHog's developer content (docs, prompts, example | ||
| code) into [Agent Skills](https://agentskills.io/specification)-compliant | ||
| skill packages. The build pipeline emits a versioned manifest plus per-skill | ||
| ZIPs, consumed by the PostHog [wizard](https://github.com/PostHog/wizard) | ||
| and the PostHog [MCP server](https://github.com/PostHog/posthog/tree/master/products/mcp). | ||
|
|
||
| User-facing intro: [README.md](README.md). Contributor handbook: | ||
| [CONTRIBUTING.md](CONTRIBUTING.md). | ||
|
|
||
| ## What lives where | ||
|
|
||
| | Concern | Where | | ||
| |---|---| | ||
| | Skill source content | `context/skills/<name>/` | | ||
| | Skill descriptor / CLI role declarations | `context/skills/<name>/config.yaml` | | ||
| | Build pipeline | `scripts/lib/` (skill generator, build phases, change router) | | ||
| | Build entrypoints | `scripts/build.js` (full) and `scripts/dev-server.js` (partial / watch) | | ||
| | Tests | `scripts/lib/tests/` and `scripts/plugins/tests/` (vitest) | | ||
| | Manifest output | `dist/skills/manifest.json`, `dist/skills/skill-menu.json` (CLI entries live under `cliEntries`) | | ||
| | Per-skill ZIPs | `dist/skills/<id>.zip` | | ||
|
|
||
| ## The `cli:` block (read [CONTRIBUTING.md](CONTRIBUTING.md) before editing) | ||
|
|
||
| Every skill's `config.yaml` may declare an optional `cli:` block that tells | ||
| the wizard whether and how to expose the skill as a CLI command. The full | ||
| schema, the YAML→command mapping table, and the promotion criterion for | ||
| `role: command` live in [CONTRIBUTING.md](CONTRIBUTING.md). Quick shape: | ||
|
|
||
| ```yaml | ||
| cli: | ||
| role: command # command | skill | internal | ||
| parentCommand: audit # optional — nests this command under another | ||
| command: events # the user-typed word; required when role is command | ||
| ``` | ||
|
|
||
| The parser is `parseCliBlock` in `scripts/lib/skill-generator.js`. It | ||
| enforces: | ||
|
|
||
| - `role` is one of `command`, `skill`, `internal` (default: `skill` | ||
| if no `cli:` block is set at all) | ||
| - `command` and `parentCommand` are kebab-case, 2–20 characters | ||
| - Neither field is a yargs reserved word (`help`, `version`, `completion`) | ||
| or a wizard internal flag (`playground`, `benchmark`, `yara-report`, | ||
| `local-mcp`, `ci`, `skill`) | ||
| - `recommended` (optional, boolean) marks a leaf as pre-highlighted in the | ||
| family picker — `wizard <family>` → Enter runs the marked leaf. Picker | ||
| still opens (discovery + consent); the recommended leaf just sorts that | ||
| option first so a single Enter runs it. | ||
|
|
||
| Failures throw at build time, before drift can ship to the wizard. | ||
|
|
||
| **Flat vs. family rule:** a public command is flat when there's only one | ||
| option today, a family when the user must pick. Don't pre-create | ||
| `wizard migrate <vendor>` while there's only one vendor — that's forced | ||
| abstraction. Restructure to a family when a second vendor lands. See | ||
| [CONTRIBUTING.md § Flat vs. family](CONTRIBUTING.md#flat-vs-family--the-convention). | ||
|
|
||
| ## When you're about to change a `cli:` block | ||
|
|
||
| 1. Read [CONTRIBUTING.md § Promotion criterion for `role: command`](CONTRIBUTING.md#promotion-criterion-for-role-command). | ||
| 2. Run `npm test` — the parser's test suite (`scripts/lib/tests/cli-block.test.js`) | ||
| covers every naming-convention case. | ||
| 3. Run `npm run build` — confirm the entry appears (or disappears) under | ||
| `cliEntries` inside `dist/skills/skill-menu.json` with the values you | ||
| expect. | ||
| 4. The wizard resolves new entries at runtime, so no wizard release is | ||
| required unless the change needs wizard-side hooks (custom outro, | ||
| content blocks, abort cases). | ||
| 5. **Flag the wizard maintainer:** the wizard ships a committed | ||
| `docs/cli.md` auto-generated from the manifest. When the wizard | ||
| upgrades to a release containing your change, someone needs to run | ||
| `pnpm docs:cli` over there to refresh it. Note this in your PR | ||
| description or open a tracking issue in the wizard repo. | ||
|
|
||
| ## Commands | ||
|
|
||
| ```bash | ||
| npm install # Install dependencies | ||
| npm test # vitest run (parsers, expander, plugins, cli block) | ||
| npm run build # Full build: emits dist/skills/<id>.zip + manifests | ||
| npm run dev # Partial-rebuild dev server with watch | ||
| ``` | ||
|
|
||
| ## Repository conventions | ||
|
|
||
| - Skill content lives in markdown, never in JS/TS. The build pipeline reads | ||
| YAML configs and stitches markdown together; it doesn't generate prose. | ||
| - The `cli:` block is the **single source of truth** for the wizard's | ||
| command surface for any skill. Don't duplicate command names in the | ||
| wizard repo; they're derived from the manifest. | ||
| - `additionalProperties: false` is set on the JSON Schema — adding a new | ||
| field to the manifest shape is a coordinated change (bump the schema, | ||
| bump consumer types in the wizard). See | ||
| [PostHog/wizard CONTRIBUTING.md](https://github.com/PostHog/wizard/blob/main/CONTRIBUTING.md) | ||
| for the wizard-side contract. | ||
|
|
||
| ## Companion projects | ||
|
|
||
| - **[wizard](https://github.com/PostHog/wizard)** — the CLI that consumes | ||
| the manifest at build time and turns each `role: command` entry into a | ||
| registered command. | ||
| - **[warlock](https://github.com/PostHog/warlock)** — the security scanner | ||
| used by the wizard. Unrelated to skill content but lives alongside in | ||
| the same engineering scope. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,250 @@ | ||
| # Contributing to context-mill | ||
|
|
||
| This is the contributor doc for `context/skills/` and the | ||
| `scripts/` build pipeline. For docs aimed at consumers of the published | ||
| manifest, see the [README](README.md). | ||
|
|
||
| ## How skills get into the wizard CLI | ||
|
|
||
| Every skill ships with a `config.yaml`. An optional `cli:` block on that | ||
| config tells the PostHog wizard whether and how this skill appears as a | ||
| command. The block is parsed in `scripts/lib/skill-generator.js` and | ||
| emitted as `cliEntries` inside `dist/skills/skill-menu.json`. The wizard | ||
| fetches `skill-menu.json` at runtime and registers each entry as a | ||
| command, so adding a new skill-backed command is a context-mill release | ||
| — no wizard release needed. | ||
|
|
||
| ### The `cli:` block schema | ||
|
|
||
| ```yaml | ||
| type: docs-only | ||
| description: Audit captured events | ||
| cli: | ||
| role: command # command | skill | internal | ||
| parentCommand: audit # the command this skill nests under (optional) | ||
| command: events # the user-typed word; required when role is command | ||
| recommended: true # optional — pre-highlight this leaf in the family picker | ||
| ``` | ||
|
|
||
| Three values for `role`: | ||
|
|
||
| | Role | Where it shows up | | ||
| |---|---| | ||
| | `command` | Registered as `wizard <parentCommand> <command>` (or `wizard <command>` if no parent). The user-facing CLI. | | ||
| | `skill` | Reachable only via `wizard skill <id>`. The full discoverable set. | | ||
| | `internal` | Hidden everywhere. Only reachable via `wizard --skill=<id>` (a dev escape hatch). Useful for in-progress skills that aren't ready to expose. | | ||
|
|
||
| Skills with **no** `cli:` block default to the `skill` role — they're | ||
| discoverable via `wizard skill list` but don't get a top-level command. | ||
|
|
||
| ### Flat vs. family — the convention | ||
|
|
||
| > A command is **flat** when there's only one option today, | ||
| > **a family** when the user must pick among multiple distinct things. | ||
|
|
||
| Don't pre-create a family form for a single-option command. If only one | ||
| migration vendor exists, the command is `wizard migrate` — not | ||
| `wizard migrate statsig`. When a second vendor arrives, restructure to | ||
| a family at that moment and document the UX change in the wizard's | ||
| release notes. Forced abstraction (`wizard migrate <vendor>` with one | ||
| vendor) is worse than the breaking change you'd cause later — that | ||
| change is real and worth notifying users about explicitly. | ||
|
|
||
| ### Migrating a flat command into a family | ||
|
|
||
| When a flat command needs to grow into a family (a second option arrived, | ||
| the original is being renamed, etc.), the `cli:` block restructures like | ||
| this: | ||
|
|
||
| ```yaml | ||
| # Before — flat command. Registers `wizard investigate`. | ||
| cli: | ||
| role: command | ||
| command: investigate | ||
|
|
||
| # After — family with a subcommand. Registers `wizard investigate events`. | ||
| cli: | ||
| role: command | ||
| parentCommand: investigate | ||
| command: events | ||
| ``` | ||
|
|
||
| **This is a breaking change for users.** Anyone scripting the old flat | ||
| form (e.g. `wizard investigate` in CI) will break the moment the new | ||
| manifest ships — the parent name now expects a subcommand and opens the | ||
| family picker. Treat this exactly like any other breaking CLI change: | ||
|
|
||
| 1. Land the YAML change in context-mill. | ||
| 2. Call out the migration explicitly in the wizard release notes for the | ||
| release that picks up the new manifest — what the old command was, | ||
| what the new shape is, and what users need to change. | ||
| 3. If the old flat name is still meaningful as a default leaf of the new | ||
| family, mark that leaf `recommended: true` so `wizard investigate` → | ||
| Enter still runs the intended action with one keystroke. | ||
|
|
||
| Going the other way — collapsing a family back to a flat command — works | ||
| the same way and is also a breaking change. Don't do it casually. | ||
|
|
||
| ### Naming rule — no shorthand for product names | ||
|
|
||
| Use the **full PostHog product name** with hyphens, not abbreviations. | ||
|
|
||
| | | Good | Bad | | ||
| |---|---|---| | ||
| | Feature flags audit | `wizard audit feature-flags` | `wizard audit flags` | | ||
| | Session replay audit | `wizard audit session-replay` | `wizard audit replay` | | ||
| | Revenue analytics | `wizard revenue-analytics` | `wizard revenue` | | ||
| | Web analytics | `wizard web-analytics` | `wizard web` | | ||
| | LLM analytics | `wizard llm-analytics` | `wizard llms` | | ||
|
|
||
| The kebab-case / length / reserved-word checks in `parseCliBlock` | ||
| enforce the mechanics; this rule is the naming taste layer on top of | ||
| them. Users typing the full product name once is cheap; getting them | ||
| to relearn an abbreviation we changed our mind on later is not. | ||
|
|
||
| ### Mapping table — YAML on the left, registered command on the right | ||
|
|
||
| ```yaml | ||
| # 1. Flat command (single option today) | ||
| cli: → wizard revenue-analytics | ||
| role: command | ||
| command: revenue-analytics | ||
|
|
||
| # 2. Nested command inside an existing family | ||
| cli: → wizard audit feature-flags | ||
| role: command | ||
| parentCommand: audit | ||
| command: feature-flags | ||
|
|
||
| # 3. Recommended leaf — pre-highlighted in the family picker | ||
| cli: → wizard audit all | ||
| role: command Pre-highlighted in the | ||
| parentCommand: audit family picker, so | ||
| command: all `wizard audit` → Enter | ||
| recommended: true runs this leaf. | ||
|
|
||
| # 4. Skill-only (reachable via `wizard skill <id>`) | ||
| cli: → wizard skill <skill-id> | ||
| role: skill | ||
| ``` | ||
|
|
||
| The block can live at the **group level** (defaults for every variant) or | ||
| inside a **single variant** (overrides the group-level defaults). When | ||
| `role: command` and `command` is omitted, the variant id fills in as the | ||
| command name — except for the magic `id: all` variant, which collapses to | ||
| the group key and so requires an explicit `command` at the group level. | ||
|
|
||
| `cli:` only configures the **command shape** — the verbs the user types. | ||
| Flags and positional arguments live on the wizard side | ||
| (`ProgramConfig.cliOptions`), not here. | ||
|
|
||
| ### What `recommended: true` does (and doesn't do) | ||
|
|
||
| `recommended: true` controls **picker pre-highlighting**, not auto-run. When | ||
| the user invokes a family parent with no subcommand, the wizard always | ||
| opens an interactive picker over the family's children — the | ||
| recommended child is sorted to the top so a single Enter keystroke | ||
| runs it. The picker still appears (so the user sees every option before | ||
| committing). Set `recommended` on the leaf you'd want a user typing | ||
| `wizard <family>` to invoke if they don't change the selection. At most | ||
| one leaf per family should be marked. | ||
|
|
||
| ## Promotion criterion for `role: command` | ||
|
|
||
| The wizard's command surface is **curated, not inclusive**. Every command | ||
| is one we're willing to teach in our docs, announce, and support | ||
| for end users — not just every skill we've authored. | ||
|
|
||
| A skill should be promoted to `role: command` when **all** of these are | ||
| true: | ||
|
|
||
| 1. **It's user-facing, not infrastructure.** The skill represents a setup, | ||
| audit, or migration workflow an end user would reasonably invoke | ||
| directly. Internal helpers and scaffolding skills stay at `role: skill`. | ||
| 2. **The name reads naturally.** `wizard audit events` is obvious. `wizard | ||
| do-the-thing-with-events` is not. If you have to explain the command in | ||
| the docs before someone could guess what it does, the name needs more | ||
| work or the skill belongs at `role: skill` until it does. | ||
| 3. **It's stable.** The command surface is hard to deprecate without breaking | ||
| users. If the skill is still iterating on what it does or how it | ||
| prompts the agent, ship it as `role: skill` first. Promote when the shape | ||
| has held for a release or two. | ||
| 4. **It plays well with the family it lives in.** If `parentCommand: | ||
| audit`, the skill should slot alongside the other audits at the same | ||
| level of abstraction. Don't put a one-off in an existing family just | ||
| because the words overlap. | ||
| 5. **A wizard maintainer has reviewed the role change.** Adding to the | ||
| command surface is a permanent commitment to that name. Loop in the wizard | ||
| docs team / maintainers on PRs that change a skill to `role: command`. | ||
|
|
||
| When in doubt, ship as `role: skill`. Promoting from skill to command is | ||
| cheap; demoting from command to skill breaks user scripts. | ||
|
|
||
| ## Adding a new skill | ||
|
|
||
| The base path is the same regardless of the skill's CLI role: | ||
|
|
||
| 1. Create `context/skills/<your-skill>/`. | ||
| 2. Add a `config.yaml` declaring `type`, `description`, `variants`, etc. | ||
| See an existing skill (e.g. `audit-events`, `migrate`) for the shape. | ||
| 3. Add a `description.md` template and any `references/*.md` files. | ||
| 4. If the skill should be a wizard command, add a `cli:` block per the | ||
| schema above. | ||
| 5. Run `npm test && npm run build`. The build emits the new skill into | ||
| `dist/skills/<your-skill>.zip` and lists it in the manifest. | ||
|
|
||
| ## Adding a new command | ||
|
|
||
| When you've decided your skill meets the `role: command` criterion: | ||
|
|
||
| 1. Add the `cli:` block to the skill's `config.yaml` with `role: | ||
| command`, the right `parentCommand` (if it nests under an existing | ||
| family), and `command`. | ||
| 2. Confirm `npm run build` emits the entry under `cliEntries` inside | ||
| `dist/skills/skill-menu.json` with the right `parentCommand` / | ||
| `command` values. The wizard picks it up on its next invocation | ||
| (no wizard release needed). | ||
| 3. No wizard PR is needed for skill-backed public commands. If you also | ||
| need wizard-side hooks (custom outro, content blocks, abort cases), | ||
| that's a wizard PR — but the CLI registration is handled by the | ||
| manifest. | ||
|
|
||
| ### Heads up: wizard's `docs/cli.md` needs regeneration | ||
|
|
||
| The wizard ships a committed `docs/cli.md` auto-generated from the | ||
| manifest. When the wizard upgrades to a release containing your new | ||
| `cli:` block, **the wizard maintainer must run `pnpm docs:cli` in the | ||
| wizard repo** to refresh that file. Open a tracking issue on the wizard | ||
| side (or flag it in the wizard release PR) so it doesn't get skipped. | ||
|
|
||
| If you're the wizard maintainer on the receiving end: any change to | ||
| `cli-manifest.bootstrap.json` or to which manifest version the wizard | ||
| consumes is a signal to regenerate. See | ||
| [PostHog/wizard CONTRIBUTING.md](https://github.com/PostHog/wizard/blob/main/CONTRIBUTING.md#when-to-regenerate-docscli-md). | ||
|
|
||
| ## What goes here vs. in the wizard repo | ||
|
|
||
| | If you're changing… | …PR goes to | | ||
| |---|---| | ||
| | Skill markdown content | `context-mill` | | ||
| | Skill `config.yaml` (including `cli:` blocks) | `context-mill` | | ||
| | Skill-generation scripts (`scripts/lib/`) | `context-mill` | | ||
| | YARA-X security rules | `warlock` | | ||
| | Wizard runner pipeline, TUI, agent runtime | `wizard` | | ||
| | Wizard-native programs (doctor, mcp, source-maps) | `wizard` | | ||
| | Wizard CLI factories and bin.ts wiring | `wizard` | | ||
|
|
||
| If a change crosses two repos, ship the context-mill PR first so the | ||
| manifest is published before the wizard tries to consume it. | ||
|
|
||
| ## Where to look for more | ||
|
|
||
| - Skill schema details: `scripts/lib/skill-generator.js` | ||
| (`parseCliBlock`, `expandSkillGroups`, JSDoc typedef for the `cli:` block) | ||
| - CLI entries emit: `scripts/lib/build-phases.js` (`generateCliEntries`) | ||
| - Tests for the cli block parser: `scripts/lib/tests/cli-block.test.js` | ||
| - The wizard's side of the contract: [PostHog/wizard CONTRIBUTING.md](https://github.com/PostHog/wizard/blob/main/CONTRIBUTING.md) | ||
|
|
||
| Questions: drop a note in | ||
| [#team-docs-and-wizard](https://posthog.slack.com/archives/C09GTQY5RLZ) or | ||
| open an issue. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move this to agents.md so all agents will read i think