Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
b4cb33e
feat: complete whet CLI implementation (v0.1.0)
ortizeg Feb 9, 2026
2bff4a4
fix: add mypy overrides and type annotations for CI
ortizeg Feb 10, 2026
fe3949c
feat: expand whet install to include agents and settings
ortizeg Feb 10, 2026
a3552b6
fix: settings engine handles real-world settings files
ortizeg Feb 10, 2026
4eb81af
fix: persist target platform selection to disk (#5)
ortizeg Feb 10, 2026
5bf59a8
Skills refactor: trim bloat, add model-evaluation + pydantic-ai, push…
ortizeg Jul 23, 2026
3af8c08
P1: remove agents/, promote data-pipelines into skills (#7)
ortizeg Jul 24, 2026
b72548b
P2: progressive disclosure (thin index + references/), install tiers …
ortizeg Jul 24, 2026
a70b117
Add whet install --prune (manifest-scoped, safe alongside other tools…
ortizeg Jul 24, 2026
7339eba
Fix archetype skill composition (inference archetype had no fastapi) …
ortizeg Jul 24, 2026
b32e8e9
Make every archetype generate a working project, and gate it in CI (#11)
ortizeg Jul 24, 2026
bd6b969
P0: whet init installs archetype skills; de-stale archetype docs (#12)
ortizeg Jul 24, 2026
f11734e
P1: compose the skills the archetypes' templates actually use (#13)
ortizeg Jul 24, 2026
a902822
P2: migrate pixi manifests from [project] to [workspace] (#14)
ortizeg Jul 24, 2026
3d2efbf
P2: split the 9 remaining skills over 350 lines (#15)
ortizeg Jul 24, 2026
4dae138
Make template/ the source of truth for documented directory trees (#16)
ortizeg Jul 24, 2026
9ab5dd6
Enforce the skill-authoring rules in CI (#17)
ortizeg Jul 24, 2026
282db7a
Demote vscode and github-repo-setup to the extra tier (#18)
ortizeg Jul 24, 2026
4b03f22
chore(release): 0.3.0
ortizeg Jul 24, 2026
feab866
fix(packaging): ship skills, archetypes, and settings in the wheel
ortizeg Jul 24, 2026
95640a0
Merge main into release/0.3.0
ortizeg Jul 24, 2026
d2ae0cb
fix(ci): make the required 'docs' check reachable
ortizeg Jul 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
19 changes: 14 additions & 5 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -1,20 +1,25 @@
name: Deploy Documentation
name: Documentation

on:
push:
branches: [main]
pull_request:
branches: [main, develop]

permissions:
contents: read
pages: write
id-token: write

# Keyed by ref so concurrent PR doc builds do not serialize behind each other,
# while repeated pushes to the same branch still supersede one another.
concurrency:
group: "pages"
group: pages-${{ github.ref }}
cancel-in-progress: false

jobs:
build:
# Job id is `docs` so it satisfies the required status check of that name.
docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand All @@ -30,20 +35,24 @@ jobs:
- name: Install dependencies
run: uv sync --extra docs

# --strict fails on broken internal links, which is how the dangling
# agent-page links were caught rather than shipped.
- name: Build docs
run: uv run mkdocs build
run: uv run mkdocs build --strict

- name: Upload artifact
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: actions/upload-pages-artifact@v3
with:
path: ./site

deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
needs: docs
steps:
- name: Deploy to GitHub Pages
id: deployment
Expand Down
29 changes: 29 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,32 @@ jobs:

- name: Run type checking
run: uv run mypy src/whet/ tests/ --strict

archetypes:
# Renders every archetype and verifies the *generated* project is usable.
# Without this, a template can ship code that never imports, crashes on the
# first step, or fails its own lint gate — none of which the unit tests notice.
name: Archetypes render and pass their own gates
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v4
with:
version: "latest"

- name: Set up Python
run: uv python install 3.11

- name: Install dependencies
run: uv sync --all-extras

- name: Render each archetype and validate the generated project
run: uv run pytest tests/test_archetype_templates.py -v

- name: Validate archetype skill composition
run: uv run pytest tests/test_archetypes.py -v

- name: Enforce skill-authoring rules
run: uv run pytest tests/test_skill_hygiene.py -v
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ venv/
env/
.pixi/

# Ruff / mypy caches
.ruff_cache/
.mypy_cache/

# IDE
.vscode/
.idea/
Expand Down Expand Up @@ -42,6 +46,10 @@ checkpoints/

# Data
data/
# ...but archetype templates ship data/ scaffolding that must be committed,
# otherwise a scaffolded project is missing directories its README documents.
!archetypes/*/template/**/data/
!archetypes/*/template/**/data/**
*.h5
*.hdf5
*.tfrecord
Expand Down
99 changes: 99 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Changelog

All notable changes to this project are documented in this file.

The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this
project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). While the
version is below 1.0.0, breaking changes are released in a minor bump.

## [0.3.0] - 2026-07-24

A library-wide audit of all three pillars — skills, agents, and archetypes — measured
against Anthropic's Agent Skills guidance. Always-resident skill context is down 73%,
the agents concept is gone, every archetype generates a project that runs, and the
authoring rules are now enforced in CI rather than by discipline.

### Removed

- **The `agents/` directory and the agent concept.** The six "agents" were never Claude
Code subagents: four had no YAML frontmatter (so they installed with their H1 as the
entire description and were effectively untriggerable), all six installed into
`.claude/skills/` rather than `.claude/agents/`, `agent.toml` was never parsed by any
code path, and both "blocking" `action.yml` files ran `pixi` against a repo with no
`pixi.toml`. Roughly 77% of their content duplicated existing skills, often diverging
from them. GSD owns agency; whet owns domain knowledge, and knowledge is a skill.
- **`whet install --agents-only` and `--skills-only` flags**, along with
`discover_agents()` and the `agents_dir` config field.
- **Skills `dvc` and `cv-model-selection`.** Guidance that referenced DVC as a generic
concept was rewritten tool-neutral rather than left pointing at a deleted skill.

### Added

- **`model-evaluation`** — detection mAP/IoU via `supervision`, confusion matrices,
per-class and per-size breakdowns, deployment-threshold selection, per-slice failure
analysis, and eval-as-CI regression gates.
- **`pydantic-ai`** — typed LLM/VLM structured outputs and VLM-in-the-loop auto-labeling.
- **`data-pipelines`** — storage-format selection, group-aware leakage-preventing dataset
splitting, schema evolution, and data-quality validation.
- **`whet install --prune`** — removes skills that no longer exist upstream. Scoped by a
`.whet-manifest.json` recording what whet installed, so skills placed in the same
directory by other tools are never touched.
- **`whet install --include-extras`** and a `tier` field in `skill.toml`. Skills outside
the flagship path are opt-in: `aws-sagemaker`, `github-repo-setup`, `gradio`,
`huggingface`, `kubernetes`, `mlflow`, `vscode`.
- **`whet init --with-recommended` / `--no-skills`.**
- **Progressive disclosure** across the library: 29 of 32 skills are now a thin index
plus a `references/` directory loaded on demand — 155 reference files in total.
- **CI enforcement.** A new `archetypes` job renders every archetype and runs the
toolchain against the generated project, plus guards for skill-authoring rules
(line ceiling, reference-link integrity, orphaned references, one-level-deep
references, tables of contents, trigger-first and third-person descriptions,
listing budget) and for archetype composition and documented directory trees.

### Changed

- **`whet init` now installs the archetype's skills** (with their `references/`) instead
of printing a `whet add ...` hint. An archetype's value over a folder copy is the skill
set it composes, so leaving that as homework made `[skills]` advisory.
- **Skill `pydantic-strict` renamed to `pydantic`.**
- **All skill descriptions rewritten** as third-person, trigger-first "use this skill
when…" statements with explicit disambiguation between adjacent skills. They were topic
summaries, which under-trigger.
- **Every archetype template now generates a project that runs.** Previously four of six
generated only a four-file stub, and the two with templates produced projects that could
not import their own entry point, crashed on the first training step, or shipped no ONNX
despite being named for it. Template files went from 19 to 125.
- **pixi is the canonical environment manager** for generated projects; manifests migrated
from the deprecated `[project]` table to `[workspace]`.
- **Archetype skill composition** now reflects what each template actually ships — every
archetype requires `pixi` and `code-quality`, and `cv-inference-service` finally
requires `fastapi`.

### Fixed

- **The published wheel shipped no skills.** `[tool.hatch.build.targets.wheel]` packaged
only `src/whet`, and the bundled directories were resolved relative to the repository
root — so `uv tool install whet` produced a CLI that reported "No skills found" outside
a source checkout. The wheel now force-includes `skills/`, `archetypes/`, and
`settings/` under `whet/_data/`, and a single resolver prefers the packaged copy while
falling back to the repo layout for development.
- Template files under a `data/` path were silently excluded by the repository's own
`.gitignore`, so a clean checkout produced projects missing `data/raw`, `data/processed`,
and an entire Hydra `data` config group.
- `.ipynb` files were not treated as text by the scaffold engine, so `${package_name}`
inside a notebook was never substituted.
- Adapters now install a skill's `references/` directory (Claude, Antigravity) or inline
it (Cursor, Copilot), so deep-dive links resolve on every platform.
- Documented directory trees in archetype READMEs and docs pages are regenerated from the
templates; 78 documented-but-absent files were removed.

## [0.2.1] - 2026-07-23

Published to PyPI outside this repository's tag-based release workflow; no corresponding
git tag exists. Recorded here for continuity.

## [0.1.0] - 2026-02-09

Initial whet CLI implementation.

[0.3.0]: https://github.com/ortizeg/whet/releases/tag/v0.3.0
104 changes: 90 additions & 14 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ Distribution: `uvx whet` (one-shot) / `uv tool install whet` (permanent)

### What whet provides

- **30 Skills** — Best-practice knowledge modules (PyTorch Lightning, Pydantic, Docker, FastAPI, Hugging Face, AWS SageMaker, Gradio, Kubernetes, etc.)
- **6 Agents** — Pre-configured behavioral profiles (Expert Coder, ML Engineer, DevOps/Infra, Data Engineer, Code Review, Test Engineer)
- **32 Skills** — Best-practice knowledge modules (PyTorch Lightning, Pydantic, Docker, FastAPI, Hugging Face, AWS SageMaker, Gradio, Kubernetes, Model Evaluation, etc.)
- **6+ Archetypes** — Complete project templates for common CV/ML project types
- **CLI** — `whet add`, `whet install`, `whet list`, `whet search`, `whet doctor`
- **Multi-platform** — Claude Code, Google Antigravity, Cursor, GitHub Copilot
Expand All @@ -29,14 +28,13 @@ whet/
│ ├── core/ # Domain models (Pydantic: skill, config)
│ └── registry/ # Skill discovery, search, dependency resolution
├── skills/ # 25+ skill definitions (SKILL.md + README.md + skill.toml)
├── agents/ # 4+ agent definitions (SKILL.md + README.md + agent.toml)
├── archetypes/ # 6+ project templates (README.md + archetype.toml + template/)
├── settings/ # Pre-built settings templates (claude.json, etc.)
├── docs/ # MkDocs Material documentation
├── tests/ # Self-tests (dynamic discovery, no hardcoded counts)
├── .github/workflows/ # CI/CD (test, lint, docs) — uses uv
├── pyproject.toml # uv-managed, publishable to PyPI
├── justfile # Task runner (replaces pixi tasks)
├── justfile # Task runner (wraps uv commands)
└── mkdocs.yml # Documentation site config
```

Expand All @@ -63,22 +61,99 @@ uv run mypy src/whet/ tests/ --strict

## How to Add a New Skill

1. Create `skills/<name>/SKILL.md` — Must start with YAML frontmatter (`name`, `description`), be >500 chars, include code examples and headers
2. Create `skills/<name>/skill.toml` — Machine-readable metadata (category, tags, deps, compatibility)
1. Create `skills/<name>/SKILL.md` — YAML frontmatter (`name`, `description`), >500 chars, code examples and headers
2. Create `skills/<name>/skill.toml` — Machine-readable metadata (category, tier, tags, deps, compatibility)
3. Create `skills/<name>/README.md` — Must explain purpose (include words like "when", "use", "purpose")
4. Create `docs/skills/<name>.md` — Documentation page (Purpose, When to Use, Key Patterns, Anti-Patterns)
5. Add nav entry to `mkdocs.yml` under Skills section

Tests discover skills dynamically — no hardcoded lists to update.

## How to Add a New Agent
### Writing the `description` (this is the trigger)

1. Create `agents/<name>/SKILL.md` — Must be >500 chars
2. Create `agents/<name>/agent.toml` — Agent metadata (type: advisory/blocking, tags)
3. Create `agents/<name>/README.md` — Agent overview
4. If blocking agent: create `agents/<name>/action.yml`
5. Create `docs/agents/<name>.md` — Documentation page
6. Add nav entry to `mkdocs.yml` under Agents section
Only the frontmatter is preloaded; the `description` is the entire discovery mechanism.
Write it third-person and trigger-first, and be a little "pushy":

```
description: >
Use this skill when <concrete situations>. Reach for it any time you would otherwise
<the manual thing>, even if the user doesn't say "<tool>" explicitly. Not for
<adjacent thing> (see <other-skill>).
```

Always disambiguate against adjacent skills so two skills never compete for the same trigger.

### Progressive disclosure (keep SKILL.md thin)

A SKILL.md loads fully into context when triggered and **stays there for the session**, so
every line is a recurring token cost. Structure a skill as an index plus on-demand detail:

```
skills/<name>/
├── SKILL.md # ~120-200 line index: core patterns, conventions, anti-patterns
└── references/ # topic files, loaded only when needed
├── <topic-a>.md
└── <topic-b>.md
```

Rules:
- Keep `SKILL.md` **under 500 lines** (target 120–200 for a split skill).
- Keep the 80%-case patterns, conventions, and anti-patterns **inline**; move depth to `references/`.
- End `SKILL.md` with a `## Deep dives` list giving each reference a *"read this when…"* trigger.
- **One level deep only** — a reference file must never link to another reference file.
- Reference files over 100 lines start with a table of contents; name them descriptively.

The installer copies `references/` for directory-based platforms (Claude, Antigravity) and
inlines them for flat-file platforms (Cursor, Copilot), so no content is lost either way.

### Scaffolding a project (`whet init`)

`whet init <archetype>` renders the template **and installs the archetype's required
skills** into the new project, so it is usable immediately:

```bash
whet init pytorch-training-project
whet init pytorch-training-project --with-recommended # + recommended skills
whet init pytorch-training-project --no-skills # scaffold files only
```

An archetype's value over a plain folder copy is the skill set it composes, so leaving
that as a printed hint made the `[skills]` list advisory. A name in `archetype.toml` that
no longer resolves is reported rather than silently skipped.

### Keeping an install in sync (`--prune`)

`whet install` copies skills but does not remove ones that were deleted upstream, so a
stale skill can linger in `.claude/skills/` long after it left the repo. `--prune` fixes
that:

```bash
whet install --prune
```

Prune is **manifest-scoped**. Each install writes `.whet-manifest.json` into the target
directory recording which skills whet owns there. Prune only ever removes names from that
manifest, so skills installed by other tools (GSD, `interface-design`, hand-written ones)
that share the same directory are never touched. A skill merely filtered out of a run
(`--category`, or an `extra` skill without `--include-extras`) is not an orphan either —
only skills that no longer exist in `skills/` are removed.

Installs predating the manifest are invisible to prune; clear those once with
`whet remove <name>`.

### Core vs extra tier

`tier = "core"` (default) installs with `whet install`. `tier = "extra"` marks a skill as
opt-in — it is skipped unless the user passes `--include-extras`. Use `extra` for skills
that are real but outside the flagship path, so they don't dilute the default trigger surface.

## Companion skills (not shipped by whet)

whet does not ship a general product-UI ruleset — `gradio` covers ML demos only. For
application/dashboard UI work, use the external **`interface-design`** skill alongside whet
(upstream: https://github.com/Dammyjay93/interface-design). Its `SKILL.md` is self-contained;
`references/` only holds design-system templates. It also provides `/design-review` and
`/design-deslop`.

## How to Add a New Archetype

Expand Down Expand Up @@ -109,7 +184,8 @@ description: >
[skill]
name = "skill-name"
version = "1.0.0"
category = "cv-ml" # core | cv-ml | infra | experiment-tracking
category = "cv-ml" # core | cv-ml | infra | cloud | experiment-tracking
tier = "core" # core (default, installed) | extra (opt-in via --include-extras)
tags = ["tag1", "tag2"]

[dependencies]
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,10 @@ whet install --global
# Add specific skills to your project
whet add pytorch-lightning wandb hydra-config

# Scaffold a new project
# Scaffold a new project (also installs the archetype's required skills)
whet init pytorch-training-project
whet init pytorch-training-project --with-recommended # + recommended skills
whet init pytorch-training-project --no-skills # files only
```

## What is whet?
Expand All @@ -41,8 +43,7 @@ whet is a CLI tool that installs curated expert skill definitions into AI coding

### Flagship Collection: CV/ML

- **30 Skills** — PyTorch Lightning, Pydantic, Docker, ONNX, TensorRT, OpenCV, FastAPI, Hugging Face, AWS SageMaker, Gradio, Kubernetes, and more
- **6 Agents** — Expert Coder, ML Engineer, DevOps/Infra, Data Engineer, Code Review, Test Engineer
- **32 Skills** — PyTorch Lightning, Pydantic, Docker, ONNX, TensorRT, OpenCV, FastAPI, Hugging Face, AWS SageMaker, Gradio, Kubernetes, Model Evaluation, and more
- **6+ Archetypes** — Training pipelines, inference services, notebooks, packages

### Multi-Platform
Expand Down Expand Up @@ -102,7 +103,7 @@ just typecheck

## Contributing

Contributions welcome! See [CLAUDE.md](CLAUDE.md) for how to add new skills, agents, or archetypes.
Contributions welcome! See [CLAUDE.md](CLAUDE.md) for how to add new skills or archetypes.

## License

Expand Down
Loading
Loading