diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index ad548e0..31e8b0a 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -44,6 +44,9 @@ jobs: - name: Skill-gate library unit tests run: node --test scripts/skill-gate-lib.test.mjs + - name: doc-classify unit tests + run: node --test plugins/doc-sweep/hooks/doc-classify.test.mjs + # Quality layer: every skill must have fresh eval artifacts meeting its # threshold. Deterministic, no LLM, no Anthropic auth (artifacts are # produced at author time by /skill-gate and committed). diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/.openspec.yaml b/openspec/changes/archive/2026-07-04-unify-doc-classification/.openspec.yaml new file mode 100644 index 0000000..251bcba --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/.openspec.yaml @@ -0,0 +1,2 @@ +schema: superpowers-bridge +created: 2026-07-03 diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/brainstorm.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/brainstorm.md new file mode 100644 index 0000000..cbea4a5 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/brainstorm.md @@ -0,0 +1,121 @@ +# Brainstorm — unify doc classification + +Raw capture of the design conversation (adversarial review → decision chain). Reorganized into +structured sections in `design.md`. + +## Background + +Triggered by an adversarial review of the just-merged `docs-staleness-ci` feature. The reviewer +asked: is `docs-ci-check.sh` useful, is it too specific, should it be node, and does it "miss out +on boundaries" — ideally the doc-sweep skills and scripts should share the same pattern logic. + +Grounding the review in the files surfaced a concrete bug, not just a smell. "What is a doc?" is +currently defined in **three** places that disagree: + +| Source | Used by | `.claude/**/*.md` | `CHANGELOG.md` | `docs/**` | +|---|---|---|---|---| +| `audience-rules-base.md` (prose) | the skills (revise/audit) | doc ("all `*.md` under `.claude/`") | overlay-only | overlay-only | +| `is_doc()` in `revise-push-guard.sh` | the hook | NOT a doc | doc | doc | +| `is_doc()` in `docs-ci-check.sh` | the CI check | NOT a doc | doc | doc | + +Consequences: +- **Divergence bug**: a PR that edits only `.claude/context/audience-rules.md` — doc-sweep's own + canonical doc — is classified as a non-doc change with no docs touched, so `docs-ci-check.sh` + **fails it**. The guard flags you for editing the file that defines what a doc is. Untested. +- **Copy-paste**: `is_doc` + the excludeDirs loop are duplicated verbatim in the two shell files; + they will drift. +- **Hybrid bash+node smell**: `docs-ci-check.sh` wraps four separate `node -e` one-liners around + bash (re-parsing the config JSON twice) — worst of both. +- **Crudeness**: the check only knows "a non-doc changed and no doc was touched"; it can't tell if + docs were actually *warranted*. The `[skip docs]` hatch is the tell. Real value only for the + cases the local hook can't reach (humans, fork PRs, non-doc-sweep contributors); marginal on a + solo repo. Kept as advisory, not a required blocker. + +Nuance that shaped scope: the **skills** and **scripts** don't do the same classification. Scripts +answer a binary *is this path a doc?* (doc vs non-doc). Skills answer *which audience does this +known doc serve?* (Claude vs human), driven by the prose table — they have no `is_doc`/`docMode`. +So "one function for everything" is a category error; the genuinely shareable thing is the +**doc-file-set** (which globs count as docs) + `excludeDirs`. + +## Decision chain + +**Q1 — Direction: unify up, simplify down, or leave it?** +Two directions weighed. (A) Unify up: one shared declarative doc-file-set + one classifier module, +fixing the divergence. (B) Simplify down: accept it's a crude nag, drop `docMode`, classify docs as +"any `*.md` or under `docs/`", delete the enum. Absent the divergence bug, B would win for a solo +repo — but the bug tips it: the fix and the unification are the same work. +→ **Decision: A (unify up), scoped tightly.** + +**Q2 — Scope: where does the single source live, and does it touch skill runtime?** +Options: (1) scripts + shared source only; (2) also wire the skills to read `docPatterns` +programmatically; (3) just dedupe the scripts + fix `.claude/**`, no declarative source. +Option 2 rejected — the skills don't do doc-vs-non-doc classification, so wiring them to +`docPatterns` is a forced fit and would churn their eval benchmarks. Option 3 leaves two +definitions. +→ **Decision: (1)** — add a machine-readable `docPatterns` block to the audience-rules +(base + overlay) as the single doc-file-set source; one shared `doc-classify.mjs` consumed by +BOTH scripts; fix `.claude/**`; retire `docMode`. Skills stay prose-driven (unchanged runtime) +with a consistent machine-readable twin. + +**Q3 — Where do the patterns physically live at runtime? (single source vs no new parser)** +Reading `docPatterns` straight from `audience-rules.md` would need a markdown-embedded-YAML parser +in node (new complexity). But `excludeDirs` already establishes the pattern: install persists it in +`audience-rules.md` (human-authoritative) and **mirrors** it into the per-install config JSON, which +the scripts read via `JSON.parse`. Reuse that exact mechanism for `docPatterns`. +→ **Decision:** `audience-rules.md` is the human source (install reads/writes it); the config JSON +is the machine-read mirror; the classifier reads `docPatterns`/`excludeDirs` from the config, else a +built-in default. No new parser; consistent with today's `excludeDirs` flow. The CI vendored +`docs-ci.json` carries the mirrored patterns; funbox's own no-config dogfood run uses the built-in +default (which now includes `.claude/**`). + +**Q4 — `docMode` retirement: alias, hard-retire, or keep both?** +→ **Decision: hard retire.** The classifier understands only `docPatterns` + a built-in default; +`docMode` removed from both scripts, the config schema, and both install skills (which now write +`docPatterns`). Accepted risk: an old config still carrying `docMode` silently falls back to the +default set until regenerated — fine, effectively single-user. + +**Q5 — node vs bash.** +→ **Decision:** git plumbing (merge-base, diff, log, per-commit `[skip docs]`) stays bash; the +classification + JSON + glob matching move into one `doc-classify.mjs` module both scripts shell +into once. Node is already a hard dependency ("no jq"); a module is unit-testable and shared rather +than copy-pasted. A reusable `uses:` action was previously rejected on supply-chain grounds; the +classifier is vendored alongside the check script instead. + +**Q6 — Exempt categories (raised mid-apply).** Idea: predefined defaults where the guard doesn't +enforce — e.g. a test-only change, or "other things like that." This is the precision upgrade the +adversarial review's "crude nag" critique wanted: doc / non-doc becomes doc / exempt / doc-requiring. +Three sub-decisions: (a) **fold into this change** (the classifier is being built now — cheap +extension) vs a separate follow-up → fold in; (b) **model** — a simple `exemptPatterns` glob list +(reuses the classifier's ignore mechanism, distinct from `excludeDirs` by *intent*) vs a richer +named-category policy (tests/ci/deps/generated toggles) → simple `exemptPatterns` (YAGNI); (c) +**default breadth** — tests only vs tests + lockfiles + CI → **tests only, rest configurable** via +`exemptPatterns`. Mechanics: exempt is evaluated after `excludeDirs`, before `docPatterns`; an exempt +path is dropped from `nonDoc` and doesn't set `docChanged`, so a test-only PR passes without an ack +while tests + real code still enforces. + +## Design shape (validated) + +- **`doc-classify.mjs`** — file list on stdin, optional `--config `; reads + `docPatterns` + `excludeDirs` (or built-in default: `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, + `docs/**`, `.claude/**/*.md`); emits `{"nonDoc":[…],"docChanged":bool}`. Tiny in-house glob + matcher (`*`, `**`), no external deps. Unit-tested with `node --test`. +- **`revise-push-guard.sh` / `docs-ci-check.sh`** — become thin git wrappers that pipe changed + files to the classifier; drop their duplicated `is_doc`/`docMode`/config-parsing. +- **audience-rules** — gains a `docPatterns:` block co-located with `excludeDirs`. +- **Install skills** — copy `doc-classify.mjs` alongside their script; write `docPatterns` instead + of `docMode`. + +## Trade-offs / risks + +- Hard-retire `docMode` → stale configs fall back to default until regenerated (accepted). +- Both scripts now depend on invoking a node **script file** (not just `node -e`) — vendored copies + must ship `doc-classify.mjs`. +- Per-commit `[skip docs]` logic in the hook may call the classifier per commit; ranges are small, + acceptable. +- The underlying "crude nag" limitation is unchanged — this change fixes correctness + duplication, + not the heuristic's inherent imprecision. Staleness check stays advisory. + +## Non-goals + +Wiring skills' runtime to `docPatterns`; changing `[skip docs]` semantics; any LLM-in-CI judgement; +altering the merge-base baseline or advisory/blocking posture. diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/design.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/design.md new file mode 100644 index 0000000..253f558 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/design.md @@ -0,0 +1,106 @@ +## Context + +doc-sweep classifies "what is a doc" in three disagreeing places: the audience-rules prose (the +skills' source of truth), and a verbatim-duplicated `is_doc()` in each of `revise-push-guard.sh` +(the local hook) and `docs-ci-check.sh` (the CI check). The scripts key off a `docMode` enum +(`minimal`/`with-skill`/`default`) hardcoded as bash globs. That set omits `.claude/**/*.md`, which +the audience-rules base explicitly calls Claude-facing docs — so a PR editing only +`.claude/context/audience-rules.md` is scored as "code changed, no docs" and the CI check fails it. +The duplication also guarantees future drift. Constraints from the repo: node is already a hard +dependency ("no jq"); the CI check must stay deterministic and secret-free; `excludeDirs` already +establishes a "persist in audience-rules, mirror into the per-install config JSON" flow the scripts +read via `JSON.parse`; a `uses:`-style shared action was previously rejected on supply-chain grounds +(the CI script is vendored into consumer repos instead). + +## Goals / Non-Goals + +**Goals** +- One definition of the script-side doc-file-set, fixing the `.claude/**` divergence bug. +- Kill the copy-pasted `is_doc` across the two scripts. +- Keep the classifier deterministic, dependency-free, and unit-testable. +- Preserve the existing `excludeDirs` mechanism rather than invent a parallel one. + +**Non-Goals** +- Wiring the skills' runtime to `docPatterns` — they classify by *audience* (Claude vs human) among + known docs, not doc-vs-non-doc, so there is no `is_doc` to share. They stay prose-driven. +- Changing `[skip docs]` semantics, the merge-base baseline, or the advisory (non-blocking) posture. +- Fully solving whether docs are *warranted* — `exemptPatterns` (D7) coarsely reduces false + positives for known no-doc categories (tests by default), but the check still can't judge whether + an arbitrary code change needs docs; `[skip docs]` remains the escape hatch for the rest. + +## Decisions + +**D1 — Unify up, not simplify down.** Two directions: (A) one shared declarative doc-set + shared +classifier; (B) accept the crude nag and reduce docs to "any `*.md` or `docs/`", deleting `docMode`. +Absent the bug, B would win for a solo repo; the divergence bug tips it to A because the fix and the +unification are the same work. + +**D2 — Scope: scripts + shared source; skills untouched at runtime.** Rejected wiring the skills to +read `docPatterns` (forced fit — they don't do doc-vs-non-doc classification, and it would churn +their eval benchmarks). Rejected script-dedupe-only (leaves two definitions). Chosen: a shared +declarative `docPatterns` source + one classifier the scripts consume; skills keep prose rules with a +consistent machine-readable twin. + +**D3 — Patterns live in audience-rules, mirrored to the config JSON (no new parser).** Reading +`docPatterns` straight from markdown would need a YAML-in-markdown parser in node. Instead reuse the +established `excludeDirs` flow: `audience-rules.md` is the human-authoritative source that the +install skill reads/writes and **mirrors** into the per-install config JSON; the classifier reads +`docPatterns`/`excludeDirs` from that JSON, else a built-in default. The vendored CI `docs-ci.json` +carries the mirror; funbox's no-config dogfood run uses the built-in default. + +**D4 — Hard-retire `docMode`.** The classifier understands only `docPatterns` + the built-in +default. Removed from both scripts, the config schema, and both installers (which now write +`docPatterns`). Alternatives (alias `docMode`→presets; keep both) rejected to avoid perpetuating the +two-ways-to-say-it problem. Accepted breaking edge: a stale config still carrying `docMode` falls +back to the default set until regenerated. + +**D5 — node module for classification, bash for git.** git plumbing (merge-base, diff, log, +per-commit `[skip docs]`) stays bash; classification + JSON + glob matching move into one +`doc-classify.mjs` that both scripts shell into once. A tiny in-house glob matcher (`*`, `**`) avoids +any external dependency. Vendored alongside each installed script. + +**D6 — Classifier interface.** `doc-classify.mjs` reads a newline-separated file list on stdin, +takes optional `--config `, and emits `{"nonDoc":[…],"docChanged":bool}` on stdout. Config +resolution: `docPatterns`/`excludeDirs`/`exemptPatterns` from `--config` JSON if present, else +built-in defaults (`CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**`, `.claude/**/*.md`). The +hook additionally invokes it per-commit for its `[skip docs]` per-commit rule (small ranges, +acceptable cost). + +**D7 — Exempt categories (three-way classification).** Add an `exemptPatterns` glob set for +first-party changes that don't require docs, evaluated AFTER `excludeDirs` and BEFORE `docPatterns`; +an exempt path is neither doc nor non-doc (dropped from `nonDoc`, doesn't set `docChanged`). Default: +common test globs (`**/*.test.*`, `**/*.spec.*`, `**/test/**`, `**/tests/**`, `**/__tests__/**`, +`**/*_test.go`, `**/*_test.py`); a configured list replaces the default. This directly addresses the +adversarial review's "crude nag" critique — a test-only PR passes without an ack, while tests + +`src/app.js` still enforces. *Alternatives:* a richer named-category policy (tests/ci/deps/generated +toggles) — rejected as over-built (YAGNI); merging into `excludeDirs` — rejected to preserve the +vendored-vs-doesn't-need-docs *intent* distinction, even though the mechanism is the same "drop from +nonDoc." Default kept to tests only; projects add lockfiles/CI/etc. via `exemptPatterns`. + +## Risks / Trade-offs + +- **Stale `docMode` config → default fallback** → acceptable (effectively single-user); installers + regenerate configs with `docPatterns`. +- **Vendored copies must now ship two files** (`docs-ci-check.sh` + `doc-classify.mjs`) → installers + copy both; uninstall removes both. +- **In-house glob matcher could mis-handle an exotic pattern** → keep the supported syntax explicit + (`*` within a segment, `**` across segments, literals) and unit-test the corners; `docPatterns` + authors stick to that vocabulary. +- **Per-commit classifier calls in the hook loop** → ranges are small; if ever hot, batch later. +- **Heuristic still crude** → unchanged by design; the check stays advisory, `[skip docs]` remains + the escape hatch. + +## Migration Plan + +1. Land `doc-classify.mjs` + unit tests; wire the node test into CI. +2. Refactor both scripts to delegate to it; update their shell test suites (incl. an + audience-rules-only-PR regression that must pass). +3. Add `docPatterns` to the audience-rules; update both install skills to vendor the classifier and + write `docPatterns` (drop `docMode`); regenerate their eval benchmarks. +4. Update doc-sweep README/CHANGELOG; funbox dogfood already runs no-config → picks up the fixed + default automatically. +5. Rollback: revert is additive — restore the inline `is_doc` and `docMode` reads; no data migration. + +## Open Questions + +None blocking. Whether to later expose `docPatterns` to the skills' runtime is deferred (see D2). diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/plan.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/plan.md new file mode 100644 index 0000000..7c79e6e --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/plan.md @@ -0,0 +1,472 @@ +# Unify Doc Classification Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Collapse the three disagreeing "what is a doc?" definitions into one shared, declarative classifier so the two guard scripts stop duplicating logic and stop misclassifying `.claude/**/*.md`. + +**Architecture:** A new dependency-free node module `doc-classify.mjs` owns doc/non-doc/excluded classification (built-in default doc-set + optional `docPatterns`/`excludeDirs` from a config JSON). `docs-ci-check.sh` and `revise-push-guard.sh` become thin git-plumbing wrappers that pipe changed files to it. `docMode` is removed everywhere; install skills vendor the module and record `docPatterns`. + +**Tech Stack:** Bash (git plumbing), Node ≥20 ESM (classifier + `node --test`), GitHub Actions, OpenSpec. + +## Global Constraints + +- Shell files (`*.sh`) MUST stay LF (`.gitattributes`); pass `bash -n` + ShellCheck (keep existing `# shellcheck disable=SC2086` where word-splitting is intentional). +- No external/non-builtin node dependencies anywhere in doc-sweep scripts. JSON via node, never `jq`. +- `allowed-tools` in SKILL.md must stay scoped (no bare/wildcard `Bash`). +- New skills/changed skills must pass `claude plugin validate` and the skill-gate (`evals/benchmark.json` ≥ 0.9, hash-fresh). +- Node module id: current Claude model is `claude-opus-4-8[1m]` (for any benchmark `model` field). + +--- + +### Task 1: Shared classifier module `doc-classify.mjs` + +**Files:** +- Create: `plugins/doc-sweep/hooks/doc-classify.mjs` +- Test: `plugins/doc-sweep/hooks/doc-classify.test.mjs` + +**Interfaces:** +- Produces (exported for tests): `globToRegExp(glob: string): RegExp`, `classify(files: string[], opts: {docPatterns?: string[], excludeDirs?: string[]}): {nonDoc: string[], docChanged: boolean}`, `DEFAULT_DOC_PATTERNS: string[]`. +- CLI: reads newline-separated paths on stdin, optional `--config ` (JSON `{docPatterns?, excludeDirs?}`), prints `{"nonDoc":[...],"docChanged":bool}` to stdout. + +- [ ] **Step 1: Write the failing test** + +```js +// plugins/doc-sweep/hooks/doc-classify.test.mjs +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { globToRegExp, classify, DEFAULT_DOC_PATTERNS } from './doc-classify.mjs'; + +test('globToRegExp: * stays within a segment', () => { + assert.ok(globToRegExp('README*.md').test('README.md')); + assert.ok(globToRegExp('**/README*.md').test('sub/README.dev.md')); + assert.ok(!globToRegExp('README*.md').test('sub/README.md')); +}); + +test('globToRegExp: ** spans directories', () => { + assert.ok(globToRegExp('.claude/**/*.md').test('.claude/context/audience-rules.md')); + assert.ok(globToRegExp('docs/**').test('docs/api/ref.md')); +}); + +test('.claude markdown is a doc under the default set', () => { + const r = classify(['.claude/context/audience-rules.md'], {}); + assert.equal(r.docChanged, true); + assert.deepEqual(r.nonDoc, []); +}); + +test('default globs: README/docs/CHANGELOG are docs, src is not', () => { + const r = classify(['README.md', 'docs/x.md', 'CHANGELOG.md', 'src/app.js'], {}); + assert.equal(r.docChanged, true); + assert.deepEqual(r.nonDoc, ['src/app.js']); +}); + +test('docPatterns override replaces the default', () => { + const r = classify(['README.md', 'docs/x.md'], { docPatterns: ['docs/**'] }); + assert.deepEqual(r.nonDoc, ['README.md']); + assert.equal(r.docChanged, true); +}); + +test('excludeDirs paths are neither doc nor non-doc', () => { + const r = classify(['vendor/lib/a.js', 'vendor/lib/README.md'], { excludeDirs: ['vendor'] }); + assert.deepEqual(r.nonDoc, []); + assert.equal(r.docChanged, false); +}); + +test('DEFAULT_DOC_PATTERNS includes a .claude glob', () => { + assert.ok(DEFAULT_DOC_PATTERNS.some((p) => p.includes('.claude'))); +}); + +test('test-only change is exempt (empty nonDoc, docChanged false)', () => { + const r = classify(['src/app.test.js', 'tests/unit/thing.js'], {}); + assert.deepEqual(r.nonDoc, []); + assert.equal(r.docChanged, false); +}); + +test('tests alongside real code still enforce', () => { + const r = classify(['src/app.test.js', 'src/app.js'], {}); + assert.deepEqual(r.nonDoc, ['src/app.js']); +}); + +test('excludeDirs wins over exempt and doc matching', () => { + const r = classify(['vendor/x.test.js', 'vendor/README.md'], { excludeDirs: ['vendor'] }); + assert.deepEqual(r.nonDoc, []); + assert.equal(r.docChanged, false); +}); + +test('configured exemptPatterns replaces the default', () => { + const r = classify(['a.test.js'], { exemptPatterns: ['**/*.gen.js'] }); + assert.deepEqual(r.nonDoc, ['a.test.js']); // .test.js no longer exempt +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `node --test plugins/doc-sweep/hooks/doc-classify.test.mjs` +Expected: FAIL — `Cannot find module './doc-classify.mjs'`. + +- [ ] **Step 3: Write minimal implementation** + +```js +// plugins/doc-sweep/hooks/doc-classify.mjs +import { readFileSync } from 'node:fs'; + +// Doc globs. `**/` is an optional prefix (matches root and nested); `**` at the tail +// matches everything under a dir. Kept consistent with the audience-rules base +// (all *.md under .claude/ are Claude-facing docs). +export const DEFAULT_DOC_PATTERNS = [ + '**/CLAUDE*.md', + '**/README*.md', + '**/CHANGELOG.md', + 'docs/**', + '**/docs/**', + '.claude/**/*.md', + '**/.claude/**/*.md', +]; + +// First-party changes that do not require docs (distinct from excludeDirs, which is vendored). +// Evaluated after excludeDirs and before docPatterns. Default: common test files. +export const DEFAULT_EXEMPT_PATTERNS = [ + '**/*.test.*', + '**/*.spec.*', + '**/test/**', + '**/tests/**', + '**/__tests__/**', + '**/*_test.go', + '**/*_test.py', +]; + +// Translate a glob to an anchored RegExp. Supported: `**/` (optional dir prefix), +// `**` (across segments), `*` (within one segment), literals. No braces/char-classes. +export function globToRegExp(glob) { + let re = ''; + for (let i = 0; i < glob.length; i++) { + const c = glob[i]; + if (c === '*' && glob[i + 1] === '*') { + i++; // consume second * + if (glob[i + 1] === '/') { i++; re += '(?:.*/)?'; } // **/ -> optional dir prefix + else re += '.*'; // ** -> anything incl. / + } else if (c === '*') { + re += '[^/]*'; + } else if ('.+^${}()|[]\\/'.includes(c)) { + re += '\\' + c; + } else { + re += c; + } + } + return new RegExp('^' + re + '$'); +} + +export function classify(files, { docPatterns, excludeDirs, exemptPatterns } = {}) { + const docRes = ((docPatterns && docPatterns.length) ? docPatterns : DEFAULT_DOC_PATTERNS).map(globToRegExp); + const exemptRes = ((exemptPatterns && exemptPatterns.length) ? exemptPatterns : DEFAULT_EXEMPT_PATTERNS).map(globToRegExp); + const excludes = excludeDirs || []; + const nonDoc = []; + let docChanged = false; + for (const f of files) { + if (!f) continue; + if (excludes.some((ex) => f === ex || f.startsWith(ex + '/'))) continue; // excluded (vendored) + if (exemptRes.some((r) => r.test(f))) continue; // exempt (e.g. tests) + if (docRes.some((r) => r.test(f))) docChanged = true; // documentation + else nonDoc.push(f); // doc-requiring + } + return { nonDoc, docChanged }; +} + +function main() { + const argv = process.argv.slice(2); + let cfgPath = null; + for (let i = 0; i < argv.length; i++) if (argv[i] === '--config') cfgPath = argv[++i]; + let opts = {}; + if (cfgPath) { + try { + const c = JSON.parse(readFileSync(cfgPath, 'utf8')); + opts = { docPatterns: c.docPatterns, excludeDirs: c.excludeDirs, exemptPatterns: c.exemptPatterns }; + } catch { /* fall back to defaults */ } + } + let input = ''; + process.stdin.setEncoding('utf8'); + process.stdin.on('data', (d) => (input += d)); + process.stdin.on('end', () => { + const files = input.split('\n').map((s) => s.trim()).filter(Boolean); + process.stdout.write(JSON.stringify(classify(files, opts))); + }); +} + +// Run as CLI only when invoked directly (not when imported by tests). +if (import.meta.url === `file://${process.argv[1]}` || process.argv[1]?.endsWith('doc-classify.mjs')) { + main(); +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `node --test plugins/doc-sweep/hooks/doc-classify.test.mjs` +Expected: PASS (all tests). + +- [ ] **Step 5: Smoke-test the CLI** + +Run: `printf 'src/app.js\n.claude/context/audience-rules.md\n' | node plugins/doc-sweep/hooks/doc-classify.mjs` +Expected: `{"nonDoc":["src/app.js"],"docChanged":true}` + +- [ ] **Step 6: Commit** + +```bash +git add plugins/doc-sweep/hooks/doc-classify.mjs plugins/doc-sweep/hooks/doc-classify.test.mjs +git commit -m "feat(doc-sweep): add shared doc-classify.mjs classifier" +``` + +--- + +### Task 2: `docs-ci-check.sh` delegates to the classifier + +**Files:** +- Modify: `plugins/doc-sweep/hooks/docs-ci-check.sh` +- Test: `plugins/doc-sweep/hooks/test-docs-ci-check.sh` + +**Interfaces:** +- Consumes: `doc-classify.mjs` CLI (`{nonDoc, docChanged}` JSON). + +- [ ] **Step 1: Add a failing regression test** — append to `test-docs-ci-check.sh` before `exit $fail`: + +```bash +# .claude markdown counts as a doc (regression: was misclassified non-doc) +repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" src/app.js; commitfile "$repo" .claude/context/audience-rules.md +run "$base" "$repo"; assert_pass $? ".claude/*.md change satisfies the check" + +# test-only change is exempt → passes without an ack +repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" src/app.test.js +run "$base" "$repo"; assert_pass $? "test-only change passes (exempt)" + +# tests + real code still enforces +repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" src/app.test.js; commitfile "$repo" src/app.js +run "$base" "$repo"; assert_fail $? "tests + src still enforces" +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `bash plugins/doc-sweep/hooks/test-docs-ci-check.sh` +Expected: FAIL on `.claude/*.md change satisfies the check` (current inline `is_doc` misses `.claude/**`). + +- [ ] **Step 3: Replace inline classification with the classifier.** In `docs-ci-check.sh`, delete the `is_doc()` function, the `docmode`/`excludes` config block, and the `while read ... is_doc` loop. Resolve the script dir and call the module: + +```bash +here="$(cd "$(dirname "$0")" && pwd)" +cfg_arg=""; [ -n "${1:-}" ] && [ -f "$1" ] && cfg_arg="--config $1" +# shellcheck disable=SC2086 +result="$(printf '%s\n' "$changed" | node "$here/doc-classify.mjs" $cfg_arg 2>/dev/null)" || { warn "classify failed; passing (fail-open)"; pass; } +docchanged="$(printf '%s' "$result" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write(String(JSON.parse(s).docChanged)))' 2>/dev/null)" +nondoc="$(printf '%s' "$result" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write((JSON.parse(s).nonDoc||[]).join("\n")))' 2>/dev/null)" +[ -z "$nondoc" ] && pass +[ "$docchanged" = "true" ] && pass +has_ack && pass +``` + +Keep the failure message loop reading `$nondoc` (now newline-separated). + +- [ ] **Step 4: Run tests to verify pass** + +Run: `bash plugins/doc-sweep/hooks/test-docs-ci-check.sh` +Expected: all `ok:` including the new regression. + +- [ ] **Step 5: Lint** + +Run: `bash -n plugins/doc-sweep/hooks/docs-ci-check.sh && shellcheck plugins/doc-sweep/hooks/docs-ci-check.sh` +Expected: no errors. + +- [ ] **Step 6: Commit** + +```bash +git add plugins/doc-sweep/hooks/docs-ci-check.sh plugins/doc-sweep/hooks/test-docs-ci-check.sh +git commit -m "refactor(doc-sweep): docs-ci-check delegates to doc-classify.mjs" +``` + +--- + +### Task 3: `revise-push-guard.sh` delegates to the classifier + +**Files:** +- Modify: `plugins/doc-sweep/hooks/revise-push-guard.sh` +- Test: `plugins/doc-sweep/hooks/test-revise-push-guard.sh` + +**Interfaces:** +- Consumes: `doc-classify.mjs` CLI. + +- [ ] **Step 1: Add a failing regression test** — append before `exit $fail`: + +```bash +# .claude markdown-only change since marker → allow (regression) +repo="$(mkrepo)"; mark "$repo"; commitfile "$repo" .claude/context/audience-rules.md +out="$(run 'git push' "$repo" "$no_cfg")"; assert_allow "$out" ".claude/*.md change allows push" + +# test-only change since marker → allow (exempt) +repo="$(mkrepo)"; mark "$repo"; commitfile "$repo" src/app.test.js +out="$(run 'git push' "$repo" "$no_cfg")"; assert_allow "$out" "test-only change allows push (exempt)" +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `bash plugins/doc-sweep/hooks/test-revise-push-guard.sh` +Expected: FAIL on `.claude/*.md change allows push`. + +- [ ] **Step 3: Replace inline classification.** Remove `is_doc()`, the `docmode` read, and the `while read ... is_doc` loop that builds `$nondoc`. Compute `$nondoc` via the module: + +```bash +here="$(cd "$(dirname "$0")" && pwd)" +cfg_arg=""; [ -n "$cfg" ] && [ -f "$cfg" ] && cfg_arg="--config $cfg" +# shellcheck disable=SC2086 +nondoc="$(printf '%s\n' "$changed" | node "$here/doc-classify.mjs" $cfg_arg 2>/dev/null \ + | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write((JSON.parse(s).nonDoc||[]).join(" ")))' 2>/dev/null)" +``` + +For the per-commit `[skip docs]` loop, replace the inner per-commit `is_doc` scan with a classifier call on that commit's files: + +```bash +cfiles="$(git diff-tree --no-commit-id --name-only -r "$c" 2>/dev/null)" +cnon="$(printf '%s\n' "$cfiles" | node "$here/doc-classify.mjs" $cfg_arg 2>/dev/null \ + | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write((JSON.parse(s).nonDoc||[]).join("\n")))' 2>/dev/null)" +[ -n "$cnon" ] && { unacked=1; break; } +``` + +Keep `trigger`/`excludeDirs`→config, bypass, marker, and fail-open logic. Note `excludeDirs` now flows to the classifier via the config file (still read into `$cfg`). + +- [ ] **Step 4: Run tests to verify pass** + +Run: `bash plugins/doc-sweep/hooks/test-revise-push-guard.sh` +Expected: all `ok:` including both `[skip docs]` cases and the new `.claude` regression. + +- [ ] **Step 5: Lint** + +Run: `bash -n plugins/doc-sweep/hooks/revise-push-guard.sh && shellcheck plugins/doc-sweep/hooks/revise-push-guard.sh` +Expected: no errors. + +- [ ] **Step 6: Commit** + +```bash +git add plugins/doc-sweep/hooks/revise-push-guard.sh plugins/doc-sweep/hooks/test-revise-push-guard.sh +git commit -m "refactor(doc-sweep): push-guard delegates to doc-classify.mjs; retire inline is_doc" +``` + +--- + +### Task 4: Retire `docMode`; declare `docPatterns` in audience-rules + +**Files:** +- Modify: `plugins/doc-sweep/context/audience-rules.md` (default overlay) +- Verify: no remaining `docMode` in `plugins/doc-sweep/hooks/*.sh` + +- [ ] **Step 1: Grep for stragglers** + +Run: `grep -rn docMode plugins/doc-sweep/hooks/ || echo "clean"` +Expected: `clean` (Tasks 2–3 removed script usages). Any hit → remove it. + +- [ ] **Step 2: Add the `docPatterns` block** to `audience-rules.md`, co-located with the excludeDirs concept: + +```markdown +## Machine-readable doc-file-set + +The guard scripts (CI check + push hook) classify paths via `doc-classify.mjs`, which reads a +`docPatterns` glob list (the machine-readable twin of the audience table above) and an +`exemptPatterns` list of first-party changes that don't require docs (default: tests). Defaults when +unset: docs = `**/CLAUDE*.md`, `**/README*.md`, `**/CHANGELOG.md`, `docs/**`, `.claude/**/*.md`; +exempt = common test globs. Installers persist the project's choices here (mirrored into the +per-install config JSON, like `excludeDirs`): + + docPatterns: + - "**/CLAUDE*.md" + - "**/README*.md" + - "**/CHANGELOG.md" + - "docs/**" + - ".claude/**/*.md" + exemptPatterns: # first-party paths that don't require docs (default: tests) + - "**/*.test.*" + - "**/*.spec.*" + - "**/test/**" + - "**/tests/**" + - "**/__tests__/**" +``` + +- [ ] **Step 3: Commit** + +```bash +git add plugins/doc-sweep/context/audience-rules.md +git commit -m "docs(doc-sweep): declare machine-readable docPatterns; retire docMode" +``` + +--- + +### Task 5: Install skills vendor the classifier + write `docPatterns` + +**Files:** +- Modify: `plugins/doc-sweep/skills/install-docs-ci/SKILL.md`, `.../install-docs-ci/evals/evals.json` +- Modify: `plugins/doc-sweep/skills/install-revise-hook/SKILL.md`, `.../install-revise-hook/evals/evals.json` + +- [ ] **Step 1: install-docs-ci SKILL.md** — in step 4 (copy script), also copy `../../hooks/doc-classify.mjs` to `.github/doc-sweep/doc-classify.mjs`; change the config write from `docMode` to `docPatterns`; in Uninstall, also delete the vendored `doc-classify.mjs`. Update the summary's "Doc-file set" line to reflect `docPatterns`. + +- [ ] **Step 2: install-revise-hook SKILL.md** — in step 3 (copy hook), also copy `../../hooks/doc-classify.mjs` next to the hook; step 5 config writes `docPatterns` not `docMode`; Uninstall deletes the copied `doc-classify.mjs`. + +- [ ] **Step 3: Update both `evals/evals.json`** — replace assertions mentioning `docMode` with `docPatterns`, and add an assertion that the installer copies `doc-classify.mjs` alongside the script and removes it on uninstall. + +- [ ] **Step 4: Validate skill frontmatter** + +Run: `claude plugin validate plugins/doc-sweep` +Expected: passes with warnings (version only). + +- [ ] **Step 5: Regenerate benchmarks** + +Run: `/skill-gate plugins/doc-sweep/skills/install-docs-ci` then `/skill-gate plugins/doc-sweep/skills/install-revise-hook` +Then: `node scripts/check-skill-gate.mjs` +Expected: all skills pass (≥ 0.9). + +- [ ] **Step 6: Commit** + +```bash +git add plugins/doc-sweep/skills/install-docs-ci plugins/doc-sweep/skills/install-revise-hook +git commit -m "feat(doc-sweep): installers vendor doc-classify.mjs and record docPatterns" +``` + +--- + +### Task 6: CI wiring, docs, validation + +**Files:** +- Modify: `.github/workflows/validate.yml` +- Modify: `plugins/doc-sweep/README.md`, `plugins/doc-sweep/CHANGELOG.md` + +- [ ] **Step 1: Wire the node test into CI** — in `validate.yml`, next to the existing `node --test` steps, add: + +```yaml + - name: doc-classify unit tests + run: node --test plugins/doc-sweep/hooks/doc-classify.test.mjs +``` + +- [ ] **Step 2: Update README + CHANGELOG** — README: replace `docMode` doc-file-set copy with `docPatterns`; document `exemptPatterns` (test-only changes pass without an ack; configurable); note a single `doc-classify.mjs` backs both guards. CHANGELOG: add a "Unify doc classification" entry noting the `.claude/**` fix, `docMode` retirement (BREAKING for stale configs), the shared module, and the new test-exempt behavior. + +- [ ] **Step 3: Full local gate** + +Run: +```bash +node --test plugins/doc-sweep/hooks/doc-classify.test.mjs \ + && bash plugins/doc-sweep/hooks/test-docs-ci-check.sh \ + && bash plugins/doc-sweep/hooks/test-revise-push-guard.sh \ + && node scripts/validate-marketplace.mjs \ + && node scripts/check-skill-gate.mjs \ + && openspec validate --strict --all \ + && node scripts/check-openspec-hygiene.mjs +``` +Expected: all pass. + +- [ ] **Step 4: Confirm funbox dogfood picks up the fix** — funbox's `docs-staleness.yml` runs the check with no config, so the built-in default (incl. `.claude/**`) applies automatically; no workflow change needed. Note this in the CHANGELOG entry. + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/validate.yml plugins/doc-sweep/README.md plugins/doc-sweep/CHANGELOG.md +git commit -m "ci+docs(doc-sweep): run doc-classify tests; document docPatterns" +``` + +--- + +## Self-Review + +- **Spec coverage:** `doc-classification` capability → Task 1 (module, default incl. `.claude/**`, glob, config resolution, **exemptPatterns** incl. test-only-exempt / tests+code-enforces / excludeDirs-wins). `docs-staleness-ci` MODIFIED (delegate + vendor + docPatterns; test-only-passes scenario) → Tasks 2, 5. `revise-docs-push-guard` MODIFIED (delegate, retire docMode, vendor, docPatterns) → Tasks 3, 4, 5. No spec requirement left without a task. +- **Placeholder scan:** none — all code steps carry real code. +- **Type consistency:** `classify()` returns `{nonDoc, docChanged}` everywhere; scripts read `.nonDoc`/`.docChanged` verbatim; `globToRegExp`/`DEFAULT_DOC_PATTERNS`/`DEFAULT_EXEMPT_PATTERNS` names match between module and tests. diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/proposal.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/proposal.md new file mode 100644 index 0000000..80450b9 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/proposal.md @@ -0,0 +1,63 @@ +## Why + +"What is a doc?" is defined in three places that disagree: the audience-rules prose (used by the +skills), and a copy-pasted `is_doc()` in each of `revise-push-guard.sh` and `docs-ci-check.sh`. The +scripts classify `.claude/**/*.md` as **non-docs**, but the audience-rules call them docs — so a PR +that edits only `.claude/context/audience-rules.md` (doc-sweep's own canonical doc) **fails** the +docs-staleness check. The duplicated classifier will keep drifting. This change collapses the +script-side definition to a single shared, declarative source. + +## What Changes + +- **New shared classifier** `plugins/doc-sweep/hooks/doc-classify.mjs` — one node module that, + given a file list, classifies each path as doc / non-doc / excluded / exempt using `docPatterns`, + `excludeDirs`, and `exemptPatterns` sets (or built-in defaults). Contains a small in-house glob + matcher (`*`, `**`), no external deps; unit-tested. +- **Exempt categories that don't require docs** — a new `exemptPatterns` glob set (default: common + test files: `**/*.test.*`, `**/*.spec.*`, `**/test/**`, `**/tests/**`, `**/__tests__/**`, + `**/*_test.go`, `**/*_test.py`; configurable per project). A change whose only non-doc paths are + exempt passes the guards **without** an ack; a doc-requiring path alongside it still enforces. + This upgrades the check from a crude "any code change needs docs" to "only doc-worthy code needs + docs." Distinct from `excludeDirs` (vendored/external) by intent. +- **Built-in default doc-set now includes `.claude/**/*.md`** (alongside `CLAUDE*.md`, `README*.md`, + `CHANGELOG.md`, `docs/**`) — **fixes the divergence bug**. +- **Both scripts become thin git-plumbing wrappers** — `revise-push-guard.sh` and + `docs-ci-check.sh` drop their duplicated `is_doc`/`docMode`/config-parsing and delegate + classification to `doc-classify.mjs`. Git logic (merge-base, diff, log, per-commit `[skip docs]`) + stays in bash. +- **`docMode` is hard-retired** (**BREAKING** for any config still carrying it — it silently falls + back to the default set until regenerated). Removed from both scripts, the config schema, and both + install skills, which now record `docPatterns`. +- **`docPatterns` block added to the audience-rules** (base default overlay), co-located with + `excludeDirs`, as the human-authoritative twin of the prose table; mirrored into the per-install + config JSON exactly like `excludeDirs` (no new markdown parser). +- **Install skills vendor `doc-classify.mjs`** alongside their script. + +Non-goals: wiring the skills' runtime to `docPatterns` (they stay prose-driven; they classify by +audience, not doc-vs-non-doc); changing `[skip docs]` semantics, the merge-base baseline, or the +advisory posture; any LLM-in-CI judgement. + +## Capabilities + +### New Capabilities +- `doc-classification`: the shared doc/non-doc/excluded/exempt classifier — its `docPatterns`, + `excludeDirs`, and `exemptPatterns` inputs, the built-in defaults (doc-set including + `.claude/**/*.md`; exempt-set covering common tests), the glob semantics, and the + config-resolution order. Owned by `doc-classify.mjs`. + +### Modified Capabilities +- `docs-staleness-ci`: the CI check delegates classification to `doc-classification` instead of an + inline `is_doc`/`docMode`; the installer vendors `doc-classify.mjs` and records `docPatterns`. +- `revise-docs-push-guard`: the hook delegates classification to `doc-classification`; `docMode` + removed; the installer vendors `doc-classify.mjs` and records `docPatterns`. + +## Impact + +- **New**: `plugins/doc-sweep/hooks/doc-classify.mjs` + `doc-classify.test.mjs` (node --test). +- **Modified**: `revise-push-guard.sh`, `docs-ci-check.sh` and their shell test suites; both + install skills (`install-revise-hook`, `install-docs-ci`) SKILL.md + regenerated eval benchmarks; + `context/audience-rules-base.md` / `audience-rules.md` (add `docPatterns`); doc-sweep + `README.md`/`CHANGELOG.md`; the two modified capability specs. +- **Validation**: new node test wired into CI (alongside the existing `node --test` suites); scripts + still pass `bash -n` + ShellCheck; skills must clear `claude plugin validate` + skill-gate. +- No new runtime dependency beyond node (already required); no secrets. diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/retrospective.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/retrospective.md new file mode 100644 index 0000000..81bd49f --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/retrospective.md @@ -0,0 +1,86 @@ +# Retrospective — unify-doc-classification + +## §0 Evidence + +- **Commits:** 15 on branch (`178c249..HEAD`); 2 propose, 13 implementation/fix/docs. +- **Diff size:** 25 files, +1591 / −345; ~16 non-artifact files touched. +- **Tasks:** 19/19 `[x]`. +- **Subagents dispatched:** ~19 — 6 implementers, 4 fix subagents, 6 per-task adversarial reviewers, + 1 final whole-branch adversarial review (opus), 2 benchmark eval-runners. +- **New external dependencies:** 0 (classifier is `node:` builtins only). +- **OpenSpec validate at archive:** 5/5 strict pass. +- **Test coverage signal:** `doc-classify.test.mjs` 13 cases; `test-docs-ci-check.sh` + + `test-revise-push-guard.sh` shell suites; both install-skill benchmarks 1.0 (18/18, 19/19). +- **Post-merge bugs:** n/a (pre-merge). +- **Commit chain:** propose → exempt fold-in → classifier → 2 script refactors (+2 fixes) → + docMode retire/audience-rules → installers (+fix) → benchmarks → CI/docs → final-review fixes. + +## §1 Wins + +- **The adversarial reviews earned their cost.** Three real defects were caught that all-green tests + missed: the merge-commit `[skip docs]` bypass (`09d7491`), the unguarded downstream JSON parse that + silently masked bad classifier output (`72f6b14`), and the install-reconfigure path leaving the + vendored classifier missing → permanent silent fail-open (`5f94af9`). None had a failing test + before review; each got one after. +- **The originating bug is fixed and regression-locked:** `.claude/**/*.md` now classifies as docs in + both guards, with tests in all three suites. +- **Single source of truth achieved:** one `doc-classify.mjs`, one default set, consumed identically + by both guards (final review confirmed no divergence in invocation or parsing). +- **Scope expansion handled cleanly:** the user's mid-apply `exemptPatterns` idea was folded into the + design (brainstorm Q6, D7) before any code, not bolted on. + +## §2 Misses + +- 🟡 **Two script refactors (Tasks 2, 3) both needed fix rounds** for robustness gaps the plan's + example code carried (unquoted `$cfg_arg`, unguarded parse, merge-commit diff-tree). The plan + transcribed working-but-fragile snippets; the adversarial reviews are what caught them. +- 📌 **Doc/code drift appeared twice** (audience-rules exempt list missing go/py globs → `c0f5b33`; + README/CHANGELOG claiming `exemptPatterns` is installer-configurable → `d66e650`) — ironic for a + change whose whole point is killing drift. Both caught (controller + final review) and fixed. +- 📌 **One eval assertion was miswritten** (required `docPatterns` for the default choice, which the + skill correctly omits) — caught during benchmark regen, corrected. + +## §3 Plan deviations + +- Task 2 rewrote a pre-existing `docMode:minimal` test to its `docPatterns` equivalent (docMode + retired) — a necessary, disclosed deviation. +- Fix rounds added tests/behavior beyond the plan (merge-commit handling, JSON-parse guard, + trailing-slash normalization, empty-config fallback, reconfigure re-copy) — all from adversarial + findings, all net improvements to the plan's baseline. + +## §4 Skill / workflow compliance + +Apply-phase skills for the superpowers-bridge schema: +- `using-git-worktrees` — ✓ (Step 0 detection: already on the dedicated feature branch; worked in + place to preserve the schema's "complete cycle in one PR" property — a legitimate work-in-place path). +- `subagent-driven-development` — ✓ (fresh implementer per task, per-task review, final review). +- `test-driven-development` (transitive) — ✓ (each task added failing tests first). +- `requesting-code-review` (transitive) — ✓ (per-task + final whole-branch review; made explicitly + adversarial per user directive). +- `finishing-a-development-branch` — ✓ (PR step). + +### Deliberately Skipped Skills +None. Every apply-phase skill was used. + +## §5 Surprises + +- **The local grep crashes on `grep -F '[skip docs]'`** (SIGABRT) on this Windows Git Bash — surfaced + in the *prior* change and honored here by keeping all matching in `node`. A reminder that "it's just + a grep" isn't portable. +- **The plan's own example code was the main defect source**, not misunderstanding — the reviewers + found fragility in code the plan handed the implementers verbatim. Adversarial review of + plan-mandated code is worth it even when implementers transcribe faithfully. + +## §6 Promote candidates → long-term learning + +- [ ] 📌 Plan example code is a starting point, not vetted — adversarially review plan-mandated + snippets, not just implementer output. + → **Promote to** CLAUDE.md / schema note + > **Why**: 3 of 4 fix rounds this cycle addressed fragility in code the plan supplied verbatim. + > **How to apply**: when a plan hands complete code, the per-task review should probe that code's + > edge cases (quoting, error paths, merge/edge git behavior), not assume the plan vetted them. +- [ ] 📌 A "kill the drift" change is itself drift-prone — check doc↔code↔spec default lists match + exactly before closing. + > **Why**: two doc/code drifts appeared in this very change. + > **How to apply**: when a change centralizes a default (a pattern list, a config schema), grep the + > default across code, bundled docs, README, and specs as an explicit verify step. diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/doc-classification/spec.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/doc-classification/spec.md new file mode 100644 index 0000000..395f528 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/doc-classification/spec.md @@ -0,0 +1,106 @@ +## ADDED Requirements + +### Requirement: Single shared classifier module + +doc-sweep SHALL provide a single node module (`doc-classify.mjs`) that classifies a list of file +paths into documentation, non-documentation, and excluded, and SHALL be the **sole** doc/non-doc +classifier used by both the docs-staleness CI check and the revise-docs push-guard hook (neither +SHALL carry its own inline classification logic). The module SHALL read a newline-separated list of +file paths on stdin, accept an optional `--config ` argument, and emit on stdout a JSON object +`{ "nonDoc": [], "docChanged": }` where `nonDoc` lists the changed non-doc, +non-excluded paths and `docChanged` is true iff at least one changed path is a documentation file. +The module SHALL have no external (non-builtin) dependencies and SHALL parse JSON with node. + +#### Scenario: Both scripts delegate to the module + +- **WHEN** the CI check or the push-guard hook needs to classify changed files +- **THEN** it pipes the file list to `doc-classify.mjs` and acts on the returned `nonDoc`/`docChanged` result, rather than applying its own inline doc/non-doc patterns + +#### Scenario: Output shape + +- **WHEN** the module is given a mix of doc, non-doc, and excluded paths +- **THEN** it emits `{ "nonDoc": [...only the non-doc non-excluded paths...], "docChanged": true|false }` as JSON on stdout + +### Requirement: Built-in default doc-set includes `.claude/**` + +When no `docPatterns` is configured, the module SHALL treat a path as a documentation file iff it +matches the built-in default set: `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**`, and +`.claude/**/*.md` (at any directory depth). This default SHALL be consistent with the audience-rules +base, under which all `*.md` files beneath `.claude/` are Claude-facing documentation. + +#### Scenario: A .claude markdown change counts as a doc + +- **WHEN** the only changed path is `.claude/context/audience-rules.md` and no `docPatterns` override is configured +- **THEN** the module classifies it as a documentation file (`docChanged` true, `nonDoc` empty) + +#### Scenario: Default doc globs + +- **WHEN** the changed set is `README.md`, `docs/api/x.md`, and `CHANGELOG.md` with no override +- **THEN** all three classify as documentation and `nonDoc` is empty + +### Requirement: Configurable patterns and glob semantics + +The module SHALL resolve `docPatterns`, `excludeDirs`, and `exemptPatterns` from the `--config` JSON +when present, otherwise from the built-in defaults (`excludeDirs` defaulting to empty). For +`docPatterns` and `exemptPatterns`, a **non-empty** configured list replaces the corresponding +built-in default; an **empty or omitted** list falls back to the built-in default (an empty list +does NOT mean "match nothing" — that would enforce on every changed path). Glob matching SHALL +support `*` (matches within a single path segment) and `**` (matches across segments), plus literal +segments. A path under any `excludeDirs` entry (the entry itself or a descendant) SHALL be treated +as **excluded** — neither documentation nor non-documentation — and SHALL NOT appear in `nonDoc` +nor set `docChanged`. An `excludeDirs` entry with a trailing slash SHALL be normalized (the trailing +slash stripped) before comparison, so `["vendor/"]` excludes `vendor/a.js` the same as `["vendor"]`. + +#### Scenario: docPatterns override replaces the default + +- **WHEN** the config sets `docPatterns` to `["docs/**"]` and the changed set is `README.md` and `docs/x.md` +- **THEN** only `docs/x.md` is a doc; `README.md` is non-doc (the default globs no longer apply) + +#### Scenario: Empty docPatterns falls back to the default + +- **WHEN** the config sets `docPatterns` to `[]` and the changed set is `README.md` +- **THEN** the built-in default applies and `README.md` classifies as documentation (`docChanged` true, `nonDoc` empty) + +#### Scenario: Excluded directory is ignored + +- **WHEN** the config sets `excludeDirs` to `["vendor"]` and the changed set is `vendor/lib/a.js` and `vendor/lib/README.md` +- **THEN** both paths are excluded — `nonDoc` is empty and `docChanged` is false + +#### Scenario: excludeDirs entry with a trailing slash still excludes + +- **WHEN** the config sets `excludeDirs` to `["vendor/"]` and the changed path is `vendor/a.js` +- **THEN** the path is excluded — `nonDoc` is empty and `docChanged` is false + +#### Scenario: `**` spans directories + +- **WHEN** `docPatterns` contains `.claude/**/*.md` and the changed path is `.claude/context/audience-rules.md` +- **THEN** the path matches and classifies as documentation + +### Requirement: Exempt patterns for changes that do not require docs + +The module SHALL support an `exemptPatterns` glob list identifying first-party changes that do not +require documentation (distinct from `excludeDirs`, which marks vendored/external paths). A path +matching `exemptPatterns` — evaluated AFTER `excludeDirs` and BEFORE `docPatterns` — SHALL be treated +as **exempt**: it SHALL NOT appear in `nonDoc` and SHALL NOT set `docChanged`. When `exemptPatterns` +is empty or omitted, the built-in default `exemptPatterns` SHALL apply, covering common test files: +`**/*.test.*`, `**/*.spec.*`, `**/test/**`, `**/tests/**`, `**/__tests__/**`, `**/*_test.go`, +`**/*_test.py`. A **non-empty** configured `exemptPatterns` replaces the default (an empty list +falls back to the default rather than exempting nothing). The consequence is that a change whose +only non-doc, non-excluded paths are exempt yields an empty `nonDoc` and therefore passes the +staleness guards without an acknowledgment, while any doc-requiring path alongside it still +enforces. + +#### Scenario: Test-only change is exempt + +- **WHEN** the changed set is `src/app.test.js` and `tests/unit/foo_spec.rb` (matching the default exempt globs) with no override +- **THEN** both are exempt — `nonDoc` is empty and `docChanged` is false + +#### Scenario: Tests alongside real code still enforce + +- **WHEN** the changed set is `src/app.test.js` and `src/app.js` +- **THEN** the test file is exempt but `src/app.js` remains non-doc, so `nonDoc` is `["src/app.js"]` + +#### Scenario: excludeDirs wins over exempt and doc matching + +- **WHEN** a path is under a configured `excludeDirs` entry AND would also match `exemptPatterns` or `docPatterns` +- **THEN** it is excluded (evaluated first) and counts as neither doc, non-doc, nor exempt diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/docs-staleness-ci/spec.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/docs-staleness-ci/spec.md new file mode 100644 index 0000000..9fbcb87 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/docs-staleness-ci/spec.md @@ -0,0 +1,82 @@ +## MODIFIED Requirements + +### Requirement: Deterministic PR-time staleness check + +doc-sweep SHALL provide a deterministic, no-LLM, no-secret check that runs on a pull request +and evaluates documentation staleness against the **merge base** of the PR (no committed marker +or persisted state). The check SHALL delegate classification of each file changed between the merge +base and the PR head to the shared `doc-classification` module (`doc-classify.mjs`) — it SHALL NOT +carry its own inline doc/non-doc patterns. Files under configured excluded directories are ignored +entirely (neither doc nor non-doc), per that module. The check SHALL **fail** if and only if at least +one non-doc, non-excluded file changed, no doc file changed, and no acknowledgment is present. +Otherwise it SHALL **pass**. When it fails, it SHALL emit a message that names the offending non-doc +paths and states every way to clear it (update docs, or add a `[skip docs]` line to any commit +message in the PR range or to the PR body). The check SHALL parse the event/diff with `node` (not +`jq`), and SHALL be self-contained in a doc-sweep-shipped script paired with the vendored +`doc-classify.mjs`. + +#### Scenario: Code-only change fails + +- **WHEN** a PR changes at least one non-doc, non-excluded file, changes no doc file, and carries no acknowledgment +- **THEN** the check fails and names the offending non-doc paths and the ways to clear it + +#### Scenario: Code plus docs passes + +- **WHEN** a PR changes non-doc files and also changes at least one doc-set file +- **THEN** the check passes with no acknowledgment required + +#### Scenario: Docs-only change passes + +- **WHEN** a PR changes only doc-set files (or only excluded files) +- **THEN** the check passes + +#### Scenario: A .claude doc change satisfies the check + +- **WHEN** a PR changes a non-doc file and also changes `.claude/context/audience-rules.md` +- **THEN** the shared classifier counts the `.claude/**/*.md` change as a doc and the check passes + +#### Scenario: Test-only change passes without an ack + +- **WHEN** a PR changes only files matching the shared classifier's `exemptPatterns` (e.g. `**/*.test.*`) and no doc-requiring file +- **THEN** the classifier returns an empty `nonDoc` and the check passes with no acknowledgment required + +#### Scenario: Excluded paths are ignored + +- **WHEN** a PR changes only files under a configured excluded directory +- **THEN** those files count as neither doc nor non-doc and the check passes + +### Requirement: Manual installer skill scaffolds the workflow + +doc-sweep SHALL provide a manual, model-non-invocable skill (`install-docs-ci`, +`disable-model-invocation: true`, with scoped `allowed-tools`) that installs the check only when +a user runs it, mirroring the `install-revise-hook` pattern. On a **fresh install** it SHALL +scaffold a GitHub Actions workflow file into the target repository's `.github/workflows/` that +invokes the doc-sweep-shipped check script on `pull_request`, vendor **both** the check script and +the shared `doc-classify.mjs` module under `.github/doc-sweep/`, collect the documentation-file set +(recorded as `docPatterns`, NOT the retired `docMode`) and any excluded directories, and print a +structured summary: the workflow path, the doc-set, the ack tokens, that blocking is the maintainer's +branch-protection choice, and how to reconfigure or uninstall by re-running the skill. When an +install already exists it SHALL offer Reconfigure / Uninstall / Cancel and SHALL be idempotent (it +SHALL NOT duplicate the workflow). Uninstall SHALL remove the workflow, the vendored check script, +and the vendored `doc-classify.mjs`. Nothing SHALL be installed until the user runs the skill and +confirms. + +#### Scenario: Fresh install scaffolds, vendors the classifier, and summarizes + +- **WHEN** a user runs the installer and confirms scoping choices +- **THEN** a `pull_request` workflow is written to `.github/workflows/`, both the check script and `doc-classify.mjs` are vendored under `.github/doc-sweep/`, a config recording `docPatterns` (not `docMode`) is written, and a structured summary with reconfigure/uninstall instructions is printed + +#### Scenario: Idempotent re-run + +- **WHEN** the installer is run again in a repo that already has the workflow +- **THEN** it does not duplicate the workflow and offers Reconfigure / Uninstall / Cancel + +#### Scenario: Uninstall removes the vendored classifier too + +- **WHEN** the user chooses uninstall +- **THEN** the scaffolded workflow, the vendored check script, and the vendored `doc-classify.mjs` are all removed, leaving other workflows and settings intact + +#### Scenario: Plugin install alone is inert + +- **WHEN** doc-sweep is installed but the installer skill has not been run +- **THEN** no workflow is scaffolded and no pull request is gated diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/revise-docs-push-guard/spec.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/revise-docs-push-guard/spec.md new file mode 100644 index 0000000..a04420f --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/specs/revise-docs-push-guard/spec.md @@ -0,0 +1,86 @@ +## MODIFIED Requirements + +### Requirement: Opt-in interactive installer + +doc-sweep SHALL provide a manual, model-non-invocable skill that installs the guard only +when a user runs it. On a **fresh install** the installer SHALL collect, via interactive +prompts: settings location (user-global vs project), repo applicability (all repos vs +doc-sweep-enabled only), the documentation-file set (recorded as `docPatterns`, NOT the retired +`docMode`), the **trigger event** (exactly one of `push` or `commit`, with `push` recommended as +default), and bypass/uninstall confirmation. It SHALL then copy the hook script **and the shared +`doc-classify.mjs` module** to a stable, version-independent path, write the chosen configuration +(including `trigger` and `docPatterns`), and merge an idempotent `PreToolUse`/`Bash` hook into +the selected `settings.json` without overwriting unrelated hooks. After writing the hook the +installer SHALL offer to seed the review marker — seed `HEAD` now (reported as an assumption, +with no review performed), run `revise-docs-and-mark` now, or leave it unseeded with a +warning that the next guarded action will block. The installer SHALL finally print a +structured summary: the settings/hook/config paths, the trigger, doc-set, repo scope, marker +state, behavior caveats (only Claude-driven git is gated, `node` is required, the hook fails +open), the bypass tokens, and how to edit or uninstall by re-running the skill. When an +install already exists, the installer SHALL offer Reconfigure / Uninstall / Cancel; +Reconfigure SHALL re-ask the choices pre-filled with the current config, rewrite the config +(and the matcher only if the hook path changed), and leave the marker untouched. + +#### Scenario: Fresh install seeds and summarizes + +- **WHEN** a user runs the installer, selects scoping options including a trigger, and chooses to seed the marker +- **THEN** the hook script and `doc-classify.mjs` are copied to a stable path, a config capturing the choices (including `trigger` and `docPatterns`) is written, a `PreToolUse`/`Bash` entry is added, the marker is set to HEAD, and a structured summary with edit/uninstall instructions is printed + +#### Scenario: Trigger is chosen at install + +- **WHEN** the user selects `commit` as the trigger +- **THEN** the written config records `trigger: "commit"` and the summary reports that commit (not push) is gated + +#### Scenario: Reconfigure an existing install + +- **WHEN** the installer detects an existing install and the user chooses Reconfigure +- **THEN** it re-asks the choices pre-filled, rewrites the config, leaves the review marker unchanged, and prints the updated summary + +#### Scenario: Idempotent re-run + +- **WHEN** the installer is run again in a repo/scope that already has the hook installed +- **THEN** it does not duplicate the hook entry and offers Reconfigure / Uninstall / Cancel + +#### Scenario: Uninstall + +- **WHEN** the user chooses uninstall +- **THEN** the `PreToolUse` entry, the copied hook script, the copied `doc-classify.mjs`, and the config are removed, leaving other settings and the marker file intact + +### Requirement: Configurable staleness gate + +The installed hook SHALL run on `PreToolUse` for `Bash` calls and SHALL gate the git +subcommand named by its configured `trigger` (`push` default, or `commit`). It SHALL deny +the gated command if and only if at least one non-documentation file changed in the range +from the last `revise-docs` marker to `HEAD`. When it denies, it SHALL return a reason that +names the gated verb and instructs the operator to run `revise-docs-and-mark`, commit any +changes, and retry. If only documentation files (or nothing) changed since the marker, it +SHALL allow the command. Doc/non-doc/excluded classification SHALL be delegated to the shared +`doc-classification` module (`doc-classify.mjs`) using the configured `docPatterns` and +`excludeDirs` (or that module's built-in default) — the hook SHALL NOT carry its own inline +doc/non-doc patterns and SHALL NOT read a `docMode`. A command that is not the configured trigger +SHALL be allowed without inspection. + +#### Scenario: Non-doc change blocks the configured trigger + +- **WHEN** the configured trigger command is attempted and a non-doc, non-excluded file changed since the marker +- **THEN** the hook denies it with a reason naming the gated verb and directing the user to run `revise-docs-and-mark`, commit, then retry + +#### Scenario: Doc-only change allows + +- **WHEN** the configured trigger command is attempted and only doc-set files changed since the marker +- **THEN** the hook allows it + +#### Scenario: A .claude doc change allows + +- **WHEN** the only change since the marker is to `.claude/context/audience-rules.md` +- **THEN** the shared classifier counts it as a doc and the hook allows the command + +#### Scenario: Commit trigger ignores push + +- **WHEN** the configured trigger is `commit` and the Bash command is a `git push` +- **THEN** the hook allows the call without inspection + +#### Scenario: Non-trigger command ignored + +- **WHEN** the Bash command is not the configured trigger subcommand +- **THEN** the hook allows the call without inspection diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/tasks.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/tasks.md new file mode 100644 index 0000000..d10b90e --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/tasks.md @@ -0,0 +1,33 @@ +## 1. Shared classifier module + +- [x] 1.1 Create `plugins/doc-sweep/hooks/doc-classify.mjs` (LF): reads a newline-separated file list on stdin, accepts optional `--config `, emits `{"nonDoc":[...],"docChanged":bool}` on stdout; no external deps +- [x] 1.2 Implement the in-house glob matcher supporting `*` (within a segment) and `**` (across segments) plus literals; unit-test its corners +- [x] 1.3 Built-in default doc-set = `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**`, `.claude/**/*.md`; `docPatterns`/`excludeDirs`/`exemptPatterns` from `--config` JSON override the defaults; excluded paths are neither doc nor non-doc +- [x] 1.4 Add `exemptPatterns` (evaluated after `excludeDirs`, before `docPatterns`; dropped from `nonDoc`, doesn't set `docChanged`); built-in default = common test globs (`**/*.test.*`, `**/*.spec.*`, `**/test/**`, `**/tests/**`, `**/__tests__/**`, `**/*_test.go`, `**/*_test.py`) +- [x] 1.5 Add `plugins/doc-sweep/hooks/doc-classify.test.mjs` (`node --test`): default globs, `.claude/**` now a doc, docPatterns override, excludeDirs, `**` spans dirs, output shape, **test-only change exempt**, **tests + real code still non-doc**, **excludeDirs wins over exempt** +- [x] 1.6 Confirm `node --test plugins/doc-sweep/hooks/doc-classify.test.mjs` passes + +## 2. Refactor both scripts to delegate + +- [x] 2.1 `docs-ci-check.sh`: replace inline `is_doc`/`docMode`/excludeDirs parsing with a single pipe of changed files to `doc-classify.mjs --config `; act on `nonDoc`/`docChanged`; keep merge-base, diff, ack, fail-open logic +- [x] 2.2 `revise-push-guard.sh`: replace inline `is_doc`/`docMode` with the classifier for both the range classification and the per-commit `[skip docs]` non-doc detection; keep trigger/bypass/marker/fail-open logic +- [x] 2.3 Update `test-docs-ci-check.sh` and `test-revise-push-guard.sh` to work through the classifier; add regression cases: an `.claude/context/audience-rules.md`-only change passes, and a **test-only change** (e.g. `src/app.test.js`) passes without an ack while tests + `src/app.js` still enforces +- [x] 2.4 `bash -n` + ShellCheck clean on both scripts; both shell test suites green + +## 3. Retire docMode + declare docPatterns + +- [x] 3.1 Remove all `docMode` handling from both scripts and the config schema (classifier owns classification) +- [x] 3.2 Add `docPatterns:` and `exemptPatterns:` blocks to `context/audience-rules.md` (default overlay), co-located with `excludeDirs`, documenting the machine-readable doc-set + the default test-exempt set as the twin of the prose table + +## 4. Install skills + +- [x] 4.1 `install-revise-hook` SKILL.md: copy `doc-classify.mjs` alongside the hook; write `docPatterns` (not `docMode`); uninstall removes the vendored classifier; update its `evals/evals.json` assertions accordingly +- [x] 4.2 `install-docs-ci` SKILL.md: vendor `doc-classify.mjs` under `.github/doc-sweep/`; write `docPatterns`; uninstall removes it; update its `evals/evals.json` assertions +- [x] 4.3 Regenerate both skills' `evals/benchmark.json` via `/skill-gate` so they clear the threshold; `node scripts/check-skill-gate.mjs` passes + +## 5. CI wiring, docs, validation + +- [x] 5.1 Wire `node --test plugins/doc-sweep/hooks/doc-classify.test.mjs` into `.github/workflows/validate.yml` alongside the existing `node --test` suites +- [x] 5.2 Update doc-sweep `README.md` (docPatterns replaces docMode; `exemptPatterns`/test-only-passes; classifier note) and `CHANGELOG.md` +- [x] 5.3 Run `node scripts/validate-marketplace.mjs`, `claude plugin validate plugins/doc-sweep`, `openspec validate --strict --all`, and `node scripts/check-openspec-hygiene.mjs`; fix findings +- [x] 5.4 Confirm funbox's own `docs-staleness.yml` (no-config) now treats `.claude/**` as docs via the fixed default diff --git a/openspec/changes/archive/2026-07-04-unify-doc-classification/verify.md b/openspec/changes/archive/2026-07-04-unify-doc-classification/verify.md new file mode 100644 index 0000000..180645b --- /dev/null +++ b/openspec/changes/archive/2026-07-04-unify-doc-classification/verify.md @@ -0,0 +1,53 @@ +# Verification — unify-doc-classification + +Post-implementation verification of the completed branch (merge-base `178c249` → HEAD, 15 commits), +executed via subagent-driven-development with per-task adversarial reviews + a final whole-branch +adversarial review on the most capable model. + +## 1. Structural validation +`openspec validate --strict --all` → **5/5 pass** (`doc-classification`, `docs-staleness-ci`, +`revise-docs-push-guard`, `doc-scope-exclusion`, `skill-eval-gate`). ✅ + +## 2. Task completion +`tasks.md` → **19/19 `[x]`**. No tasks left open. ✅ + +## 3. Delta spec sync state +Delta specs under `changes/unify-doc-classification/specs/`: +- `doc-classification` (new) → ✗ Needs sync (will be created on archive) +- `docs-staleness-ci` (MODIFIED) → ✗ Needs sync (folds into living spec on archive) +- `revise-docs-push-guard` (MODIFIED) → ✗ Needs sync (folds into living spec on archive) + +All three sync at archive (`openspec archive`). Expected pre-archive state. + +## 4. Design/specs coherence +Spot-checked: design D5 (node module + bash git plumbing) ↔ `doc-classification` "Single shared +classifier module"; D3 (patterns in audience-rules, mirrored to config JSON) ↔ installer specs + +`context/audience-rules.md` block; D7 (exemptPatterns three-way) ↔ "Exempt patterns" requirement + +the docs-staleness-ci test-only scenario. No drift. ✅ (The final review confirmed the DEFAULT +`docPatterns`/`exemptPatterns` match across `doc-classify.mjs`, `audience-rules.md`, README, and the +specs.) + +## 5. Implementation signal +Working tree clean (only gitignored `.superpowers/` scratch untracked). All code changes committed +across the 15-commit range `178c249..HEAD`. ✅ + +## 6. Front-door routing leak +`ls docs/superpowers/specs/*.md` → none. Brainstorm/plan output was routed to the change directory +(`brainstorm.md`, `plan.md`), not `docs/superpowers/`. ✅ + +## 7. Deferred dogfood vs automated-test equivalence +`plan.md` has **0** `[~]` deferred tasks. Every behavior is covered by an automated test: +`doc-classify.test.mjs` (13 cases incl. exempt/precedence/trailing-slash/empty-config), +`test-docs-ci-check.sh` (incl. `.claude/**`, test-only, config-path-with-spaces, malformed-output +fail-open), `test-revise-push-guard.sh` (incl. `.claude/**`, test-only, merge-commit bypass, +per-commit `[skip docs]`). The only un-automated gate is **ShellCheck** (not installable on the dev +box) — covered by `validate.yml` in CI. No coverage gap. + +## Overall Decision +- [x] ✅ PASS +- [ ] ⚠️ PASS WITH WARNINGS +- [ ] ❌ FAIL + +Full local gate green: doc-classify unit (13/13), docs-ci-check + revise-push-guard shell suites, +marketplace policy, skill-gate (7/7), openspec strict (5/5). openspec hygiene is intentionally RED +only because the change is fully implemented but not yet archived — it goes green on archive. diff --git a/openspec/specs/doc-classification/spec.md b/openspec/specs/doc-classification/spec.md new file mode 100644 index 0000000..134227a --- /dev/null +++ b/openspec/specs/doc-classification/spec.md @@ -0,0 +1,116 @@ +# doc-classification Specification + +## Purpose +Define the single shared doc/non-doc/excluded/exempt classifier (`doc-classify.mjs`) that both +doc-sweep guards — the docs-staleness CI check and the revise-docs push hook — delegate to, so +"what counts as a doc" is defined exactly once and cannot drift between them. It owns the +`docPatterns` / `excludeDirs` / `exemptPatterns` inputs, the built-in defaults (doc-set including +`.claude/**/*.md`; exempt-set covering common tests so test-only changes pass without a `[skip docs]` +ack), the glob semantics (`*`, `**`), and the excludeDirs → exemptPatterns → docPatterns precedence. +Dependency-free node, parsed with `node` (no `jq`). +## Requirements +### Requirement: Single shared classifier module + +doc-sweep SHALL provide a single node module (`doc-classify.mjs`) that classifies a list of file +paths into documentation, non-documentation, and excluded, and SHALL be the **sole** doc/non-doc +classifier used by both the docs-staleness CI check and the revise-docs push-guard hook (neither +SHALL carry its own inline classification logic). The module SHALL read a newline-separated list of +file paths on stdin, accept an optional `--config ` argument, and emit on stdout a JSON object +`{ "nonDoc": [], "docChanged": }` where `nonDoc` lists the changed non-doc, +non-excluded paths and `docChanged` is true iff at least one changed path is a documentation file. +The module SHALL have no external (non-builtin) dependencies and SHALL parse JSON with node. + +#### Scenario: Both scripts delegate to the module + +- **WHEN** the CI check or the push-guard hook needs to classify changed files +- **THEN** it pipes the file list to `doc-classify.mjs` and acts on the returned `nonDoc`/`docChanged` result, rather than applying its own inline doc/non-doc patterns + +#### Scenario: Output shape + +- **WHEN** the module is given a mix of doc, non-doc, and excluded paths +- **THEN** it emits `{ "nonDoc": [...only the non-doc non-excluded paths...], "docChanged": true|false }` as JSON on stdout + +### Requirement: Built-in default doc-set includes `.claude/**` + +When no `docPatterns` is configured, the module SHALL treat a path as a documentation file iff it +matches the built-in default set: `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**`, and +`.claude/**/*.md` (at any directory depth). This default SHALL be consistent with the audience-rules +base, under which all `*.md` files beneath `.claude/` are Claude-facing documentation. + +#### Scenario: A .claude markdown change counts as a doc + +- **WHEN** the only changed path is `.claude/context/audience-rules.md` and no `docPatterns` override is configured +- **THEN** the module classifies it as a documentation file (`docChanged` true, `nonDoc` empty) + +#### Scenario: Default doc globs + +- **WHEN** the changed set is `README.md`, `docs/api/x.md`, and `CHANGELOG.md` with no override +- **THEN** all three classify as documentation and `nonDoc` is empty + +### Requirement: Configurable patterns and glob semantics + +The module SHALL resolve `docPatterns`, `excludeDirs`, and `exemptPatterns` from the `--config` JSON +when present, otherwise from the built-in defaults (`excludeDirs` defaulting to empty). For +`docPatterns` and `exemptPatterns`, a **non-empty** configured list replaces the corresponding +built-in default; an **empty or omitted** list falls back to the built-in default (an empty list +does NOT mean "match nothing" — that would enforce on every changed path). Glob matching SHALL +support `*` (matches within a single path segment) and `**` (matches across segments), plus literal +segments. A path under any `excludeDirs` entry (the entry itself or a descendant) SHALL be treated +as **excluded** — neither documentation nor non-documentation — and SHALL NOT appear in `nonDoc` +nor set `docChanged`. An `excludeDirs` entry with a trailing slash SHALL be normalized (the trailing +slash stripped) before comparison, so `["vendor/"]` excludes `vendor/a.js` the same as `["vendor"]`. + +#### Scenario: docPatterns override replaces the default + +- **WHEN** the config sets `docPatterns` to `["docs/**"]` and the changed set is `README.md` and `docs/x.md` +- **THEN** only `docs/x.md` is a doc; `README.md` is non-doc (the default globs no longer apply) + +#### Scenario: Empty docPatterns falls back to the default + +- **WHEN** the config sets `docPatterns` to `[]` and the changed set is `README.md` +- **THEN** the built-in default applies and `README.md` classifies as documentation (`docChanged` true, `nonDoc` empty) + +#### Scenario: Excluded directory is ignored + +- **WHEN** the config sets `excludeDirs` to `["vendor"]` and the changed set is `vendor/lib/a.js` and `vendor/lib/README.md` +- **THEN** both paths are excluded — `nonDoc` is empty and `docChanged` is false + +#### Scenario: excludeDirs entry with a trailing slash still excludes + +- **WHEN** the config sets `excludeDirs` to `["vendor/"]` and the changed path is `vendor/a.js` +- **THEN** the path is excluded — `nonDoc` is empty and `docChanged` is false + +#### Scenario: `**` spans directories + +- **WHEN** `docPatterns` contains `.claude/**/*.md` and the changed path is `.claude/context/audience-rules.md` +- **THEN** the path matches and classifies as documentation + +### Requirement: Exempt patterns for changes that do not require docs + +The module SHALL support an `exemptPatterns` glob list identifying first-party changes that do not +require documentation (distinct from `excludeDirs`, which marks vendored/external paths). A path +matching `exemptPatterns` — evaluated AFTER `excludeDirs` and BEFORE `docPatterns` — SHALL be treated +as **exempt**: it SHALL NOT appear in `nonDoc` and SHALL NOT set `docChanged`. When `exemptPatterns` +is empty or omitted, the built-in default `exemptPatterns` SHALL apply, covering common test files: +`**/*.test.*`, `**/*.spec.*`, `**/test/**`, `**/tests/**`, `**/__tests__/**`, `**/*_test.go`, +`**/*_test.py`. A **non-empty** configured `exemptPatterns` replaces the default (an empty list +falls back to the default rather than exempting nothing). The consequence is that a change whose +only non-doc, non-excluded paths are exempt yields an empty `nonDoc` and therefore passes the +staleness guards without an acknowledgment, while any doc-requiring path alongside it still +enforces. + +#### Scenario: Test-only change is exempt + +- **WHEN** the changed set is `src/app.test.js` and `tests/unit/foo_spec.rb` (matching the default exempt globs) with no override +- **THEN** both are exempt — `nonDoc` is empty and `docChanged` is false + +#### Scenario: Tests alongside real code still enforce + +- **WHEN** the changed set is `src/app.test.js` and `src/app.js` +- **THEN** the test file is exempt but `src/app.js` remains non-doc, so `nonDoc` is `["src/app.js"]` + +#### Scenario: excludeDirs wins over exempt and doc matching + +- **WHEN** a path is under a configured `excludeDirs` entry AND would also match `exemptPatterns` or `docPatterns` +- **THEN** it is excluded (evaluated first) and counts as neither doc, non-doc, nor exempt + diff --git a/openspec/specs/docs-staleness-ci/spec.md b/openspec/specs/docs-staleness-ci/spec.md index 7ef5a5d..7da1a7d 100644 --- a/openspec/specs/docs-staleness-ci/spec.md +++ b/openspec/specs/docs-staleness-ci/spec.md @@ -14,15 +14,16 @@ workflow, so there is no external action reference to trust. doc-sweep SHALL provide a deterministic, no-LLM, no-secret check that runs on a pull request and evaluates documentation staleness against the **merge base** of the PR (no committed marker -or persisted state). The check SHALL classify each file changed between the merge base and the -PR head as a documentation file or a non-documentation file using the configured doc-file set -(default: `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**`); files under configured -excluded directories SHALL be ignored entirely (neither doc nor non-doc). The check SHALL -**fail** if and only if at least one non-doc, non-excluded file changed, no doc file changed, -and no acknowledgment is present. Otherwise it SHALL **pass**. When it fails, it SHALL emit a -message that names the offending non-doc paths and states every way to clear it (update docs, or -add a `[skip docs]` line to any commit message in the PR range or to the PR body). The check SHALL parse the -event/diff with `node` (not `jq`), and SHALL be self-contained in a doc-sweep-shipped script. +or persisted state). The check SHALL delegate classification of each file changed between the merge +base and the PR head to the shared `doc-classification` module (`doc-classify.mjs`) — it SHALL NOT +carry its own inline doc/non-doc patterns. Files under configured excluded directories are ignored +entirely (neither doc nor non-doc), per that module. The check SHALL **fail** if and only if at least +one non-doc, non-excluded file changed, no doc file changed, and no acknowledgment is present. +Otherwise it SHALL **pass**. When it fails, it SHALL emit a message that names the offending non-doc +paths and states every way to clear it (update docs, or add a `[skip docs]` line to any commit +message in the PR range or to the PR body). The check SHALL parse the event/diff with `node` (not +`jq`), and SHALL be self-contained in a doc-sweep-shipped script paired with the vendored +`doc-classify.mjs`. #### Scenario: Code-only change fails @@ -39,6 +40,16 @@ event/diff with `node` (not `jq`), and SHALL be self-contained in a doc-sweep-sh - **WHEN** a PR changes only doc-set files (or only excluded files) - **THEN** the check passes +#### Scenario: A .claude doc change satisfies the check + +- **WHEN** a PR changes a non-doc file and also changes `.claude/context/audience-rules.md` +- **THEN** the shared classifier counts the `.claude/**/*.md` change as a doc and the check passes + +#### Scenario: Test-only change passes without an ack + +- **WHEN** a PR changes only files matching the shared classifier's `exemptPatterns` (e.g. `**/*.test.*`) and no doc-requiring file +- **THEN** the classifier returns an empty `nonDoc` and the check passes with no acknowledgment required + #### Scenario: Excluded paths are ignored - **WHEN** a PR changes only files under a configured excluded directory @@ -75,27 +86,30 @@ doc-sweep SHALL provide a manual, model-non-invocable skill (`install-docs-ci`, `disable-model-invocation: true`, with scoped `allowed-tools`) that installs the check only when a user runs it, mirroring the `install-revise-hook` pattern. On a **fresh install** it SHALL scaffold a GitHub Actions workflow file into the target repository's `.github/workflows/` that -invokes the doc-sweep-shipped check script on `pull_request`, collect the doc-file set and any -excluded directories, and print a structured summary: the workflow path, the doc-set, the ack -tokens, that blocking is the maintainer's branch-protection choice, and how to reconfigure or -uninstall by re-running the skill. When an install already exists it SHALL offer -Reconfigure / Uninstall / Cancel and SHALL be idempotent (it SHALL NOT duplicate the workflow). -Nothing SHALL be installed until the user runs the skill and confirms. - -#### Scenario: Fresh install scaffolds and summarizes +invokes the doc-sweep-shipped check script on `pull_request`, vendor **both** the check script and +the shared `doc-classify.mjs` module under `.github/doc-sweep/`, collect the documentation-file set +(recorded as `docPatterns`, NOT the retired `docMode`) and any excluded directories, and print a +structured summary: the workflow path, the doc-set, the ack tokens, that blocking is the maintainer's +branch-protection choice, and how to reconfigure or uninstall by re-running the skill. When an +install already exists it SHALL offer Reconfigure / Uninstall / Cancel and SHALL be idempotent (it +SHALL NOT duplicate the workflow). Uninstall SHALL remove the workflow, the vendored check script, +and the vendored `doc-classify.mjs`. Nothing SHALL be installed until the user runs the skill and +confirms. + +#### Scenario: Fresh install scaffolds, vendors the classifier, and summarizes - **WHEN** a user runs the installer and confirms scoping choices -- **THEN** a `pull_request` workflow calling the shipped check script is written to `.github/workflows/`, and a structured summary with reconfigure/uninstall instructions is printed +- **THEN** a `pull_request` workflow is written to `.github/workflows/`, both the check script and `doc-classify.mjs` are vendored under `.github/doc-sweep/`, a config recording `docPatterns` (not `docMode`) is written, and a structured summary with reconfigure/uninstall instructions is printed #### Scenario: Idempotent re-run - **WHEN** the installer is run again in a repo that already has the workflow - **THEN** it does not duplicate the workflow and offers Reconfigure / Uninstall / Cancel -#### Scenario: Uninstall +#### Scenario: Uninstall removes the vendored classifier too - **WHEN** the user chooses uninstall -- **THEN** the scaffolded workflow file is removed, leaving other workflows and settings intact +- **THEN** the scaffolded workflow, the vendored check script, and the vendored `doc-classify.mjs` are all removed, leaving other workflows and settings intact #### Scenario: Plugin install alone is inert diff --git a/openspec/specs/revise-docs-push-guard/spec.md b/openspec/specs/revise-docs-push-guard/spec.md index f26e2ff..d033437 100644 --- a/openspec/specs/revise-docs-push-guard/spec.md +++ b/openspec/specs/revise-docs-push-guard/spec.md @@ -13,10 +13,11 @@ skill is untouched. The hook is deterministic, fails open, and parses JSON with doc-sweep SHALL provide a manual, model-non-invocable skill that installs the guard only when a user runs it. On a **fresh install** the installer SHALL collect, via interactive prompts: settings location (user-global vs project), repo applicability (all repos vs -doc-sweep-enabled only), the documentation-file set, the **trigger event** (exactly one of -`push` or `commit`, with `push` recommended as default), and bypass/uninstall confirmation. -It SHALL then copy the hook script to a stable, version-independent path, write the chosen -configuration (including `trigger`), and merge an idempotent `PreToolUse`/`Bash` hook into +doc-sweep-enabled only), the documentation-file set (recorded as `docPatterns`, NOT the retired +`docMode`), the **trigger event** (exactly one of `push` or `commit`, with `push` recommended as +default), and bypass/uninstall confirmation. It SHALL then copy the hook script **and the shared +`doc-classify.mjs` module** to a stable, version-independent path, write the chosen configuration +(including `trigger` and `docPatterns`), and merge an idempotent `PreToolUse`/`Bash` hook into the selected `settings.json` without overwriting unrelated hooks. After writing the hook the installer SHALL offer to seed the review marker — seed `HEAD` now (reported as an assumption, with no review performed), run `revise-docs-and-mark` now, or leave it unseeded with a @@ -31,7 +32,7 @@ Reconfigure SHALL re-ask the choices pre-filled with the current config, rewrite #### Scenario: Fresh install seeds and summarizes - **WHEN** a user runs the installer, selects scoping options including a trigger, and chooses to seed the marker -- **THEN** the hook is copied to a stable path, a config capturing the choices (including `trigger`) is written, a `PreToolUse`/`Bash` entry is added, the marker is set to HEAD, and a structured summary with edit/uninstall instructions is printed +- **THEN** the hook script and `doc-classify.mjs` are copied to a stable path, a config capturing the choices (including `trigger` and `docPatterns`) is written, a `PreToolUse`/`Bash` entry is added, the marker is set to HEAD, and a structured summary with edit/uninstall instructions is printed #### Scenario: Trigger is chosen at install @@ -51,7 +52,7 @@ Reconfigure SHALL re-ask the choices pre-filled with the current config, rewrite #### Scenario: Uninstall - **WHEN** the user chooses uninstall -- **THEN** the `PreToolUse` entry, the copied hook script, and the config are removed, leaving other settings and the marker file intact +- **THEN** the `PreToolUse` entry, the copied hook script, the copied `doc-classify.mjs`, and the config are removed, leaving other settings and the marker file intact ### Requirement: Snapshot owned by a guard wrapper, not the base skill @@ -131,10 +132,11 @@ the gated command if and only if at least one non-documentation file changed in from the last `revise-docs` marker to `HEAD`. When it denies, it SHALL return a reason that names the gated verb and instructs the operator to run `revise-docs-and-mark`, commit any changes, and retry. If only documentation files (or nothing) changed since the marker, it -SHALL allow the command. A documentation file is one matching the configured doc-file set -(default: `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**`); files under configured -excluded directories SHALL be treated as neither doc nor non-doc (ignored entirely). A -command that is not the configured trigger SHALL be allowed without inspection. +SHALL allow the command. Doc/non-doc/excluded classification SHALL be delegated to the shared +`doc-classification` module (`doc-classify.mjs`) using the configured `docPatterns` and +`excludeDirs` (or that module's built-in default) — the hook SHALL NOT carry its own inline +doc/non-doc patterns and SHALL NOT read a `docMode`. A command that is not the configured trigger +SHALL be allowed without inspection. #### Scenario: Non-doc change blocks the configured trigger @@ -146,6 +148,11 @@ command that is not the configured trigger SHALL be allowed without inspection. - **WHEN** the configured trigger command is attempted and only doc-set files changed since the marker - **THEN** the hook allows it +#### Scenario: A .claude doc change allows + +- **WHEN** the only change since the marker is to `.claude/context/audience-rules.md` +- **THEN** the shared classifier counts it as a doc and the hook allows the command + #### Scenario: Commit trigger ignores push - **WHEN** the configured trigger is `commit` and the Bash command is a `git push` diff --git a/plugins/doc-sweep/CHANGELOG.md b/plugins/doc-sweep/CHANGELOG.md index 7fab297..1a5ba4a 100644 --- a/plugins/doc-sweep/CHANGELOG.md +++ b/plugins/doc-sweep/CHANGELOG.md @@ -10,6 +10,35 @@ For what the plugin does and how to use it, see [README.md](README.md). ## Notable additions +**Unify doc classification** (`unify-doc-classification`, 2026-07) + +- Both guards previously carried their own copy of the doc-matching logic; a `.claude/**` + misclassification (non-`.md` and nested files under `.claude/` were not reliably recognized as + docs) had drifted between them. Fixed by extracting a single shared module, + `hooks/doc-classify.mjs`, that both the CI check (`docs-ci-check.sh`) and the push guard + (`revise-push-guard.sh`) now delegate to — one glob-matching implementation, one default doc + set, no room for the two guards to disagree again. Installers vendor the classifier alongside + the guard script they scaffold. +- **`docMode` is retired** in favor of a plain **`docPatterns`** glob array (**BREAKING** for any + hand-written or previously-scaffolded config still using the old `docMode` field — it is no + longer read; re-run the relevant install skill, or rename the field to `docPatterns` in + `doc-sweep-revise.json` / `docs-ci.json`, to keep a custom doc-file set in effect). Omitting + `docPatterns` falls back to the module's built-in default, which now correctly includes + `.claude/**/*.md`. +- New **`exemptPatterns`** concept: first-party changes that never require docs, matched before + `docPatterns`. The built-in default covers common test globs (`*.test.*`, `*.spec.*`, + `test/**`, `tests/**`, `__tests__/**`, `*_test.go`, `*_test.py`) — a change whose only non-doc + files are exempt now clears both the CI check and the push guard *without* a `[skip docs]` ack, + so test-only PRs and pushes are never blocked. A change that mixes an exempt file with real + source still enforces on the source file. Unlike `docPatterns` (an installer prompt), + `exemptPatterns` isn't prompted for — configure it by hand-editing the per-install config JSON + that the classifier reads via `--config`. +- Funbox's own `docs-staleness.yml` needs no change to pick this up — it runs the check with no + config, so the fixed built-in default (including `.claude/**`) applies automatically. +- New `node --test` suite, `hooks/doc-classify.test.mjs`, covering glob translation, default + classification, `docPatterns`/`exemptPatterns` overrides, `excludeDirs` precedence, and the + test-only-exempt behavior; wired into `validate.yml`. + **Docs-staleness CI check** (`add-docs-staleness-ci`, 2026-07) - New PR-time GitHub Actions check (`hooks/docs-ci-check.sh`) that **fails** a pull request when diff --git a/plugins/doc-sweep/README.md b/plugins/doc-sweep/README.md index bfa5689..61148bb 100644 --- a/plugins/doc-sweep/README.md +++ b/plugins/doc-sweep/README.md @@ -90,12 +90,25 @@ merge base, so there's no marker or state to maintain. A pull request clears the check when any one of these is true: - a documentation file was updated, or +- every non-doc file that changed is **exempt** (see below), or - a **`[skip docs]`** token (mirroring `[skip ci]`) appears in any commit message in the PR, or - a **`[skip docs]`** line appears in the PR description (editable in the browser — no rebase). So a legitimate code-only change — a bug fix that needs no docs — is never a hard blocker; you just add `[skip docs]`. +Both this check and the push guard below delegate classification to a single shared module, +`hooks/doc-classify.mjs`, so "what counts as a doc" is defined once and can't drift between the +two guards. It takes a **`docPatterns`** glob list (default: `CLAUDE*.md`, `README*.md`, +`CHANGELOG.md`, anything under `docs/`, and any `.md` under `.claude/`) and an **`exemptPatterns`** +glob list — first-party changes that never require docs, matched *before* `docPatterns`. The +built-in default covers common test globs (`*.test.*`, `*.spec.*`, `test/**`, `tests/**`, +`__tests__/**`, `*_test.go`, `*_test.py`), so a change that only touches tests passes without a +`[skip docs]` ack — but a commit that mixes a test file with real source still enforces on the +source file. `docPatterns` is an installer prompt (see "Doc-file set" below); `exemptPatterns` +isn't prompted — configure it by hand-editing the per-install config JSON that the classifier +reads via `--config`. + Nothing is installed automatically. To set it up, run: ```text @@ -133,10 +146,17 @@ The installer is interactive and asks you four questions before writing anything 2. **Repo applicability** — all repos, or only repos that have `doc-sweep` set up (a `CLAUDE.md` or `.claude/context/audience-rules.md`). User-global installs default to doc-sweep-enabled repos only. -3. **Doc-file set** — which files count as "documentation" and won't trigger the guard: - - `default`: `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**` +3. **Doc-file set** — which files count as "documentation" (`docPatterns`) and won't trigger the + guard: + - `default`: `CLAUDE*.md`, `README*.md`, `CHANGELOG.md`, `docs/**`, and any `.md` under + `.claude/` - `with-skill`: same as default, plus `SKILL.md` files - `minimal`: `CLAUDE.md` and `README.md` only + + A custom choice is recorded as a `docPatterns` glob list (replacing the module's built-in + default, not adding to it); `default` omits `docPatterns` from the config so the shared + module's own default applies. Independent of this choice, `exemptPatterns` (see above) always + applies — test-only changes clear the guard without an ack regardless of the doc-file set. 4. **Bypass and uninstall** — the installer confirms the bypass token and how to remove the guard. diff --git a/plugins/doc-sweep/context/audience-rules.md b/plugins/doc-sweep/context/audience-rules.md index 79fbc1d..13e3891 100644 --- a/plugins/doc-sweep/context/audience-rules.md +++ b/plugins/doc-sweep/context/audience-rules.md @@ -15,3 +15,39 @@ Shared files (`CLAUDE.md`, `README.md`, scripts, code comments) should target th primary environment and stay consistent with it. As a default, prefer POSIX `sh`/`bash` syntax and paths; keep machine-specific or OS-specific snippets (Windows drive letters, PowerShell, personal tool paths) in the `.local.md` twin rather than the shared files. + +## Machine-readable doc-file-set + +The guard scripts (CI check + push hook) classify paths via `doc-classify.mjs`, which reads a +`docPatterns` glob list (the machine-readable twin of the audience table above) and an +`exemptPatterns` list of first-party changes that don't require docs (default: tests). Defaults when +unset: docs = `**/CLAUDE*.md`, `**/README*.md`, `**/CHANGELOG.md`, `docs/**`, `.claude/**/*.md`; +exempt = common test globs. + +Both installers (`install-docs-ci`, `install-revise-hook`) offer a doc-file-set choice of +`default`, `with-skill`, or `minimal`. When a project keeps the **default**, `docPatterns` is +omitted from the per-install config JSON entirely (and nothing is written here) — the classifier +falls back to its built-in default list above. When a project picks a **custom** set +(`with-skill` or `minimal`), the installer persists the chosen glob list here, the same way +`excludeDirs` is persisted, and mirrors the identical list into the per-install config JSON. For +example, `with-skill` would be persisted as: + + docPatterns: + - "**/CLAUDE*.md" + - "**/README*.md" + - "**/CHANGELOG.md" + - "docs/**" + - ".claude/**/*.md" + - "**/SKILL.md" + +`exemptPatterns` (first-party paths that don't require docs) is not currently exposed as an +installer choice; only the built-in default applies: + + exemptPatterns: + - "**/*.test.*" + - "**/*.spec.*" + - "**/test/**" + - "**/tests/**" + - "**/__tests__/**" + - "**/*_test.go" + - "**/*_test.py" diff --git a/plugins/doc-sweep/hooks/doc-classify.mjs b/plugins/doc-sweep/hooks/doc-classify.mjs new file mode 100644 index 0000000..785e00b --- /dev/null +++ b/plugins/doc-sweep/hooks/doc-classify.mjs @@ -0,0 +1,88 @@ +import { readFileSync } from 'node:fs'; + +// Doc globs. `**/` is an optional prefix (matches root and nested); `**` at the tail +// matches everything under a dir. Kept consistent with the audience-rules base +// (all *.md under .claude/ are Claude-facing docs). +export const DEFAULT_DOC_PATTERNS = [ + '**/CLAUDE*.md', + '**/README*.md', + '**/CHANGELOG.md', + 'docs/**', + '**/docs/**', + '.claude/**/*.md', + '**/.claude/**/*.md', +]; + +// First-party changes that do not require docs (distinct from excludeDirs, which is vendored). +// Evaluated after excludeDirs and before docPatterns. Default: common test files. +export const DEFAULT_EXEMPT_PATTERNS = [ + '**/*.test.*', + '**/*.spec.*', + '**/test/**', + '**/tests/**', + '**/__tests__/**', + '**/*_test.go', + '**/*_test.py', +]; + +// Translate a glob to an anchored RegExp. Supported: `**/` (optional dir prefix), +// `**` (across segments), `*` (within one segment), literals. No braces/char-classes. +export function globToRegExp(glob) { + let re = ''; + for (let i = 0; i < glob.length; i++) { + const c = glob[i]; + if (c === '*' && glob[i + 1] === '*') { + i++; // consume second * + if (glob[i + 1] === '/') { i++; re += '(?:.*/)?'; } // **/ -> optional dir prefix + else re += '.*'; // ** -> anything incl. / + } else if (c === '*') { + re += '[^/]*'; + } else if ('.+^${}()|[]\\/'.includes(c)) { + re += '\\' + c; + } else { + re += c; + } + } + return new RegExp('^' + re + '$'); +} + +export function classify(files, { docPatterns, excludeDirs, exemptPatterns } = {}) { + const docRes = ((docPatterns && docPatterns.length) ? docPatterns : DEFAULT_DOC_PATTERNS).map(globToRegExp); + const exemptRes = ((exemptPatterns && exemptPatterns.length) ? exemptPatterns : DEFAULT_EXEMPT_PATTERNS).map(globToRegExp); + const excludes = (excludeDirs || []).map((ex) => ex.replace(/\/+$/, '')); + const nonDoc = []; + let docChanged = false; + for (const f of files) { + if (!f) continue; + if (excludes.some((ex) => f === ex || f.startsWith(ex + '/'))) continue; // excluded (vendored) + if (exemptRes.some((r) => r.test(f))) continue; // exempt (e.g. tests) + if (docRes.some((r) => r.test(f))) docChanged = true; // documentation + else nonDoc.push(f); // doc-requiring + } + return { nonDoc, docChanged }; +} + +function main() { + const argv = process.argv.slice(2); + let cfgPath = null; + for (let i = 0; i < argv.length; i++) if (argv[i] === '--config') cfgPath = argv[++i]; + let opts = {}; + if (cfgPath) { + try { + const c = JSON.parse(readFileSync(cfgPath, 'utf8')); + opts = { docPatterns: c.docPatterns, excludeDirs: c.excludeDirs, exemptPatterns: c.exemptPatterns }; + } catch { /* fall back to defaults */ } + } + let input = ''; + process.stdin.setEncoding('utf8'); + process.stdin.on('data', (d) => (input += d)); + process.stdin.on('end', () => { + const files = input.split('\n').map((s) => s.trim()).filter(Boolean); + process.stdout.write(JSON.stringify(classify(files, opts))); + }); +} + +// Run as CLI only when invoked directly (not when imported by tests). +if (import.meta.url === `file://${process.argv[1]}` || process.argv[1]?.endsWith('doc-classify.mjs')) { + main(); +} diff --git a/plugins/doc-sweep/hooks/doc-classify.test.mjs b/plugins/doc-sweep/hooks/doc-classify.test.mjs new file mode 100644 index 0000000..35da19e --- /dev/null +++ b/plugins/doc-sweep/hooks/doc-classify.test.mjs @@ -0,0 +1,76 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { globToRegExp, classify, DEFAULT_DOC_PATTERNS } from './doc-classify.mjs'; + +test('globToRegExp: * stays within a segment', () => { + assert.ok(globToRegExp('README*.md').test('README.md')); + assert.ok(globToRegExp('**/README*.md').test('sub/README.dev.md')); + assert.ok(!globToRegExp('README*.md').test('sub/README.md')); +}); + +test('globToRegExp: ** spans directories', () => { + assert.ok(globToRegExp('.claude/**/*.md').test('.claude/context/audience-rules.md')); + assert.ok(globToRegExp('docs/**').test('docs/api/ref.md')); +}); + +test('.claude markdown is a doc under the default set', () => { + const r = classify(['.claude/context/audience-rules.md'], {}); + assert.equal(r.docChanged, true); + assert.deepEqual(r.nonDoc, []); +}); + +test('default globs: README/docs/CHANGELOG are docs, src is not', () => { + const r = classify(['README.md', 'docs/x.md', 'CHANGELOG.md', 'src/app.js'], {}); + assert.equal(r.docChanged, true); + assert.deepEqual(r.nonDoc, ['src/app.js']); +}); + +test('docPatterns override replaces the default', () => { + const r = classify(['README.md', 'docs/x.md'], { docPatterns: ['docs/**'] }); + assert.deepEqual(r.nonDoc, ['README.md']); + assert.equal(r.docChanged, true); +}); + +test('excludeDirs paths are neither doc nor non-doc', () => { + const r = classify(['vendor/lib/a.js', 'vendor/lib/README.md'], { excludeDirs: ['vendor'] }); + assert.deepEqual(r.nonDoc, []); + assert.equal(r.docChanged, false); +}); + +test('DEFAULT_DOC_PATTERNS includes a .claude glob', () => { + assert.ok(DEFAULT_DOC_PATTERNS.some((p) => p.includes('.claude'))); +}); + +test('test-only change is exempt (empty nonDoc, docChanged false)', () => { + const r = classify(['src/app.test.js', 'tests/unit/thing.js'], {}); + assert.deepEqual(r.nonDoc, []); + assert.equal(r.docChanged, false); +}); + +test('tests alongside real code still enforce', () => { + const r = classify(['src/app.test.js', 'src/app.js'], {}); + assert.deepEqual(r.nonDoc, ['src/app.js']); +}); + +test('excludeDirs wins over exempt and doc matching', () => { + const r = classify(['vendor/x.test.js', 'vendor/README.md'], { excludeDirs: ['vendor'] }); + assert.deepEqual(r.nonDoc, []); + assert.equal(r.docChanged, false); +}); + +test('configured exemptPatterns replaces the default', () => { + const r = classify(['a.test.js'], { exemptPatterns: ['**/*.gen.js'] }); + assert.deepEqual(r.nonDoc, ['a.test.js']); // .test.js no longer exempt +}); + +test('excludeDirs entry with a trailing slash still excludes', () => { + const r = classify(['vendor/a.js'], { excludeDirs: ['vendor/'] }); + assert.deepEqual(r.nonDoc, []); + assert.equal(r.docChanged, false); +}); + +test('empty docPatterns falls back to the built-in default', () => { + const r = classify(['README.md'], { docPatterns: [] }); + assert.equal(r.docChanged, true); + assert.deepEqual(r.nonDoc, []); +}); diff --git a/plugins/doc-sweep/hooks/docs-ci-check.sh b/plugins/doc-sweep/hooks/docs-ci-check.sh index 6ffbadd..64a443d 100644 --- a/plugins/doc-sweep/hooks/docs-ci-check.sh +++ b/plugins/doc-sweep/hooks/docs-ci-check.sh @@ -19,13 +19,7 @@ set -uo pipefail warn(){ printf 'docs-ci-check: %s\n' "$1" >&2; } pass(){ exit 0; } -# --- config (optional): docMode + excludeDirs, same shape as the push guard --- -docmode="default"; excludes="" -cfg="${1:-}" -if [ -n "$cfg" ] && [ -f "$cfg" ]; then - docmode="$(node -e 'try{process.stdout.write(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).docMode||"default")}catch(e){process.stdout.write("default")}' "$cfg" 2>/dev/null || echo default)" - excludes="$(node -e 'try{const a=JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).excludeDirs;process.stdout.write(Array.isArray(a)?a.join("\n"):"")}catch(e){}' "$cfg" 2>/dev/null || echo)" -fi +here="$(cd "$(dirname "$0")" && pwd)" git rev-parse --is-inside-work-tree >/dev/null 2>&1 || { warn "not a git work tree; passing"; pass; } @@ -62,44 +56,16 @@ $(git log --format=%B "${mb}..HEAD" 2>/dev/null || true)" changed="$(git diff --name-only "${mb}..HEAD" 2>/dev/null)" || { warn "cannot diff ${mb}..HEAD; passing (fail-open)"; pass; } [ -n "$changed" ] || pass # nothing changed -is_doc(){ # $1 = path; doc per $docmode (identical classification to revise-push-guard.sh) - case "$docmode" in - minimal) - case "$1" in CLAUDE.md|*/CLAUDE.md|README.md|*/README.md) return 0;; esac ;; - with-skill) - case "$1" in SKILL.md|*/SKILL.md) return 0;; esac - case "$1" in CLAUDE*.md|*/CLAUDE*.md|README*.md|*/README*.md|CHANGELOG.md|*/CHANGELOG.md|docs/*|*/docs/*) return 0;; esac ;; - *) # default - case "$1" in CLAUDE*.md|*/CLAUDE*.md|README*.md|*/README*.md|CHANGELOG.md|*/CHANGELOG.md|docs/*|*/docs/*) return 0;; esac ;; - esac - return 1 -} - -nondoc=""; docchanged=0 -while IFS= read -r f; do - [ -n "$f" ] || continue - skip=0 - if [ -n "$excludes" ]; then - while IFS= read -r ex; do - [ -n "$ex" ] || continue - case "$f" in "$ex"/*|"$ex") skip=1; break;; esac - done </dev/null)" || { warn "classify failed; passing (fail-open)"; pass; } +parsed="$(printf '%s' "$result" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{const o=JSON.parse(s);process.stdout.write(String(o.docChanged)+"\n"+((o.nonDoc||[]).join("\n")))}catch(e){process.exit(1)}})' 2>/dev/null)" || { warn "classify produced unexpected output; passing (fail-open)"; pass; } +docchanged="$(printf '%s' "$parsed" | head -1)" +nondoc="$(printf '%s' "$parsed" | tail -n +2)" # Pass if no non-doc changed, or a doc changed, or the change is acknowledged. [ -z "$nondoc" ] && pass -[ "$docchanged" = 1 ] && pass +[ "$docchanged" = "true" ] && pass has_ack && pass # Otherwise: code changed, no docs, no ack → fail with guidance. @@ -107,8 +73,9 @@ has_ack && pass echo "Docs staleness check failed." echo echo "Non-doc file(s) changed in this PR but no documentation was updated:" - # shellcheck disable=SC2086 - for f in $nondoc; do echo " - $f"; done + while IFS= read -r f; do [ -n "$f" ] && echo " - $f"; done </dev/null || echo default)" reposcope="$(node -e 'try{process.stdout.write(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).repoScope||"all")}catch(e){process.stdout.write("all")}' "$cfg" 2>/dev/null || echo all)" trigger="$(node -e 'try{process.stdout.write(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).trigger||"push")}catch(e){process.stdout.write("push")}' "$cfg" 2>/dev/null || echo push)" - excludes="$(node -e 'try{const a=JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).excludeDirs;process.stdout.write(Array.isArray(a)?a.join("\n"):"")}catch(e){}' "$cfg" 2>/dev/null || echo)" fi # Only gate the configured trigger subcommand (push by default). @@ -75,37 +74,12 @@ fi changed="$(git diff --name-only "$range" 2>/dev/null)" || allow [ -n "$changed" ] || allow # nothing new since marker -is_doc(){ # $1 = path; doc per $docmode - case "$docmode" in - minimal) - case "$1" in CLAUDE.md|*/CLAUDE.md|README.md|*/README.md) return 0;; esac ;; - with-skill) - case "$1" in SKILL.md|*/SKILL.md) return 0;; esac - case "$1" in CLAUDE*.md|*/CLAUDE*.md|README*.md|*/README*.md|CHANGELOG.md|*/CHANGELOG.md|docs/*|*/docs/*) return 0;; esac ;; - *) # default - case "$1" in CLAUDE*.md|*/CLAUDE*.md|README*.md|*/README*.md|CHANGELOG.md|*/CHANGELOG.md|docs/*|*/docs/*) return 0;; esac ;; - esac - return 1 -} +# Classify via the shared module (delegates docPatterns/excludeDirs/exemptPatterns). +here="$(cd "$(dirname "$0")" && pwd)" +cfg_arg=(); [ -n "$cfg" ] && [ -f "$cfg" ] && cfg_arg=(--config "$cfg") -nondoc="" -while IFS= read -r f; do - [ -n "$f" ] || continue - skip=0 - if [ -n "$excludes" ]; then - while IFS= read -r ex; do - [ -n "$ex" ] || continue - case "$f" in "$ex"/*|"$ex") skip=1; break;; esac - done </dev/null)" || result="" +nondoc="$(printf '%s' "$result" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{const o=JSON.parse(s);process.stdout.write((o.nonDoc||[]).join(" "))}catch(e){process.stdout.write("")}})' 2>/dev/null)" || nondoc="" has_skip(){ # $1 = commit sha; true if its message carries the shared [skip docs] token git log -1 --format=%B "$1" 2>/dev/null | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{process.exit(/\[skip docs\]/i.test(s)?0:1)})' 2>/dev/null @@ -120,23 +94,20 @@ if [ -n "$nondoc" ]; then while IFS= read -r c; do [ -n "$c" ] || continue has_skip "$c" && continue # this commit is acknowledged - while IFS= read -r f; do - [ -n "$f" ] || continue - skip=0 - if [ -n "$excludes" ]; then - while IFS= read -r ex; do - [ -n "$ex" ] || continue - case "$f" in "$ex"/*|"$ex") skip=1; break;; esac - done </dev/null) -CF - [ "$unacked" = 1 ] && break + # Merge commits (2+ parents): a plain `-r` first-parent-style diff prints NOTHING for a + # merge, so a non-doc file introduced only by the merge (e.g. a conflict resolution) would + # silently fall through to allow. Use the combined-diff form (`-c`) instead, which surfaces + # only merge-unique content — files that differ from EVERY parent — so a clean auto-merge + # with no merge-unique change still reports nothing (no over-flagging every merged-in file). + # Non-merge commits are untouched: same `-r` diff as before. + if git rev-parse -q --verify "${c}^2" >/dev/null 2>&1; then + cfiles="$(git diff-tree --no-commit-id --name-only -r -c "$c" 2>/dev/null)" + else + cfiles="$(git diff-tree --no-commit-id --name-only -r "$c" 2>/dev/null)" + fi + cresult="$(printf '%s\n' "$cfiles" | node "$here/doc-classify.mjs" ${cfg_arg[@]+"${cfg_arg[@]}"} 2>/dev/null)" || cresult="" + cnon="$(printf '%s' "$cresult" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{const o=JSON.parse(s);process.stdout.write((o.nonDoc||[]).join("\n"))}catch(e){process.stdout.write("")}})' 2>/dev/null)" || cnon="" + [ -n "$cnon" ] && { unacked=1; break; } done </dev/null) CL diff --git a/plugins/doc-sweep/hooks/test-docs-ci-check.sh b/plugins/doc-sweep/hooks/test-docs-ci-check.sh index 7a5e124..b1f5dd1 100644 --- a/plugins/doc-sweep/hooks/test-docs-ci-check.sh +++ b/plugins/doc-sweep/hooks/test-docs-ci-check.sh @@ -55,9 +55,45 @@ run "$base" "$repo"; assert_pass $? "no changes passes" repo="$(mkrepo)"; commitfile "$repo" src/app.js run "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef" "$repo"; assert_pass $? "unresolvable base fails open" -# 11. minimal docMode: CHANGELOG counts as non-doc → fail +# 11. custom docPatterns config: CHANGELOG no longer in the doc set → fail repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" CHANGELOG.md -cfg="$(mktemp)"; echo '{"docMode":"minimal"}' > "$cfg" -run "$base" "$repo" "" "$cfg"; assert_fail $? "minimal docMode: CHANGELOG is non-doc" +cfg="$(mktemp)"; echo '{"docPatterns":["**/CLAUDE*.md","**/README*.md"]}' > "$cfg" +run "$base" "$repo" "" "$cfg"; assert_fail $? "custom docPatterns: CHANGELOG is non-doc" + +# .claude markdown counts as a doc (regression: was misclassified non-doc) +repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" src/app.js; commitfile "$repo" .claude/context/audience-rules.md +run "$base" "$repo"; assert_pass $? ".claude/*.md change satisfies the check" + +# test-only change is exempt → passes without an ack +repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" src/app.test.js +run "$base" "$repo"; assert_pass $? "test-only change passes (exempt)" + +# tests + real code still enforces +repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" src/app.test.js; commitfile "$repo" src/app.js +run "$base" "$repo"; assert_fail $? "tests + src still enforces" + +# config path containing spaces is honored (not word-split into the default fallback) +repo="$(mkrepo)"; base="$(basesha "$repo")"; commitfile "$repo" vendor/x.js +cfgdir="$(mktemp -d)/with space"; mkdir -p "$cfgdir"; cfg="$cfgdir/cfg.json" +echo '{"excludeDirs":["vendor"]}' > "$cfg" +run "$base" "$repo" "" "$cfg"; assert_pass $? "config path with spaces is honored (exclusion applies)" + +# malformed classifier output fails open with a warning (unguarded JSON.parse would silently +# yield empty strings and exit 0 with no diagnostic instead) +malrepo="$(mkrepo)"; malbase="$(basesha "$malrepo")"; commitfile "$malrepo" src/app.js +maldir="$(mktemp -d)" +cp "$SCRIPT" "$maldir/docs-ci-check.sh" +cat > "$maldir/doc-classify.mjs" <<'EOF' +process.stdout.write('{"nonDoc":["x"'); +process.exit(0); +EOF +malout="$(cd "$malrepo" && DOCS_CI_BASE="$malbase" DOCS_CI_PR_BODY="" bash "$maldir/docs-ci-check.sh" 2>&1 >/dev/null)" +malrc=$? +if [ "$malrc" -eq 0 ] && printf '%s' "$malout" | grep -qi 'unexpected output'; then + echo "ok: malformed classifier output fails open with a warning" +else + echo "FAIL(expected pass+warning): malformed classifier output fails open with a warning (rc=$malrc, stderr=$malout)" + fail=1 +fi exit $fail diff --git a/plugins/doc-sweep/hooks/test-revise-push-guard.sh b/plugins/doc-sweep/hooks/test-revise-push-guard.sh index e9a579d..52a3f89 100644 --- a/plugins/doc-sweep/hooks/test-revise-push-guard.sh +++ b/plugins/doc-sweep/hooks/test-revise-push-guard.sh @@ -41,7 +41,7 @@ out="$(run 'DOC_SWEEP_REVISE_SKIP=1 git push' "$repo" "$no_cfg")"; assert_allow # 6. doc-sweep-only self-skip in repo without CLAUDE.md markers → allow repo2="$(mktemp -d)"; git -C "$repo2" init -q; git -C "$repo2" config user.email t@t; git -C "$repo2" config user.name t echo x > "$repo2/a.js"; git -C "$repo2" add -A; git -C "$repo2" commit -qm i -cfg6="$(mktemp)"; echo '{"docMode":"default","repoScope":"doc-sweep-only"}' > "$cfg6" +cfg6="$(mktemp)"; echo '{"repoScope":"doc-sweep-only"}' > "$cfg6" out="$(run 'git push' "$repo2" "$cfg6")"; assert_allow "$out" "doc-sweep-only self-skip" # 7. error/fail-open: cwd not a repo → allow @@ -59,10 +59,11 @@ out="$(run 'git push --no-verify' "$repo" "$no_cfg")"; assert_allow "$out" "--no repo="$(mkrepo)"; mark "$repo"; commitfile "$repo" docs/api/ref.md out="$(run 'git push' "$repo" "$no_cfg")"; assert_allow "$out" "deep docs/ path allows push" -# 11. minimal docMode: CHANGELOG change is non-doc → deny +# 11. custom docPatterns: CHANGELOG no longer in the doc set → deny (docMode is retired; +# this is the docPatterns equivalent of the old "minimal" docMode) repo="$(mkrepo)"; mark "$repo"; commitfile "$repo" CHANGELOG.md -cfg11="$(mktemp)"; echo '{"docMode":"minimal","repoScope":"all"}' > "$cfg11" -out="$(run 'git push' "$repo" "$cfg11")"; assert_deny "$out" "minimal docMode: CHANGELOG is non-doc" +cfg11="$(mktemp)"; echo '{"repoScope":"all","docPatterns":["**/CLAUDE*.md","**/README*.md"]}' > "$cfg11" +out="$(run 'git push' "$repo" "$cfg11")"; assert_deny "$out" "custom docPatterns: CHANGELOG is non-doc" # --- Task 1: configurable trigger --- @@ -95,4 +96,52 @@ out="$(run 'git push' "$repo" "$no_cfg")"; assert_allow "$out" "[skip docs] in c commitfile "$repo" other.js out="$(run 'git push' "$repo" "$no_cfg")"; assert_deny "$out" "un-acked later non-doc commit still denies" +# --- shared classifier (doc-classify.mjs) parity regressions --- + +# .claude markdown-only change since marker → allow (regression) +repo="$(mkrepo)"; mark "$repo"; commitfile "$repo" .claude/context/audience-rules.md +out="$(run 'git push' "$repo" "$no_cfg")"; assert_allow "$out" ".claude/*.md change allows push" + +# test-only change since marker → allow (exempt) +repo="$(mkrepo)"; mark "$repo"; commitfile "$repo" src/app.test.js +out="$(run 'git push' "$repo" "$no_cfg")"; assert_allow "$out" "test-only change allows push (exempt)" + +# --- Finding 1 regression: merge-commit bypass in the per-commit [skip docs] loop --- +# `git diff-tree --no-commit-id --name-only -r ` prints nothing for merge commits, +# so a non-doc file introduced only by a merge (e.g. a conflict resolution) used to fall through +# to allow even though it's genuinely un-acked. Build two branches that truly conflict on +# README.md, merge with --no-ff, resolve the conflict AND introduce a new non-doc file +# (sneaky.js) in the merge commit itself. + +# 12. merge-commit-introduced non-doc, no [skip docs] anywhere → deny +repo="$(mkrepo)" +( cd "$repo" && echo base > README.md && git add . && git commit -qm "add readme" ) +mark "$repo" +base_branch="$(git -C "$repo" symbolic-ref --short HEAD)" +( cd "$repo" && git checkout -qb feature-a ) +( cd "$repo" && echo line-a >> README.md && git commit -qam "readme change a" ) +( cd "$repo" && git checkout -q "$base_branch" && git checkout -qb feature-b ) +( cd "$repo" && echo line-b >> README.md && git commit -qam "readme change b" ) +( cd "$repo" && git checkout -q "$base_branch" && git merge --no-ff -q feature-a -m "merge feature-a" ) +( cd "$repo" && git merge --no-ff feature-b -m "merge feature-b" >/dev/null 2>&1 + echo resolved > README.md && echo sneaky > sneaky.js && git add -A \ + && git commit -qm "merge feature-b: resolve conflict, add sneaky.js" ) +out="$(run 'git push' "$repo" "$no_cfg")"; assert_deny "$out" "merge-commit-introduced non-doc denies" + +# 13. companion: identical conflict/merge, but every commit (branches + both merges) carries +# [skip docs] → allow (proves this is the ack rule, not a blanket merge block) +repo="$(mkrepo)" +( cd "$repo" && echo base > README.md && git add . && git commit -qm "add readme" ) +mark "$repo" +base_branch="$(git -C "$repo" symbolic-ref --short HEAD)" +( cd "$repo" && git checkout -qb feature-a ) +( cd "$repo" && echo line-a >> README.md && git commit -qam "readme change a [skip docs]" ) +( cd "$repo" && git checkout -q "$base_branch" && git checkout -qb feature-b ) +( cd "$repo" && echo line-b >> README.md && git commit -qam "readme change b [skip docs]" ) +( cd "$repo" && git checkout -q "$base_branch" && git merge --no-ff -q feature-a -m "merge feature-a [skip docs]" ) +( cd "$repo" && git merge --no-ff feature-b -m "merge feature-b [skip docs]" >/dev/null 2>&1 + echo resolved > README.md && echo sneaky > sneaky.js && git add -A \ + && git commit -qm "merge feature-b: resolve conflict, add sneaky.js [skip docs]" ) +out="$(run 'git push' "$repo" "$no_cfg")"; assert_allow "$out" "merge-commit non-doc with [skip docs] on every commit allows" + exit $fail diff --git a/plugins/doc-sweep/skills/install-docs-ci/SKILL.md b/plugins/doc-sweep/skills/install-docs-ci/SKILL.md index 986ca14..50fff51 100644 --- a/plugins/doc-sweep/skills/install-docs-ci/SKILL.md +++ b/plugins/doc-sweep/skills/install-docs-ci/SKILL.md @@ -36,12 +36,16 @@ hook recognizes, so the two guards share one vocabulary. 1. **Detect an existing install.** Look for the scaffolded workflow at `${CLAUDE_PROJECT_DIR}/.github/workflows/doc-sweep-docs.yml` (and the vendored script at - `.github/doc-sweep/docs-ci-check.sh`). + `.github/doc-sweep/docs-ci-check.sh` plus its classifier at + `.github/doc-sweep/doc-classify.mjs` — both files, not just the script, are part of what + constitutes an install). - If **no install** is found → proceed to step 2 (fresh install). - If an install **is found**, offer three choices via `AskUserQuestion`: - **Reconfigure** — re-ask the choices in step 2 pre-filled from the existing - `.github/doc-sweep/docs-ci.json`; rewrite that config and re-copy the script; leave the + `.github/doc-sweep/docs-ci.json`; rewrite that config, re-copy the script, and + unconditionally re-copy `doc-classify.mjs` alongside it (idempotent — this self-heals a + missing or stale classifier even when nothing else about the install changed); leave the workflow file in place (rewrite it only if its path/name changed); print the summary (step 6). Stop. - **Uninstall** — follow the Uninstall section below. Stop. @@ -49,9 +53,15 @@ hook recognizes, so the two guards share one vocabulary. 2. **Collect scope (AskUserQuestion).** Ask both in one prompt, with defaults called out: - - **Doc-file set** — `default` (CLAUDE*.md, README*.md, CHANGELOG.md, docs/**), + - **Doc-file set** — `default` (matches `doc-classify.mjs`'s built-in list: `CLAUDE*.md`, + `README*.md`, `CHANGELOG.md`, files under `docs/`, and any `.md` under `.claude/`), `with-skill` (also treats SKILL.md as a doc), or `minimal` (CLAUDE.md + README.md only). - Recommend `default`. Recorded as `docMode` (identical meaning to the push-guard config). + Recommend `default`. If the user picks `default`, do **not** transcribe this parenthetical + into a `docPatterns` list — omit `docPatterns` from the config JSON entirely (step 4) so + `doc-classify.mjs`'s own built-in default (documented in `context/audience-rules.md`) + applies. If the user picks `with-skill` or `minimal` (a custom set), record the concrete + glob list as `docPatterns` and persist it into `.claude/context/audience-rules.md` the same + way `excludeDirs` is persisted (step 4). - **Excluded directories** — confirm the vendored/generated dirs whose changes should be ignored (neither doc nor non-doc). Recorded as `excludeDirs`. @@ -67,15 +77,31 @@ hook recognizes, so the two guards share one vocabulary. - **Known vendor names**: any of `vendor`, `third_party`, `Pods`, `bower_components`, `node_modules` existing as a root directory. -4. **Copy the check script and write the config.** `mkdir -p` `.github/doc-sweep/`, then: +4. **Copy the check script and classifier, and write the config.** `mkdir -p` `.github/doc-sweep/`, then: - Copy this skill's bundled `../../hooks/docs-ci-check.sh` to `${CLAUDE_PROJECT_DIR}/.github/doc-sweep/docs-ci-check.sh` (keep it executable; it must stay LF). Vendoring the script keeps the check self-contained — no external action ref, no runtime download. + - Also copy this skill's bundled `../../hooks/doc-classify.mjs` to + `${CLAUDE_PROJECT_DIR}/.github/doc-sweep/doc-classify.mjs` — the **same directory** as the + script above, since `docs-ci-check.sh` resolves the classifier next to itself + (`$here/doc-classify.mjs`). Without it the check fails open (passes) on every PR. + - If the user chose a **custom** doc-file set in step 2 (`with-skill` or `minimal`), persist + the chosen `docPatterns` glob list into `.claude/context/audience-rules.md` the same way + `excludeDirs` is persisted: append a `docPatterns:` block if the file exists and doesn't + already have one, update it in place if it does, or create the file with a brief header + comment if it doesn't exist yet. If the user kept the **default**, leave `audience-rules.md` + untouched for `docPatterns`. - Write `${CLAUDE_PROJECT_DIR}/.github/doc-sweep/docs-ci.json`: - ```json - { "docMode": "", "excludeDirs": [] } - ``` + - Default doc-file set — omit `docPatterns` entirely: + ```json + { "excludeDirs": [] } + ``` + - Custom doc-file set (`with-skill` or `minimal`) — include the concrete glob list, + mirroring what was just persisted to `audience-rules.md`: + ```json + { "docPatterns": [], "excludeDirs": [] } + ``` 5. **Scaffold the workflow (idempotent).** Write `${CLAUDE_PROJECT_DIR}/.github/workflows/doc-sweep-docs.yml` (do not overwrite an unrelated @@ -112,8 +138,9 @@ hook recognizes, so the two guards share one vocabulary. ─────────────────────────────────────────── Workflow file : Check script : + Classifier : Config file : - Doc-file set : + Doc-file set : (docPatterns: ) Excluded dirs : Ack token : [skip docs] (in a commit message or the PR description) @@ -129,7 +156,8 @@ hook recognizes, so the two guards share one vocabulary. ## Uninstall Delete the scaffolded workflow `${CLAUDE_PROJECT_DIR}/.github/workflows/doc-sweep-docs.yml`, the -vendored `${CLAUDE_PROJECT_DIR}/.github/doc-sweep/docs-ci-check.sh`, and its -`docs-ci.json`. Remove the now-empty `.github/doc-sweep/` directory if nothing else remains. -Leave all other workflows and files untouched. Confirm what was removed (workflow path, script -path, config path). The change takes effect once you commit the removal. +vendored `${CLAUDE_PROJECT_DIR}/.github/doc-sweep/docs-ci-check.sh` and its vendored +`doc-classify.mjs`, and the `docs-ci.json` config. Remove the now-empty `.github/doc-sweep/` +directory if nothing else remains. Leave all other workflows and files untouched. Confirm what +was removed (workflow path, script path, classifier path, config path). The change takes effect +once you commit the removal. diff --git a/plugins/doc-sweep/skills/install-docs-ci/evals/benchmark.json b/plugins/doc-sweep/skills/install-docs-ci/evals/benchmark.json index b0af01c..5121151 100644 --- a/plugins/doc-sweep/skills/install-docs-ci/evals/benchmark.json +++ b/plugins/doc-sweep/skills/install-docs-ci/evals/benchmark.json @@ -3,103 +3,25 @@ "pass_rate": 1.0, "threshold": 0.9, "model": "claude-opus-4-8[1m]", - "source_hash": "sha256:625d1a0d8d25baa351c06556b4d53278cff0a5478f9d22c299f34fa440d219d5", + "source_hash": "sha256:cb680e3306f388e40d590fc5d67831563a127f90d9627af13aef8caef3545f6e", "results": [ - { - "eval_id": 1, - "text": "Vendors the check script docs-ci-check.sh into the project's .github/doc-sweep/ directory", - "passed": true, - "evidence": "Copied bundled hooks/docs-ci-check.sh to $ws/.github/doc-sweep/docs-ci-check.sh; diff -q vs source = identical" - }, - { - "eval_id": 1, - "text": "Writes a config JSON at .github/doc-sweep/docs-ci.json capturing docMode AND an excludeDirs array", - "passed": true, - "evidence": "docs-ci.json = {docMode:\"default\", excludeDirs:[]}; node JSON.parse confirmed docMode + Array.isArray(excludeDirs)" - }, - { - "eval_id": 1, - "text": "Scaffolds a pull_request workflow at .github/workflows/doc-sweep-docs.yml that runs the vendored check script with checkout fetch-depth:0", - "passed": true, - "evidence": "workflow has on:pull_request, actions/checkout@v6 fetch-depth:0, and run: bash .github/doc-sweep/docs-ci-check.sh .github/doc-sweep/docs-ci.json" - }, - { - "eval_id": 1, - "text": "Prints a structured summary listing the workflow/script/config paths, the doc-file set, the [skip docs] token, the branch-protection note, and how to edit/uninstall", - "passed": true, - "evidence": "step-6 summary listed workflow/script/config paths, doc-file set=default, [skip docs] token, required-status-check note, and re-run /doc-sweep:install-docs-ci" - }, - { - "eval_id": 1, - "text": "Does not scaffold anything until the user confirms", - "passed": true, - "evidence": "SKILL.md gates scaffolding on confirmation ('Nothing is installed until you run this and confirm'); scope collected via AskUserQuestion before any copy/write; detection step is read-only" - }, - { - "eval_id": 1, - "text": "Re-running the install does not duplicate the workflow (idempotent)", - "passed": true, - "evidence": "Ran full install a SECOND time: exactly 1 workflow file and grep -c 'docs-staleness:' = 1 both times; rewrite-in-place, no duplicate" - }, - { - "eval_id": 2, - "text": "Deletes the scaffolded workflow .github/workflows/doc-sweep-docs.yml", - "passed": true, - "evidence": "BEFORE find listed the workflow; Uninstall rm -f executed; AFTER only the unrelated other.yml remained" - }, - { - "eval_id": 2, - "text": "Deletes the vendored check script and the docs-ci.json config under .github/doc-sweep/", - "passed": true, - "evidence": "Both .github/doc-sweep/docs-ci-check.sh and docs-ci.json rm -f'd; now-empty .github/doc-sweep/ rmdir'd; no doc-sweep files remain" - }, - { - "eval_id": 2, - "text": "Leaves unrelated workflows and files untouched", - "passed": true, - "evidence": "AFTER: .github/workflows/other.yml still present; unrelated README.md and src.txt still exist; only the three doc-sweep artifacts removed" - }, - { - "eval_id": 2, - "text": "Confirms what was removed (workflow, script, config paths)", - "passed": true, - "evidence": "SKILL.md Uninstall section directs confirming the three removed paths (workflow, script, config), all verified deleted" - }, - { - "eval_id": 3, - "text": "Detects the existing install and offers Reconfigure / Uninstall / Cancel rather than a blind fresh install", - "passed": true, - "evidence": "Detection keyed off existing doc-sweep-docs.yml + vendored script; on hit routes to AskUserQuestion with exactly Reconfigure/Uninstall/Cancel; fresh-install path skipped" - }, - { - "eval_id": 3, - "text": "On Reconfigure, rewrites the docs-ci.json config with docMode:\"minimal\" while preserving the other choices", - "passed": true, - "evidence": "Before {docMode:default, excludeDirs:[vendor]} -> After {docMode:minimal, excludeDirs:[vendor]}; docMode switched, excludeDirs preserved verbatim" - }, - { - "eval_id": 3, - "text": "Leaves the workflow file in place (rewrites it only if its path or name changed)", - "passed": true, - "evidence": "Path/name unchanged; git diff --stat of the workflow produced no output; git status --porcelain showed only the config modified" - }, - { - "eval_id": 4, - "text": "Scans for likely-vendored directories (submodules, non-root package manifests, known vendor names) and presents candidates for confirmation", - "passed": true, - "evidence": "All three signals fired: .gitmodules -> external/dep; non-root package.json -> frontend (root excluded); known names -> vendor; candidate set {external/dep, frontend, vendor} presented" - }, - { - "eval_id": 4, - "text": "Records the confirmed excludeDirs list into the .github/doc-sweep/docs-ci.json config", - "passed": true, - "evidence": "docs-ci.json excludeDirs=[external/dep, frontend, vendor]; functionally verified — PRs touching only those exited 0, an unexcluded src.js exited 1" - }, - { - "eval_id": 4, - "text": "Reads an existing excludeDirs list from .claude/context/audience-rules.md if one is already present rather than re-prompting", - "passed": true, - "evidence": "Pre-created audience-rules.md with excludeDirs list; skill read the three entries verbatim into docs-ci.json with no scan/re-prompt path" - } + { "eval_id": 1, "text": "Vendors the check script docs-ci-check.sh into the project's .github/doc-sweep/ directory", "passed": true, "evidence": "Copied hooks/docs-ci-check.sh to /.github/doc-sweep/docs-ci-check.sh; diff vs source empty (byte-identical), executable, LF-only" }, + { "eval_id": 1, "text": "Also copies doc-classify.mjs alongside docs-ci-check.sh into .github/doc-sweep/ so the classifier resolves at runtime", "passed": true, "evidence": "Copied hooks/doc-classify.mjs to /.github/doc-sweep/doc-classify.mjs (same dir); diff vs source empty" }, + { "eval_id": 1, "text": "For the default doc-file set, writes a config JSON at .github/doc-sweep/docs-ci.json that OMITS docPatterns (the classifier's built-in default applies) and records an excludeDirs array", "passed": true, "evidence": "Final docs-ci.json = {\"excludeDirs\":[]} — docPatterns correctly omitted for the default choice per SKILL.md, so the built-in default applies" }, + { "eval_id": 1, "text": "Scaffolds a pull_request workflow at .github/workflows/doc-sweep-docs.yml that runs the vendored check script with checkout fetch-depth:0", "passed": true, "evidence": "Wrote doc-sweep-docs.yml with on:pull_request, fetch-depth:0, run: bash .github/doc-sweep/docs-ci-check.sh .github/doc-sweep/docs-ci.json" }, + { "eval_id": 1, "text": "Prints a structured summary listing the workflow/script/classifier/config paths, the doc-file set, the [skip docs] token, the branch-protection note, and how to edit/uninstall", "passed": true, "evidence": "Summary included all four paths, 'Doc-file set: default (docPatterns: built-in default)', [skip docs] token, branch-protection note, and re-run /doc-sweep:install-docs-ci" }, + { "eval_id": 1, "text": "Does not scaffold anything until the user confirms", "passed": true, "evidence": "find before any action showed only README.md — no .github created prior to the post-confirmation copy/write steps" }, + { "eval_id": 1, "text": "Re-running the install does not duplicate the workflow (idempotent)", "passed": true, "evidence": "Second invocation detected the existing install and branched to Reconfigure/Uninstall/Cancel; workflow dir still had exactly one file" }, + { "eval_id": 2, "text": "Deletes the scaffolded workflow .github/workflows/doc-sweep-docs.yml", "passed": true, "evidence": "Pre-uninstall find listed doc-sweep-docs.yml; post-uninstall it is gone" }, + { "eval_id": 2, "text": "Deletes the vendored check script, the vendored doc-classify.mjs, and the docs-ci.json config under .github/doc-sweep/", "passed": true, "evidence": "All three removed; .github/doc-sweep/ empty then rmdir'd" }, + { "eval_id": 2, "text": "Leaves unrelated workflows and files untouched", "passed": true, "evidence": "Seeded other-ci.yml + NOTES.md/README.md all present and unmodified after uninstall" }, + { "eval_id": 2, "text": "Confirms what was removed (workflow, script, classifier, config paths)", "passed": true, "evidence": "Confirmation listed all four deleted absolute paths (Workflow/Script/Classifier/Config)" }, + { "eval_id": 3, "text": "Detects the existing install and offers Reconfigure / Uninstall / Cancel rather than a blind fresh install", "passed": true, "evidence": "Pre-check found workflow+script+classifier present → DETECTED branch offering the three choices; selected Reconfigure" }, + { "eval_id": 3, "text": "On Reconfigure, rewrites the docs-ci.json config with docPatterns set to the minimal glob list (e.g. CLAUDE*.md and README*.md only) while preserving the other choices", "passed": true, "evidence": "Before {excludeDirs:[build]} → after {docPatterns:[CLAUDE*.md,README*.md], excludeDirs:[build]}; excludeDirs preserved; also persisted to audience-rules.md" }, + { "eval_id": 3, "text": "Leaves the workflow file in place (rewrites it only if its path or name changed)", "passed": true, "evidence": "diff of pre/post workflow snapshot empty — byte-identical, not rewritten" }, + { "eval_id": 3, "text": "On Reconfigure, unconditionally re-copies doc-classify.mjs alongside docs-ci-check.sh into .github/doc-sweep/ (self-heals a missing or stale classifier) even though the workflow file itself is left in place", "passed": true, "evidence": "Corrupted doc-classify.mjs before reconfigure; after, diff vs bundled source empty — unconditionally re-copied/self-healed" }, + { "eval_id": 4, "text": "Scans for likely-vendored directories (submodules, non-root package manifests, known vendor names) and presents candidates for confirmation", "passed": true, "evidence": "All three signals fired: .gitmodules → external/thirdlib; non-root packages/sub-app/package.json; root vendor/ dir" }, + { "eval_id": 4, "text": "Records the confirmed excludeDirs list into the .github/doc-sweep/docs-ci.json config", "passed": true, "evidence": "docs-ci.json = {excludeDirs:[vendor, packages/sub-app, external/thirdlib]}, matching scanned candidates" }, + { "eval_id": 4, "text": "Reads an existing excludeDirs list from .claude/context/audience-rules.md if one is already present rather than re-prompting", "passed": true, "evidence": "With pre-existing audience-rules excludeDirs:[legacy_vendor], the flow read it silently; final config = {excludeDirs:[legacy_vendor]} (did NOT re-scan/add vendor)" } ] } diff --git a/plugins/doc-sweep/skills/install-docs-ci/evals/evals.json b/plugins/doc-sweep/skills/install-docs-ci/evals/evals.json index c5def8c..9704442 100644 --- a/plugins/doc-sweep/skills/install-docs-ci/evals/evals.json +++ b/plugins/doc-sweep/skills/install-docs-ci/evals/evals.json @@ -6,9 +6,10 @@ "prompt": "Install the doc-sweep docs-staleness CI check for this project with the default doc-file set.", "assertions": [ "Vendors the check script docs-ci-check.sh into the project's .github/doc-sweep/ directory", - "Writes a config JSON at .github/doc-sweep/docs-ci.json capturing docMode AND an excludeDirs array", + "Also copies doc-classify.mjs alongside docs-ci-check.sh into .github/doc-sweep/ so the classifier resolves at runtime", + "For the default doc-file set, writes a config JSON at .github/doc-sweep/docs-ci.json that OMITS docPatterns (the classifier's built-in default applies) and records an excludeDirs array", "Scaffolds a pull_request workflow at .github/workflows/doc-sweep-docs.yml that runs the vendored check script with checkout fetch-depth:0", - "Prints a structured summary listing the workflow/script/config paths, the doc-file set, the [skip docs] token, the branch-protection note, and how to edit/uninstall", + "Prints a structured summary listing the workflow/script/classifier/config paths, the doc-file set, the [skip docs] token, the branch-protection note, and how to edit/uninstall", "Does not scaffold anything until the user confirms", "Re-running the install does not duplicate the workflow (idempotent)" ], @@ -19,9 +20,9 @@ "prompt": "Uninstall the doc-sweep docs-staleness CI check from this project.", "assertions": [ "Deletes the scaffolded workflow .github/workflows/doc-sweep-docs.yml", - "Deletes the vendored check script and the docs-ci.json config under .github/doc-sweep/", + "Deletes the vendored check script, the vendored doc-classify.mjs, and the docs-ci.json config under .github/doc-sweep/", "Leaves unrelated workflows and files untouched", - "Confirms what was removed (workflow, script, config paths)" + "Confirms what was removed (workflow, script, classifier, config paths)" ], "files": [] }, @@ -30,8 +31,9 @@ "prompt": "I already set up the docs CI check here but I want to switch the doc-file set to minimal. Reconfigure it.", "assertions": [ "Detects the existing install and offers Reconfigure / Uninstall / Cancel rather than a blind fresh install", - "On Reconfigure, rewrites the docs-ci.json config with docMode:\"minimal\" while preserving the other choices", - "Leaves the workflow file in place (rewrites it only if its path or name changed)" + "On Reconfigure, rewrites the docs-ci.json config with docPatterns set to the minimal glob list (e.g. CLAUDE*.md and README*.md only) while preserving the other choices", + "Leaves the workflow file in place (rewrites it only if its path or name changed)", + "On Reconfigure, unconditionally re-copies doc-classify.mjs alongside docs-ci-check.sh into .github/doc-sweep/ (self-heals a missing or stale classifier) even though the workflow file itself is left in place" ], "files": [] }, diff --git a/plugins/doc-sweep/skills/install-revise-hook/SKILL.md b/plugins/doc-sweep/skills/install-revise-hook/SKILL.md index e87c523..724273e 100644 --- a/plugins/doc-sweep/skills/install-revise-hook/SKILL.md +++ b/plugins/doc-sweep/skills/install-revise-hook/SKILL.md @@ -28,17 +28,21 @@ parse the event JSON. 1. **Detect an existing install.** Look for a `revise-push-guard` entry in the user (`~/.claude/settings.json`) and project (`${CLAUDE_PROJECT_DIR}/.claude/settings.json`) - hooks by scanning for any `command` value that contains `doc-sweep-revise-push.sh`. + hooks by scanning for any `command` value that contains `doc-sweep-revise-push.sh` (the + copied hook script and the `doc-classify.mjs` copied next to it are both part of what + constitutes an install). - If **no install** is found → proceed to step 2 (fresh install). - If an install **is found**, offer three choices via `AskUserQuestion`: - **Reconfigure** — re-ask all choices from step 2 pre-filled with the values read from the existing config JSON; then rewrite the config; re-copy the hook script only if the - target path changed; if and only if the hook target path or settings location changed, - remove the old `PreToolUse` matcher entry (the one whose `command` references the old - hook path/file) and add a new one pointing to the updated paths — otherwise leave the - matcher as-is; leave the review marker file untouched; print the structured summary - (step 8). Stop. + target path changed, but unconditionally re-copy `doc-classify.mjs` into that (possibly + unchanged) directory regardless of whether the path changed (idempotent — this self-heals + a missing or stale classifier even when the hook path itself is unchanged); if and only if + the hook target path or settings location changed, remove the old `PreToolUse` matcher + entry (the one whose `command` references the old hook path/file) and add a new one + pointing to the updated paths — otherwise leave the matcher as-is; leave the review marker + file untouched; print the structured summary (step 8). Stop. - **Uninstall** — follow the Uninstall section below. Stop. - **Cancel** — do nothing and exit. Stop. @@ -51,9 +55,15 @@ parse the event JSON. - **Repo applicability** — `all` (guard fires in every repo) vs `doc-sweep-only` (the hook self-skips repos without a `CLAUDE.md` or `.claude/context/audience-rules.md`). Recommend `doc-sweep-only` for user-global installs. - - **Doc-file set** — `default` (CLAUDE*.md, README*.md, CHANGELOG.md, docs/**), + - **Doc-file set** — `default` (matches `doc-classify.mjs`'s built-in list: `CLAUDE*.md`, + `README*.md`, `CHANGELOG.md`, files under `docs/`, and any `.md` under `.claude/`), `with-skill` (also treats SKILL.md as a doc), or `minimal` (CLAUDE.md + README.md only). - Recommend `default`. + Recommend `default`. If the user picks `default`, do **not** transcribe this parenthetical + into a `docPatterns` list — omit `docPatterns` from the config JSON entirely (step 5) so + `doc-classify.mjs`'s own built-in default (documented in `context/audience-rules.md`) + applies. If the user picks `with-skill` or `minimal` (a custom set), record the concrete + glob list as `docPatterns` and persist it into `.claude/context/audience-rules.md` the same + way `excludeDirs` is persisted (step 4). - **Trigger event** — exactly one of: `push` (recommended; one prompt per share) or `commit` (stricter; prompts on nearly every commit). Record as `trigger` in the config. - **Bypass + uninstall** — confirm the bypass tokens (`DOC_SWEEP_REVISE_SKIP=1` or @@ -67,6 +77,12 @@ parse the event JSON. (`mkdir -p` the `hooks/` dir first.) Use an absolute path; do not rely on `${CLAUDE_PLUGIN_ROOT}` expanding inside settings.json. + Also copy this skill's bundled `../../hooks/doc-classify.mjs` into the **same directory** as + the hook above (i.e. `~/.claude/hooks/doc-classify.mjs` or + `${CLAUDE_PROJECT_DIR}/.claude/hooks/doc-classify.mjs`), since `revise-push-guard.sh` resolves + the classifier next to itself (`$here/doc-classify.mjs`). Without it the hook fails open + (allows the action) on every guarded command. + 4. **Scan for vendored directories and persist `excludeDirs`.** Before writing the config, check whether `.claude/context/audience-rules.md` already @@ -98,10 +114,22 @@ parse the event JSON. - third_party ``` + d. If the user chose a **custom** doc-file set in step 2 (`with-skill` or `minimal`), persist + the chosen glob list as a `docPatterns:` block in `.claude/context/audience-rules.md` the + same way: append the block if the file exists and doesn't already have one, update it in + place if it does, or create the file with a brief header comment if it doesn't exist yet. + If the user kept the **default**, do not write a `docPatterns:` block here. + 5. **Write the config** next to the copied hook as `doc-sweep-revise.json`: - ```json - { "docMode": "", "repoScope": "", "trigger": "", "excludeDirs": [] } - ``` + - Default doc-file set — omit `docPatterns` entirely: + ```json + { "repoScope": "", "trigger": "", "excludeDirs": [] } + ``` + - Custom doc-file set (`with-skill` or `minimal`) — include the concrete glob list, + mirroring what was persisted to `audience-rules.md` in step 4d: + ```json + { "docPatterns": [], "repoScope": "", "trigger": "", "excludeDirs": [] } + ``` The `excludeDirs` array must match the list persisted in step 4. 6. **Merge the hook into settings.json (idempotent).** Read the chosen settings.json @@ -137,9 +165,10 @@ parse the event JSON. ───────────────────────────────────────── Settings file : Hook script : + Classifier : Config file : Trigger : - Doc-file set : + Doc-file set : (docPatterns: ) Repo scope : Excluded dirs : Marker state : (assumption) | seeded by revise-docs-and-mark | unseeded (next gated action will block)> @@ -157,7 +186,7 @@ parse the event JSON. Remove the `PreToolUse` matcher block whose `command` value contains `doc-sweep-revise-push.sh` from the settings.json where it was found, then delete the -copied hook script and its config JSON. Leave all other settings, hooks, and the review -marker file untouched. Confirm what was removed (settings file path, hook path, config -path). The marker file is intentionally left in place so a reinstall can seed from it or -ignore it. +copied hook script, the copied `doc-classify.mjs` next to it, and the config JSON. Leave +all other settings, hooks, and the review marker file untouched. Confirm what was removed +(settings file path, hook path, classifier path, config path). The marker file is +intentionally left in place so a reinstall can seed from it or ignore it. diff --git a/plugins/doc-sweep/skills/install-revise-hook/evals/benchmark.json b/plugins/doc-sweep/skills/install-revise-hook/evals/benchmark.json index 9e8fef4..f3a043e 100644 --- a/plugins/doc-sweep/skills/install-revise-hook/evals/benchmark.json +++ b/plugins/doc-sweep/skills/install-revise-hook/evals/benchmark.json @@ -1,111 +1,28 @@ { "skill": "install-revise-hook", - "pass_rate": 0.9411764705882353, + "pass_rate": 1.0, "threshold": 0.9, - "model": "claude-sonnet-4-6", - "source_hash": "sha256:a90ae56e2ff5ea8e6e20066d9d4615fe146405b63d5d4c0b55e8b2c11235f283", + "model": "claude-opus-4-8[1m]", + "source_hash": "sha256:0d98ec25d5605f26ef7a89edc24cd8c6e10acecadf75ff716e7a396638cd0e4b", "results": [ - { - "eval_id": 1, - "text": "Copies the hook script to a stable path under the project's .claude/hooks/", - "passed": true, - "evidence": "eval-1 outputs: .claude/hooks/doc-sweep-revise-push.sh present (-rwxr-xr-x), real bash script" - }, - { - "eval_id": 1, - "text": "Writes a config JSON capturing docMode/repoScope AND trigger:\"commit\" AND an excludeDirs array", - "passed": true, - "evidence": "doc-sweep-revise.json = {docMode:default, repoScope:doc-sweep-only, trigger:commit, excludeDirs:[]}" - }, - { - "eval_id": 1, - "text": "Merges a PreToolUse matcher:Bash hook entry into the project settings.json referencing the copied hook + config, preserving any existing settings", - "passed": true, - "evidence": "settings.json has a PreToolUse/Bash matcher whose command references the hook + config paths" - }, - { - "eval_id": 1, - "text": "Offers to seed the review marker on this fresh install (seed-now / review-now / leave) rather than silently leaving it unset", - "passed": true, - "evidence": "transcript Step 7 offers seed/review/leave and seeds the marker; doc-sweep-revise-marker output contains a HEAD SHA" - }, - { - "eval_id": 1, - "text": "Prints a structured summary listing the settings/hook/config paths, the trigger, marker state, bypass tokens, and how to edit/uninstall", - "passed": true, - "evidence": "transcript Step 8 summary lists settings/hook/config paths, Trigger=commit, marker state, bypass tokens, edit/uninstall line" - }, - { - "eval_id": 1, - "text": "Re-running the install does not duplicate the hook entry (idempotent)", - "passed": false, - "evidence": "Executor stated idempotency would apply but did not actually perform a second run; burden of proof not met" - }, - { - "eval_id": 2, - "text": "Removes the guard PreToolUse entry (the one referencing doc-sweep-revise-push.sh) from settings.json", - "passed": true, - "evidence": "settings-after-uninstall.json no longer contains the Bash/doc-sweep-revise-push.sh PreToolUse entry" - }, - { - "eval_id": 2, - "text": "Deletes the copied hook script and the config JSON", - "passed": true, - "evidence": "deleted-files.txt lists hook script + config JSON deleted; no hook file in eval-2 outputs" - }, - { - "eval_id": 2, - "text": "Leaves unrelated settings (other hooks/keys) untouched", - "passed": true, - "evidence": "settings-after-uninstall.json retains theme:dark and the unrelated PreToolUse/Write matcher" - }, - { - "eval_id": 2, - "text": "Leaves the per-clone review marker file in place (does not delete it)", - "passed": true, - "evidence": "doc-sweep-revise-marker still present after uninstall; deleted-files.txt marks it preserved" - }, - { - "eval_id": 3, - "text": "Detects the existing install and offers Reconfigure / Uninstall / Cancel rather than doing a blind fresh install", - "passed": true, - "evidence": "transcript Step 1 detects doc-sweep-revise-push.sh and offers Reconfigure/Uninstall/Cancel" - }, - { - "eval_id": 3, - "text": "On Reconfigure, rewrites the config JSON with the new trigger while preserving the other choices", - "passed": true, - "evidence": "doc-sweep-revise.json trigger changed to commit; docMode/repoScope/excludeDirs unchanged" - }, - { - "eval_id": 3, - "text": "Leaves the review marker file untouched during reconfigure (does not re-seed it)", - "passed": true, - "evidence": "doc-sweep-revise-marker SHA unchanged from the pre-existing install" - }, - { - "eval_id": 3, - "text": "Updates the settings.json matcher only if the hook path or settings location changed", - "passed": true, - "evidence": "path + settings location unchanged → no matcher update; settings.json command string unmodified" - }, - { - "eval_id": 4, - "text": "Scans for likely-vendored directories (submodules, non-root package manifests, known vendor names) and presents candidates for confirmation", - "passed": true, - "evidence": "transcript Step 4 shows all three signals (.gitmodules, non-root package.json, known names) producing deduped candidates for confirmation" - }, - { - "eval_id": 4, - "text": "Persists the confirmed excludeDirs list into .claude/context/audience-rules.md", - "passed": true, - "evidence": "audience-rules.md output contains the excludeDirs YAML list with the four confirmed dirs" - }, - { - "eval_id": 4, - "text": "Mirrors the same excludeDirs into the hook config JSON written next to the hook", - "passed": true, - "evidence": "doc-sweep-revise.json excludeDirs matches the four entries in audience-rules.md" - } + { "eval_id": 1, "text": "Copies the hook script to a stable path under the project's .claude/hooks/", "passed": true, "evidence": ".claude/hooks/doc-sweep-revise-push.sh present (6662 bytes, executable), byte-identical to bundled revise-push-guard.sh" }, + { "eval_id": 1, "text": "Also copies doc-classify.mjs into the same directory as the copied hook script so the classifier resolves at runtime", "passed": true, "evidence": ".claude/hooks/doc-classify.mjs (3296 bytes) copied alongside the hook, matching the bundled file" }, + { "eval_id": 1, "text": "Writes a config JSON capturing docPatterns (a glob list)/repoScope AND trigger:\"commit\" AND an excludeDirs array", "passed": true, "evidence": "doc-sweep-revise.json = {repoScope:doc-sweep-only, trigger:commit, excludeDirs:[]}; docPatterns correctly omitted for the default doc-file set (built-in default applies)" }, + { "eval_id": 1, "text": "Merges a PreToolUse matcher:Bash hook entry into the project settings.json referencing the copied hook + config, preserving any existing settings", "passed": true, "evidence": "settings.json hooks.PreToolUse[0] = {matcher:Bash, hooks:[{type:command, command:' '}]}; fresh install, nothing to preserve" }, + { "eval_id": 1, "text": "Offers to seed the review marker on this fresh install (seed-now / review-now / leave) rather than silently leaving it unset", "passed": true, "evidence": "Seed-now path: git rev-parse HEAD written to $(git rev-parse --git-common-dir)/doc-sweep-revise-marker" }, + { "eval_id": 1, "text": "Prints a structured summary listing the settings/hook/classifier/config paths, the trigger, marker state, bypass tokens, and how to edit/uninstall", "passed": true, "evidence": "Summary listed real abs paths for settings/hook/classifier/config, Trigger: commit, Marker seeded at , bypass tokens, and re-run /doc-sweep:install-revise-hook" }, + { "eval_id": 1, "text": "Re-running the install does not duplicate the hook entry (idempotent)", "passed": true, "evidence": "grep -c of the hook path in settings.json was 1 before and 1 after a simulated re-run (Reconfigure path, unchanged paths → matcher untouched)" }, + { "eval_id": 2, "text": "Removes the guard PreToolUse entry (the one referencing doc-sweep-revise-push.sh) from settings.json", "passed": true, "evidence": "Seeded 2 matchers; after node-based removal, only the unrelated 'some-other-hook.sh' matcher remains" }, + { "eval_id": 2, "text": "Deletes the copied hook script, the copied doc-classify.mjs next to it, and the config JSON", "passed": true, "evidence": ".claude/hooks/ empty after uninstall — hook, doc-classify.mjs, and config all removed" }, + { "eval_id": 2, "text": "Leaves unrelated settings (other hooks/keys) untouched", "passed": true, "evidence": "settings.json still has someOtherSetting:true, the unrelated PreToolUse matcher, and the PostToolUse block byte-for-byte" }, + { "eval_id": 2, "text": "Leaves the per-clone review marker file in place (does not delete it)", "passed": true, "evidence": "doc-sweep-revise-marker still present after uninstall with the original seeded SHA" }, + { "eval_id": 3, "text": "Detects the existing install and offers Reconfigure / Uninstall / Cancel rather than doing a blind fresh install", "passed": true, "evidence": "grep for the hook filename in settings.json matched → detection fired → Reconfigure branch instead of blind fresh install" }, + { "eval_id": 3, "text": "On Reconfigure, rewrites the config JSON with the new trigger while preserving the other choices", "passed": true, "evidence": "Before {repoScope:all, trigger:push, excludeDirs:[]} → after trigger:commit; repoScope and excludeDirs preserved" }, + { "eval_id": 3, "text": "Leaves the review marker file untouched during reconfigure (does not re-seed it)", "passed": true, "evidence": "Marker SHA identical before and after reconfigure" }, + { "eval_id": 3, "text": "Updates the settings.json matcher only if the hook path or settings location changed", "passed": true, "evidence": "Path/location unchanged → PreToolUse matcher block byte-identical before/after (no edit)" }, + { "eval_id": 3, "text": "On Reconfigure, unconditionally re-copies doc-classify.mjs next to the hook script (self-heals a missing or stale classifier) even when the hook path itself is unchanged and the hook script is not re-copied", "passed": true, "evidence": "mtime: doc-classify.mjs changed after reconfigure (re-copied) while doc-sweep-revise-push.sh mtime unchanged (not re-copied)" }, + { "eval_id": 4, "text": "Scans for likely-vendored directories (submodules, non-root package manifests, known vendor names) and presents candidates for confirmation", "passed": true, "evidence": "Three signals fired: .gitmodules → vendor/thirdlib; non-root packages/app/package.json; root vendor/ dir" }, + { "eval_id": 4, "text": "Persists the confirmed excludeDirs list into .claude/context/audience-rules.md", "passed": true, "evidence": "audience-rules.md written with excludeDirs: block listing vendor and packages/app" }, + { "eval_id": 4, "text": "Mirrors the same excludeDirs into the hook config JSON written next to the hook", "passed": true, "evidence": "config = {repoScope:all, trigger:push, excludeDirs:[vendor, packages/app]}; end-to-end check via real doc-classify.mjs --config produced {nonDoc:[src/index.js], docChanged:true} — vendored paths excluded" } ] } diff --git a/plugins/doc-sweep/skills/install-revise-hook/evals/evals.json b/plugins/doc-sweep/skills/install-revise-hook/evals/evals.json index 87723ca..a7b09d3 100644 --- a/plugins/doc-sweep/skills/install-revise-hook/evals/evals.json +++ b/plugins/doc-sweep/skills/install-revise-hook/evals/evals.json @@ -6,10 +6,11 @@ "prompt": "Install the revise-docs guard for this project: project-scoped settings, default doc-file set, guard only doc-sweep-enabled repos, and gate on git commit (not push).", "assertions": [ "Copies the hook script to a stable path under the project's .claude/hooks/", - "Writes a config JSON capturing docMode/repoScope AND trigger:\"commit\" AND an excludeDirs array", + "Also copies doc-classify.mjs into the same directory as the copied hook script so the classifier resolves at runtime", + "Writes a config JSON capturing docPatterns (a glob list)/repoScope AND trigger:\"commit\" AND an excludeDirs array", "Merges a PreToolUse matcher:Bash hook entry into the project settings.json referencing the copied hook + config, preserving any existing settings", "Offers to seed the review marker on this fresh install (seed-now / review-now / leave) rather than silently leaving it unset", - "Prints a structured summary listing the settings/hook/config paths, the trigger, marker state, bypass tokens, and how to edit/uninstall", + "Prints a structured summary listing the settings/hook/classifier/config paths, the trigger, marker state, bypass tokens, and how to edit/uninstall", "Re-running the install does not duplicate the hook entry (idempotent)" ], "files": [] @@ -19,7 +20,7 @@ "prompt": "Uninstall the revise-docs guard from this project.", "assertions": [ "Removes the guard PreToolUse entry (the one referencing doc-sweep-revise-push.sh) from settings.json", - "Deletes the copied hook script and the config JSON", + "Deletes the copied hook script, the copied doc-classify.mjs next to it, and the config JSON", "Leaves unrelated settings (other hooks/keys) untouched", "Leaves the per-clone review marker file in place (does not delete it)" ], @@ -32,7 +33,8 @@ "Detects the existing install and offers Reconfigure / Uninstall / Cancel rather than doing a blind fresh install", "On Reconfigure, rewrites the config JSON with the new trigger while preserving the other choices", "Leaves the review marker file untouched during reconfigure (does not re-seed it)", - "Updates the settings.json matcher only if the hook path or settings location changed" + "Updates the settings.json matcher only if the hook path or settings location changed", + "On Reconfigure, unconditionally re-copies doc-classify.mjs next to the hook script (self-heals a missing or stale classifier) even when the hook path itself is unchanged and the hook script is not re-copied" ], "files": [] },