feat(0.21.8): bstack skills audit — skill registry audit (Phase 6c)

[broomva]

Summary

bstack skills audit — the second skills-monorepo meta-tool (after graduate). Crystallizes the "Skill Registry Audit" pattern (3/3 in the bstack-engine ledger). Adapts Steipete's skill-cleaner algorithm for Claude Code + bstack.

5 reports

• Budget: ceil(utf8/4) token cost of all descriptions vs ceiling (default 2% of 1M); over-budget flag
• Duplicates: same name across >1 realpath (symlink-deduped)
• Registry coherence: companion-skills.yaml vs installed (missing + unregistered)
• Unused: no trace in ~/.claude/projects/**/*.jsonl within --months window
• Roots: count per root

--json + env overrides (BSTACK_AUDIT_ROOTS/BSTACK_DIR/BSTACK_AUDIT_LOG_GLOB) for hermetic tests.

Real run findings (2026-05-26)

362 skills / 331 unique across 3 roots; budget 223% of 2% ceiling (corroborates the 2026-05-25 skill-cleaner 269% reading); 31 cross-root duplicates.

Files

File	Change
scripts/skill-audit.py	NEW (~250 lines)
tests/skill-audit.test.sh	NEW — 9 hermetic tests, all pass
bin/bstack-skills	audit) dispatch + usage
VERSION	0.21.7 → 0.21.8

Test plan

• 9/9 hermetic tests pass (realpath-dedupe, duplicate detection, registry coherence, unused detection, budget flag, JSON validity, 5-section human report)
• real run against live roots dogfooded — surfaces budget pressure + duplicates correctly

Summary by CodeRabbit

Release Notes

• New Features

  • Introduced bstack skills audit subcommand to analyze your skill inventory with comprehensive reports including budget estimates, duplicate detection, registry coherence validation, and skill usage tracking.
  • Supports --json for structured output and --no-logs for isolated testing environments.

• Version

  • Updated to 0.21.8

[coderabbitai]

Warning

Review limit reached

@broomva, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 55 minutes and 2 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 534c247b-4887-49ff-a9de-7160b6592d16

📥 Commits

Reviewing files that changed from the base of the PR and between 54d8679 and 3711d54.

📒 Files selected for processing (2)

• scripts/skill-audit.py
• tests/skill-audit.test.sh

📝 Walkthrough

Walkthrough

This PR introduces the bstack skills audit subcommand that discovers and analyzes skill registry health. It discovers SKILL.md metadata, calculates token budgets, detects duplicates, validates registry coherence, scans session logs for usage patterns, and reports findings via JSON or human-readable output. Version 0.21.8 with comprehensive testing.

Changes

Skill Audit Subcommand Feature (Phase 6c)

Layer / File(s)	Summary
Version and Release Documentation VERSION, CHANGELOG.md	Version bumped from 0.21.7 to 0.21.8. Changelog documents the new audit subcommand, its outputs (budget, duplicates, registry coherence, usage analysis, per-root counts), env-based configuration, and new tooling files.
CLI Integration and Audit Dispatch bin/bstack-skills	The bstack skills CLI advertises audit [--json] [--no-logs] in usage help and dispatches to python3 $BSTACK_DIR/scripts/skill-audit.py, forwarding arguments and flags.
Skill Discovery and Metadata Extraction scripts/skill-audit.py (init, discovery, parsing)	Script initializes with pyyaml dependency check and default skill roots. Implements frontmatter parsing from SKILL.md (YAML extraction with fallbacks), token-cost calculation from description text, and skill discovery with symlink realpath de-duplication.
Registry Validation and Usage Scanning scripts/skill-audit.py (registry, usage)	Registry loading reads companion-skills.yaml for registered skill names with safe YAML parse handling. Usage scanning computes time cutoff, reads session-log JSONL via glob, builds per-skill regex patterns, and returns observed skills for unused-skill detection.
Audit Orchestration and Output scripts/skill-audit.py (main)	Parses CLI args and env overrides; discovers skills; computes budget totals and duplicates; compares discovered vs. registered names; optionally scans logs; aggregates per-root counts; and emits JSON or formatted human-readable report.
Test Suite Setup and Fixtures tests/skill-audit.test.sh (harness, fixtures, helper)	Bash test harness with strict mode and test counters. Hermetic fixture creates temporary skill roots, duplicate scenarios, fake registry YAML, and session log referencing one skill. Helper function invokes auditor with fixture configuration.
Test: CLI Help and JSON Output Validation tests/skill-audit.test.sh (T1–T2)	Verifies CLI help advertises audit command and validates JSON output parsing, including correct unique_names count with realpath deduplication.
Test: Duplicate Detection tests/skill-audit.test.sh (T3–T4)	Confirms duplicates are detected for skills across distinct realpaths and symlinked duplicates are not reported (realpath dedup in effect).
Test: Registry Coherence tests/skill-audit.test.sh (T5)	Asserts installed-but-unregistered and registered-but-missing skills are identified correctly.
Test: Usage Detection from Logs tests/skill-audit.test.sh (T6)	Validates skills in session logs are marked used; skills absent from logs are marked unused.
Test: Budget Enforcement and Flags tests/skill-audit.test.sh (T7–T8)	Validates over-budget token counts trigger error message. Confirms --no-logs skips usage scanning and emits "skipped" message.
Test: Human Report Format tests/skill-audit.test.sh (T9)	Asserts human-readable output contains all five expected section headers.
Test: Cleanup and Results tests/skill-audit.test.sh (cleanup)	Removes temporary fixture and reports aggregated test results, exiting nonzero if any test failed.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A new audit tool hops in with flair,
Discovering skills floating everywhere,
It counts the tokens, checks what's not there,
Then reports all findings with utmost care! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: introducing the bstack skills audit subcommand for skill registry auditing as Phase 6c of version 0.21.8.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/skill-audit-cli

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[broomva]

P20 Cross-Review — PASS (8/10)

Strata B fresh-context adversarial review.

Verified correct: realpath-dedupe (symlink deduped, content-identical-distinct-paths kept), regex escaping + no-false-positive, mtime cutoff, BROOMVA_ROOT per C1, all 48 real SKILL.md parsed with zero silent drops, 9/9 hermetic tests genuine.

Findings (both edge-input nits, neither in real corpus):

• B1: --chars-per-token 0 → ZeroDivisionError → fixed this commit (clamp to max(1,…) + regression test T9b)
• B2: unquoted-colon description: → silent empty parse → benign, 0 occurrences across all 48 workspace skills (real descriptions use quoting/folding); documented, no fix needed

Nested sub-skills (content-engine/skills/_{) correctly excluded by design (not independently installed/registered).}

Verdict: PASS, mergeable. 10/10 tests pass.