Add offline marketplace validator#87
Merged
Merged
Conversation
Single Python script run via 'python3 scripts/validate.py' covering:
- marketplace.json + plugin.json shape via jsonschema
- SKILL.md frontmatter required keys + name/directory match
- YAML parse for every .yaml under marketplace/plugins
- Path references (${CLAUDE_PLUGIN_ROOT}, compliance/<framework>/) resolve
- bash -n on fenced bash blocks with placeholder sanitization
- Baseline regression: skills present on master but missing from HEAD
without a BREAKING.md entry fail the run
No credentials, no network, ~3s runtime. Tier-A only; sandbox dry-run
and golden-prompt tiers deferred until justified by a concrete failure.
Refs refractionPOINT/tracking#4229
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/validate.py— Tier-1 offline validation for the plugin marketplace, no credentials, ~3s runtimemaster.gitignorefor working-notes files (.action.md,actions.md)Why
The repo currently has no automated testing. PR #85 (merged 2026-04-30) silently carved ~25 skills out of
lc-essentialsinto a new plugin without a migration note — there was no mechanism to surface that to a reviewer. This validator would have flagged it.What it checks
marketplace.json+ eachplugin.jsonvia jsonschema (required fields, semver, source paths exist, names match)namematches directory.yamlundermarketplace/plugins/${CLAUDE_PLUGIN_ROOT}/...,compliance/<framework>/...) resolve to real filesbash -non fenced ```bash blocks (placeholders sanitized)masterbut missing from HEAD without an entry inBREAKING.md→ hard error; cross-plugin moves → warningFirst-run findings on current master
5 errors + 11 warnings. Errors are pre-existing — to be addressed in follow-up PRs (or via baseline ignore-list when the GitHub Actions workflow lands):
lc-essentials/limacharlie-calldeleted with no replacement, noBREAKING.mdlc-essentials/web-ui-link/SKILL.md— frontmatter YAML invalidlc-essentials/investigation-creation/SKILL.md— frontmattername≠ directorylc-fundamentals/adapters/SKILL.md— ```bash fence wraps aWebFetch(...)calllc-essentials/sensor-tasking/SKILL.md— markdown leak into code blockOut of scope (follow-up)
.github/workflows/validate.ymlto run this on every PRBREAKING.mddocumenting the lc-essentials → lc-advanced-skills carve-outlimacharlie sync --dry-run(deferred — needs scoped CI org + secret)Test plan
pip install --break-system-packages pyyaml jsonschemapython3 scripts/validate.pyfrom repo root → expect 5 errors / 11 warnings against current masterRefs refractionPOINT/tracking#4229