feat: validate skills against Agent Skills spec across harnesses - #351
Conversation
Publish the shared skills tree to two more harnesses. A Codex-native plugin manifest and marketplace catalog let Codex users install the plugin from this repo, and a root Gemini CLI extension manifest serves the skills folder that Gemini auto-discovers. A new skillsCheck gate target validates the skills tree against the Agent Skills spec, since Codex and Gemini load those folders directly and a frontmatter name that drifts from its directory ships a skill that silently fails to load. The version-bump check now names every version-carrying manifest, and the manifest tests enforce that all of them agree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019As9A3ugixLQiAvcat1ZKk
Adversarial review of the validator found documents that passed the gate but fail in a strict YAML loader: a key with no space after the colon, duplicated keys, and block-scalar values that dodge the presence and length checks. The parser now mirrors the loader where it matters, unquotes quoted scalars, strips a leading byte-order mark, validates symlinked skill folders instead of skipping them, and reports a SKILL.md that is not a regular file as a finding rather than crashing. Each fixed case carries a regression test, and the README notes that Gemini installs the extension from the latest release snapshot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019As9A3ugixLQiAvcat1ZKk
The build-graph page is regenerated since the new skillsCheck target changes the graph that docs/graph.md renders. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019As9A3ugixLQiAvcat1ZKk
|
🤖 Zuke AI review 🔎 security review —
|
| Finding | Reason |
|---|---|
| New Codex marketplace manifest may broaden installation surface without validation of its trust model | The new Codex catalog is a local path to the same checked-in ./plugins/zuke directory, and the diff adds explicit manifest/version checks plus a skillsCheck gate rather than removing or weakening existing install-time validation. |
The change mostly adds new skill-validation and manifest coverage, but it also introduces a potential authorization/policy drift issue by advertising a new Codex marketplace entry without a corresponding signed/validated installation path in the reviewed diff.
|
🤖 Zuke AI review 🔎 generic review —
|
| Finding | Reason |
|---|---|
| New skills validator is not exercised for hidden files and nested directories in the tree walk | checkSkillTree() intentionally scans only immediate directories under skills/, and the repo’s documented contract and tests treat skills/ as a flat set of top-level skills with no recursion or nonstandard children to validate. |
The change is generally well-contained, but there is one maintainability/test coverage gap around the new skills validator’s incomplete spec handling.
What & why
This PR adds validation of the
skills/tree against the Agent Skills specification, catching a critical class of silent failures: when a skill's directory name drifts from its frontmatternamefield.The problem: Zuke's skills are now served to three harnesses — Claude Code (via plugin), Codex (via native marketplace), and Gemini CLI (via extension manifest). All three load skills directly from the
skills/folder and require the frontmatternameto match the directory name. A rename that touches only one side (e.g., renaming the folder but forgetting to update the frontmatter) ships a skill that silently fails to load in the other harnesses — the worst kind of quiet breakage.The solution:
build/skill_check.ts: a spec validator that parses YAML frontmatter strictly (matching what real loaders reject), checks required fields (name,description), enforces the spec's shape and length limits, and validates the critical name/directory match.skillsChecktarget tozuke.tsthat runs this validator overskills/as part of thecigate..codex-plugin/plugin.json,.agents/plugins/marketplace.json) and Gemini CLI (gemini-extension.json), all carrying the same version.pluginVersionCheckto verify all four version-carrying manifests stay in sync.The validator is deliberately not a full YAML parser — it errs strict where YAML is strict (rejecting
key:valuewithout a space, detecting duplicated keys, handling quoted scalars) while remaining simple and auditable.Related issues
Enables safe distribution of skills to Codex and Gemini CLI without silent load failures.
Checklist
feat: ...).deno task cipasses locally.tests/skill_check_test.ts(199 lines, 11 test cases covering parser, validation, tree traversal, symlinks, and the realskills/tree),tests/integration/skill_check_test.ts(68 lines, CLI integration), andtests/plugin_manifest_test.tsextended with Codex and Gemini assertions.README.md(Codex and Gemini CLI installation),AGENTS.md(new manifests,skillsCheckin CI gate).any,ascasts, or!assertions in new code.https://claude.ai/code/session_019As9A3ugixLQiAvcat1ZKk