docs(skill): rewrite ecp skill pack, A/B-validated on Haiku#646
Merged
Conversation
- ecp SKILL.md: backtick real commands in the description map
("definition→`ecp find`") — the bare arrow form ("where→find",
"trace-flow→processes") made Haiku invent verbs (ecp where,
ecp trace-flow) in 2/9 trigger trials; 0/9 after. Collapse the four
directives into Core rules: drop the ambiguous-name trap (the CLI
error already instructs --file/--kind; no-skill control passed 3/3)
and the warm-attach mechanics prose (the result caveat from PR #504
self-instructs; control passed 3/3). Remove stale HIGH/CRITICAL
risk_level wording (feature removed in PR #592). Body -12%.
- ecp ECP.md: same stale HIGH/CRITICAL fix.
- ecp-onboard SKILL.md: merge when-to-use triggers into description —
skill loaders surface only description, so the ONBOARDING-link
trigger never fired (baseline 0/3 → 3/3 after).
- lint-skill.sh: stop requiring when-to-use (it codified the
invisible-field bug); regression case in test-lint-frontmatter.sh.
- codex ecp SKILL.md: adopt the validated command-map description
(who-calls trials 0/3 → 3/3); collapse the three layers.
A/B method: claude -p --model haiku --setting-sources project from a
clean dir, 11 scenarios x 3 arms (no-skill control / current /
rewritten), n=3-6 per cell, flagged outputs read manually.
Contributor
ecp impact cache (0 symbols) — internal, used by
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Rewrites all three ecp skills (
docs/skills/ecp,docs/skills/ecp-onboard,skill_sample/codex/ecp) following skill-writing TDD: baseline (RED) runs on Haiku first, rewrite targets only the observed failures, then re-test (GREEN). Net -30 lines, main skill body -12%.A/B method
claude -p --model haiku --setting-sources projectfrom a clean dir (contamination-checked control), 11 scenarios × 3 arms (no-skill control / current / rewritten), n=3–6 per cell, every flagged output read manually. Scenarios cover: trigger-stage verb selection (4), command selection with body loaded (4), output interpretation with canned real-format fixtures (3).Findings → changes
where→find,trace-flow→processes) makes Haiku invent verbsecp where,ecp trace-flowin 2/9 trigger trialsdefinition→\ecp find`` — 0/9 afteradd --file or --kind; no-skill control passed 3/3resultcaveat (PR #504) self-instructs; control passed 3/3HIGH/CRITICALrisk wording is stalewhen-to-use, a field skill loaders never surfacedescriptionlint-skill.shno longer requireswhen-to-use(it codified the bug); regression case addedecp grep,ecp refs,ecp callers)ecp impact; three-layer structure collapsedLoad-bearing content kept verbatim (control failed without it, current passed): Quick Reference tables,
impact --literal, the orphan cypher example (copied verbatim by Haiku 6/6), lower-bound cross-check rule, honest-miss red-line.Notes
tests/skill/test-gen-cli-ref.shfails identically on unmodifiedmain— pre-existing, untouched here.docs/skills/ecp-onboard/SKILL.md(last updated 2026-05-24); that PR needs a rebase regardless.~/.claude/skills/ecpcopy syncs at next release per convention; its locally-tuned description is upstreamed here.