Trim skill tokens: progressive disclosure (~55% resident cut) - #88
Conversation
…tokens) The flow skill's SKILL.md is loaded resident by the Skill tool and re-billed as cached tokens on every turn, making it the dominant contributor to flow's per-session token cost (~27.7K tokens, multiplied across a long session). Measured attribution: the skill body is ~95% of flow's fixed per-session overhead; the SessionStart injection (~375 tok) and UserPromptSubmit anchor (~58 tok) are rounding error by comparison. Split SKILL.md into a lean resident core (~50 KB / ~12.6K tokens, down from ~111 KB / ~27.7K) plus twelve on-demand references/*.md files holding rarely-needed workflows (playbooks, owners, upgrade, weekly review/archive, tagging, session-binding detail, project intake, first-run setup, brief templates, work_dir recipe, cross-task transcripts & field edits). The core keeps a one-line trigger pointing at each reference, so a workflow's full text loads only when needed — paid once, not every turn. Resident prose was also tightened. No workflow content removed. Plumbing: embed changed from `//go:embed skill/SKILL.md` to the whole `skill/` dir (embed.FS); Harness.InstallSkill now takes an fs.FS and walks the tree, writing SKILL.md next to references/ under ~/.claude/skills/flow/. UninstallSkill already RemoveAll's the dir. No-info-loss guards: content assertions now run against the full corpus (SKILL.md + references) via skillCorpus, so relocating a section can't silently drop it; TestSkillReferencesAreReachable forbids orphan references; TestSkillCoreIsLean caps the resident core to prevent re-inflation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
anshulsao
left a comment
There was a problem hiding this comment.
Clean split, and the partition is the right one — passive/always-on sections (KB scoop, scope-creep, close-out, substantive-work, anti-patterns, bootstrap contract) stayed resident, only trigger-gated workflows moved out, and each moved section kept its trigger phrases inline in the core so the model still knows when to load it.
Verified independently, not just from the PR body:
- Diffed every heading in the base
SKILL.mdagainst the new core + all 12 references — zero base headings dropped. - Core is 50,387 B, under the 54 KB
TestSkillCoreIsLeanceiling;fs.Sub(...,"skill")strips the prefix correctly so the walk yieldsSKILL.md/references/<x>.md; no danglingembeddedSkillsymbol survives. - The two new tests map 1:1 onto the split's real risks (re-inflation, orphans), and content assertions now run against the full corpus so a relocation can't silently drop a section.
Approving. One non-blocking [question] inline — a future-only upgrade-hygiene note, doesn't gate.
What this changes (behavior)
BEFORE (monolith resident) AFTER (lean core + on-demand refs)
--------------------------- -----------------------------------
Session start Session start
| |
v v
loads SKILL.md ~111KB/~27.7K tok loads SKILL.md [~ 50KB / ~12.6K tok] [~ 55% cut]
| |
v v
whole manual RESIDENT lean core RESIDENT
| |
v v
EVERY turn re-billed ~27.7K x turns trigger fires? --no--> nothing else loaded
(playbooks/owners/tags all resident |
even if the session never uses them) +--yes--> [+ Read references/<x>.md] paid ONCE
Install / embed pipeline
BEFORE AFTER
------ -----
//go:embed skill/SKILL.md //go:embed skill [~ whole dir -> embed.FS]
| |
v v
embeddedSkill []byte [+ skillFiles() = fs.Sub(fs,"skill")]
| |
v v
InstallSkill(content []byte) [~ InstallSkill(files fs.FS)]
| |
v v
os.WriteFile(SKILL.md) [+ fs.WalkDir -> write each file, preserving rel path]
| |
v v
~/.claude/skills/flow/SKILL.md ~/.claude/skills/flow/SKILL.md + references/*.md [+ 12 files]
Frontend: no UI — backend/tooling-only PR.
| } | ||
| return nil | ||
| base := filepath.Dir(p) | ||
| return fs.WalkDir(files, ".", func(rel string, d fs.DirEntry, walkErr error) error { |
There was a problem hiding this comment.
[question] InstallSkill now writes every file in the embed tree but never prunes files already on disk that aren't in it.
Both upgrade paths (maybeAutoUpgradeSkill and skill install --force / flow skill update) call this in place — only the explicit flow skill uninstall does a RemoveAll. Harmless for this PR (the split adds references/ fresh, overwrites SKILL.md, keeps VERSION — nothing is stranded today). But the first future rename/removal of a reference (e.g. tags.md -> tagging.md) will leave the old file orphaned on disk across a flow skill update. Not blocking — fine to leave as a known follow-up, or RemoveAll references/ before the walk if you'd rather update be self-cleaning. Your call.
Why
Users reported flow accounts for ~21% of their weekly Claude usage. Measured attribution (via the same real-transcript token counts
flow statsreads): the skill body loaded by the Skill tool is ~95% of flow's fixed per-session token cost — ~27.7K tokens, and because it sits resident it's re-billed as cache-read on every turn, so the effective cost is ~27.7K × turns. The SessionStart injection (~375 tok) and UserPromptSubmit anchor (~58 tok) are rounding error next to it. So the one lever that matters is shrinking what stays resident.What
Progressive disclosure: a lean resident
SKILL.md(~50 KB / ~12.6K tokens, down from ~111 KB / ~27.7K — 55% cut) plus twelve on-demandreferences/*.mdfiles holding rarely-needed workflows:playbooks·owners·do-advanced(live-guard / Accessibility /--with) ·binding·tags·templates·reviews(archive + weekly) ·intake-project·setup(first-run) ·upgrade·workdir·commands-advanced(transcripts + field edits).The core keeps a one-line trigger pointing at each reference, so a workflow's full text loads only when actually needed — paid once, not every turn. Resident prose was also tightened. No workflow content was removed — sections were relocated verbatim (then lightly compressed only in the resident core).
Plumbing
//go:embed skill/SKILL.mdto the wholeskill/dir (embed.FS).Harness.InstallSkill([]byte)→InstallSkill(fs.FS); the claude impl walks the tree and writesSKILL.mdnext toreferences/under~/.claude/skills/flow/.UninstallSkillalreadyRemoveAlls the dir; theVERSIONsidecar is left untouched.No-info-loss guards (new/changed tests)
SKILL.md+ all references) viaskillCorpus, so relocating a section can't silently drop it.TestSkillReferencesAreReachable— everyreferences/*.mdmust be pointed at from the core (no orphans).TestSkillCoreIsLean— caps the resident core (54 KB) to prevent re-inflation.mainSKILL.md still appears in the corpus; a realflow skill installinto an isolated$HOMElays down all 13 files with every reference reachable.Testing
go test ./...passes on the touched packages (app, harness, stats). One unrelated pre-existing failure remains:TestCmdListTasksSinceTodayfails identically on cleanmain(time-dependent, date rolled over) — out of scope here.🤖 Generated with Claude Code