test: add generator proof-of-behavior gates (Short + classic-manifest path hygiene)#228
Merged
Merged
Conversation
Regression guard over the full NewRootCmd tree; 0 violations today. Reuses the existing walkCommands/commandPath helpers. Closes the one Proof-D slice not already covered by destructive_annotations_test.go. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Validates every Path/IDPath/GroupPath token in specs/classic/resources.yaml is a clean single URL segment and the constructed /JSSResource/ URL is well-formed. Parses clean today; guards the hand-maintained manifest against typo'd tokens that nothing else catches. The design's Proof B. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses final-review nit: the prior comment could be misread as implying hidden commands' children are also skipped; walkCommands descends into them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses local-review finding (1): the manifest proof's floor check guards against an empty parse but not against the validator logic itself being weakened. A permanent table test keeps the 'prove the gate can fail' property after the dev-time negative control was removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ktn-jamf
force-pushed
the
worktree-feat+generator-proof-gates
branch
from
0cf3a7e to
76651e6
Compare
neilmartin83
approved these changes
May 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two offline generator-verification gates, inspired by an audit of the cli-printing-press "proof of behavior" layer. Both are pure
go testregression guards — no runtime cost, no new CI wiring, no production or generated code touched.TestProofAllCommandsHaveShort(internal/commands/) — fails CI if any command in theNewRootCmdtree ships without a non-emptyShort.Shortdrives--help, the docs site (commands -o json), and agent/MCP summaries; a blank one renders a broken-looking command. 0 violations today — this keeps it that way as new generated/hand-written commands land.TestProofClassicManifestPathsWellFormed+TestIsCleanPathSegment(generator/classic/) — fails CI if a hand-edited path token inspecs/classic/resources.yamlhas a stray space/slash/control char. That typo otherwise produces a broken/JSSResource/…URL that nothing catches until it 404s against a live instance (or inmake smoke).Scope note (why only two gates)
A pre-implementation audit found 3 of the 4 originally-designed proofs already exist and are implemented as well or better:
scripts/lint-dead-code/pro_resources_test.go::TestBackupResources_AllKeysRegistereddestructive_annotations_test.goSo only the two genuine gaps were built. The audit + reasoning are recorded in
docs/superpowers/specs/2026-05-31-generator-proof-gates-design.md(with a scope-revision banner) anddocs/superpowers/plans/.Heads-up for reviewers
The
Shortgate walks the entire command tree (hand-written, Protect, Platform — not just generated), so it polices every command, not only generator output. Reasonable invariant, but slightly broader than "generator verification." Easy to scope to generated-only if preferred.Test Plan
go test ./...passes (exit 0, no failures)golangci-lint run ./internal/commands/... ./generator/classic/...→ 0 issuesTestIsCleanPathSegmentis a permanent guard)git diff --stat main..HEADconfirms zero changes underinternal/commands/pro/generated/,internal/commands/platform/generated/, orspecs/🤖 Generated with Claude Code