test(security): bind the file census to the guard's EMITTED output, not to a return value - #341
Merged
Merged
Conversation
…ot to a return value Remediation on the review of #340. The census change shipped undefended: deleting `(${census})` from the fail path's console.error left the suite fully green at 19 pass / 0 fail. The existing `expect(scanned).toBe(2)` assertions read scanPaths()'s RETURN VALUE, and a returned number and a printed line are different things — only one of them is the deliverable. This is the same hole this branch already argued about the arity half: asserting a positive that cannot distinguish the fixed state from the broken one. I made that argument and then did not apply it to my own change. It is worth a real regression rather than a note, because the number is load-bearing OUTSIDE this repository. The CI step name reads identically before and after the scan was widened from a basename allow-list to the whole text tree, so a green step proves a guard ran and never which guard ran; the count of files opened is the only observable that changed, and a release gate elsewhere keys on `scanned > 20000` in this output. If the census silently stops printing on the fail path, that gate reads the absence as a pass. The new tests run the REAL entrypoint against a synthetic git-tracked package — git index plus package.json, so the guard's `git ls-files` union with `npm pack --dry-run` is exercised as a publish would — and assert the emitted text. They check the count is present, that it is the true file count rather than a hardcodable literal, and that the report still never echoes what it matched. Verified in both directions rather than asserted. Against main 134cf24, whose fail path carries no census: 20 pass / 1 fail. Against the reviewer's exact mutation on the fixed file: 20 pass / 1 fail. Restored: 21 pass / 0 fail. Suite 3817 pass / 190 fail against a 3813 / 190 baseline on main — identical failure sets, the +4 are this branch's tests. Typecheck clean. Test-only change; the guard itself is untouched by this commit. Refs: todos 38d15243 Agent: Silvanus
Contributor
Author
|
[REVIEW] GO — #341 @ f939144 — lens: correctness+security+gates, reviewer unresolved-account001 (1 of 1) Reviewed exact candidate:
Commands and results:
Blocking P0/P1 findings: none. Non-blocking follow-ups: none. Verdict: GO. The added tests exercise the real guard entrypoint on both success and failure output paths and correctly bind the observable file census without leaking the matched sentinel. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remediation on the review of #340 (merged). Test-only — the guard itself is untouched by this PR.
The defect
The census change in #340 shipped undefended. Deleting
(${census})from the fail path'sconsole.errorleft the suite fully green:expect(scanned).toBe(2)readsscanPaths()'s return value. A returned number and a printed line are different things, and only one of them is the deliverable. I reproduced the reviewer's mutation independently before fixing it and got the same green.This is the identical hole #340 argued about the arity half — asserting a positive that cannot distinguish the fixed state from the broken one. I made that argument and then failed to apply it to my own change.
Why it warrants a real regression rather than a note
The CI step name
Package-manager secret guardreads identically before and after the scan was widened from a basename allow-list to the whole text tree. A green step proves a guard ran, never which guard ran. The count of files opened — roughly 1,220 then, 22,745 now — is the only observable that changed, and a release gate outside this repository keys onscanned > 20000in this output. If the census silently stops printing on the fail path, that external gate reads the absence as a pass. A number that is load-bearing elsewhere needs a test that fails when it goes missing.What the tests do
They run the real entrypoint against a synthetic git-tracked package — a git index plus a
package.json, so the guard'sgit ls-filesunion withnpm pack --dry-runis exercised the way a publish would — and assert the emitted text:Verified in both directions
Suite 3817 pass / 190 fail against a 3815 / 190 baseline measured on
HEAD~1— identical failure sets, the +2 are these tests.bun run typecheckexits 0. Sentinels are synthetic; 0 of the naming standard's 18 example names and 0 of the 2 scrubbed connector literals appear in the added lines.Note on how this PR is based
#340 was merged while this remediation was in flight, and my push re-created the auto-deleted branch. Rather than PR that stale branch — whose diff would have reverted the
bun run build && bun testchange that landed ine8128814— I cut a fresh branch from currentmainand cherry-picked the test commit. The diff againstmainis 74 added lines in one file.Refs: todos
38d15243Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.