Fix main: invert the reporter-session assertions, and widen the test trigger - #433
Merged
Conversation
…file they read main went red on 076cbf3. `production_config.test.ts` asserted that the deploy workflow wires the reporter-session keyring through the secret store, and the cleanup that merged in 076cbf3 removed exactly that wiring. The test was describing a state we deliberately left behind. Inverted rather than deleted. The assertions are the record of what was removed, and a deleted test cannot notice the wiring coming back: HANDS_FEEDBACK_REPORTER_SESSION_KEYS must not appear wrangler secret put FEEDBACK_REPORTER_... must not appear vars.HANDS_FEEDBACK_REPORTER_SESSION_ must not appear Validate reporter-session rollout inputs must not appear A suite of `doesNotMatch` passes on an empty string, on a renamed file, and on a read that quietly returned nothing, so it keeps a positive control anchoring it to a workflow that was actually read. The environment binding moves to its own test, because it outlived the case it was living in: the environment is named for the retired feature but is not part of it, and its branch policy is what confines this deploy to main. It is now matched at job-key indentation instead of requiring `jobs:\n deploy:\n environment:` to be adjacent. That adjacency is what broke here - a comment was added above the line explaining why it must never be deleted, and the test failed because of the explanation. A test that goes red when someone documents the thing it protects gets "fixed" by deleting the documentation. Both new cases were mutation-checked: removing the environment binding reds the binding test, re-adding a keyring reference reds the absence test, and restoring the file greens both. Why nobody saw it before the merge: the pull_request paths list here enumerated the files this job *reads in order to run*, which is not the same set as the files its tests *make assertions about*. production_config.test.ts opens deploy-hands-server.yml, that path matched nothing in the list, and a PR touching only it therefore ran no tests at all - six green checks, none of them this suite. The push-to-main trigger has no paths filter, by an earlier deliberate decision, and that is what caught it, one merge too late. The deploy workflow is now in the list with the distinction written down. "No test ran" and "every test passed" are the same green on a pull request page. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deriving 'which files do these tests assert on' is itself a list that can miss one, and it misses in the direction of green - the same failure this entry exists to fix, one level up. A wildcard has nothing to miss. Cost measured rather than assumed: the last eight runs of this job took 28-42s, median 35. Thirty-five seconds on an unrelated workflow edit is affordable waste; a guard that silently does not run is an unobservable failure. `paths` is a performance optimisation and was being used as a correctness boundary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ector cors_allow_headers.test.ts scans container/src and container/package.json because the container has no cors() today and its own suite does not run here. Without container/** a PR touching only container/ does not run it - and a PR introducing the first cors() call is, almost by definition, a PR touching only container/. The instrument would have been absent from exactly the change it exists to catch, which is the trigger condition task #97 currently rests on. Its own `registrations.length > 0` guard is correctly written and would not have helped. A denominator only means anything in a run that happens. Second instance in one census, which is why these entries are wildcards rather than a list derived again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
main is red on
076cbf3. This fixes it.What broke
production_config.test.tsasserted that the deploy workflow wires thereporter-session keyring through the secret store. The cleanup merged in
076cbf3removed exactly that wiring, so the test was describing a state we deliberately
left behind.
What this does
Inverts the assertions rather than deleting them — they are the record of what
was removed, and a deleted test cannot notice the wiring coming back. Since a suite
of
doesNotMatchpasses on an empty string, on a renamed file, and on a read thatreturned nothing, it keeps a positive control anchoring it to a workflow that was
actually read.
Splits the environment binding into its own test. It outlived the case it lived
in: the environment is named for the retired feature but is not part of it. It is now
matched at job-key indentation rather than requiring
jobs:→deploy:→environment:to be adjacent — that adjacency is what broke, because a comment wasadded above the line explaining why it must never be deleted. A test that goes red
when someone documents the thing it protects gets "fixed" by deleting the
documentation.
Both new cases were mutation-checked:
environment:bindingWhy it wasn't caught before the merge
The
pull_requestpaths list enumerated the files this job reads in order to run.That is not the same set as the files its tests make assertions about.
production_config.test.tsopensdeploy-hands-server.yml; that path matched nothingin the list; a PR touching only it ran no tests at all. Six green checks, none of them
this suite.
The push-to-main trigger has no paths filter — an earlier deliberate decision — and
that is what caught it, one merge too late. It is a net, not a gate.
The fix is a wildcard, not a longer list
The first version of this PR named
deploy-hands-server.yml. That is still a derivedlist — "which files do these tests assert on" — and a derivation that misses one is
silent in the direction of green, which is the same failure this entry exists to fix,
one level up.
.github/workflows/**has nothing to miss.Cost measured rather than assumed: the last eight runs of this job took 28–42s,
median 35. Thirty-five seconds on an unrelated workflow edit is affordable waste; a
guard that silently does not run is an unobservable failure.
pathsis a performanceoptimisation and was being used as a correctness boundary.
(
.github/workflows/worker-tests.ymlis now subsumed by the wildcard and left in placeas documentation of why this job watches its own definition.)
Known residual, stated rather than papered over: this still does not cover a test
asserting on a file outside
worker/**and.github/workflows/**. That is bettersettled by a one-off repo-wide census than by a standing mechanism that misses silently.
"No test ran" and "every test passed" are the same green on a pull request page.
Verification