Skip to content

fix: make the new-chain test guard fail CI instead of exiting 0 - #118

Merged
kuzdogan merged 1 commit into
mainfrom
fix/chain-test-guard-exit-code
Jul 30, 2026
Merged

fix: make the new-chain test guard fail CI instead of exiting 0#118
kuzdogan merged 1 commit into
mainfrom
fix/chain-test-guard-exit-code

Conversation

@kuzdogan

Copy link
Copy Markdown
Member

Summary

The "there needs to be at least one passing test" guard for new chains lived in a suite-level after() hook in tests/chain-tests.test.ts. When it throws, Node reports the suite as failed but leaves the failure count at 0 — and Node >= 22.22 (CI runs node-version: "22") derives the process exit code from those counters. So the runner exited 0 and the test-new-chain job went green on a run where the guard had actually fired.

Seen on #117, run 30430598954:

✖ Test Supported Chains (466.288187ms)
ℹ tests 2 / pass 1 / fail 0
✖ failing tests:
  Error: There needs to be at least one passing test. Did you forget to add a test for your new chain with the id(s) 641230?

…and the step still succeeded.

Reproduced with a minimal test (one passing it, one throwing suite-level after):

Node exit code
20.12.2 1
22.5.1 1
22.22.0 0
23.5.0 0

A failing before() hook or a genuinely failing it() still exits 1, so this is specific to after. Neither downstream safety net catches it: the junit XML has no failing <testcase> (a hook failure isn't one), and mikepenz/action-junit-report runs with fail_on_failure: false.

Change

Move the guard out of the hook into a real test registered after the per-chain loop, so anyTestsPass is already settled when it runs. It now exits non-zero and appears as a failing testcase in the junit report / PR annotations.

🤖 Generated with Claude Code

The "at least one passing test" guard lived in a suite-level after() hook.
A throwing hook is reported as a failed suite but leaves the failure count
at 0, and Node >=22.22 (CI uses node-version: "22") derives its exit code
from the counters — so the runner exited 0 and the job went green on a run
where the guard had fired. Node 20.x and 22.5 still exit 1, which is why
this went unnoticed.

Moving the check into a real test registered after the per-chain loop makes
it exit non-zero again, and makes it show up as a failing testcase in the
junit report instead of disappearing from it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kuzdogan
kuzdogan merged commit 51c98be into main Jul 30, 2026
2 checks passed
@kuzdogan
kuzdogan deleted the fix/chain-test-guard-exit-code branch July 30, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants