Skip to content

test(ci): bind the Windows shard assertion to an executable command (#1185) #1301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lidge-jun merged 2 commits into from
Aug 9, 2026
+18 −1
Merged

test(ci): bind the Windows shard assertion to an executable command (#1185) #1301

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1e8e88c
test(ci): bind the Windows shard assertion to an executable command (…
lidge-jun Aug 8, 2026
e239b96
test(ci): require the Windows test step to be unconditional
lidge-jun Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion tests/ci-workflows.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,15 @@ function count(text: string, fragment: string): number {
return text.split(fragment).length - 1;
}

/** Match an executable shell line, not a fragment that could appear in echo or a comment. */
function hasExactShellCommand(run: string | undefined, expected: string): boolean {
return (run ?? "")
.split(/\r?\n/)
.map(line => line.trim())
.filter(line => line.length > 0 && !line.startsWith("#"))
.includes(expected);
Comment on lines +47 to +48

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject PowerShell block-commented commands

When the Windows Test step wraps the command in a PowerShell block comment such as <#\nbun test ...\n#>, this filter retains the command line and hasExactShellCommand returns true, even though the step executes no tests; the step has no explicit shell, so Windows uses PowerShell. This leaves the same silent no-test regression the new assertion is intended to prevent. Compare the trimmed run value directly with the expected single-line command, or otherwise account for PowerShell block comments rather than filtering only lines beginning with #.

Useful? React with 👍 / 👎.

}

Comment on lines +42 to +50

@coderabbitai coderabbitai Bot Aug 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Reject exact command text that is not executable.

hasExactShellCommand checks only trimmed text lines. It returns true for a command inside a PowerShell block comment, an uncalled function, or a disabled shell branch. For example, this Windows step contains the exact line but does not run the test:

run: |
  <#
  bun test --isolate tests --shard=${{ matrix.shard }}/4
  #>

The step has no if, so lines 182-184 accept it. The Windows workflow currently uses a single-line run, so compare the trimmed run value directly in this Windows-specific check. If multiline scripts must remain supported, add shell-aware parsing and regression coverage for non-executable blocks.

Proposed fix 
-    const windowsTestSteps = winSteps.filter(step => hasExactShellCommand(step.run, windowsTestCommand));
+    const windowsTestSteps = winSteps.filter(
+      step => step.run?.trim() === windowsTestCommand,
+    );

Also applies to: 176-184

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/ci-workflows.test.ts` around lines 42 - 50, Update the Windows-specific
checks around hasExactShellCommand so they compare the trimmed run value
directly against the expected single-line command, rather than accepting
matching lines inside multiline or non-executable blocks. Preserve the existing
command validation for the Windows workflow and adjust the affected checks near
the Windows step.

function expectSecureLinuxKeyringBootstrap(workflow: string): void {
const smokeStep = workflow
.split("- name: OS keyring create/read/delete smoke")[1]
Expand Down Expand Up @@ -164,7 +173,15 @@ describe("GitHub Actions hardening", () => {
// the runner's disk and the suite passes against a tree that no longer
// exists in git.
const winSteps = (ci.jobs?.["platform-windows"] as { steps?: { if?: string; run?: string }[] })?.steps ?? [];
expect(winSteps.some(step => step.run?.includes(`--shard=\${{ matrix.shard }}/${windowsShards.length}`))).toBe(true);
const windowsTestCommand = `bun test --isolate tests --shard=\${{ matrix.shard }}/${windowsShards.length}`;
expect(hasExactShellCommand(`echo ${windowsTestCommand}`, windowsTestCommand)).toBe(false);
// Binding the assertion to an executable line is only half the guarantee: a
// step carrying the exact command still runs nothing under `if: false`, and
// the suite would stay green against a Windows leg that never tests. Require
// the matching step to be unconditional.
const windowsTestSteps = winSteps.filter(step => hasExactShellCommand(step.run, windowsTestCommand));
expect(windowsTestSteps.length).toBeGreaterThan(0);
expect(windowsTestSteps.every(step => step.if === undefined)).toBe(true);
expect(winSteps.some(step => step.if === "runner.environment == 'self-hosted'"
&& step.run?.includes("git clean -xffd"))).toBe(true);

Expand Down
Loading