test(ci): split platform/fixture tests into an informational CODEHUB_PLATFORM lane#192
Merged
Merged
Conversation
…PLATFORM lane
Audit P1 follow-up to the package collapse: isolate the vendor/runtime/
fixture tests so environment flakiness (the Windows chmod/path/file-url
failures that dominated the collapse PR) can't mask or block logic-
regression signal on the release-gating `test` job.
Mechanism — env-var `{ skip }` gate (generalizes the existing
CODEHUB_INTEGRATION idiom in sagemaker-embedder.integration.test.ts).
Chosen over a file-rename/glob split and over `--test-skip-pattern`
because BOTH of those break on the Node 20 floor: `--test-skip-pattern`
is `added: v22.1.0`, and `node --test` glob expansion is post-20 (the
repo's double-quoted globs would match nothing on 20). The env gate uses
only core-stable test-runner APIs and behaves identically on 20/22/24 and
on Windows cmd + POSIX shells. Zero package.json / invocation changes —
both lanes run the identical `pnpm -r test`; only CODEHUB_PLATFORM differs.
Tagged (verified per-case, not file-level — interleaved logic cases stay
in the required lane):
- mcp/run-smoke.test.ts — all 25 ToolResult-shape smokes (whole file).
- cli/setup-embeddings.test.ts — the weight-download suite (whole file).
- ingestion/wasm-grammar-resolution.test.ts — the 15 vendored-.wasm
disk-presence cases (the php->php_only/cobol routing logic stays).
- cli/scip-downloader.test.ts — the 6 download/extract+chmod mechanic
cases (pin-mismatch, UnsupportedPlatform, placeholder-hash, SHA-mismatch,
dotnet-SDK, install ordering stay as logic).
- cli/embedder-downloader.test.ts — the 5 download-mechanic cases
(Sha256MismatchError contract + retry/backoff policy stay as logic).
CI: new `test-platform` job (3 OS × Node 22) runs the same suite with
CODEHUB_PLATFORM=1 + `continue-on-error: true`, NOT a required check — a
Windows fixture quirk turns it amber without blocking. The required
`test` job runs the identical command with the gate off and carries the
logic-regression signal alone. Also fixed a pre-existing useTemplate lint
nit in init.test.ts touched by the same sweep.
Verified: logic lane (default) full suite exit 0 with platform cases
skipped (cli 11 skipped, mcp 25, ingestion's 15-case wasm suite); platform
lane (CODEHUB_PLATFORM=1) runs full counts (cli 303, mcp 218, ingestion
577), exit 0; lint + typecheck + banned-strings 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Audit P1 follow-up to the package collapse (#189): isolate the vendor/runtime/fixture tests into a separate, informational CI lane so environment flakiness can't mask or block the logic-regression signal on the release-gating
testjob.This is the structural answer to the Windows chmod/path/file-url churn that dominated #189 — those tests are legitimate (they caught real bugs), but they fail for environment reasons unrelated to OpenCodeHub's logic, so they shouldn't sit on the required check.
Mechanism: env-var
{ skip }gateA per-suite/per-case
{ skip: platformSkip }keyed onCODEHUB_PLATFORM, generalizing the existingCODEHUB_INTEGRATIONidiom insagemaker-embedder.integration.test.ts.Chosen over the two alternatives because both break on the Node 20 floor:
--test-skip-patternisadded: v22.1.0— absent on Node 20.node --testglob expansion is post-20, so a*.platform.test.tsfile-glob split would match nothing on Node 20 (vacuous pass).The env gate uses only core-stable test-runner APIs, behaves identically on Node 20/22/24 and on Windows cmd + POSIX shells, and needs zero package.json / invocation changes — both lanes run the identical
pnpm -r test; onlyCODEHUB_PLATFORMdiffers.What's tagged (verified per-case, not file-level)
Interleaved logic cases stay in the required lane:
mcp/run-smoke.test.tscli/setup-embeddings.test.tsingestion/wasm-grammar-resolution.test.ts.wasmdisk-presence casescli/scip-downloader.test.tscli/embedder-downloader.test.tsCI
New
test-platformjob (3 OS × Node 22) runs the same suite withCODEHUB_PLATFORM=1+continue-on-error: true, not a required check. A Windows fixture quirk turns it amber without blocking a logic-correct PR or the release-please PR. The requiredtestjob runs the identical command with the gate off and carries the logic-regression signal alone.Verified
CODEHUB_PLATFORM=1): full counts run (cli 303, mcp 218, ingestion 577), exit 0.Also fixes a pre-existing
useTemplatelint nit ininit.test.tstouched by the same sweep.🤖 Generated with Claude Code