feat: CI gate: run the existing test suite and typecheck on every PR in

[andrei-hasna]

Objective

CI gate: run the existing test suite and typecheck on every PR in evals — the repo has tests but no CI workflow at all [loop-infinity-300pr]

REPO: https://github.com/hasna/evals

WHAT TO CHANGE
This repo has a real test suite and a test script, but there is no .github/workflows directory at all, so nothing runs on a pull request. Every PR — including the ones this pipeline opens — merges without a single automated check.

Add a single workflow at .github/workflows/ci.yml that runs on pull_request and on push to the default branch.

DONE LOOKS LIKE

• .github/workflows/ci.yml exists and runs, in one job: checkout, oven-sh/setup-bun, bun install --frozen-lockfile, bun run typecheck (only if the repo defines that script), bun test.
• Pin the actions to a major version tag (for example actions/checkout@v4, oven-sh/setup-bun@v2) rather than @master.
• The workflow must actually be green on this PR. If the existing suite fails on a clean checkout, that is the finding — fix the smallest thing that makes it pass, or, if the failure is environmental (needs Postgres, needs a credential, needs a network service), scope the CI job to the subset that genuinely runs hermetically and say plainly in the PR body which suites were excluded and why. Do NOT make it green with continue-on-error, || true, or by skipping tests.
• No production source changed beyond what is needed to make the gate honest.

VERIFY

• The workflow run on this PR is green in GitHub Actions.
• bun install --frozen-lockfile && bun test reproduces the same result locally.

EVIDENCE
Checked 2026-07-29 against GitHub HEAD: the .github/workflows contents were fetched through the GitHub API and grepped for test and typecheck invocations. Observed state: no .github/workflows directory exists at HEAD (GitHub API returned 404). If CI has since been added or extended upstream, close this task as already-done with a one-line comment rather than opening an empty PR.

PROCESS: work in a branch off the default branch, one focused change, conventional commit, open a PR. Do not bundle unrelated cleanups. If the repo has a CHANGELOG, add an entry.

Verification

• policy source: base ccb79e8 (immutable commit — agent-proof)
• ⚠️ GATE-INTEGRITY: agent touched verify-bearing config — REVIEW (.github/workflows/ci.yml)
• containment: env — allowlist env, non-login shell, run-scoped HOME (registry auth seeded for install)
• install: pass
• typecheck: pass
• build: FAIL
• test: pass
• doctor (ci): ok — 11 checks passed (1 advisory)

---

Run run_13f6adedc9d6 · backend codewith · task ce4acc7a-35cd-4a88-b338-b09cf0e385c3
🏭 Generated by @hasnaxyz/factory

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[andrei-hasna]

Closing in favour of #10, which carries the same CI workflow plus the fix that makes it pass.

Both PRs came from separate drain waves ~28 minutes apart and add an equivalent workflow (checkout, setup-bun, frozen install, typecheck, bun test, on pull_request + push to main). This one is RED; #10 is GREEN.

The cause is a real defect, not flake. src/adapters/mcp-adapter.test.ts used mock.module() to fake the MCP SDK. Bun's mock.module is process-global, so when the whole suite runs in one process the fake leaks into later integration tests. Run 30459881185 fails on 'calls named tool and returns text output' and 'concatenates multiple text content blocks' for that reason. That failure only appears when the full suite runs together — which is precisely what CI does and what nobody was doing locally.

#10 root-causes it: it deletes the module mocks and stands up a real stdio MCP fixture server (src/adapters/mcp-test-server.fixture.ts), so the adapter is exercised over a real transport. That is a strictly stronger test, not a weaker one.

Merging this PR would have installed a gate that fails on main from day one.