ci: add macOS to test + e2e matrix

ualtinok · ualtinok · commit f2ff19df02ee · 2026-05-06T14:24:17.000+02:00
Two new layers of macOS CI coverage. macOS GH Actions runners are free for
public repos and run in parallel with Linux/Windows, so wall-clock release
time is unchanged.

Phase A — macOS unit + plugin tests (Rust + bun)
  tests.yml:    new `rust-macos` job
  release.yml:  new `test-macos` job (gates the e2e suite)

Until now, the build-darwin-* jobs only ran `cargo build --release`, never
`cargo test`, so macOS-specific code paths (FSEvents, /var canonicalization,
broken-symlink fallback, bash_background SIGTERM behavior, Apple Silicon
codegen) had zero CI coverage. Tag-time release builds were the only place
the macOS binary ever got produced, with no test signal at all.

Phase B — macOS native e2e
  _e2e-suite.yml:        new `e2e-macos` job
  tests/macos-e2e/run.sh: new host-setup script (mirrors Dockerfile.linux-x64)
  tests/docker/test-e2e.sh: refactored to be platform-aware via
                            AFT_E2E_PLATFORM=linux|macos. The shared scenario
                            logic now reads the broken-ONNX path
                            (libonnxruntime.so vs libonnxruntime.dylib),
                            mock-server location, and platform display from
                            env so we don't fork the harness.

Single source of truth: macOS uses the same scenario script as Linux Docker,
just on a real macOS-arm64 host with native install of OpenCode + Bun +
aimock. Catches FSEvents watcher behavior, /var vs /private/var
canonicalization, broken-symlink-chain fallback, and .dylib loading paths
for ONNX (probe order /opt/homebrew/lib then /usr/local/lib differs from
Linux /usr/local/lib).

Verified locally:
  - yaml.safe_load on all 3 workflow files: clean
  - actionlint .github/workflows/*.yml: clean
  - bash -n on tests/{docker,macos-e2e}/test-e2e.sh + run.sh: clean
  - cargo fmt --check: clean
  - cargo clippy --workspace --all-targets -- -D warnings: clean
  - bun test (opencode-plugin): 679 pass / 2 skip / 0 fail
  - bun test (pi-plugin):       380 pass / 0 fail
diff --git a/.github/workflows/_e2e-suite.yml b/.github/workflows/_e2e-suite.yml
@@ -1,8 +1,9 @@
 name: E2E Suite
 
 # Reusable workflow that runs the full E2E test matrix:
-#   - Linux Docker e2e — OpenCode + plugin + binary stack against aimock
-#   - Windows native e2e — same stack on real Windows runner (no containers)
+#   - Linux Docker e2e   — OpenCode + plugin + binary stack against aimock
+#   - macOS native e2e   — same stack on a real macOS-arm64 runner
+#   - Windows native e2e — same stack on a real Windows runner (no containers)
 #
 # Called from:
 #   - tests.yml (PR + main push) — catches regressions at PR time
@@ -13,9 +14,10 @@ name: E2E Suite
 # IMPORTANT: any change to bridge transport, bash spawning, ONNX install,
 # locking, or platform-conditional code paths SHOULD touch the matching
 # integration test or e2e scenario here. The Linux harness has caught real
-# regressions before; the Windows e2e is here to extend that coverage to
-# issue-#26-class Windows-specific bugs (bash timeouts, lock recovery, path
-# separators).
+# regressions before; the Windows e2e covers issue-#26-class Windows-specific
+# bugs (bash timeouts, lock recovery, path separators); the macOS e2e covers
+# FSEvents watcher behavior, /var vs /private/var symlink canonicalization,
+# .dylib loading paths for ONNX, and Apple Silicon native codegen for aft.
 
 on:
   workflow_call:
@@ -114,3 +116,45 @@ jobs:
         env:
           AFT_BINARY_PATH: ${{ github.workspace }}\target\x86_64-pc-windows-msvc\release\aft.exe
           AFT_PLUGIN_DIST: ${{ github.workspace }}\packages\opencode-plugin\dist
+
+  # ---------------------------------------------------------------------------
+  # macOS native e2e — exercises the same OpenCode + plugin + binary stack on
+  # a real macOS-arm64 runner. No containers (GH Actions macOS runners can't
+  # run Docker reliably). Uses the same shared bash harness as Linux Docker
+  # via tests/macos-e2e/run.sh, which sets up host deps natively and then
+  # invokes tests/docker/test-e2e.sh with AFT_E2E_PLATFORM=macos.
+  #
+  # Catches: FSEvents watcher behavior, /var vs /private/var canonicalization,
+  # broken-symlink-chain fallback, .dylib loading, and Apple Silicon codegen.
+  # ---------------------------------------------------------------------------
+  e2e-macos:
+    name: E2E (macOS native)
+    runs-on: macos-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: dtolnay/rust-toolchain@stable
+
+      - uses: Swatinem/rust-cache@v2
+
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - name: Build AFT binary (macOS native)
+        run: cargo build --release -p agent-file-tools
+
+      - name: Install workspace deps + build aft-bridge
+        run: |
+          bun install --frozen-lockfile
+          bun run --cwd packages/aft-bridge build
+
+      - name: Build OpenCode plugin dist
+        run: bun run --cwd packages/opencode-plugin build
+
+      - name: Run macOS E2E suite
+        run: bash tests/macos-e2e/run.sh
+        env:
+          AFT_BINARY_PATH: ${{ github.workspace }}/target/release/aft
+          AFT_PLUGIN_DIST: ${{ github.workspace }}/packages/opencode-plugin/dist
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -65,13 +65,55 @@ jobs:
       - name: Build all JS packages (gate publish)
         run: bun run --filter '*' build
 
-  # Run the full E2E matrix (Linux Docker + Windows native) at release time.
-  # Reuses the same workflow tests.yml runs at PR time — single source of
-  # truth, so PR-time and release-time e2e can never drift. Build/publish
-  # jobs below `needs:` this job, so an e2e regression blocks the release.
+  # macOS unit-test gate — mirrors the Linux `test` job's cargo + plugin test
+  # coverage on a real macOS runner. The build-darwin-* jobs below only run
+  # `cargo build --release`, not `cargo test`, so this is the only place where
+  # macOS-specific code paths are actually exercised at release time:
+  #   - FSEvents watcher behavior (different coalescing latency from inotify)
+  #   - /var vs /private/var symlink canonicalization
+  #   - Broken-symlink-chain fallback in context.rs
+  #   - bash_background SIGTERM behavior on Apple Silicon
+  # Lint and `bun run --filter '*' build` only run in the Linux `test` job —
+  # they are platform-independent so duplicating them on macOS adds no signal.
+  test-macos:
+    name: Test (macOS)
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: dtolnay/rust-toolchain@stable
+
+      - uses: Swatinem/rust-cache@v2
+
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - name: Rust tests
+        run: cargo test --workspace
+
+      - name: Install JS dependencies
+        run: bun install
+
+      - name: Build aft-bridge (consumers need its dist/ for typecheck)
+        run: bun run --cwd packages/aft-bridge build
+
+      - name: TypeScript typecheck
+        run: bun run typecheck
+
+      - name: Plugin tests
+        run: bun run test
+        env:
+          AFT_CACHE_DIR: ${{ runner.temp }}/aft-cache
+
+  # Run the full E2E matrix (Linux Docker + Windows native + macOS native) at
+  # release time. Reuses the same workflow tests.yml runs at PR time — single
+  # source of truth, so PR-time and release-time e2e can never drift.
+  # Build/publish jobs below `needs:` this job, so an e2e regression blocks
+  # the release.
   e2e:
     name: E2E
-    needs: [test]
+    needs: [test, test-macos]
     uses: ./.github/workflows/_e2e-suite.yml
 
   publish-crates:
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -91,6 +91,50 @@ jobs:
           # ~/.cache to avoid cross-test pollution.
           AFT_CACHE_DIR: ${{ runner.temp }}/aft-cache
 
+  # ---------------------------------------------------------------------------
+  # Rust + plugin tests on macOS
+  # Catches macOS-specific code paths: FSEvents watcher behavior (different
+  # coalescing latency from inotify), /var vs /private/var symlink
+  # canonicalization, broken-symlink-chain fallback, bash_background SIGTERM
+  # behavior, and Apple Silicon-specific Rust compilation. The build-darwin-*
+  # jobs in release.yml only run `cargo build` — this is the only place we
+  # actually execute tests on macOS in CI.
+  # ---------------------------------------------------------------------------
+  rust-macos:
+    name: Unit tests (macOS)
+    runs-on: macos-latest
+    timeout-minutes: 25
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: dtolnay/rust-toolchain@stable
+
+      - uses: Swatinem/rust-cache@v2
+
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - name: Install workspace deps
+        run: bun install --frozen-lockfile
+
+      - name: Build aft-bridge dist (workspace consumers depend on it)
+        run: bun run --cwd packages/aft-bridge build
+
+      - name: Cargo build (debug — needed by plugin e2e tests)
+        run: cargo build -p agent-file-tools
+
+      - name: Cargo test
+        run: cargo test --workspace
+
+      - name: Bun typecheck
+        run: bun run typecheck
+
+      - name: Bun test (all packages)
+        run: bun run test
+        env:
+          AFT_CACHE_DIR: ${{ runner.temp }}/aft-cache
+
   # ---------------------------------------------------------------------------
   # Rust integration tests on Windows
   # Catches platform-conditional code paths (#[cfg(target_os = "windows")])
diff --git a/tests/docker/test-e2e.sh b/tests/docker/test-e2e.sh
@@ -1,6 +1,10 @@
 #!/usr/bin/env bash
 # ------------------------------------------------------------------
-# E2E test: AFT plugin running inside OpenCode on Linux x64
+# E2E test: AFT plugin running inside OpenCode.
+#
+# Used by both:
+#   - tests/docker/Dockerfile.linux-x64 (Linux Docker E2E in CI)
+#   - tests/macos-e2e/run.sh             (macOS native E2E in CI)
 #
 # Uses aimock for deterministic OpenAI-compatible mock LLM.
 # Simulates a realistic multi-turn agent session that exercises:
@@ -11,6 +15,14 @@
 #
 # Each scenario runs a full OpenCode session with 8 tool call turns,
 # giving background threads enough time to build indices.
+#
+# Platform-specific behavior is controlled by the AFT_E2E_PLATFORM env
+# var (defaults to "linux"):
+#   AFT_E2E_PLATFORM=linux  →  fake libonnxruntime.so in /usr/local/lib
+#   AFT_E2E_PLATFORM=macos  →  fake libonnxruntime.dylib in /tmp
+# Each platform's runner script is responsible for installing OpenCode,
+# Bun, aimock, writing configs, and placing the AFT binary + plugin
+# before invoking this script.
 # ------------------------------------------------------------------
 
 set -euo pipefail
@@ -22,7 +34,28 @@ NC='\033[0m'
 
 PASS=0
 FAIL=0
-PLUGIN_LOG="/tmp/aft-plugin.log"
+PLUGIN_LOG="${AFT_E2E_PLUGIN_LOG:-/tmp/aft-plugin.log}"
+PLATFORM="${AFT_E2E_PLATFORM:-linux}"
+
+# Platform-specific paths for the broken-ONNX scenario.
+case "$PLATFORM" in
+    linux)
+        FAKE_ORT_PATH="/usr/local/lib/libonnxruntime.so"
+        PLATFORM_DISPLAY="Linux x64 (Debian)"
+        ;;
+    macos)
+        # /tmp on macOS is a symlink to /private/tmp; we use a path the
+        # plugin can find via DYLD_LIBRARY_PATH or the AFT-managed cache.
+        # We don't drop into /usr/local/lib because SIP-protected paths
+        # need root, and macOS GH Actions runners don't grant it.
+        FAKE_ORT_PATH="${RUNNER_TEMP:-/tmp}/libonnxruntime.dylib"
+        PLATFORM_DISPLAY="macOS native"
+        ;;
+    *)
+        echo "Unknown AFT_E2E_PLATFORM: $PLATFORM (expected linux|macos)" >&2
+        exit 2
+        ;;
+esac
 
 check() {
     local label="$1"
@@ -49,8 +82,13 @@ warn_check() {
     fi
 }
 
+# AFT_E2E_MOCK_SERVER points to the mock-server.js entry. The Docker setup
+# places it at /test/mock-server.js; the macOS runner places it relative to
+# the repo checkout. Both wire this through env so we don't hardcode paths.
+MOCK_SERVER="${AFT_E2E_MOCK_SERVER:-/test/mock-server.js}"
+
 start_aimock() {
-    node /test/mock-server.js > /tmp/aimock.log 2>&1 &
+    node "$MOCK_SERVER" > /tmp/aimock.log 2>&1 &
     AIMOCK_PID=$!
     for i in $(seq 1 15); do
         if curl -s http://127.0.0.1:4010/v1/models > /dev/null 2>&1; then
@@ -92,7 +130,7 @@ run_opencode_session() {
 }
 
 echo "════════════════════════════════════════"
-echo "  AFT E2E Test — Linux x64 (Debian)"
+echo "  AFT E2E Test — $PLATFORM_DISPLAY"
 echo "════════════════════════════════════════"
 echo ""
 
@@ -181,45 +219,60 @@ cat /tmp/aimock.log 2>/dev/null | sed 's/^/    /' || echo "    (empty)"
 stop_aimock
 
 # ══════════════════════════════════════════════════════════════════
-# Scenario 2: Broken libonnxruntime.so (reproduces issue #4)
-# A fake .so in /usr/local/lib that the plugin detects and sets
-# as ORT_DYLIB_PATH — the binary should NOT crash when loading it.
+# Scenario 2: Broken ONNX Runtime library (reproduces issue #4)
+# A fake shared library that the plugin detects and sets as
+# ORT_DYLIB_PATH — the binary should NOT crash when loading it.
+# Library suffix and path differ per platform; FAKE_ORT_PATH was
+# resolved at the top of the script.
 # ══════════════════════════════════════════════════════════════════
 
 echo ""
-echo "── Scenario 2: Broken libonnxruntime.so (issue #4) ──"
+echo "── Scenario 2: Broken ONNX Runtime library (issue #4) ──"
 echo ""
 
-# Install fake broken .so
-echo "not a real shared library" > /usr/local/lib/libonnxruntime.so
-chmod 755 /usr/local/lib/libonnxruntime.so
-echo "  Installed fake libonnxruntime.so in /usr/local/lib"
+# Install fake broken library at $FAKE_ORT_PATH (linux: /usr/local/lib/libonnxruntime.so,
+# macos: $RUNNER_TEMP/libonnxruntime.dylib).
+echo "not a real shared library" > "$FAKE_ORT_PATH"
+chmod 755 "$FAKE_ORT_PATH"
+echo "  Installed fake $(basename "$FAKE_ORT_PATH") at $FAKE_ORT_PATH"
 
 rm -f "$PLUGIN_LOG"
 
 start_aimock
 check "aimock started (s2)" "curl -s http://127.0.0.1:4010/v1/models > /dev/null 2>&1"
 
-echo "Running session with broken .so..."
+echo "Running session with broken library..."
 RESULT_FILE="/tmp/result-scenario2.txt"
-run_opencode_session \
-    "Read the file src/main.py and then grep for all function definitions." \
-    "$RESULT_FILE" \
+# On macOS, the AFT plugin probes a fixed list of system paths
+# (/usr/local/lib, /opt/homebrew/lib) for libonnxruntime.dylib. Since
+# we cannot write into /usr/local/lib on a vanilla GH Actions runner
+# without sudo, we point ORT_DYLIB_PATH directly at our fake instead.
+# Linux scenario keeps the implicit /usr/local/lib detection path.
+if [ "$PLATFORM" = "macos" ]; then
+    ORT_DYLIB_PATH="$FAKE_ORT_PATH" \
+    run_opencode_session \
+        "Read the file src/main.py and then grep for all function definitions." \
+        "$RESULT_FILE"
+else
+    run_opencode_session \
+        "Read the file src/main.py and then grep for all function definitions." \
+        "$RESULT_FILE"
+fi
 
 EXIT_CODE=$?
 
-check "session completed (broken .so)" "[ $EXIT_CODE -eq 0 ] || [ $EXIT_CODE -eq 124 ]"
-warn_check "no crash (broken .so)" "! grep -qi 'Binary crashed\|SIGABRT\|panicked' '$RESULT_FILE' 2>/dev/null"
-check "no plugin crash (broken .so)" "! grep -qi 'SIGABRT\|thread.*panicked' '$PLUGIN_LOG' 2>/dev/null"
+check "session completed (broken lib)" "[ $EXIT_CODE -eq 0 ] || [ $EXIT_CODE -eq 124 ]"
+warn_check "no crash (broken lib)" "! grep -qi 'Binary crashed\|SIGABRT\|panicked' '$RESULT_FILE' 2>/dev/null"
+check "no plugin crash (broken lib)" "! grep -qi 'SIGABRT\|thread.*panicked' '$PLUGIN_LOG' 2>/dev/null"
 
-# Verify the plugin detected the system .so
+# Verify the plugin detected the system library
 warn_check "system ORT detected" "grep -q 'ONNX Runtime found at system path\|ORT_DYLIB_PATH' '$PLUGIN_LOG' 2>/dev/null"
 
 echo ""
 echo "  Plugin log (last 30 lines):"
 tail -30 "$PLUGIN_LOG" 2>/dev/null | sed 's/^/    /' || echo "    (empty)"
 
-rm -f /usr/local/lib/libonnxruntime.so
+rm -f "$FAKE_ORT_PATH"
 stop_aimock
 
 # ══════════════════════════════════════════════════════════════════
diff --git a/tests/macos-e2e/README.md b/tests/macos-e2e/README.md
@@ -0,0 +1,46 @@
+# macOS native E2E
+
+End-to-end test for the AFT plugin running inside OpenCode on a real macOS
+host (no containers, no QEMU). Runs as `e2e-macos` in the reusable
+`.github/workflows/_e2e-suite.yml` workflow, alongside the Linux Docker and
+Windows native E2E jobs.
+
+## What this catches
+
+- FSEvents watcher behavior (different coalescing latency from Linux inotify;
+  the v0.19.5 release fixed two flaky watcher tests with exactly this shape).
+- `/var` vs `/private/var` symlink canonicalization in `context.rs`.
+- Broken-symlink-chain fallback in `context.rs`.
+- macOS dylib loading (`.dylib` extension, `/usr/local/lib` and
+  `/opt/homebrew/lib` probe paths) for ONNX Runtime — distinct from the Linux
+  `.so` / `/usr/local/lib` path.
+- Apple Silicon native ARM64 codegen for the `aft` binary itself.
+
+## How it works
+
+`run.sh` performs the host setup that `Dockerfile.linux-x64` does on Linux
+(install OpenCode, Bun, aimock, write configs, place locally-built AFT binary
++ plugin dist), then invokes the shared `tests/docker/test-e2e.sh` harness
+with `AFT_E2E_PLATFORM=macos`. The shared harness reads platform-specific
+paths (e.g. `libonnxruntime.dylib` vs `libonnxruntime.so`) from env so we
+don't fork the scenario logic.
+
+## Local invocation
+
+The harness expects `AFT_BINARY_PATH` and `AFT_PLUGIN_DIST` to point at the
+locally-built AFT binary and OpenCode plugin dist:
+
+```bash
+cargo build --release -p agent-file-tools
+bun run --cwd packages/aft-bridge build
+bun run --cwd packages/opencode-plugin build
+
+AFT_BINARY_PATH="$PWD/target/release/aft" \
+AFT_PLUGIN_DIST="$PWD/packages/opencode-plugin/dist" \
+bash tests/macos-e2e/run.sh
+```
+
+The script writes its OpenCode config under `$RUNNER_TEMP/aft-e2e-xdg/opencode/`
+(or `$TMPDIR/aft-e2e-xdg/opencode/` outside CI) and the test project under
+`$RUNNER_TEMP/aft-e2e-project/`, leaving your real `~/.config/opencode/`
+untouched.
diff --git a/tests/macos-e2e/run.sh b/tests/macos-e2e/run.sh
