Skip to content

Commit 435705f

Browse files
rustyconoverclaude
andcommitted
ci: stream the native sqllogictest report instead of a rolled-up count
run-integration.sh ran each .test in its own unittest invocation and printed only a per-file PASS/SKIP/FAIL tally, hiding that the suite executes ~8700 individual query assertions. Run the whole suite in a single unittest invocation (as `make test_launcher` does) so the CI log streams the runner's native output: a `[i/N] (..%): test/...` line per file and the final `All tests passed (.. N assertions in M test cases)` summary. Out-of-scope tests are already dropped at staging, so the glob never matches them; a failed assertion exits non-zero and fails the job. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 3b02c0d commit 435705f

2 files changed

Lines changed: 18 additions & 24 deletions

File tree

ci/README.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,13 @@ extension from the Haybarn community channel:
2626
5. **Run**[`run-integration.sh`](run-integration.sh) stages the preprocessed
2727
tree, points the four `VGI_*_WORKER` env vars at the worker (via `launch:`,
2828
which amortises JVM cold-start across the run) and the three catalog
29-
[`wrappers/`](wrappers), warms the extension cache once, then runs each
30-
`.test` and tallies pass / skip / fail.
29+
[`wrappers/`](wrappers), warms the extension cache once, then runs the whole
30+
suite in a **single `unittest` invocation** (as `make test_launcher` does).
31+
The CI log streams the runner's native report — a `[i/N] (..%): test/...`
32+
line per file and the final `All tests passed (.. N assertions in M test
33+
cases)` summary (thousands of assertions across the ~185 files) — so you can
34+
see the tests actually ran. Any failed assertion exits non-zero and fails
35+
the job.
3136

3237
## Transport lanes
3338

ci/run-integration.sh

Lines changed: 11 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -116,25 +116,14 @@ EOF
116116
"$HAYBARN_UNITTEST" "test/_warm.test" >/dev/null 2>&1 || echo "::warning::extension warm step did not fully succeed"
117117
rm -f "$STAGE/test/_warm.test"
118118

119-
echo "Running suite ..."
120-
pass=0; skip=0; fail=0; failed=()
121-
while IFS= read -r t; do
122-
rel="${t#"$STAGE"/}"
123-
if out=$("$HAYBARN_UNITTEST" "$rel" 2>&1); then :; fi
124-
if grep -q "All tests passed" <<<"$out"; then
125-
pass=$((pass + 1))
126-
elif grep -qE "All tests were skipped|No tests ran" <<<"$out"; then
127-
skip=$((skip + 1))
128-
else
129-
fail=$((fail + 1)); failed+=("$rel")
130-
echo "----- FAIL: $rel -----"
131-
printf '%s\n' "$out" | tail -40
132-
fi
133-
done < <(find "$STAGE/test" -name '*.test' | sort)
134-
135-
echo "=================================================="
136-
echo "PASS=$pass SKIP=$skip FAIL=$fail"
137-
if [ "$fail" -gt 0 ]; then
138-
printf ' failed: %s\n' "${failed[@]}"
139-
exit 1
140-
fi
119+
# Run the whole suite in ONE unittest invocation (as `make test_launcher`
120+
# does), streaming the runner's native sqllogictest report: a `[i/N] (..%):
121+
# test/...` progress line per file and the final
122+
# `All tests passed (.. N assertions in M test cases)` summary (a failure
123+
# prints the offending query + a `M test cases | K failed` summary). This keeps
124+
# the CI log showing that the tests actually ran — and how many assertions —
125+
# rather than a rolled-up count. Out-of-scope tests were already dropped at
126+
# staging, so the glob never matches them; any failed assertion exits non-zero
127+
# and fails the job (via `set -e`).
128+
echo "Running suite (single invocation — native sqllogictest report) ..."
129+
"$HAYBARN_UNITTEST" "test/sql/integration/*"

0 commit comments

Comments
 (0)