ci: multi-transport SQL E2E (subprocess + http + unix)#1
Merged
Conversation
The haybarn sqllogictest suite (test/sql/*.test) previously ran only over the
subprocess/stdio transport. The vgi extension picks its transport from the
worker LOCATION string, so the same suite exercises a different transport just
by changing what VGI_ICAL_WORKER resolves to.
- ci/run-integration.sh: parameterize by TRANSPORT (subprocess|http|unix).
* http: boot `java -jar <jar> --http --host 127.0.0.1 --port <p>`, wait for
GET /health to return 200, set VGI_ICAL_WORKER=http://127.0.0.1:<p>.
* unix: boot `java -jar <jar> --unix <sock>`, wait for the socket to appear,
set VGI_ICAL_WORKER=unix://<sock>.
For http/unix the worker is started by the script (not DuckDB), so it is
launched with cwd = the stage dir where the committed .ics fixtures live, so
the relative `test/sql/data/*.ics` paths resolve identically to subprocess.
trap-kill the worker on exit. Keep the warm `INSTALL vgi FROM community` step.
- test.yml: turn the integration job into a transport matrix
(subprocess, http, unix), named "SQL E2E (<transport>)". Unit/build/lint and
the boot smoke test are untouched.
- ci/README.md: document the transport matrix, the worker --http/--host/--port
(+ /health, PORT:<n>) and --unix flags from the vgi Java SDK, fixture/cwd
handling, and the haybarn "skip on error_message matching 'HTTP'" caveat.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… suite)
The vgi extension's HTTP worker transport uses DuckDB's HTTP client, which lives
in the httpfs extension; without it, ATTACH over an http:// LOCATION raises
"VGI HTTP transport requires the httpfs extension". On the standalone
haybarn-unittest runner, auto-load/auto-install is off, so httpfs must be
installed and loaded explicitly.
Before this fix the http matrix leg was GREEN but DECEPTIVE: every test was
silently skipped. Some haybarn-unittest builds carry a built-in
`skip on error_message matching 'HTTP'` rule, and the httpfs-missing
BinderException contains the substring "HTTP", so the runner skipped the whole
suite instead of failing it ("All tests were skipped (total skipped 3)").
Fix:
- test/sql/*.test: add `require httpfs` (preprocess-require.awk already rewrites
it to `INSTALL httpfs FROM core; LOAD httpfs;`). httpfs is a no-op for the
subprocess/unix transports, keeping the suite transport-agnostic.
- run-integration.sh: also warm `INSTALL httpfs FROM core` alongside vgi.
- ci/README.md: document the httpfs requirement and the silent-skip trap.
Verified locally: subprocess / http / unix all "All tests passed (49 assertions
in 3 test cases)" — http now genuinely runs the suite rather than skipping it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds multi-transport coverage to the haybarn SQL E2E suite (
test/sql/*.test).Previously it ran only over subprocess/stdio. The
vgiextension picks itstransport from the worker
LOCATION, so the same suite now runs once pertransport via a CI matrix.
ci/run-integration.shis parameterized byTRANSPORT(subprocess|http|unix).java -jar <jar> --http --host 127.0.0.1 --port 18110, waitsfor
GET /health200, setsVGI_ICAL_WORKER=http://127.0.0.1:18110.java -jar <jar> --unix <sock>, waits for the socket, setsVGI_ICAL_WORKER=unix://<sock>.cwd = the stage dir so relative
test/sql/data/*.icsfixtures resolve.INSTALL vgi FROM communitystep is kept.test.yml: integration job becomesstrategy.matrix.transport: [subprocess, http, unix],named
SQL E2E (<transport>). Unit/build/lint + boot smoke test untouched.ci/README.md: documents the matrix, the vgi Java SDK--http/--host/--port(+
/health,PORT:<n>) and--unixflags, and fixture/cwd handling.Local validation: subprocess GREEN (43 assertions), unix GREEN (43 assertions).
The http leg is validated on the Linux CI runner (a local
haybarn-unittestmasks it via a built-in
skip on error_message matching 'HTTP'rule; the workeritself serves
/healthand the RPC/initendpoint correctly).🤖 Generated with Claude Code