Add periodic volume tests - #1647
Conversation
The volume integration tests (standalone volume-content access, no sandbox) are gated behind ENABLE_VOLUME_TESTS and never run in CI. A prod-side auth regression on that path (inc-2026-08-04) therefore went uncaught for ~24h. Add a scheduled workflow that runs the JS + Python suites every 6h with the volume tests enabled, and pings Slack on failure. Per-PR CI is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — straightforward, additive CI change with no production code touched.
What was reviewed: verified the new enable-volume-tests input correctly threads through to the ENABLE_VOLUME_TESTS env var that both packages/js-sdk/tests/setup.ts and packages/python-sdk/tests/conftest.py already gate on; confirmed per-PR CI is unchanged since the input defaults to false; checked the pinned rtCamp/action-slack-notify SHA and SLACK_WEBHOOK" secret usage matches the existing pattern in release.yml`.
Extended reasoning...
Overview
This PR adds a new scheduled workflow (periodic_tests.yml) that runs the existing JS and Python SDK test workflows every 6 hours with a new enable-volume-tests input, which exports ENABLE_VOLUME_TESTS=1 so the previously CI-skipped volume integration tests run periodically. It also adds a Slack failure notification. No application/SDK source code is touched — only GitHub Actions workflow YAML.
Security risks
None of consequence. The workflow only reads existing repo secrets (E2B_API_KEY, SLACK_WEBHOOK) that are already used elsewhere with the same trust boundary (reusable workflows called via workflow_call, secrets passed explicitly). The Slack notify action is pinned to a commit SHA matching the same version already used in release.yml. permissions: contents: read is set, which is appropriately minimal for this workflow.
Level of scrutiny
Low-to-moderate. This is a CI-only, additive change: it does not modify per-PR checks (the new input defaults to false in both reusable workflows, so existing behavior is unchanged), does not touch production or SDK code, and is easily reversible (just delete/disable the new workflow file). The main risk surface is "does this actually run and gate correctly," which I verified by tracing the env var into both SDKs' existing skip conditions.
Other factors
The new workflow follows established conventions in the repo (same reusable-workflow-call pattern, same pinned Slack action SHA and secret name as release.yml). No changeset is needed since no publishable packages changed (the changeset-bot comment confirms this is expected). No outstanding review comments exist on the PR.
1 participant
What
New scheduled workflow (
.github/workflows/periodic_tests.yml) that runs the JS + Python SDK suites every 6h with the volume integration tests enabled, and pings Slack on failure.Why
The volume integration tests (standalone volume-content access, no sandbox) are gated behind
ENABLE_VOLUME_TESTSand never run in CI. When that path broke in prod (inc-2026-08-04, invalid token audience), nothing caught it for ~24h. This adds a periodic backstop.How
enable-volume-testsinput to the reusablejs_sdk_tests.yml/python_sdk_tests.yml; when set, an extra step exportsENABLE_VOLUME_TESTS=1.periodic_tests.ymlcalls both with the flag on. Per-PR CI is unchanged (input defaults to false).Notes (easy to tune / drop)
19 */6 * * *(every 6h) — adjust as needed.monitoring-releasesvia the existingSLACK_WEBHOOK— change the channel if there's a better one.Ref: SDK-313