feat(ci): run the fixture release flow nightly - #28
Merged
spencer-tb merged 6 commits intoJul 14, 2026
Merged
Conversation
Move the fill flags shared by all fixture releases (`-n`, `--no-html`, `--durations=100`, `--log-level=DEBUG`) out of the inline fill step in `.github/actions/build-fixtures` into a new `just fill-release` recipe. Callers append the feature's fill params, fork range and output via pass-through args; for repeated flags, the last occurrence wins. The xdist worker count is plumbed via `PYTEST_XDIST_AUTO_NUM_WORKERS` (`xdist: auto` in `evm.yaml` for all EVM types, unchanged). Reimplement `fill-nightly` as a thin wrapper around `fill-release` so the nightly fill shares the release flags. Deliberate changes to the nightly invocation: it gains `--no-html` and `--log-level=DEBUG`, and it drops flags that defaults already provide: `--dist=loadgroup` and the trailing `tests` path (both defaults in `pytest-fill.ini`), and the explicit `-n` (now supplied by `fill-release`, same value).
Replace the standalone nightly-fill workflow with a schedule trigger on `release_fixtures.yaml`: scheduled runs fill the new `nightly` feature (`--until=Amsterdam --generate-all-formats`, slow tests included) through the exact release pipeline (matrix generation, `build-fixtures`, index merge and tarball) and skip only the `release` job, so no tag or draft release is created. The nightly is thereby a daily rehearsal of the release flow itself. Scheduled runs skip when there are no new commits since the last successful scheduled run (query filtered with `event=schedule` so manual releases never advance the nightly baseline) and list the new commits in the step summary. `fail-fast` is disabled on schedule so every fork range reports. `run-name` distinguishes nightly runs from manual releases in the run history. Artifact retention replaces the FIFO rotation of the standalone workflow: the combined `fixtures_nightly.tar.gz` is kept for 5 days (roughly the last five nightly builds at a daily cadence) and split intermediates for 1 day, via a new `split_retention_days` input on the `build-fixtures` action. Manual releases keep the repo defaults. Remove the `fill-nightly` recipe: the nightly fill now goes through `fill-release` inside `build-fixtures`, gaining the fixture index and release tarball packaging that the standalone workflow lacked.
The `benchmark` key appeared twice with identical values. The two readers of this file disagree on which duplicate wins (the awk snippet in `build-evm-base` matches the first, `yaml.safe_load` keeps the last), so a future edit to only one copy would apply inconsistently. Keep the first occurrence.
Move the `evm` override check from inline bash in the release workflow's setup step into `validate_inputs`, where the rest of the dispatch input validation already lives and is unit-tested. The check becomes a YAML key lookup instead of a line-anchored grep.
Move the inline bash of the "Check for new commits" step into `.github/scripts/check_new_commits.py`, following the pattern of the other release helper scripts (`uv run`-able, unit-tested). The last-success baseline semantics, the dispatch short-circuit, the no-baseline bootstrap and the empty-commit-list handling now have unit tests (via a fake `gh` on `PATH`); the workflow step shrinks to a single command.
danceratopz
marked this pull request as draft
July 13, 2026 10:39
With the nightly skipping itself when there are no new commits, a quiet stretch longer than the five-day artifact retention would leave no live nightly tarball. Re-run the fill once the last successful scheduled run is four days old: a release-shaped artifact then always exists within the retention window, and the release pipeline keeps getting exercised during quiet periods.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🗒️ Description
This makes the cron-trigger idea from the ethereum#3100 review thread concrete: instead of adding a parallel nightly workflow, the existing
release_fixtures.yamlflow runs on a nightly schedule and stops aftercombine, so no tag or draft release is created. The nightly thereby becomes a daily rehearsal of the actual release pipeline:generate_build_matrix.py,build-fixtures, the index merge and the release tarball all get exercised every night, and the resultingfixtures_nightly.tar.gzartifact is release-shaped (merged.meta/index.jsonincluded), so it can be consumed directly.Scheduled runs fill a new
nightlyfeature infeature.yaml(--until=Amsterdam --generate-all-formats, slow tests included) and skip themselves when there are no new commits since the last successful scheduled run: the baseline query filters onevent=scheduleso manual releases never advance it, and a failed nightly keeps re-running until it goes green.fail-fastis disabled on schedule so every fork range reports,run-namedistinguishes "Nightly Fill" runs from manual releases in the run history, and artifact retention (5 days for the combined tarball, 1 day for the per-range splits) replaces the FIFO rotation of the standalone workflow.Commits, in order:
refactor(ci): Extract a sharedfill-releasejust recipe holding the flags common to all fixture releases (previously inline inbuild-fixtures); callers append feature params, fork range and output via pass-through args (last flag wins). Fork splitting needs no special handling:--dist loadgroupandtestpaths = tests/are already defaults inpytest-fill.ini.feat(ci): The workflow merge described above; deletesnightly-fill.yamland thefill-nightlyrecipe.chore(ci): Remove a duplicatebenchmarkentry fromevm.yaml(the awk reader matches the first occurrence,yaml.safe_loadkeeps the last; identical today, a landmine tomorrow).refactor(ci): Move theevmoverride validation from inline bash intogenerate_build_matrix.py, where the rest of the input validation lives and is unit-tested.refactor(ci): Extract the "Check for new commits" inline bash into.github/scripts/check_new_commits.pywith unit tests (dispatch short-circuit, baseline bootstrap, new/no commits,ghfailure), following the other release helper scripts.Deliberate trades versus the standalone nightly workflow, flagging for visibility:
combinerequires a fully green build); the baseline logic refills everything the next night.tests@releases ethereum/execution-specs#3100 it is worth oneworkflow_dispatchvalidation run on a fork, ideally also confirming that the empty-stringretention-dayson dispatch releases falls back to the repo default as expected.Net effect on the workflow YAML: the 163-line
nightly-fill.yamlis deleted andrelease_fixtures.yamlgrows by ~45 lines of schedule-gated conditionals; the overall diff is dominated by the new script and its tests.🔗 Related Issues or PRs
Targets the head branch of ethereum#3100; see the review discussion there for context.
✅ Checklist
just static<type>(<area>):, where<type>and<area>come from an approrpriateC-<type>, respectivelyA-<area>, label. The title should match the a target squash commit message.