chore: refresh EEST tests and genesis files by github-actions[bot] · Pull Request #121 · NethermindEth/gas-benchmarks

github-actions · 2026-02-20T09:11:11Z

Automated refresh of EEST tests and genesis files.

EEST tests: eest_tests/**
Genesis files: scripts/genesisfiles/**/zkevmgenesis.json (tracked with Git LFS)
Trigger: workflow_dispatch

* Add payload-dir based run name for stateful generation workflow * ci: add generation mode switch for stateful and warmup * perf: optimize generator HTTP usage and warmup patching * ci: pass nethermind genesis for perf-devnet-2 stateful generation * fix: force mainnet runtime config for perf-devnet-2 nethermind * ci: lock RPC funding address to fixed value * chore: remove noisy debug logs from warmup generation * ci: normalize generator chain arg to mainnet * ci: comment out perf-devnet-2 custom genesis args * chore: generate stateful tests for repricings_compute/perf-devnet-2 * chore: generate stateful tests for repricings_stateful/perf-devnet-2 * fix stub --------- Co-authored-by: GitHub Actions <actions@github.com>

* Initial plan * Revert genesisfiles to state before commit c8a40a4 Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> * Restore genesis files to commit c8a40a4 (correct version) Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com>

* Initial plan * Add ensure_table_schema to fill_postgres_db.py to auto-migrate missing columns like opcount Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> * Move generate_postgres_schema import to module level Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: kamilchodola <43241881+kamilchodola@users.noreply.github.com>

* Fix report generation and reduce logging noise - Fix invalid escape sequence '\>' in report_html.py DOCTYPE - Add missing --skipEmpty argument to report_html.py (matching report_tables.py) - Fix docker logs copy path in run.sh (results/docker_* -> logs/docker_*) - Add fallback to tar when zip is not available - Remove verbose stdout prints from report_html.py, report_tables.py, and utils.py * Add --filter option to run_and_post_metrics.sh * Add --warmup-count option to run_and_post_metrics.sh * Fix multitask-multi-parallel workflow - Remove push trigger (workflow_dispatch only) - Remove invalid -w flag (not supported by run.sh) - Remove obsolete warmup file input and INPUT_WARMUP env - Simplify INPUT_CLIENT (no push-event special case)

Transactions arriving via eth_sendRawTransaction were passed to testing_buildBlockV1 in network arrival order, which is not guaranteed to match the intended sequence. Now the buffer is sorted by the txIndex metadata field before every flush (both quiet-timer and group-switch).

Give heavier scenarios more time for all transactions to arrive before triggering block production via the testing endpoint.

The force-flush on eth_getTransactionByHash was bypassing the quiet period entirely, causing blocks to be built with partial transaction sets in heavy scenarios. Now only force-flush when the quiet period has already elapsed; otherwise let the normal timer handle it — the test runner will retry within its tx-wait-timeout window.

The force-flush on eth_getTransactionByHash is correct — it means the test runner believes it has finished sending. The actual problem is the quiet timer firing too early while transactions are still arriving in heavy scenarios. Bump QUIET_SECONDS from 0.5 to 2.0 (still under the 5s tx-wait-timeout) to give enough headroom.

Block production is now triggered only by: 1. eth_getTransactionByHash force-flush (test runner is done sending) 2. Group switch (new scenario's sendRawTransaction arrives) The quiet-timer monitor thread and QUIET_SECONDS constant are removed as they were a race-condition-prone heuristic that didn't add value over the two deterministic triggers above.

Tracks _LAST_SENDRAW_TS and logs gap_since_last_sendraw when eth_getTransactionByHash triggers a force flush. This gives visibility into what QUIET_SECONDS value would have been needed, and helps identify if the sender side needs optimization.

Adds an optional pytest -k filter expression that gets passed through to eest_stateful_generator.py via --parameter_filter.

Replace boolean _PENDING_OVERLAY_CONFIRMED with set-based tx hash tracking. After a testing block is produced, all tx hashes are stored and non-eth_getTransactionByHash requests are blocked until every tx is confirmed via a non-null response. This prevents chain state reads from leaking through before the overlay restore completes.

* Added space cleanup to workflow * Code cleanup * Use /mnt to download artifacts * Fixed directory creation * Fixed artifact location * Use sudo for removing temp artifacts * Use sudo for removing genesis_temp directory

Adds a config option to completely bypass the overlay restore mechanism (pause/resume, reorg between scenarios). When enabled, blocks are produced straight through without stalling. Defaults to true for now to test whether overlay restore is causing scenario failures.

Instead of restoring overlay on any non-getTxByHash request or new scenario sendRaw, only trigger the reorg when eth_getBalance is called for 0x86cf016fb873d50a7b8f31eb154c9234dd31b058. This provides a deterministic, single trigger point for overlay restore. Re-enables overlay restore (disable_overlay_restore defaults to false).

The summary job has no checkout step, so gh CLI cannot infer the repository from git context.

- Remove fork, chainspec_file, gas_bump_count, commit_results, warmup_snapshot_backend, append_to_draft inputs (all derived or hardcoded) - Derive full/partial from parameter_filter presence - Auto-append to existing draft instead of explicit toggle

…code_tracing

- Add base_release_tag input to generate-stateful-tests.yml - Download base archive to restore gas-bump.txt, funding.txt, and existing tests before partial runs - Merge base opcodes tracing with partial results for complete output - Wire base_release_tag from orchestrator to child workflow

…f-devnet-3)

- Use gzip -9 for max compression on release archives - Skip warmup dirs from artifact upload and archive in stateful_only mode - Split artifact upload into separate stateful/warmup steps - Remove obsolete repricings_compute/perf-devnet-2 and repricings_stateful/perf-devnet-2

…t integrity - Add repricings_compute/ and repricings_stateful/ to .gitignore - Remove tracked payload files (artifacts live in GitHub Releases) - Clean payload dirs before restoring base release to prevent stale runner files from leaking into archives - Fingerprint content after restore and after generation; reuse base archive byte-for-byte when content is unchanged - Fix generation summary failing when grep finds no pytest result line

… starting slot Previously only the regen path bumped _PREP_SLOT_COUNTER; partial regen replayed payloads without incrementing it, leaving mitm seeded at 0 while the chain head was past slot 5000. Resulted in "Invalid slot number (46) - slot number must exceed parent (5001)" on Amsterdam.

Previously the upload step ran on always(), so a failed/partial run would overwrite the good base asset that the setup step pre-copied with a truncated archive. Switch to success() so failures preserve the base release assets intact.

…unner Changes in generate-stateful-tests.yml: - Clean /tmp/base-restore + /tmp/base-tracing before download to avoid 'file already exists' errors on self-hosted runners that persist /tmp across runs. Add --clobber as a safety net. - Treat missing base archive as fatal (exit 1) instead of exit 0. Partial regeneration is invalid without the base, and silent continuation led to lost assets. Combined with success() gate on upload, a failure here preserves the pre-copied v1.1.0 assets intact. - Split archives exceeding 1.9 GiB into .NN.part files (split -b -d). Upload all parts instead of the single tar.gz. Reassembly on consumer side: cat *.part > archive.tar.gz && tar xzf archive.tar.gz. - Restore step detects single-file vs multi-part base, reassembles via cat for extraction, and records part filenames for byte-identical re-upload in the REUSE_BASE path. - Delete stale asset variants on the release before uploading new ones so transitions between single/multi formats don't leave orphans. Changes in release-workstream.yml: - Setup copy step uses --clobber and clears /tmp/base-assets first for the same runner-persistence reason.

…match grep -E returns 1 when no assets match, and with set -euo pipefail this killed the script before splitting could run. Capture the matches first with || true, then iterate. Surfaced as failure right after 'Archive size: 15G' line on first runs producing splittable archives.

…m repricing - Skip --address-stubs entirely for compute mode (stubs are only meaningful for stateful scenarios that reference deployed factories) - For stateful + eest_mode=repricing: always use stubs_repricing.json regardless of chain, so the same repricing scenarios run consistently - Make --stubs-file flag conditional on STUBS_FILE being non-empty

Drops perf-devnet-2 from: - generate-stateful-tests.yml: chain choice, case branch, gas-bump default, description - repricing-client.yml: ALL_TARGETS list and TEST_PATH regex - run.sh: genesis selection and setup_cmd network conditions - eest_stateful_generator.py: nethermind runtime config mapping - setup_node.py: resolve_nethermind_config_network mapping

…eters

* feat: add jochemnet chain support to stateful generator - Add jochemnet to workflow chain choices and chainspec options - Add snapshot path mapping (/mnt/sda/jochemnet/nethermind) - Use bloatnet stubs and 5000 gas bump count (same as perf-devnet-3) - Copy perf-devnet-3 chainspec as jochemnet starting point * fix: base jochemnet chainspec on mainnet, not perf-devnet-3 * feat: swap mainnet to jochemnet in release workstream matrices Also added stateful jochemnet entries to amsterdam workstreams which previously only had stateful for perf-devnet-3. * debug: add trace directory listing and cumulative file preview to opcode matching

Keep files as regular git objects to avoid LFS pointer/smudge issues that block branch checkouts.

Signed-off-by: GitHub Actions <actions@github.com>

+    uses: ./.github/workflows/repricing-client.yml
+    with:
+      client: besu
+      test: ${{ inputs.test || 'all' }}
+      runs: ${{ inputs.runs || '1' }}
+      images: ${{ inputs.images || '{"nethermind":"default","geth":"default","reth":"default","erigon":"default","besu":"default"}' }}
+      opcodes_warmup_count: ${{ inputs.opcodes_warmup_count }}
+      filter: ${{ inputs.filter || '' }}
+      fork: ${{ inputs.fork || 'osaka' }}
+    secrets: inherit
+
+  retrigger:


Add an explicit top-level permissions block in .github/workflows/repricing-besu.yml so all jobs in this workflow have least-privilege defaults.

Best fix here (without changing behavior): define minimal read access needed for normal workflow operation and explicitly grant only what’s required for dispatching workflows. Since retrigger dispatches another workflow, add actions: write. Also include contents: read as a safe baseline. Place this block at workflow root level (after on: inputs section and before concurrency:), so it applies to both repricing and retrigger unless overridden.

No imports, methods, or dependencies are needed (YAML-only change).

+    needs: repricing
+    runs-on: ubuntu-latest
+    steps:
+      - name: Retrigger workflow
+        uses: benc-uk/workflow-dispatch@v1
+        with:
+          workflow: repricing-besu.yml
+          ref: "${{ github.ref }}"
+          token: "${{ secrets.GH_TOKEN }}"
+          inputs: ${{ toJson(github.event.inputs) }}


Add explicit permissions to the workflow and constrain each job to least privilege:

Add a top-level permissions: {} to disable default GITHUB_TOKEN scopes globally.

For retrigger, add a job-level permissions block granting only what is needed to dispatch another workflow (actions: write).

For repricing (reusable workflow call), add a conservative read-only permission (contents: read) unless stronger permissions are known to be required by the called workflow. This preserves explicitness and least privilege in this file.

Change location: .github/workflows/repricing-besu.yml
Edits needed: YAML keys only; no imports, methods, or dependencies.

+    runs-on: ubuntu-latest
+    steps:
+      - name: Retrigger workflow
+        uses: benc-uk/workflow-dispatch@v1


+    runs-on: ubuntu-latest
+    outputs:
+      matrix: ${{ steps.set-matrix.outputs.matrix }}
+    steps:
+      - name: Set matrix dynamically
+        id: set-matrix
+        run: |
+          runs="${{ inputs.runs }}"
+          if ! [[ "$runs" =~ ^[0-9]+$ ]] || [ "$runs" -lt 1 ]; then
+            runs=1
+          fi
+          tests_input="${{ inputs.test }}"
+          targets=()
+
+          ALL_TARGETS=(
+            "repricings_compute/mainnet|mainnet"
+            "repricings_compute/perf-devnet-3|perf-devnet-3"
+            "repricings_stateful/mainnet|mainnet"
+            "repricings_stateful/perf-devnet-3|perf-devnet-3"
+          )
+
+          if [ -z "$tests_input" ] || [ "$tests_input" = "all" ]; then
+            targets=("${ALL_TARGETS[@]}")
+          else
+            IFS=',' read -ra requested_paths <<< "$tests_input"
+            for raw_path in "${requested_paths[@]}"; do
+              test_path="$(printf '%s' "$raw_path" | xargs)"
+              test_path="${test_path%/}"
+              [ -z "$test_path" ] && continue
+
+              echo "Processing test_path='$test_path'"
+
+              # Check if this is a parent folder (e.g. "repricings_stateful") — expand to all matching network targets
+              matched=false
+              for candidate in "${ALL_TARGETS[@]}"; do
+                candidate_path="${candidate%%|*}"
+                case "$candidate_path" in
+                  "${test_path}"/*)
+                    echo "  Folder expansion: matched '$candidate_path' under '$test_path'"
+                    targets+=("$candidate")
+                    matched=true
+                    ;;
+                esac
+              done
+
+              if [ "$matched" = false ]; then
+                network="$(basename "$test_path")"
+                echo "  No folder expansion; using as-is: path='$test_path' network='$network'"
+                targets+=("${test_path}|${network}")
+              fi
+            done
+          fi
+
+          echo "Final targets (${#targets[@]}):"
+          for t in "${targets[@]}"; do echo "  - $t"; done
+
+          if [ "${#targets[@]}" -eq 0 ]; then
+            echo "No test targets selected"
+            exit 1
+          fi
+
+          matrix_elements=()
+
+          for ((i=1; i<=runs; i++)); do
+            for target in "${targets[@]}"; do
+              test_path="${target%%|*}"
+              network="${target##*|}"
+              element="{'run': '$i', 'test_path': '$test_path', 'network': '$network'}"
+              matrix_elements+=("$element")
+            done
+          done
+
+          matrix="{\"include\": [$(IFS=,; echo "${matrix_elements[*]}")]}"
+
+          echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
+
+  build:


Add an explicit top-level permissions block in .github/workflows/repricing-client.yml so all jobs inherit minimal token access by default.

Best fix (without changing behavior): define read-only baseline permissions at workflow root:

contents: read (typical for checkout/repo reads)

packages: read (recommended minimal read baseline, especially if pulling packages/images)

This addresses CodeQL’s requirement while keeping existing job logic unchanged. If any later step truly needs write access, that should be granted narrowly at the specific job level.

+    needs: set-matrix
+    runs-on:
+      - self-hosted
+      - repricing
+      - ${{ inputs.client }}
+    timeout-minutes: 10080
+
+    strategy:
+      matrix: ${{ fromJson(needs.set-matrix.outputs.matrix) }}
+      fail-fast: false
+      max-parallel: 1
+
+    env:
+      PIP_BREAK_SYSTEM_PACKAGES: "1"
+      MIN_DOCKER_API_VERSION: "1.44"
+      NUGET_ENHANCED_MAX_NETWORK_TRY_COUNT: "12"
+      NUGET_ENHANCED_NETWORK_RETRY_DELAY_MILLISECONDS: "5000"
+      NUGET_ENHANCED_NETWORK_TIMEOUT_MILLISECONDS: "300000"
+      NUGET_PACKAGES: "/data/nuget/packages"
+
+    steps:
+      - name: Ensure base tools are installed
+        run: |
+          set -euo pipefail
+          missing=()
+          docker_missing=false
+          compose_missing=false
+
+          if ! command -v git-lfs >/dev/null 2>&1; then
+            missing+=("git-lfs")
+          fi
+          if ! command -v zip >/dev/null 2>&1; then
+            missing+=("zip")
+          fi
+          if ! command -v curl >/dev/null 2>&1; then
+            missing+=("curl")
+          fi
+          if ! command -v make >/dev/null 2>&1; then
+            missing+=("make")
+          fi
+
+          if ! command -v docker >/dev/null 2>&1; then
+            docker_missing=true
+            compose_missing=true
+          else
+            if ! docker compose version >/dev/null 2>&1 && ! command -v docker-compose >/dev/null 2>&1; then
+              compose_missing=true
+            fi
+          fi
+
+          packages=("${missing[@]}")
+          if [ "$docker_missing" = true ]; then
+            packages+=("docker.io")
+          fi
+          if [ "$compose_missing" = true ]; then
+            if command -v apt-cache >/dev/null 2>&1 && apt-cache show docker-compose-plugin >/dev/null 2>&1; then
+              packages+=("docker-compose-plugin")
+            elif command -v apt-cache >/dev/null 2>&1 && apt-cache show docker-compose-v2 >/dev/null 2>&1; then
+              packages+=("docker-compose-v2")
+            fi
+          fi
+
+          if [ ${#packages[@]} -gt 0 ]; then
+            if command -v apt-get >/dev/null 2>&1; then
+              sudo apt-get update
+              sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends "${packages[@]}"
+            else
+              echo "Missing packages: ${packages[*]}. Install them on the runner or provide apt-get."
+              exit 1
+            fi
+          fi
+
+          if command -v systemctl >/dev/null 2>&1; then
+            sudo systemctl start docker >/dev/null 2>&1 || true
+          elif command -v service >/dev/null 2>&1; then
+            sudo service docker start >/dev/null 2>&1 || true
+          fi
+
+          if ! docker compose version >/dev/null 2>&1; then
+            echo "docker compose plugin is still unavailable; installing Compose v2 binary plugin..."
+            arch="$(uname -m)"
+            case "$arch" in
+              x86_64|amd64) arch="x86_64" ;;
+              aarch64|arm64) arch="aarch64" ;;
+              *)
+                echo "Unsupported architecture for Compose v2 bootstrap: $arch"
+                exit 1
+                ;;
+            esac
+            os="$(uname -s | tr '[:upper:]' '[:lower:]')"
+            compose_version="${COMPOSE_VERSION:-v2.29.7}"
+            plugin_dir="$HOME/.docker/cli-plugins"
+            mkdir -p "$plugin_dir"
+            curl -fsSL "https://github.com/docker/compose/releases/download/${compose_version}/docker-compose-${os}-${arch}" -o "$plugin_dir/docker-compose"
+            chmod +x "$plugin_dir/docker-compose"
+            mkdir -p "$HOME/.local/bin"
+            ln -sf "$plugin_dir/docker-compose" "$HOME/.local/bin/docker-compose"
+            echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+          fi
+
+          mkdir -p "$NUGET_PACKAGES"
+
+          echo "PATH=$PATH"
+          command -v docker || true
+          which -a docker || true
+          docker version --format 'CLIENT={{.Client.Version}} API={{.Client.APIVersion}} SERVER_MIN={{.Server.MinAPIVersion}}' || true
+          docker compose version || true
+          if ! docker compose version >/dev/null 2>&1; then
+            if command -v docker-compose >/dev/null 2>&1 && docker-compose version --short 2>/dev/null | grep -q '^v\?2\.'; then
+              echo "Using docker-compose v2 standalone fallback: $(command -v docker-compose)"
+            else
+              echo "ERROR: Docker Compose v2 is not available (plugin missing and no docker-compose v2 fallback)."
+              exit 1
+            fi
+          fi
+
+      - name: Ensure Docker client API compatibility
+        run: |
+          set -euo pipefail
+
+          version_ge() {
+            local lhs="$1"
+            local rhs="$2"
+            [ "$lhs" = "$rhs" ] && return 0
+            [ "$(printf '%s\n%s\n' "$lhs" "$rhs" | sort -V | tail -n 1)" = "$lhs" ]
+          }
+
+          required="${MIN_DOCKER_API_VERSION:-1.44}"
+
+          docker_bin="$(command -v docker || true)"
+          while IFS= read -r candidate; do
+            if [ -n "$candidate" ] && [[ "$candidate" != *"/.native-bin/"* ]] && [ -x "$candidate" ]; then
+              docker_bin="$candidate"
+              break
+            fi
+          done < <(which -a docker 2>/dev/null | awk '!seen[$0]++')
+
+          if [ -z "${docker_bin}" ] || [ ! -x "${docker_bin}" ]; then
+            echo "docker CLI is missing on runner."
+            exit 1
+          fi
+
+          echo "Using docker binary: ${docker_bin}"
+          echo "DOCKER_BIN=${docker_bin}" >> "$GITHUB_ENV"
+
+          if [ -n "${DOCKER_API_VERSION:-}" ]; then
+            echo "Clearing DOCKER_API_VERSION=${DOCKER_API_VERSION} to allow client/daemon negotiation."
+            echo "DOCKER_API_VERSION=" >> "$GITHUB_ENV"
+            unset DOCKER_API_VERSION
+          fi
+
+          client_api="$("${docker_bin}" version --format '{{.Client.APIVersion}}' 2>/dev/null || true)"
+          daemon_min="$("${docker_bin}" version --format '{{.Server.MinAPIVersion}}' 2>/dev/null || true)"
+
+          if [ -n "$daemon_min" ] && version_ge "$daemon_min" "$required"; then
+            required="$daemon_min"
+          fi
+
+          if [ -z "$client_api" ]; then
+            echo "Unable to read Docker client API version. Diagnostic output:"
+            "${docker_bin}" version || true
+            exit 1
+          fi
+
+          echo "Docker client API=${client_api}, required>=${required}"
+          if ! version_ge "$client_api" "$required"; then
+            echo "ERROR: Docker client API ${client_api} is too old for this daemon (requires >= ${required})."
+            echo "ERROR: Upgrade docker CLI on the self-hosted runner."
+            exit 1
+          fi
+
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          lfs: true
+
+      - name: Ensure Python is available (self-hosted)
+        run: |
+          if ! command -v python3 >/dev/null 2>&1; then
+            if command -v apt-get >/dev/null 2>&1; then
+              sudo apt-get update
+              sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3 python3-pip python3-venv
+            else
+              echo "python3 not found and apt-get is unavailable on this runner."
+              exit 1
+            fi
+          fi
+
+          if ! python3 -m pip --version >/dev/null 2>&1; then
+            if command -v apt-get >/dev/null 2>&1; then
+              sudo apt-get update
+              sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3-pip
+            else
+              echo "python3 is present but pip is unavailable and apt-get is unavailable."
+              exit 1
+            fi
+          fi
+
+          if ! command -v pip >/dev/null 2>&1; then
+            mkdir -p "$HOME/.local/bin"
+            printf '%s\n' '#!/usr/bin/env bash' 'exec python3 -m pip "$@"' > "$HOME/.local/bin/pip"
+            chmod +x "$HOME/.local/bin/pip"
+            echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+          fi
+
+          python3 --version
+          pip --version
+      - uses: actions/setup-dotnet@v4
+        with:
+          dotnet-version: '10.0.x'
+
+      - name: Ensure .NET SDK is available (self-hosted)
+        run: |
+          set -euo pipefail
+          if ! command -v dotnet >/dev/null 2>&1; then
+            echo "dotnet not found after setup-dotnet; installing fallback SDK 10.0"
+            curl -fsSL https://dot.net/v1/dotnet-install.sh -o /tmp/dotnet-install.sh
+            bash /tmp/dotnet-install.sh --channel 10.0 --quality ga --install-dir "$HOME/.dotnet"
+            echo "$HOME/.dotnet" >> "$GITHUB_PATH"
+            export PATH="$HOME/.dotnet:$PATH"
+          fi
+
+          if ! dotnet --list-sdks | grep -q '^10\.'; then
+            echo "No .NET 10 SDK found; installing fallback SDK 10.0"
+            curl -fsSL https://dot.net/v1/dotnet-install.sh -o /tmp/dotnet-install.sh
+            bash /tmp/dotnet-install.sh --channel 10.0 --quality ga --install-dir "$HOME/.dotnet"
+            echo "$HOME/.dotnet" >> "$GITHUB_PATH"
+            export PATH="$HOME/.dotnet:$PATH"
+          fi
+
+          dotnet --info
+
+      - name: Install Python dependencies
+        run: |
+          python3 -m pip install --user --ignore-installed -r requirements.txt
+
+      - name: Make run.sh executable
+        run: chmod +x ./run.sh
+
+      - name: Run benchmarks for client=${{ inputs.client }}
+        run: |
+          TEST_PATH="${{ matrix.test_path }}"
+          NETWORK="${{ matrix.network }}"
+          CLIENT="${{ inputs.client }}"
+          WARMUP_PATH="${TEST_PATH}_warmup"
+
+          WARMUP_COUNT="${{ inputs.opcodes_warmup_count }}"
+          if [ -z "$WARMUP_COUNT" ]; then
+            case "$TEST_PATH" in
+              repricings_stateful/*) WARMUP_COUNT="0" ;;
+              *) WARMUP_COUNT="1" ;;
+            esac
+            echo "Auto-resolved opcodes_warmup_count=$WARMUP_COUNT for $TEST_PATH"
+          fi
+
+          if [ ! -d "$TEST_PATH" ]; then
+            echo "Test path does not exist: $TEST_PATH"
+            exit 1
+          fi
+
+          WARMUP_ARGS=()
+          if [ -d "$WARMUP_PATH" ]; then
+            WARMUP_ARGS=(-W "$WARMUP_PATH")
+          else
+            echo "Warmup path not found: $WARMUP_PATH; continuing without explicit warmup path"
+          fi
+
+          EFFECTIVE_FILTER="${{ inputs.filter }}"
+          echo "Using filter: '${EFFECTIVE_FILTER}' for test path: $TEST_PATH"
+
+          FORK_NAME="${{ inputs.fork || 'osaka' }}"
+
+          # Per-client snapshot backend selection
+          case "$CLIENT" in
+            erigon|reth)
+              ./run.sh \
+                -t "$TEST_PATH" \
+                -n "$NETWORK" \
+                -S "zfs" \
+                -B "/data-zfs/eth/<<CLIENT>>/<<NETWORK>>" \
+                -c "$CLIENT" \
+                -r 1 \
+                -i '${{ inputs.images }}' \
+                -o "$WARMUP_COUNT" \
+                -f "$EFFECTIVE_FILTER" \
+                -K "$FORK_NAME" \
+                "${WARMUP_ARGS[@]}"
+              ;;
+            *)
+              ./run.sh \
+                -t "$TEST_PATH" \
+                -n "$NETWORK" \
+                -B "/data/snapshots/<<CLIENT>>/<<NETWORK>>" \
+                -O "/data/overlay-workspace" \
+                -c "$CLIENT" \
+                -r 1 \
+                -i '${{ inputs.images }}' \
+                -o "$WARMUP_COUNT" \
+                -f "$EFFECTIVE_FILTER" \
+                -K "$FORK_NAME" \
+                "${WARMUP_ARGS[@]}"
+              ;;
+          esac
+
+      - name: Dump latest Docker logs for ${{ inputs.client }}
+        if: always()
+        run: |
+          # Find the newest docker log file for this client
+          LATEST=$(ls -1t logs/docker_${{ inputs.client }}_*.log | head -n1 || true)
+          LATEST_SYNC=$(ls -1t logs/docker_sync_${{ inputs.client }}_*.log | head -n1 || true)
+          if [[ -n "$LATEST" ]]; then
+            echo "=== Dumping $LATEST ==="
+            cat "$LATEST"
+          else
+            echo "No docker_*.log files found for client=${{ inputs.client }}"
+          fi
+
+          if [[ -n "$LATEST_SYNC" ]]; then
+            echo "=== Dumping $LATEST_SYNC ==="
+            cat "$LATEST_SYNC"
+          else
+            echo "No docker_sync_*.log files found for client=${{ inputs.client }}"
+          fi
+
+      - name: Generate per-client report CSVs
+        run: |
+          TEST_PATH="${{ matrix.test_path }}"
+          CLIENT="${{ inputs.client }}"
+          IMAGES='${{ inputs.images }}'
+          RUNS="${{ inputs.runs }}"
+
+          python3 report_html.py \
+            --resultsPath "results" \
+            --clients "$CLIENT" \
+            --testsPath "$TEST_PATH" \
+            --runs "$RUNS" \
+            --images "$IMAGES"
+
+      - name: Publish results to DB
+        env:
+          PERFNET_DB_HOST: ${{ secrets.PERFNET_0_DB_HOST }}
+          PERFNET_DB_PORT: ${{ secrets.PERFNET_0_DB_PORT || '5432' }}
+          PERFNET_DB_USER: ${{ secrets.PERFNET_0_DB_USER }}
+          PERFNET_DB_PASSWORD: ${{ secrets.PERFNET_0_DB_PASSWORD }}
+        run: |
+          set -euo pipefail
+          TEST_PATH="${{ matrix.test_path }}"
+
+          if [[ "$TEST_PATH" =~ ^repricings_(compute|stateful)/(mainnet|perf-devnet-3)$ ]]; then
+            SUITE="${BASH_REMATCH[1]}"
+            NETWORK="${BASH_REMATCH[2]}"
+            DB_NETWORK="${NETWORK//-/_}"
+            DB_NAME="monitoring"
+          else
+            echo "Skipping DB upload for non-repricing path: $TEST_PATH"
+            exit 0
+          fi
+
+          if [ ! -d "reports" ]; then
+            echo "No reports directory found, skipping DB upload"
+            exit 0
+          fi
+
+          FORK_NAME="${{ inputs.fork || 'osaka' }}"
+          FORK_SUFFIX=$(echo "$FORK_NAME" | tr '[:upper:]' '[:lower:]' | sed -E 's#[/.-]+#_#g; s#[^a-z0-9_]#_#g; s#_+#_#g; s#^_##; s#_$##')
+          TABLE_NAME="${SUITE}_${DB_NETWORK}_${FORK_SUFFIX}"
+
+          BRANCH_NAME="${GITHUB_REF_NAME}"
+          if [[ "$BRANCH_NAME" != "main" ]]; then
+            BRANCH_SUFFIX=$(echo "$BRANCH_NAME" | tr '[:upper:]' '[:lower:]' | sed -E 's#[/.-]+#_#g; s#[^a-z0-9_]#_#g; s#_+#_#g; s#^_##; s#_$##')
+            TABLE_NAME="${TABLE_NAME}_${BRANCH_SUFFIX}"
+          fi
+
+          echo "Publishing results to table: $TABLE_NAME"
+
+          python3 fill_postgres_db.py \
+            --db-host "$PERFNET_DB_HOST" \
+            --db-port "$PERFNET_DB_PORT" \
+            --db-user "$PERFNET_DB_USER" \
+            --db-name "$DB_NAME" \
+            --table-name "$TABLE_NAME" \
+            --db-password "$PERFNET_DB_PASSWORD" \
+            --log-level DEBUG \
+            --reports-dir "reports"
+
+      - name: Zip the results folder
+        run: |
+          CLEANED_RUN=$(echo "${{ matrix.run }}" | tr -d '\n')
+          CLEANED_CLIENT=$(echo "${{ inputs.client }}" | tr -d '\n')
+          CLEANED_TEST_PATH=$(echo "${{ matrix.test_path }}" | tr -d '\n' | tr '/ ' '__' | tr -cd '[:alnum:]_.-')
+          echo "CLEANED_RUN=$CLEANED_RUN" >> $GITHUB_ENV
+          echo "CLEANED_CLIENT=$CLEANED_CLIENT" >> $GITHUB_ENV
+          echo "CLEANED_TEST_PATH=$CLEANED_TEST_PATH" >> $GITHUB_ENV
+          zip -r results-${CLEANED_RUN}-${CLEANED_CLIENT}-${CLEANED_TEST_PATH}.zip results
+
+      - name: Upload artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: results-${{ env.CLEANED_RUN }}-${{ env.CLEANED_CLIENT }}-${{ env.CLEANED_TEST_PATH }}
+          path: results-${{ env.CLEANED_RUN }}-${{ env.CLEANED_CLIENT }}-${{ env.CLEANED_TEST_PATH }}.zip
+
+      - name: Docker full prune
+        if: always()
+        run: |
+          docker ps -q | xargs -r docker stop || true
+          docker ps -aq | xargs -r docker rm -f || true
+          docker system prune -af --volumes || true


+    needs: repricing
+    runs-on: ubuntu-latest
+    steps:
+      - name: Retrigger workflow
+        uses: benc-uk/workflow-dispatch@v1
+        with:
+          workflow: repricing-nethermind.yml
+          ref: "${{ github.ref }}"
+          token: "${{ secrets.GH_TOKEN }}"
+          inputs: ${{ toJson(github.event.inputs) }}


Add an explicit permissions block at the workflow root in .github/workflows/repricing-nethermind.yml, just after concurrency and before jobs.
This is the least invasive fix and applies to all jobs in the workflow, including the reusable-workflow caller job and the retrigger job, without changing behavior.

Best single fix here:

Set minimal baseline permission:

contents: read

Keep existing logic unchanged.

No imports, methods, or dependencies are needed (YAML-only change).

+    runs-on: ubuntu-latest
+    steps:
+      - name: Retrigger workflow
+        uses: benc-uk/workflow-dispatch@v1


+    uses: ./.github/workflows/repricing-client.yml
+    with:
+      client: reth
+      test: ${{ inputs.test || 'all' }}
+      runs: ${{ inputs.runs || '1' }}
+      images: ${{ inputs.images || '{"nethermind":"default","geth":"default","reth":"default","erigon":"default","besu":"default"}' }}
+      opcodes_warmup_count: ${{ inputs.opcodes_warmup_count }}
+      filter: ${{ inputs.filter || '' }}
+      fork: ${{ inputs.fork || 'osaka' }}
+    secrets: inherit
+
+  retrigger:


Add explicit permissions to this workflow.

Best fix (minimal and safe without changing functionality):

Add a root-level permissions block with contents: read so all jobs get least-privilege baseline.

Add a job-level permissions block on retrigger with actions: write (required to dispatch workflows) and contents: read.

Keep everything else unchanged.

File/region to change:

.github/workflows/repricing-reth.yml

Insert top-level permissions after concurrency (before jobs).

Insert permissions under jobs.retrigger.

No imports/dependencies/methods are needed (YAML-only change).

+    needs: repricing
+    runs-on: ubuntu-latest
+    steps:
+      - name: Retrigger workflow
+        uses: benc-uk/workflow-dispatch@v1
+        with:
+          workflow: repricing-reth.yml
+          ref: "${{ github.ref }}"
+          token: "${{ secrets.GH_TOKEN }}"
+          inputs: ${{ toJson(github.event.inputs) }}


Add explicit permissions blocks to the workflow:

Set a restrictive default at the workflow root (applies to all jobs unless overridden), e.g. contents: read.

For the retrigger job, add a job-level override granting only what is needed to dispatch workflows: actions: write plus contents: read.

This preserves existing functionality while documenting and constraining token scope.
Only edit .github/workflows/repricing-reth.yml in the shown snippet:

Insert root-level permissions below concurrency (before jobs).

Insert permissions under retrigger job definition.

No imports, methods, or extra definitions are needed (YAML workflow file).

+    runs-on: ubuntu-latest
+    steps:
+      - name: Retrigger workflow
+        uses: benc-uk/workflow-dispatch@v1


kamilchodola and others added 30 commits February 10, 2026 10:15

Allow CI generation to continue on partial execute-remote failures

ac40c27

chore: generate stateful tests for repricings_stateful/mainnet

e6ffd71

Update warmup tests generation script to use Nethermind

3f59437

Merge branch 'main' of https://github.com/NethermindEth/gas-benchmarks

290f407

Cleanup old tests

e5011aa

fix: update volume path for geth datadir (#119)

5671485

chore: generate stateful tests for repricings_compute/mainnet

a6e0594

chore: generate stateful tests for test_account_query

8ea440a

fix: increase QUIET_SECONDS from 0.1 to 0.5

21b0f57

Give heavier scenarios more time for all transactions to arrive before triggering block production via the testing endpoint.

fix: increase default mainnet gas bump count from 2000 to 10000

9020b91

feat: expose parameter_filter input in generate-stateful-tests workflow

12959dc

Adds an optional pytest -k filter expression that gets passed through to eest_stateful_generator.py via --parameter_filter.

chore: generate stateful tests for test_account_query

2093963

chore: generate stateful tests for test_account_query

14746eb

Improved benchmark action to handle large fixtures (#122)

e26d939

* Added space cleanup to workflow * Code cleanup * Use /mnt to download artifacts * Fixed directory creation * Fixed artifact location * Use sudo for removing temp artifacts * Use sudo for removing genesis_temp directory

chore: generate stateful tests for test_account_query

bb6ac31

chore: generate stateful tests for test_account_query

ffe1f91

chore: generate stateful tests for test_account_query

eea2332

chore: generate stateful tests for test_account_query

c1569cc

kamilchodola and others added 28 commits April 2, 2026 10:57

flatDB enabling

574aa70

fix: add --repo flag to gh commands in summary job

91696a0

The summary job has no checkout step, so gh CLI cannot infer the repository from git context.

chore: update default Nethermind image to testing_build_block_with_op…

ccc50cf

…code_tracing

fix: install gh CLI on stateful-generator runner

a2cf514

chore: replace full log dump with pytest short test summary

8e8e95a

remove log arg

d32f8b3

fix: show pytest result line even when all tests pass

468f692

fix: always print pytest result line, show failures only when present

7a55c2b

chore: split amsterdam genesis into per-chain variants (mainnet / per…

6b0db65

…f-devnet-3)

fix: update stubs path from bloatnet/ to stubs/

09a17c8

fix: rename EEST to EELS in release notes template, add repo to param…

0211818

…eters

chore: remove stateful mainnet from amsterdam release workstreams

36c365e

chore: remove generator-amsterdam genesis files from LFS tracking

5e27c54

Keep files as regular git objects to avoid LFS pointer/smudge issues that block branch checkouts.

Use jochemnet stubs for jochemnet generation

17b7aa2

Always use jochemnet stubs for jochemnet generation

e4ca536

Resolve jochemnet generation paths from chainspec

a569479

chore: refresh EEST tests and genesis files

589522c

Signed-off-by: GitHub Actions <actions@github.com>

github-actions Bot force-pushed the chore/refresh-eest-tests branch from b621cf2 to 589522c Compare May 4, 2026 06:08

github-advanced-security AI found potential problems May 4, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: refresh EEST tests and genesis files - #121

chore: refresh EEST tests and genesis files#121
github-actions[bot] wants to merge 127 commits into
fix/gh-runner-spacefrom
chore/refresh-eest-tests

github-actions Bot commented Feb 20, 2026

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Check warning

Check warning

Copilot Autofix

Check warning

Check warning

Copilot Autofix

Check warning

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Check warning

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

@@ -21,6 +21,11 @@
                   fork:
                     description: 'Fork name (osaka, amsterdam)'
                     default: 'osaka'
+            permissions:
+              contents: read
+              actions: write
             concurrency:
               group: repricing-besu-${{ github.ref }}
               cancel-in-progress: true

@@ -1,5 +1,9 @@
             name: Repricing Client (Reusable)
+            permissions:
+              contents: read
+              packages: read
             on:
               workflow_call:
                 inputs:

Uh oh!

Conversation

github-actions Bot commented Feb 20, 2026

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Check warning

Uh oh!

Check warning

Copilot Autofix

Check warning

Check warning

Copilot Autofix

Check warning

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Check warning

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants