First Class Ported Static Tests Tracker

[spencer-tb]

Context

The 2,095 machine-ported tests in tests/ported_static/ started as flat stXxx/ directories with a blanket valid_from("Cancun") and, on Amsterdam, a 153-entry skip list. The end goal of this tracker is to make them first class native tests, fork-organized, minimally floored, fork-robust, and ultimately merged into the tests/<fork>/ structure properly.

Aim: the overarching aim is to make it as though it looks like they never existed, move all the ported static tests into tests/<forks> and fill them all as low as there floor allows for. Ideally from Frontier for most of them.

Phases

Note all PRs rely on each parents merging, i.e one after the other, they are built upon each other.

Removing The Ported Static Skip List

• Part 1 — enhance & un-skip Amsterdam ported static tests: fix(tests): enhance & un-skip Amsterdam ported static tests (Pt. 1) #3215
• Part 2 — un-skip the remaining 130, empty and remove the skip list + conftest machinery: fix(tests): enhance & un-skip Amsterdam ported static tests (Pt. 2) #3264

Restructuring

• Fork-folder reorg, pure moves into tests/ported_static/<fork>/<suite>/ (subject fork, decoupled from valid_from; layout enforced by conftest; fill-verified inert): refactor(tests): reorganize ported static tests into subject-fork folders spencer-tb/execution-specs#33
• Empirical valid_from floors, protected=fork.supports_protected_txs() for 1,716 tests + 1,690 markers lowered (1,128 now floor at Frontier; 119,610 full-range cases green; minimality audited — 392 floors provably equal their newest-used-opcode bound, 0 defects): fix(tests): lower ported static valid_from floors to empirically verified forks spencer-tb/execution-specs#34
• Gas-limit hygiene — 226 non-load-bearing hardcoded tx gas limits removed (individually gated, 91 load-bearing candidates rolled back): chore(tests): drop non-load-bearing hardcoded gas limits from ported static tests spencer-tb/execution-specs#35

These 3 PRs will be retargeted to forks/amsterdam sequentially as their bases merge.

Snapshot floor parity (required before archiving ethereum/tests + ethereum/legacytests, see #2556)

An audit of every ported test's floor against the per-fork post sections of the frozen snapshots (ethereum/tests HEAD tarball + legacytests/Constantinople + legacytests/Cancun) shows 1,403 tests at parity or better, and 670 tests where the snapshots covered lower forks than we currently do — archiving today would lose that coverage. Largest classes: Cancun-floored fragile tests whose originals reached Byzantium/Frontier/Istanbul, and TangerineWhistle/SpuriousDragon-floored tests whose originals expressed per-fork expectations (reaching parity there needs fork-conditional posts, not just lower markers).

• Close the 670 floor-parity regressions (per-file targets in the audit CSV)
• Re-run the parity audit clean before any archive decision

Remaining enhancement queue (floors are code-bound, not subject-bound; the enhance-ported-test recipes apply)

• 48 gas-snapshot tests → CodeGasMeasure rewrites
• 63 + 91 fragile tests (hand triage + load-bearing-limit files)
• Documented honest floors, no change planned: 98 TangerineWhistle (forward-all gas shape), 47 SpuriousDragon (EIP-161 fundamentals), 18 Berlin (EIP-2929 metadata), 19 London (EIP-3529 refund semantics)

Endgame

• Rename suites/files to native conventions (stEIP3855_push0 → eip3855_push0, meaningful names for the random/fuzzed blobs where feasible)
• Merge into the native tests/<fork>/ structure and dissolve tests/ported_static/ (reference-spec wiring, ported_from markers retained for ethereum/tests traceability)

[spencer-tb]

Related issues

Parent / endgame

• Meta: Archive ethereum/tests and ethereum/legacytests #2556 — archive ethereum/tests + ethereum/legacytests. This tracker closes the floor-parity half; Meta: Archive ethereum/tests and ethereum/legacytests #2556 phases 1–8 cover the unported filler corpus (GeneralStateTests ~507, Blockchain ~400, Transaction ~210, InvalidRLP ~20) and the release pipeline. Note its phase 1.4 references amsterdam_skip_list.txt, which fix(tests): enhance & un-skip Amsterdam ported static tests (Pt. 2) #3264 deletes — needs a refresh once the PR chain lands.

Directly on tests/ported_static/

• Static Test Fail Tracker for EIP-8037 #2601 — EIP-8037 static test fail tracker
• Update static tests for EIP-2780/EIP-8038 #3051 — update static tests for EIP-2780/EIP-8038
• Fixup magic numbers in tests/ported_static introduced in 8038 fixups #3016 — magic numbers introduced in the 8038 fixups
• Use dynamic address allocation in ported static tests #2517 — dynamic address allocation (~5,000 hardcoded addresses, pre_alloc_mutable everywhere)
• Add subset of ported static tests to json_loader CI #2516 — add a ported static subset to json_loader CI

Source tests/static/ fillers & porting pipeline

• Static tests that need update for Osaka #1576 — static tests disabled for Osaka (stMemoryStressTest → python)
• chore(tests): Port all stTimeConsuming to benchmark tests folder #1572 — port stTimeConsuming to the benchmark folder
• AI test: convert CreateOOGafterMaxCodesizeFiller #1552 — convert CreateOOGafterMaxCodesizeFiller to python
• Add tests for "reverted touch of empty account" #2174 — port TouchToEmptyAccountRevert* from legacytests (coverage/parity gap)
• Use historically-accurate per-fork block gas limits #2230 — historically-accurate per-fork block gas limits (bounds how low floors can honestly fill)
• Improve ported-from update detection in the coverage script #1581 — improve ported-from update detection in the coverage script
• codecov: use separate flags for main/internal/static tests #2382 — codecov flags for main/static/internal tests
• verify-traces: asymmetry between in-memory and disk-loaded traces for failed-during-execution txs (extend #2709) #2758 — verify-traces in-memory vs disk asymmetry (port-verification tooling)

Closed, historical context

• Open issues with ported static tests #2507 (original ported static issues), static_test(port): EIP-196 test cases coverage #2421 (EIP-196 port coverage), Review and remove (or port?) static transient storage test cases #1957 (transient storage static cases), Easy to convert test from ethereum/tests  #1505 (easy-to-convert list), feat(test-fill): --verify-traces flag for automatic trace comparison across test runs #2506 (--verify-traces)