Skip to content

add glamsterdam-devnet-7-full subdir: full suite, no repricing marker, 200M/300M, benchmarks/amsterdam - #21

Open
qu0b wants to merge 1 commit into
masterfrom
qu0b/full-suite-context
Open

add glamsterdam-devnet-7-full subdir: full suite, no repricing marker, 200M/300M, benchmarks/amsterdam#21
qu0b wants to merge 1 commit into
masterfrom
qu0b/full-suite-context

Conversation

@qu0b

@qu0b qu0b commented Jul 30, 2026

Copy link
Copy Markdown
Member

Adds a v1/glamsterdam-devnet-7-full subdir to the repricing context, per the ask in #benchmarking: fill the complete EEST benchmark suite (drop -m repricing) at 200M and 300M gas, from benchmarks/amsterdam (forks/amsterdam now merges devnet-8 spec PRs; benchmarks/amsterdam is its frozen pre-devnet-8 snapshot — exact ancestor commit 178d9f9, so the state-actor sender/delegate key constants and stub addresses are unchanged).

Test types keep their plain stateful/compute names — the full suite is distinguished by subdir. The existing v1/glamsterdam-devnet-7 (repricing-marked) subdir is untouched and keeps running in parallel.

Changes

  • test-source.{stateful,compute}.builder.yaml: same shape as the devnet-7 builders, with eest_ref: benchmarks/amsterdam, no marker, gas_benchmark_values: [200,300]. The force: true workaround flags aren't carried over (one-time fix for leftover partial fixtures; this subdir's fixtures dir is fresh). The compute not (test_blobhash and one_blob) exclusion is kept — the filler rejects that variant INVALID.
  • global.yaml: own EEST_FIXTURES_DIR (glamsterdam-devnet-7-full), following the per-subdir convention.
  • clients.yaml: byte-identical copy of devnet-7's (diff -q clean) — same instances so the two suites stay comparable; keep them in sync.
  • Workflows: the new subdir added to the subdir choice in build + run. No other machinery changes: release derives everything from the build run title, and dispatchoor/generate.sh already scans all repricing/v1/* subdirs — with no runner files yet it skips this one (verified: regen is a zero-line diff).
  • Build-only for now: the test-source.*.runner.yaml files land in a follow-up once the first build+release exists to point fixtures_url at. That follow-up should also set run timeouts for this subdir in generate.sh.

Coverage notes (verified against the branch)

  • Dropping the marker adds the unmarked tests: stateful/bloatnet/{test_call,test_create,test_erc20,test_transient_storage,depth_benchmarks}, compute/scenario/{test_mix_operations,test_transaction_types,test_unchunkified_bytecode}, and the compute/eip7928_block_level_access_lists tests (85 of 133 test functions carry the marker today).
  • test_call/test_erc20 need factory_stub/erc20_stub stub addresses the state-actor prestate doesn't provide — EEST warns and skips those (verified in address_stubs.py::parametrize_args), it does not fail the fill. Filling them later needs new state-actor entities + stub entries.
  • stateful/eip7928_block_level_access_lists stays excluded (test path is tests/benchmark/stateful/bloatnet, matching the devnet-7 scope) — can widen after a first green build.
  • Release tags embed test-type + EEST sha but not subdir, so devnet-7 and devnet-7-full releases of the same test type are distinguished only by sha/timestamp once glamsterdam-devnet-7: pin eest_ref to frozen benchmarks/amsterdam #22 lands; pass an explicit tag input on release dispatch if clearer naming is wanted.

Ordering

Dispatch benchmarkoor-build.yaml with subdir=v1/glamsterdam-devnet-7-full only after the state-actor datadir regen (see #benchmarking), so payloads are filled against the new state. Companion: #22 pins the existing devnet-7 subdir's eest_ref to the same frozen branch.

@qu0b
qu0b force-pushed the qu0b/full-suite-context branch from 3549156 to 559e325 Compare July 30, 2026 18:07
@qu0b qu0b changed the title add full-suite context: benchmarks/amsterdam, no repricing marker, 200M/300M gas glamsterdam-devnet-7: add full-suite test types (no repricing marker, 200M/300M, benchmarks/amsterdam) Jul 30, 2026
…, 200M/300M, benchmarks/amsterdam

New subdir fills the complete EEST benchmark suite (drop -m repricing)
at 200M and 300M gas from the frozen benchmarks/amsterdam branch
(forks/amsterdam now merges devnet-8 spec PRs). Test types stay
stateful/compute; the suite is distinguished by subdir. Build-only for
now: the runner test-source files land with the first release, until
then dispatchoor skips the subdir (it discovers test types from runner
files).
@qu0b
qu0b force-pushed the qu0b/full-suite-context branch from 559e325 to 1b7b496 Compare July 30, 2026 18:54
@qu0b qu0b changed the title glamsterdam-devnet-7: add full-suite test types (no repricing marker, 200M/300M, benchmarks/amsterdam) add glamsterdam-devnet-7-full subdir: full suite, no repricing marker, 200M/300M, benchmarks/amsterdam Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant