feat(test-fill): align fill-stateful with gas-benchmarks implementation

[LouisTsai-Csie]

🗒️ Description

Add a few features & refactors to the stateful filling implementation.

Commit d198221: Replace debug_setHead with FCU approach.

debug_setHead only supports in Geth, Besu and Erigon. Nethermind implements debug_resetHead that takes block hash as input. Reth has the method registered but not implemented (Link), Nimbus comments out the method (Link), no implementation either.

Consider some features (e.g., built-in opcode tracing) is now only supported in Nethermind master branch, this PR changes the debug_setHead to FCU version, in order to set the canonical chain.

Commit 0252bec: Add a new CLI flag --gas-bump-blocks.

For perf-devnet-3 snapshot, we need to bump the block gas limit to ~1T block gas limit. This is done by sending empty blocks in gas-benchmarks. User could use --gas-bump-blocks to assign number of empty block to increase the block gas limit.

In gas-benchmarks, we send 5000 empty blocks to bump the block gas limit for perf-devnet-3. For Jochmnet, we do not need to bump the gas limit, as the block gas limit is already high enough (~1 gigagas).

Commit 05f3c16: Add deterministic sender pooling.

For test_ether_transfers_onchain_receivers benchmark, to prevent the sender account being cached, we pre-fund the account via CL withdrawal. We inject 15K withdrawal into funding phase, starting with this pk range:

SENDER_BASE_KEY = int.from_bytes(
    keccak256(b"gas-repricings-private-key"), "big"
)

Commit 54f43ad: Batch tx receipt request

Batch tx receipt RPC request, to avoid large amount of RPC request at the same time. This was a bottleneck in the past when using gas-benchmarks.

Commit 8477bfe: refactor fill_stateful.py

Split the hive setup version from the original fill_stateful.py file, to keep the file small and focus on the filler feature.

These senders are then consumed by test_ether_transfers_onchain_receivers, this aligns changes in gas-benchmarks PR NethermindEth/gas-benchmarks#146

Verification

Verify the test via Hive mode, works successfully.

🔗 Related Issues or PRs

Issue #2910

✅ Checklist

• All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:

  just static

• All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
• All: Considered updating the online docs in the ./docs/ directory.
• All: Set appropriate labels for the changes (only maintainers can apply labels).
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
• Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
• Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

Cute Animal Picture

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.22%. Comparing base (3f888bc) to head (ded858f).

Additional details and impacted files

@@               Coverage Diff                @@
##           forks/amsterdam    #2923   +/-   ##
================================================
  Coverage            93.22%   93.22%           
================================================
  Files                  624      624           
  Lines                36926    36926           
  Branches              3377     3377           
================================================
  Hits                 34424    34424           
  Misses                1708     1708           
  Partials               794      794           

Flag	Coverage Δ
unittests	93.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[marioevz]

Thanks for doing this, I added a couple of comments which I feel need to be resolved before merging.

[marioevz]

We could do this once during initialization of ChainBuilderEthRPC, save fcu_version as a field of the chain builder instance, and then just use it as self.fcu_version.
This is also under the assumption that fork is always Fork and never TransitionFork.

[LouisTsai-Csie]

Same as comment

[marioevz]

Maybe out of scope for this PR, but the way this fixture is auto-used feels off.
I think a better structure would be to have a session_client_backend fixture that is session-scoped, and a client_backend fixture that is function-scoped.

The session_client_backend fixture initializes the object for the entire session, most of what _session_pre_run and client_backend fixtures do now.

The new client_backend fixture prepares and cleans up the ClientBackend object before and after every test, so what _reset_chain_between_tests currently does.

[marioevz]

See example of session_t8n and t8n session and function scoped fixtures that use the same object internally:

execution-specs/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py

Line 1093 in 7771d27

def session_t8n(

[LouisTsai-Csie]

Could I combine this refactor with the comment in a subsequent PR? They both touch ClientBackend, so a single PR would be cleaner, and the current PR already bundles lots of changes from different aspects.

[marioevz]

This call feels like we are conflating two different objects to achieve a purpose.

We could have client_backend.reset_to_start_block or similar because client_backend already knows start_block so it can do everything by itself.

This also makes the code more maintainable because if we ever modify the client_backend structure, you only have to modify methods inside of the self-contained ClientBackend class.