Skip to content

Conversation

@MartinquaXD
Copy link
Contributor

@MartinquaXD MartinquaXD commented Jan 24, 2026

Description

In order to know which gas price we have to beat at least (in case of cancellations) we made the driver scan the RPC node's mempool using the respective API as this is the ultimate source of truth.
However, this has 2 issues:

  1. not widely supported
  2. introduces latency (apparently up to 2s on mainnet at times)

Especially the latency seemingly causes us to not notify the connected solver about the tx submission at times. The submission process works as follows:

  1. driver receives a /settle call and starts the submission
  2. driver does the usual tx submission where it monitors the submission deadline and initiate the cancellation if necessary
  3. due to an issue with dead block streams the driver also monitors if the autopilot is still waiting for the response for the /settle call
  4. if the autopilot terminates the /settle call the driver only polls the submission future for 1 more second but otherwise simply stops polling it (code)

Usually the submission future and autopilot detect the breach of the submission deadline at the same time so the settle future naturally executes the cancellation logic during that grace period. However, with the latency introduced by the mempool API this grace period is often not sufficient anymore (especially on mainnet). Doing some back of the napkin calculation using logs it appears as if the driver is currently not cancelling and submitting the respective notification for ~40% of the /settle calls.

There is an argument to be made that the submission strategy should be refactored more broadly to ensure that cancellations always get initiated (instead of just stopping to poll the settle future) but this PR should at least already resolve the current issue.

Changes

Instead of using the RPC's mempool API we simply store the last successfully submitted transactions in memory. Now that we only have to lookup a key in a Dashmap the latency will be as it was before.

@MartinquaXD MartinquaXD requested a review from a team as a code owner January 24, 2026 15:45
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This PR removes the usage of the node's mempool API for determining replacement gas prices by caching the last submitted transaction in memory. While this successfully reduces latency, it introduces a potential issue. The in-memory cache is not persistent across driver restarts, which can lead to transaction submission failures if a transaction was pending before a restart. I have added a comment with details on this issue.

@MartinquaXD MartinquaXD changed the title Stop using node mempool API Replace RPC mempool API with in-memory tracking Jan 24, 2026
Copy link
Contributor

@squadgazzz squadgazzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would it work with the service restarts? If there is a pending tx in the mempool, we will never know the proper gas price, right? Should we fallback to the legacy approach then?

@MartinquaXD MartinquaXD enabled auto-merge January 26, 2026 10:47
@MartinquaXD MartinquaXD added this pull request to the merge queue Jan 26, 2026
Merged via the queue into main with commit 5ca01e7 Jan 26, 2026
19 checks passed
@MartinquaXD MartinquaXD deleted the stop-using-mempool-api branch January 26, 2026 11:19
@github-actions github-actions bot locked and limited conversation to collaborators Jan 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants