* Make alloy gas estimator configurable (#4081)
# Description
Adds the ability to configure past_blocks and reward_percentile
parameters for the EIP-1559 gas price estimator. Previously, alloy's
hardcoded defaults (10 blocks, 20th percentile) were always used.
# Changes
[x] Add configurable_alloy.rs - a gas estimator that calls
eth_feeHistory with custom parameters, then uses alloy's default
estimation algorithm
[x] Extend GasEstimatorType::Alloy config variant with optional
past-blocks and reward-percentile fields
[x] Default values match alloy's hardcoded constants (10 blocks, 20.0
percentile) for backwards compatibility
* Improve handling of unverifiable quotes (#4085)
# Description
Currently the quote verification leads to weird results - especially for
Ondo tokens. Routing these tokens requires the use of a proprietary API
which does not give out usable call data without an actual trade intent.
To adhere to the API solvers simply leave the execution plan of their
solution blank (pre interactions, regular interactions, JIT orders).
Normally this would lead to a revert in the trade simulation which would
in turn cause our system to keep the quotes but mark them as
unverifiable.
However, if the settlement contract has enough buy_tokens to pay for the
entire quoted amount the simulation will not revert but the analysis
afterwards will sniff out that the quote is giving money away for free.
This will then lead to the quote getting discarded entirely.
This causes 2 main issues:
1. it is possible to get a `fast` quote (which skips verification
alltogether) and then end up with `NoLiquidity` errors for `optimal`
quotes which is confusing
2. said `NoLiquidity` errors then prevent users from placing orders
# Changes
Since this needs to be resolved urgently I went for a relatively simple
approach where we detect whether a solution contains any execution plan
at all. Now we only discard quotes that are too inaccurate if the solver
actually tried to provide such a plan. If no plan is provided we simply
assume it's because no plan could be provided.
Note that there is still an incentive to provide verifiable calldata
because any verifiable quote will be preferred over any non-verifiable
quote. So solvers that don't make the effort to provide the calldata
will basically never win quotes for trades where it's possible to
provide calldata.
Minor other changes:
* renamed `Error::TooInaccurate` to `Error::BuffersPayForOrder` to
hopefully make the error case more self explanatory
* adjusted some getter functions to return `impl Iterator` instead of
`Vec` to avoid unnecessary cloning
## How to test
This is very hard to test with unit or e2e tests. Given how small the
actual change is I think existing e2e tests should be enough to cover
the correctness of the regular case and a deployment to prod will show
if we now indeed handle the Ondo token case better.
* Log solve request data transfer (#4082)
# Description
Some solvers reported that some requests come significantly delayed
(judging by the auction deadline). Currently we have no way to
distinguish between receiving the start of the `/solve` request and
streaming the actual data.
This PR makes this possible by making the `solve` handler take a raw
http request and stream the body afterwards.
# Changes
Instead of:
1. collecting the whole body into a `String` (including utf8 check)
2. logging that we received a request
3. putting that `String` into an `Arc` to make copying it cheap
4. deserializing the string into a `SolveRequest`
We now do:
1. receive raw http request
2. log that we received it
3. stream the body into a cheaply copyable `Bytes` type
4. log how long the data transfer took
5. deserialize raw bytes into `SolveRequest`
Since handling the raw request seems to bypass axum's request size
checks I did it manually for this endpoint.
## How to test
Existing tests should suffice
---------
Co-authored-by: ilya <ilya@cow.fi>
* Replace RPC mempool API with in-memory tracking (#4086)
# Description
In order to know which gas price we have to beat at least (in case of
cancellations) we made the driver scan the RPC node's mempool using the
respective API as this is the ultimate source of truth.
However, this has 2 issues:
1. not widely supported
2. introduces latency (apparently up to 2s on mainnet at times)
Especially the latency seemingly causes us to not notify the connected
solver about the tx submission at times. The submission process works as
follows:
1. driver receives a `/settle` call and starts the submission
2. driver does the usual tx submission where it monitors the submission
deadline and initiate the cancellation if necessary
3. due to an [issue](https://github.com/cowprotocol/services/pull/3427)
with dead block streams the driver also monitors if the autopilot is
still waiting for the response for the `/settle` call
4. if the autopilot terminates the `/settle` call the driver only polls
the submission future for 1 more second but otherwise simply stops
polling it
([code](https://github.com/cowprotocol/services/blob/main/crates/driver/src/domain/competition/mod.rs#L630-L643))
Usually the submission future and autopilot detect the breach of the
submission deadline at the same time so the settle future naturally
executes the cancellation logic during that grace period. However, with
the latency introduced by the mempool API this grace period is often not
sufficient anymore (especially on mainnet). Doing some back of the
napkin calculation using logs it appears as if the driver is currently
not cancelling and submitting the respective notification for ~40% of
the `/settle` calls.
There is an argument to be made that the submission strategy should be
refactored more broadly to ensure that cancellations always get
initiated (instead of just stopping to poll the settle future) but this
PR should at least already resolve the current issue.
# Changes
Instead of using the RPC's `mempool` API we simply store the last
successfully submitted transactions in memory. Now that we only have to
lookup a key in a `Dashmap` the latency will be as it was before.
* Optimize live orders queries based on confirmed_valid_to column (#4055)
# Description
Second part of https://github.com/cowprotocol/services/pull/4047 which
introduced optimized queries based on the introduced confirmed_valid_to
column
It is **crucial** for the database to be already migrated manually as
described in previous PR before applying this one.
# Changes
- [x] Adapt user_orders_with_quote query to use new column
- [x] Adapt solvable_orders query to use new column
## How to test
Tested on a test-db created by @MartinquaXD which contains a snapshot of
prod data. The optimized queries run significantly faster due to changes
in `orders` table and new indices.
## Related Issues
https://github.com/cowprotocol/services/pull/4021
---------
Co-authored-by: ilya <ilya@cow.fi>
Co-authored-by: José Duarte <duarte.gmj@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com>
* [TRIVIAL] Remove atty and move maplit to dev-dependencies (#4089)
# Description
While migrating the orderbook to axum I did another pass over the
dependencies and found that atty not only is deprecated, it has a
RUSTSEC because its unmaintained and a proper replacement since Rust
1.70
# Changes
- [ ] Replace deprecated atty crate with std::io::IsTerminal
(https://github.com/softprops/atty/blob/master/README.md?plain=1#L3-L7)
- [ ] Move maplit to dev-dependencies
## How to test
Compilation
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Remove useless logs (#4084)
# Description
Our services are extremely chatty which is annoying for debugging and
overwhelms our logging infra. This PR removes or strips down logs that
should not be needed.
# Changes
- removes huge structs like calldata, access lists, and duplicated
transactions - calldata is still preserved where it matters most (when
resimulating quotes, or in revert errors)
- removes tempo items that needlessly get printed in every log of the
respective trace (where it seemed useful I added 1 log that contained
the data)
- removes logs when solutions could not be merged (this is an optimistic
optimization and solutions are not expected to always be mergeable)
- downgraded some logs from `debug` to `trace` (the ones I think I never
used for any debugging but on the surface level seemed like they might
be useful eventually)
- 404 errors from `/notify` requests
* Fetch inflight orders from DB (#4087)
# Description
In order to avoid solver solutions conflicting with each other once a
solution for an order was proposed it will get removed from the auction
until its submission deadline has been reached. So far this was managed
entirely in-memory which can lead to issues whenever the autopilot gets
restarted.
# Changes
Since the DB scheme refactor a while ago we now have all the data we
need to recover inflight orders from the DB. This PR replaces the
in-memory inflight order handling by looking them up from the DB.
To make the query fast enough I added an index on the deadline column on
the `competition_auctions` table. With that the query takes ~0.1ms to
look up 10 auctions worth of inflight orders.
<details>
<summary>execution plan</summary>
```
"Unique (cost=1352.80..1352.98 rows=35 width=57) (actual time=0.041..0.043 rows=1 loops=1)"
" -> Sort (cost=1352.80..1352.89 rows=35 width=57) (actual time=0.040..0.041 rows=1 loops=1)"
" Sort Key: pte.order_uid"
" Sort Method: quicksort Memory: 25kB"
" -> Nested Loop (cost=1.86..1351.90 rows=35 width=57) (actual time=0.028..0.033 rows=1 loops=1)"
" -> Nested Loop Anti Join (cost=1.29..1339.25 rows=4 width=24) (actual time=0.023..0.028 rows=1 loops=1)"
" Join Filter: (s.solution_uid = ps.uid)"
" -> Nested Loop (cost=0.86..1171.92 rows=4 width=24) (actual time=0.013..0.020 rows=2 loops=1)"
" -> Index Scan using competition_auction_deadline on competition_auctions ca (cost=0.43..11.96 rows=5 width=8) (actual time=0.005..0.006 rows=2 loops=1)"
" Index Cond: (deadline > 24300390)"
" -> Index Scan using proposed_solutions_pkey on proposed_solutions ps (cost=0.43..231.80 rows=19 width=16) (actual time=0.003..0.006 rows=1 loops=2)"
" Index Cond: (auction_id = ca.id)"
" Filter: is_winner"
" Rows Removed by Filter: 8"
" -> Index Scan using settlements_auction_id on settlements s (cost=0.43..41.69 rows=11 width=16) (actual time=0.003..0.003 rows=0 loops=2)"
" Index Cond: (auction_id = ca.id)"
" -> Index Only Scan using proposed_trade_executions_pkey on proposed_trade_executions pte (cost=0.56..3.15 rows=1 width=73) (actual time=0.004..0.004 rows=1 loops=1)"
" Index Cond: ((auction_id = ps.auction_id) AND (solution_uid = ps.uid))"
" Heap Fetches: 1"
"Planning Time: 0.543 ms"
"Execution Time: 0.079 ms"
```
</details>
## How to test
added a new unit test for the DB query
* Stop enforcing body size limit (#4092)
# Description
The shadow competition broke because the driver is now rejecting
`/solve` requests that are too large due to this new
[code](https://github.com/cowprotocol/services/pull/4082/changes#diff-b997d6f696c5591860aef8658bb56d2a03fc4fa6b37b5e0432ce8e5e4e356aa9R61-R64).
This was surprising to me because that code was added specifically
because the new handler is bypassing the original content length
limiting layer so I would have expected huge requests to already cause
issues.
During the investigation I confirmed using the `/solve` requests stored
on S3 that recent auctions are indeed larger than. 10MB. Afterwards I
spun up a driver locally and sent that solve request to the original
code to confirm that it's indeed not throwing any errors.
I further investigated and concluded that the issue is how we build the
driver's http router. The size limiting layer is the first thing that
gets added to the router but it should actually be the last. This caused
the size limit to never go into effect.
# Changes
To resolve the issue quickly and remove this breaking change ASAP I
simply removed the new size limiting logic from the `/solve` request.
In a follow up PR I'll make the size limit configurable and fix the
router.
## How to test
manual test
* Revert "Optimize live orders queries based on confirmed_valid_to colu… (#4094)
# Description
This reverts commit 7081b03f66c5154cc84ae93f79542425803b4639. (PR
https://github.com/cowprotocol/services/pull/4055)
The migrations will be revisited as they could not be applied to prod
due to lockup and long duration.
* Add no-op placeholder migrations for the numbering to be continuous (#4096)
# Description
We needed to revert migrations V098, V099, and V097 was spelled wrongly
(lowercase v). Since then V100 has been added and it makes flyway
complain about missing interim migrations.
Adding no-op migrations is enough to keep the continuity.
# Changes
Adds no-op migrations V097, V098 and V099
* Rewrite migration V100 to be optional (#4098)
# Description
The migration V100 creates index on competition_auction_deadline on
competition_auctions. To make the prod deployment viable it needs to be
optional (IF NOT EXISTS) which will enable to apply it manually
beforehand.
# Changes
Update the V100 migration to specify IF NOT EXISTS.
## How to test
Will apply the migration manually and deploy on staging to verify.
* Move order outside market log to callsites (#4090)
# Description
The log inside the unwrap does not provide an actionable info, the lack
or order ID, from address, quote ID, make it extremely hard to follow up
on.
More context in
https://cowservices.slack.com/archives/C0375NV72SC/p1769440848303459
# Changes
- [ ] Remove the log from the unwrap
- [ ] Place it in the (seemingly) more relevant callsites
## How to test
NA
* Add configurable database connection pool size (#4097)
# Description
We had an incident where latency increase due to queries waiting for
available connections. This PR provides a configuration for that.
Adds `--db-max-connections` (env: DB_MAX_CONNECTIONS) flag to configure
the maximum database pool size. Default is 10.
# Changes
- [ ] New config for DB connection pool size
- [ ] Add it to autopilot, orderbook, refunder
## How to test
E2E + staging (?)
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix migrations introducing indexes on true_valid_to (#4095)
# Description
Reinstates https://github.com/cowprotocol/services/pull/4055 with
improved migrations that should successfully apply
to prod.
The previous migrations tended to lock-up indefinitely on prod database
when running UPDATE on all rows in the `orders` table to ensure
`true_valid_to` is not null. This was done as an additional safety layer
as these rows have been manually backfilled previously, so it is no
longer needed and turned out to be problematic.
Additionally, the index creation can take more time than is allowed for
release deployment which causes them to be aborted. It is easier to
create them manually and have the migration CREATE INDEX IF NOT EXISTS.
# Changes
- Had to move migrations from 098, 099 to 101, 102 as there was a
migration 100 merged in the meantime.
- Moves migration 098 to 101.
Removes conservative backfill of empty `true_valid_to` which caused a
lock-up on the prod database.
- Moves migration 099 to 102
Makes index creation optional (CREATE INDEX IF NOT EXISTS) as they will
be created manually, to ensure smooth deployment.
---------
Co-authored-by: ilya <ilya@cow.fi>
Co-authored-by: José Duarte <duarte.gmj@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com>
* Migrate gas estimation to match alloy's type (#4054)
> [!CAUTION]
> Review with care! The changes are non trivial and there is one
breaking change, the gas price returned by the driver no longer includes
the base fee!
# Description
Refactors gas price handling to use integer arithmetic and alloy's
native types instead of floating-point
calculations. This eliminates precision loss in gas price calculations
and better aligns with alloy's conventions.
The removal of the base fee is not a true removal, before this change,
the base fee was either 0 or the max value available, both leading to
inaccurate results. The new code removes the base fee from the type that
was being used to describe the estimations (because the base_fee isn't
an estimate, it comes in the previous block); but starts querying the
chain for the latest block so it's able to get the proper base_fee (if
available). The gas estimates themselves should suffer a big change
(since the estimate code is the same) but the effective gas price should
become more accurate due to the inclusion of the base fee in the
calculations.
# Changes
- Replace custom GasPrice1559 with alloy's Eip1559Estimation throughout
codebase
- Change GasPrice::base from FeePerGas to Option<u64> to match alloy's
base fee representation
- Migrate gas calculations from f64 to u128/U256 integer arithmetic
- Implement calc_effective_gas_price from alloy for effective gas price
calculations
- Add base_fee: Option<u64> to BlockInfo for proper EIP-1559 support
- Update API responses to return u128 directly instead of wrapped U256
- Add scaling helper methods (scaled_by_pct, scaled_by_pml) for clearer
gas price adjustments
# How to test
> [!NOTE]
> Tested on staging, starting at Mon, 19 Jan 2026 12:10:18 +0000.
> Performed a successful trade:
https://staging.explorer.cow.fi/lens/orders/0x06677572a2715cc28241a34f5d669247fba167c8d9adc3fcd338e40a3c52ea4109fbad1ea29c36dfe4f8f7baa87c5edf85e0d9f3696e28f5
1. Run existing test suite: cargo test
2. Verify gas price estimation endpoints return expected values
3. Test refunder gas price calculations with various scenarios (new tx,
replacement tx, max gas price limits)
4. Verify settlement submissions use correct gas parameters
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [TRIVIAL] Add Plasma to OpenApi (#4100)
Adds Plasma URLs to the orderbook's OpenAPI.
* Optimize order queries by using true_valid_to (#4104)
# Description
Reinstates queries from
https://github.com/cowprotocol/services/pull/4055 to accelerate live
order queries based
on the newly introduced `true_valid_to` column and its indexes.
Tested on a test-db created by @MartinquaXD which contains a snapshot of
prod data. The optimized queries run significantly faster due to changes
in `orders` table and new indices.
# Changes
- [x] Adapt user_orders_with_quote query to use new column
- [x] Adapt solvable_orders query to use new column
## How to test
Tested on the test database created by @MartinquaXD by analyzing query
plan (`EXPLAIN (ANALYZE, BUFFERS)`). In the worst case, the latency has
improved 40x (from 20s to 0.5s).
* Fix haircut logic (#4093)
# Description
The haircut feature had a critical bug where the driver-reported
`sell_amount` would exceed the user's signed one. For example:
- User signed: sell_amount = 5 ETH
- Solver proposed a solution with the same sell amount
- Driver reported: sellAmount = 5.25 ETH (with 5% haircut added)
- The settlement executed onchain, but autopilot couldn't make sense of
it due to the unexpected sell amount
- The circuit breaker also detects this and bans the solver
# Changes
1. Removed haircut from `sell_amount()` - Now returns only executed +
fee, which is the actual amount that left the user's wallet
2. Added `haircut_in_sell_token()` helper - Computes haircut amount
converted to sell token
3. Updated `custom_prices()` - Applies haircut only for quotes/scoring
purposes, making bids more conservative without affecting reported
amounts
4. Added `Jit::custom_prices()` - JIT orders don't have a haircut (for
now), so they use simple sell/buy amount derivation
## How to test
Adjusted existing and added new tests that fail on the `main` branch,
but work with the fix.
* [TRIVIAL] Drop the db solver participation guard (#4099)
# Description
Cleans up the codebase by removing the DB solver participation guard.
It's been used in a log-only mode for a while. Given the lack of demand
for this functionality, it doesn't make sense to keep it. Also, even if
it were decided to enable it, the logic would need to be reworked to
cover some edge cases, which would take some time to implement.
* [TRIVIAL] Fix playground configs (#4108)
# Description
Estimators were expecting different strings and the tx gas was missing
from the driver
# Changes
- [ ] Remove Native from gas estimators
- [ ] Add "Driver" to the native price estimators
- [ ] Add tx-gas-limit to the driver config
## How to test
Run docker compose and check if autopilot, orderbook and driver are up
Co-authored-by: Claude <noreply@anthropic.com>
* Update playground frontend Dockerfile (#4103)
# Description
Fixes Docker pnpm version error.
# Changes
Enable and prepare corepack version before running pnpm install.
# Fixes
https://github.com/cowprotocol/services/issues/4101
* Normalize ECDSA signature `v` parameter for Solidity `ecrecover` compatibility (#4107)
# Description
This PR fixes an issue where EIP-712 signatures with `v = 0` or `1`
(modern EIP-2 format) pass off-chain validation but fail on-chain
settlement with `GPv2: invalid signature`.
# Problem
Some wallets (e.g., Bitget Wallet) produce ECDSA signatures using the
modern EIP-2 format, where `v ∈ {0, 1}`, while Solidity's `ecrecover`
precompile expects the legacy format where `v ∈ {27, 28}`.
Off-chain (Alloy library): The
https://github.com/alloy-rs/core/blob/main/crates/primitives/src/signature/sig.rs
internally normalizes `v` to a boolean parity before recovery, so
signatures with `v = 0` or `1` recover correctly.
On-chain (Solidity): The `ecrecover` precompile
https://coders-errand.com/ecrecover-signature-verification-ethereum/.
When given `v = 0`, it returns `address(0)`, which triggers the
https://github.com/cowprotocol/contracts/blob/main/src/contracts/mixins/GPv2Signing.sol#L207-L208:
```
signer = ecrecover(message, v, r, s);
require(signer != address(0), "GPv2: invalid ecdsa signature");
```
This mismatch causes orders to pass orderbook validation but fail at
settlement.
# Solution
Normalize `v` to the legacy format (`27/28`) at signature parsing time
in `EcdsaSignature::from_bytes()`:
```
let normalized_v = match v {
0 | 27 => 27,
1 | 28 => 28,
_ => anyhow::bail!("invalid signature v value: {v}, expected 0, 1, 27, or 28"),
};
```
This ensures:
1. Signatures are stored with normalized `v` values
2. Both off-chain validation and on-chain `ecrecover` receive compatible
parameters
3. The fix applies to all entry points (`Signature::from_bytes`, JSON
deserialization)
# Reproducing the Issue
The issue can be verified using a real failed order and Foundry's cast
tool to call the `ecrecover` precompile directly:
Failed order:
- Order UID:
`0xb8e19962dd762067afb9f169684abfcbf2cb13bdc7a62ae2e680ebd5ce18c9bcca0c9c4a650cc4ed406d4a6dd031cdd9d4ebf0dc697a0686`
- Order hash (struct hash):
`0xb8e19962dd762067afb9f169684abfcbf2cb13bdc7a62ae2e680ebd5ce18c9bc`
- Expected signer (owner): `0xca0c9c4a650cc4ed406d4a6dd031cdd9d4ebf0dc`
- Signature:
`0xAB2E74AA0D67233ADC7B52C3B832357ED35F2052338D820D4DA66210EFA7A9684601726CB76BD26DDD958EFE291CFB57E02C39B3F60FBB8BBED1E891FB14CB5D00`
- r:
`0xAB2E74AA0D67233ADC7B52C3B832357ED35F2052338D820D4DA66210EFA7A968`
- s:
`0x4601726CB76BD26DDD958EFE291CFB57E02C39B3F60FBB8BBED1E891FB14CB5D`
- v: `0x00` ← the problem
## Step 1: Compute the EIP-712 message hash
To avoid computing it manually, I grabbed it from a Tenderly
simulation[[URL](https://dashboard.tenderly.co/cow-protocol/barn/simulator/babc6191-e15a-470c-83e0-5825b8a4501b/debugger?trace=0.0.4.1.1.0)].
```
MESSAGE_HASH="0xb8e19962dd762067afb9f169684abfcbf2cb13bdc7a62ae2e680ebd5ce18c9bc"
```
## Step 2: Test ecrecover with `v=0` (returns zero address - FAILS
on-chain)
```
cast call 0x0000000000000000000000000000000000000001 \
"${MESSAGE_HASH}0000000000000000000000000000000000000000000000000000000000000000AB2E74AA0D67233ADC7B52C3B832357ED35F2052338D820D4DA66210EFA7A9684601726CB76BD26DDD958EFE291CFB57E02C39B3F60FBB8BBED1E891FB14CB5D" \
--rpc-url https://eth.llamarpc.com
```
### Returns:
`0x0000000000000000000000000000000000000000000000000000000000000000`
## Step 3: Test ecrecover with `v=27` (returns correct signer - WORKS)
```
cast call 0x0000000000000000000000000000000000000001 \
"${MESSAGE_HASH}000000000000000000000000000000000000000000000000000000000000001bAB2E74AA0D67233ADC7B52C3B832357ED35F2052338D820D4DA66210EFA7A9684601726CB76BD26DDD958EFE291CFB57E02C39B3F60FBB8BBED1E891FB14CB5D" \
--rpc-url https://eth.llamarpc.com
```
### Returns:
`0x000000000000000000000000ca0c9c4a650cc4ed406d4a6dd031cdd9d4ebf0dc` ✅
* Fix haircut mismatch between reported and on-chain amounts (#4109)
# Description
Fixes the mismatch between driver-reported amounts and on-chain executed
amounts when the haircut is configured. Previously, the driver reported
higher buy amounts than users actually received on-chain (for sell
orders), resulting in a discrepancy that matched the configured haircut.
Root cause: `sell_amount()` and `buy_amount()` did NOT include haircut,
but `custom_prices()` (used for on-chain encoding) DID. This caused
reported amounts to differ from on-chain execution.
# Changes
Include haircut effects in `sell_amount()` and `buy_amount()` so that:
- Reported amounts include haircut
- On-chain execution matches reported amounts
- Autopilot scores based on actual (haircutted) amounts
For sell orders:
- `sell_amount()` → unchanged (user sells exactly what they signed)
- `buy_amount()` → reduced by haircut (user receives less)
For buy orders:
- `sell_amount()` → increased by haircut (user pays more)
- `buy_amount()` → unchanged (user receives exactly what they signed
for)
## How to test
Adjusted existing tests.
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Support fractional vol fee bips in orderbook (#4112)
# Description
We were rounding fractional bips (e.g. `0.3`) to zero. This PR increases
the scale so we can handle basis point values lower than 0. At this
moment we have volume fee overrides for correlated assets sets to 0.3
and it works fine in autopilot, but /quote endpoint rounds to zero
instead, so this fix is needed.
## How to test
Unit tests & I tested by deploying this branch to staging.
---------
Co-authored-by: ilya <ilya@cow.fi>
* Improve autopilot maintenance (#4113)
# Description
While looking into the degraded time to happy moo SLI it became apparent
that ethflow orders have a significantly worse SLI compared to "regular"
orders.
Ethflow orders are not harder to solve for than any other orders but
they are special in the way they enter the system. Instead of having a
REST API call that puts those orders into the DB they get placed by
calling the ethflow contract onchain. The autopilot then indexes those
events and puts them into the DB.
Since the autopilot run loop is synced to the block chain (start a new
auction right after seeing a new block) ethflow orders are comparable to
regular orders that ALWAYS get placed at the worst possible time
(immediately before cutting the auction).
Due to being overwhelmed with indexing ethflow orders because of a trade
inventive we moved ethflow indexing off of the critical path (see
[here](https://github.com/cowprotocol/services/pull/3849)) but that also
had the consequence of more ethflow orders not making it into the first
possible auction which immediately delays them at least by 12s.
# Changes
This PR puts ethflow order indexing back on the critical path while
still avoiding the issue that caused us to move it off the critical path
in the first place.
Instead of having a system where the autopilot triggers the maintenance
to happen before a new auction or after new block appearing (when
waiting for submitted solutions) with an additional background task that
checks every second for new ethflow orders that need indexing.
This PR moves autopilot maintenance (i.e. block indexing) completely
into a background task which triggers ASAP when the system sees a new
block. In order to build the auction only after the blocks have been
indexed this background tasks feeds a channel of processed blocks. The
autopilot then only has to wait for this channel to yield a block with a
high enough block number.
So the properties of the new solutions are:
* event indexing has as little delay as possible
* indexing runs concurrently so it's as fast as possible (without
speeding up the individual code paths)
* autopilot can wait for data from a given block to be processed fully
* autopilot stops waiting after a configurable amount of time to keep
running auctions even if indexing is slow for whatever reason
## How to test
Covered by existing e2e tests
* Remove ethcontract+web3+primitive-types (#4106)
Completes the Alloy migration by removing the last remaining legacy
Ethereum libraries: `ethcontract`, `web3`, and `primitive-types`. These
dependencies are no longer needed and can be fully removed, simplifying
the dependency tree.
**Key change**: The labelling layer for observability now operates at
the `Web3` wrapper level instead of directly on `DynProvider`, ensuring
the wallet state is properly preserved when creating labeled provider
instances.
# Changes
- [x] Remove `ethcontract`, `web3`, and `primitive-types` from workspace
dependencies
- [x] Delete unused legacy ethrpc implementations (`buffered.rs`,
`http.rs`, `instrumented.rs`, `alloy/conversions.rs`)
- [x] Migrate `ProviderLabelingExt` from `DynProvider` to `Web3`
wrapper, preserving wallet state across labeled instances
- [x] Clean up ethrpc module structure and simplify exports
- [x] Update imports across affected crates to use Alloy types only
- [x] Remove legacy references from contract vendoring script and test
setup
## How to test
Existing tests
* Trait‑Based Architecture for Refunder Service (#4051)
### Description
This PR follows up on #4029 and introduces a **trait‑based
architecture** for the `refunder` crate. By decoupling the
`RefundService` from concrete database and blockchain implementations,
we can now write unit tests without relying on (heavyweight) integration
tests.
### Changes
- Added a new `traits.rs` module that defines `DbRead`, `ChainRead`, and
`ChainWrite` traits to abstract the two main boundaries of the system.
- Created an `infra/` module housing the previous concrete
implementations of those traits:
- `AlloyChain` implements `ChainRead`
- `Postgres` implements `DbRead`
- Made `RefundService` generic over those traits, so we can use mocks
for unit testing it (and thes real/production implementations) as
needed.
- Extracted `identify_uids_refunding_status` into its own function, to
simplify testing.
- Moved the `RefundStatus` enum into `traits.rs` so it lives alongside
the related abstractions.
- Reorganized the service construction inside `run()` for clearer flow.
- Added a suite of unit tests that use mocks to cover a variety of
scenarios.
### How to test
Run the unit tests with:
```bash
cargo nextest run -p refunder
```
* Fix tini zombie reaping with shared process namespace (#4114)
## Summary
Adds the `-s` (subreaper) flag to tini in the Dockerfile entrypoint to
fix zombie process reaping when `shareProcessNamespace: true` is set in
Kubernetes deployments.
## Problem
Our Kubernetes deployments use `shareProcessNamespace: true` to allow
sidecar containers (like the memory monitor) to access `/proc` of the
main process. However, this causes the following warning:
```
[WARN tini (82)] Tini is not running as PID 1 and isn't registered as a child subreaper.
Zombie processes will not be re-parented to Tini, so zombie reaping won't work.
To fix the problem, use the -s option or set the environment variable TINI_SUBREAPER to register Tini as a child subreaper, or run Tini as PID 1.
```
When `shareProcessNamespace` is enabled, Kubernetes' pause container
becomes PID 1 instead of tini:
```
PID 1: pause (Kubernetes infrastructure)
├── tini -- autopilot
│ └── autopilot
└── /bin/sh -c (memory-monitor sidecar)
```
Without PID 1 status, tini cannot reap zombie (orphaned) child processes
by default.
## Solution
The `-s` flag tells tini to register as a **child subreaper** via the
`PR_SET_CHILD_SUBREAPER` prctl. This Linux kernel feature allows a
non-PID-1 process to adopt and reap orphaned descendant processes,
restoring proper zombie cleanup.
* [TRIVIAL] Fix vulnerability by bumping bytes crate version (#4121)
# Description
The `cargo audit` action complained about the `bytes` crate being
vulnerable. The recommended fix is to upgrade `bytes` to version
`1.11.1` (patch version bump).
## How to test
`cargo audit` action
* [TRIVIAL] Fix verbose log (#4120)
# Description
There was a slight oversight in
https://github.com/cowprotocol/services/pull/4084. Instead of printing
only the solver name we now print all the internals which is quite a
lot.
# Changes
* only log the solver name again
* stop logging the weth address as well
* [TRIVIAL] Remove unused error (#4124)
# Description
Removes an unused error, this could actually have made it in #4106 but
VSCode looked like an Xmas tree, I wasn't expecting this to be this
simple.
# Changes
- [ ] Removes unused error
- [ ] Removes accompanying From
* [TRIVIAL] Rename Web3's DynProvider alloy -> provider (#4123)
# Description
Renames alloy to provider. Done in a separate PR from the removal due to
the number of changes
# Changes
- [ ] `alloy` -> `provider`
* Log tracing spans in JSON logger (#4117)
# Description
In order to reduce load on the logging infra we want to switch to
structured logging (JSON). However, when we tested the current setup we
realized that the `request_id` (among other things) was not logged which
made debugging things basically impossible.
# Changes
Adjusted the custom JSON formatter to iterate over parent spans and
serialize their names and associated fields.
Conceptually the current logic is slightly awkward as the field
formatter formats the fields to JSON and later when we format the whole
log line we re-parse the formatted string to JSON and the serialize it
again. But unless this is actually causing issues when it's deployed
I'll not address in order to unblock structured logging ASAP.
## How to test
Manual tests (spans are at the end)
original version:
```
{
"timestamp":"2026-02-03T09:51:18.773085+00:00",
"level":"INFO",
"fields":{
"message":"- \"GET /api/v1/token/0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48/native_price HTTP/1.1\" 404 \"-\" \"curl/8.7.1\" 5.229292ms",
"log.target":"orderbook::api::request_summary",
"log.module_path":"warp::filters::log",
"log.file":"/Users/martin/.cargo/git/checkouts/warp-ee983b87d3028bb6/586244e/src/filters/log.rs",
"log.line":37
},
"target":"log",
"trace_id":"4aaa6c6e6e56f103d5cf975005c15d85"
```
default JSON logger:
```
{
"timestamp":"2026-02-03T09:40:00.265994Z",
"level":"INFO",
"fields":{
"message":"- \"GET /api/v1/token/0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48/native_price HTTP/1.1\" 404 \"-\" \"curl/8.7.1\" 2.721875ms",
"log.target":"orderbook::api::request_summary",
"log.module_path":"warp::filters::log",
"log.file":"/Users/martin/.cargo/git/checkouts/warp-ee983b87d3028bb6/586244e/src/filters/log.rs",
"log.line":37
},
"target":"orderbook::api::request_summary",
"span":{
"request_id":"6",
"name":"http_request"
},
"spans":[
{
"request_id":"6",
"name":"http_request"
}
]
}
```
new version:
```
{
"timestamp":"2026-02-03T09:41:25.529338+00:00",
"level":"INFO",
"fields":{
"message":"- \"GET /api/v1/token/0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48/native_price HTTP/1.1\" 404 \"-\" \"curl/8.7.1\" 6.522583ms",
"log.target":"orderbook::api::request_summary",
"log.module_path":"warp::filters::log",
"log.file":"/Users/martin/.cargo/git/checkouts/warp-ee983b87d3028bb6/586244e/src/filters/log.rs",
"log.line":37
},
"target":"log",
"trace_id":"cd6bf89be55582b19fd046b7c111f2b1",
"spans":{
"http_request":{
"request_id":"0"
}
}
}
```
* Initial Claude setup (#4122)
# Description
Add instructions for Claude to work more efficiently.
Expect .env.claude with secrets filled in. Contact me.
None of this was written by hand. To put together the order debugging
document I took transcripts of both Felix's talks on the topic and our
internal docs and threw it at Claude. Then I tried it for an order and
made some edits.
# Changes
- [x] Sets up MCP servers for main DB and analytics DB
- [x] Sets up MPC for fetch
- [x] Add CLAUDE.md instructing Claude about the project and coding
practices (formatting, style, etc.)
- [x] Adds a document telling Claude how to debug orders so it knows
what to do next time you ask him "y order {uid} not filled"
- [x] Added a comman so you can run `/debug-order
0xd997dc715a7610c75e5f97548685befacb7ea5ad878cb4bac1816903514ed84d1dffc418c0d83bd8b98ab3d2e07b83bf5439f4236981a392`
within Claude Code
## How to test
Ask Claude Coe to do stuff for you.
<!--
## Related Issues
Fixes #
-->
* Don't notify solvers about failed solutions with a haircut fee (#4115)
# Description
This PR addresses [a
comment](https://github.com/cowprotocol/services/pull/4049#pullrequestreview-3687251046)
that suggests avoiding notifying solvers on failed solutions encoding
that were configured with a haircut fee. Also, updates the metric to
easily identify the error rate of solutions with the haircut fee to
configure a new alert to take any action.
* Optimize total_surplus query (#4116)
# Description
We've been experiencing latency spikes on several endpoints, we've
pinned this down to the time it takes to acquire DB connections from the
pool; when checking RDS monitoring, the surplus query always shows up at
the top.
The current theory is that the query is a bit slower than it could be,
as more users request the main swap page, their surplus is loaded (even
if they don't request it — i.e. load the wallet modal) and if some of
these users have a larger amount of orders, they're taking up
connections that other endpoints aren't getting.
I looked into the user distribution, here are the results:
|log_bucket|bucket_start|num_users|pct|cumulative_users|cumulative_pct|
|----------|------------|---------|---|----------------|--------------|
|0|1|295329|83.86|295329|83.86|
|1|10|51334|14.58|346663|98.44|
|2|100|5071|1.44|351734|99.88|
|3|1000|306|0.09|352040|99.96|
|4|10000|106|0.03|352146|99.99|
|5|100000|28|0.01|352174|100.00|
<details><summary>Distribution Query</summary>
<p>
```sql
WITH buckets AS (
SELECT
floor(log(order_count))::int as log_bucket,
power(10, floor(log(order_count)))::int as bucket_start,
count(*) as num_users
FROM (
SELECT owner, count(*) as order_count
FROM orders
GROUP BY owner
) sub
GROUP BY 1, 2
)
SELECT
log_bucket,
bucket_start,
num_users,
round(100.0 * num_users / sum(num_users) over(), 2) as pct,
sum(num_users) over(order by log_bucket) as cumulative_users,
round(100.0 * sum(num_users) over(order by log_bucket) / sum(num_users) over(), 2) as cumulative_pct
FROM buckets
ORDER BY log_bucket;
```
</p>
</details>
However, if it was just this, it would be too simple. Depending on the
user, they might have no orders and only onchain orders (note that the
max number of onchain orders is around ~10k), some will have skewed data
distributions across tables too, leading to analyzing and optimizing
this query a bit tricky.
There are two crucial changes to the query — removing ARRAY_AGG and
adding indexes — the first makes it so that the DB does not have to
materialize a potentially big array in memory, which would otherwise
lead to bad estimations too; the second provides better "access paths"
to some of the information the query requires.
### RDS Stats (1h)
Before
<img width="2374" height="438" alt="Screenshot 2026-02-05 at 11-12-43
CloudWatch eu-central-1"
src="https://github.com/user-attachments/assets/06326a41-dfd5-4799-a0de-b0ab15a6349b"
/>
After
<img width="2368" height="1228" alt="Screenshot 2026-02-05 at 11-14-53
CloudWatch eu-central-1"
src="https://github.com/user-attachments/assets/7822d4c9-35bd-45c7-aecc-c5a271c7aded"
/>
# Changes
- [ ] Replace the query with the optimized one
- [ ] Create indexes (already done to avoid issues during migration)
## Query Plans
<details><summary>Before</summary>
<p>
```
Aggregate (cost=1033006.55..1033006.56 rows=1 width=8) (actual time=130139.025..130143.456 rows=1 loops=1)
-> Append (cost=1032650.26..1033006.51 rows=2 width=100) (actual time=31175.499..130090.776 rows=10998 loops=1)
-> Subquery Scan on "*SELECT* 1" (cost=1032650.26..1032856.24 rows=1 width=100) (actual time=31175.498..130085.109 rows=10998 loops=1)
-> Nested Loop (cost=1032650.26..1032856.23 rows=1 width=100) (actual time=31175.497..130079.454 rows=10998 loops=1)
Join Filter: (t.order_uid = o.uid)
InitPlan 3 (returns $5)
-> Finalize Aggregate (cost=1032601.57..1032601.58 rows=1 width=32) (actual time=30761.210..30765.632 rows=1 loops=1)
-> Gather (cost=1032601.35..1032601.56 rows=2 width=32) (actual time=30678.267..30689.666 rows=3 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (cost=1031601.35..1031601.36 rows=1 width=32) (actual time=30675.862..30675.863 rows=1 loops=3)
-> Parallel Bitmap Heap Scan on orders (cost=24591.99..1030861.56 rows=295913 width=57) (actual time=278.099..30619.932 rows=226281 loops=3)
Recheck Cond: (owner = '\x10dad59905d93ca37cd25a35f25349cb5956ba8e'::bytea)
Rows Removed by Index Recheck: 737927
Heap Blocks: exact=15162 lossy=55571
-> Bitmap Index Scan on order_owner (cost=0.00..24414.44 rows=710192 width=0) (actual time=271.994..271.994 rows=749143 loops=1)
Index Cond: (owner = '\x10dad59905d93ca37cd25a35f25349cb5956ba8e'::bytea)
InitPlan 4 (returns $6)
-> Aggregate (cost=47.00..47.01 rows=1 width=32) (actual time=0.021..0.022 rows=1 loops=1)
-> Bitmap Heap Scan on onchain_placed_orders (cost=4.09..46.97 rows=11 width=57) (actual time=0.017..0.018 rows=0 loops=1)
Recheck Cond: (sender = '\x10dad59905d93ca37cd25a35f25349cb5956ba8e'::bytea)
-> Bitmap Index Scan on order_sender (cost=0.00..4.08 rows=11 width=0) (actual time=0.013..0.013 rows=0 loops=1)
Index Cond: (sender = '\x10dad59905d93ca37cd25a35f25349cb5956ba8e'::bytea)
-> Nested Loop (cost=1.12..136.08 rows=2 width=188) (actual time=31172.849..109529.707 rows=10914 loops=1)
-> Index Scan using orders_pkey on orders o (cost=0.56..85.71 rows=10 width=123) (actual time=31172.113..101635.999 rows=678843 loops=1)
Index Cond: (uid = ANY (array_cat($5, $6)))
-> Index Only Scan using order_rewards_pkey on order_execution oe (cost=0.56..5.01 rows=3 width=65) (actual time=0.011..0.011 rows=0 loops=678843)
Index Cond: (order_uid = o.uid)
Heap Fetches: 170
-> Index Scan using trade_order_uid on trades t (cost=0.56..0.74 rows=2 width=81) (actual time=0.416..0.416 rows=1 loops=10914)
Index Cond: (order_uid = oe.order_uid)
SubPlan 1
-> Index Scan using auction_prices_pkey on auction_prices ap (cost=0.58..35.00 rows=16 width=11) (actual time=1.428..1.428 rows=1 loops=9675)
Index Cond: ((auction_id = oe.auction_id) AND (token = o.buy_token))
SubPlan 2
-> Index Scan using auction_prices_pkey on auction_prices ap_1 (cost=0.58..35.00 rows=16 width=11) (actual time=1.596..1.596 rows=1 loops=1323)
Index Cond: ((auction_id = oe.auction_id) AND (token = o.sell_token))
-> Subquery Scan on "*SELECT* 2" (cost=2.09..150.26 rows=1 width=100) (actual time=0.010..0.013 rows=0 loops=1)
-> Nested Loop (cost=2.09..150.25 rows=1 width=100) (actual time=0.009..0.012 rows=0 loops=1)
Join Filter: (j.uid = t_1.order_uid)
-> Nested Loop (cost=1.54..79.45 rows=1 width=192) (actual time=0.009..0.010 rows=0 loops=1)
-> Nested Loop Anti Join (cost=0.98..74.32 rows=1 width=127) (actual time=0.009..0.010 rows=0 loops=1)
-> Index Scan using jit_user_order_creation_timestamp on jit_orders j (cost=0.42..33.62 rows=8 width=127) (actual time=0.008..0.009 rows=0 loops=1)
Index Cond: (owner = '\x10dad59905d93ca37cd25a35f25349cb5956ba8e'::bytea)
-> Index Only Scan using orders_pkey on orders o_1 (cost=0.56..5.08 rows=1 width=57) (never executed)
Index Cond: (uid = j.uid)
Heap Fetches: 0
-> Index Only Scan using order_rewards_pkey on order_execution oe_1 (cost=0.56..5.11 rows=3 width=65) (never executed)
Index Cond: (order_uid = j.uid)
Heap Fetches: 0
-> Index Scan using trade_order_uid on trades t_1 (cost=0.56..0.74 rows=2 width=81) (never executed)
Index Cond: (order_uid = oe_1.order_uid)
SubPlan 5
-> Index Scan using auction_prices_pkey on auction_prices ap_2 (cost=0.58..35.00 rows=16 width=11) (never executed)
Index Cond: ((auction_id = oe_1.auction_id) AND (token = j.buy_token))
SubPlan 6
-> Index Scan using auction_prices_pkey on auction_prices ap_3 (cost=0.58..35.00 rows=16 width=11) (never executed)
Index Cond: ((auction_id = oe_1.auction_id) AND (token = j.sell_token))
Planning Time: 4.788 ms
Execution Time: 130157.850 ms
```
</p>
</details>
<details><summary>After</summary>
<p>
```
Aggregate (cost=14614.00..14614.01 rows=1 width=8) (actual time=1901.439..1902.539 rows=1 loops=1)
Buffers: shared hit=11838 read=4075
I/O Timings: shared read=2240.469
-> Gather Merge (cost=14557.86..14604.91 rows=404 width=136) (actual time=1900.725..1902.018 rows=917 loops=1)
Workers Planned: 2
Workers Launched: 2
Buffers: shared hit=11838 read=4075
I/O Timings: shared read=2240.469
-> Sort (cost=13557.83..13558.25 rows=168 width=136) (actual time=762.591..762.628 rows=306 loops=3)
Sort Key: "*SELECT* 2".uid
Sort Method: quicksort Memory: 49kB
Buffers: shared hit=11838 read=4075
I/O Timings: shared read=2240.469
Worker 0: Sort Method: quicksort Memory: 115kB
Worker 1: Sort Method: quicksort Memory: 25kB
-> Parallel Append (cost=2.25..13551.63 rows=168 width=136) (actual time=2.247..762.040 rows=306 loops=3)
Buffers: shared hit=11824 read=4075
I/O Timings: shared read=2240.469
-> Subquery Scan on "*SELECT* 2" (cost=24.42..13550.79 rows=53 width=136) (actual time=3.496..1892.483 rows=709 loops=1)
Buffers: shared hit=9702 read=3415
I/O Timings: shared read=1854.511
-> Nested Loop Left Join (cost=24.42..13550.26 rows=53 width=136) (actual time=3.495..1892.175 rows=709 loops=1)
Buffers: shared hit=9702 read=3415
I/O Timings: shared read=1854.511
-> Nested Loop (cost=23.84..13414.32 rows=53 width=155) (actual time=1.455..762.309 rows=709 loops=1)
Buffers: shared hit=8050 read=1521
I/O Timings: shared read=736.402
-> Nested Loop (cost=23.28..12573.83 rows=195 width=245) (actual time=1.418..667.512 rows=709 loops=1)
Buffers: shared hit=5387 read=1337
I/O Timings: shared read=647.346
-> Nested Loop (cost=22.73..9169.44 rows=796 width=180) (actual time=1.397..584.403 rows=711 loops=1)
Buffers: shared hit=2649 read=1211
I/O Timings: shared read=570.801
-> Bitmap Heap Scan on onchain_placed_orders opo (cost=22.17..2343.76 rows=796 width=57) (actual time=0.699..6.255 rows=711 loops=1)
Recheck Cond: (sender = '\x8ef4fb956d0cb06ca9e3db76040f08154e8d0122'::bytea)
Buffers: shared hit=56 read=248
I/O Timings: shared read=2.028
-> Bitmap Index Scan on order_sender (cost=0.00..21.97 rows=796 width=0) (actual time=0.044..0.044 rows=711 loops=1)
Index Cond: (sender = '\x8ef4fb956d0cb06ca9e3db76040f08154e8d0122'::bytea)
Buffers: shared hit=4
-> Index Scan using orders_pkey on orders o (cost=0.56..8.57 rows=1 width=123) (actual time=0.812..0.812 rows=1 loops=711)
Index Cond: (uid = opo.uid)
Filter: (owner <> '\x8ef4fb956d0cb06ca9e3db76040f08154e8d0122'::bytea)
Buffers: shared hit=2593 read=963
I/O Timings: shared read=568.773
-> Index Only Scan using order_rewards_pkey on order_execution oe (cost=0.56..4.25 rows=3 width=65) (actual time=0.115..0.115 rows=1 loops=711)
Index Cond: (order_uid = o.uid)
Heap Fetches: 16
Buffers: shared hit=2738 read=126
I/O Timings: shared read=76.545
-> Index Only Scan using trades_covering on trades t (cost=0.56..4.29 rows=2 width=81) (actual time=0.132..0.133 rows=1 loops=709)
Index Cond: (order_uid = o.uid)
Heap Fetches: 0
Buffers: shared hit=2663 read=184
I/O Timings: shared read=89.056
-> Index Scan using auction_prices_pkey on auction_prices ap (cost=0.58..31.94 rows=16 width=40) (actual time=1.590..1.590 rows=1 loops=709)
Index Cond: ((auction_id = oe.auction_id) AND (token = CASE o.kind WHEN 'sell'::orderkind THEN o.buy_token ELSE o.sell_token END))
Buffers: shared hit=1652 read=1894
I/O Timings: shared read=1118.109
-> Subquery Scan on "*SELECT* 1" (cost=2.25..4428.09 rows=350 width=136) (actual time=3.233..393.256 rows=208 loops=1)
Buffers: shared hit=2119 read=660
I/O Timings: shared read=385.958
-> Nested Loop (cost=2.25..4424.59 rows=350 width=136) (actual time=3.233..393.181 rows=208 loops=1)
Join Filter: (t_1.order_uid = o_1.uid)
Buffers: shared hit=2119 read=660
I/O Timings: shared read=385.958
-> Nested Loop Left Join (cost=1.70..3742.63 rows=156 width=149) (actual time=3.212..369.737 rows=208 loops=1)
Buffers: shared hit=1320 read=618
I/O Timings: shared read=364.992
-> Nested Loop (cost=1.12..3316.62 rows=156 width=188) (actual time=1.266..28.471 rows=208 loops=1)
Buffers: shared hit=849 read=49
I/O Timings: shared read=26.775
-> Index Only Scan using orders_owner_covering on orders o_1 (cost=0.56..187.36 rows=638 width=123) (actual time=1.247..2.194 rows=210 loops=1)
Index Cond: (owner = '\x8ef4fb956d0cb06ca9e3db76040f08154e8d0122'::bytea)
Heap Fetches: 1
Buffers: shared hit=24 read=7
I/O Timings: shared read=2.038
-> Index Only Scan using order_rewards_pkey on order_execution oe_1 (cost=0.56..4.87 rows=3 width=65) (actual time=0.124..0.124 rows=1 loops=210)
Index Cond: (order_uid = o_1.uid)
Heap Fetches: 4
Buffers: shared hit=825 read=42
I/O Timings: shared read=24.736
-> Index Scan using auction_prices_pkey on auction_prices ap_1 (cost=0.58..34.99 rows=16 width=40) (actual time=1.639..1.639 rows=1 loops=208)
Index Cond: ((auction_id = oe_1.auction_id) AND (token = CASE o_1.kind WHEN 'sell'::orderkind THEN o_1.buy_token ELSE o_1.sell_token END))
Buffers: shared hit=471 read=569
I/O Timings: shared read=338.217
-> Index Only Scan using trades_covering on trades t_1 (cost=0.56..4.29 rows=2 width=81) (actual time=0.109..0.109 rows=1 loops=208)
Index Cond: (order_uid = oe_1.order_uid)
Heap Fetches: 0
Buffers: shared hit=799 read=42
I/O Timings: shared read=20.966
-> Subquery Scan on "*SELECT* 3" (cost=2.67..94.60 rows=1 width=136) (actual time=0.012..0.014 rows=0 loops=1)
Buffers: shared hit=3
-> Nested Loop Left Join (cost=2.67..94.59 rows=1 width=136) (actual time=0.011..0.013 rows=0 loops=1)
Buffers: shared hit=3
-> Nested Loop (cost=2.09..91.83 rows=1 width=159) (actual time=0.011..0.013 rows=0 loops=1)
Buffers: shared hit=3
-> Nested Loop (cost=1.54..87.55 rows=1 width=208) (actual time=0.011..0.012 rows=0 loops=1)
Buffers: shared hit=3
-> Nested Loop Anti Join (cost=0.98..82.50 rows=1 width=127) (actual time=0.011..0.011 rows=0 loops=1)
Buffers: shared hit=3
-> Index Scan using jit_user_order_creation_timestamp on jit_orders j (cost=0.42..37.21 rows=9 width=127) (actual time=0.010..0.010 rows=0 loops=1)
Index Cond: (owner = '\x8ef4fb956d0cb06ca9e3db76040f08154e8d0122'::bytea)
Buffers: shared hit=3
-> Index Only Scan using orders_pkey on orders o_2 (cost=0.56..5.02 rows=1 width=57) (never executed)
Index Cond: (uid = j.uid)
Heap Fetches: 0
-> Index Only Scan using trades_covering on trades t_2 (cost=0.56..5.03 rows=2 width=81) (never executed)
Index Cond: (order_uid = j.uid)
Heap Fetches: 0
-> Index Only Scan using order_rewards_pkey on order_execution oe_2 (cost=0.56..4.25 rows=3 width=65) (never executed)
Index Cond: (order_uid = t_2.order_uid)
Heap Fetches: 0
-> Index Scan using auction_prices_pkey on auction_prices ap_2 (cost=0.58..35.01 rows=16 width=40) (never executed)
Index Cond: ((auction_id = oe_2.auction_id) AND (token = CASE j.kind WHEN 'sell'::orderkind THEN j.buy_token ELSE j.sell_token END))
Planning:
Buffers: shared hit=892 read=21
I/O Timings: shared read=14.023
Planning Time: 18.375 ms
Execution Time: 1902.681 ms
```
</p>
</details>
## How to test
Due to the fact that floating point addition is not commutative and the
order specified in the old query is not deterministic (the ORDER BY uid
is merely an approximation that matches) the validation script leaves
some room for differences, 1e-9 to be precise.
<details><summary>Validation script</summary>
<p>
```
#!/usr/bin/env python3
"""
Compare original and optimized surplus queries for correctness.
Picks random addresses and verifies both queries return identical results.
"""
import os
import sys
import psycopg2
from psycopg2 import sql
from decimal import Decimal
# Connection settings - override with environment variables
DB_CONFIG = {
"host": os.getenv("DB_HOST", "localhost"),
"port": os.getenv("DB_PORT", "5432"),
"dbname": os.getenv("DB_NAME", "your_database"),
"user": os.getenv("DB_USER", "your_user"),
"password": os.getenv("DB_PASSWORD", ""),
}
ORIGINAL_QUERY = """
WITH regular_orders AS (
SELECT ARRAY_AGG(uid) AS ids FROM orders WHERE owner = $1
),
onchain_orders AS (
SELECT ARRAY_AGG(uid) AS ids FROM onchain_placed_orders WHERE sender = $1
),
trade_components AS (
SELECT
CASE kind
WHEN 'sell' THEN t.buy_amount
WHEN 'buy' THEN t.sell_amount - t.fee_amount
END AS trade_amount,
CASE kind
WHEN 'sell' THEN (t.sell_amount - t.fee_amount) * o.buy_amount / o.sell_amount
WHEN 'buy' THEN t.buy_amount * o.sell_amount / o.buy_amount
END AS limit_amount,
o.kind,
CASE kind
WHEN 'sell' THEN (SELECT price FROM auction_prices ap WHERE ap.token = o.buy_token AND ap.auction_id = oe.auction_id)
WHEN 'buy' THEN (SELECT price FROM auction_prices ap WHERE ap.token = o.sell_token AND ap.auction_id = oe.auction_id)
END AS surplus_token_native_price
FROM orders o
JOIN trades t ON o.uid = t.order_uid
JOIN order_execution oe ON o.uid = oe.order_uid
WHERE o.uid = ANY(ARRAY_CAT((SELECT ids FROM regular_orders), (SELECT ids FROM onchain_orders)))
UNION ALL
SELECT
CASE j.kind
WHEN 'sell' THEN t.buy_amount
WHEN 'buy' THEN t.sell_amount - t.fee_amount
END AS trade_amount,
CASE j.kind
WHEN 'sell' THEN (t.sell_amount - t.fee_amount) * j.buy_amount / j.sell_amount
WHEN 'buy' THEN t.buy_amount * j.sell_amount / j.buy_amount
END AS limit_amount,
j.kind,
CASE j.kind
WHEN 'sell' THEN (SELECT price FROM auction_prices ap WHERE ap.token = j.buy_token AND ap.auction_id = oe.auction_id)
WHEN 'buy' THEN (SELECT price FROM auction_prices ap WHERE ap.token = j.sell_token AND ap.auction_id = oe.auction_id)
END AS surplus_token_native_price
FROM jit_orders j
JOIN trades t ON j.uid = t.order_uid
JOIN order_execution oe ON j.uid = oe.order_uid
WHERE j.owner = $1 AND NOT EXISTS (
SELECT 1
FROM orders o
WHERE o.uid = j.uid
)
),
trade_surplus AS (
SELECT
CASE kind
WHEN 'sell' THEN (trade_amount - limit_amount) * surplus_token_native_price
WHEN 'buy' THEN (limit_amount - trade_amount) * surplus_token_native_price
END / POWER(10, 18) AS surplus_in_wei
FROM trade_components
)
SELECT
COALESCE(SUM(surplus_in_wei), 0) AS total_surplus_in_wei
FROM trade_surplus;
"""
OPTIMIZED_QUERY = """
WITH trade_components AS (
-- Regular orders: join trades first, then order_execution
SELECT
o.uid,
CASE o.kind
WHEN 'sell' THEN t.buy_amount
WHEN 'buy' THEN t.sell_amount - t.fee_amount
END AS trade_amount,
CASE o.kind
WHEN 'sell' THEN (t.sell_amount - t.fee_amount) * o.buy_amount / o.sell_amount
WHEN 'buy' THEN t.buy_amount * o.sell_amount / o.buy_amount
END AS limit_amount,
o.kind,
ap.price AS surplus_token_native_price
FROM orders o
JOIN trades t ON t.order_uid = o.uid
JOIN order_execution oe ON oe.order_uid = t.order_uid
LEFT JOIN auction_prices ap
ON ap.auction_id = oe.auction_id
AND ap.token = CASE o.kind WHEN 'sell' THEN o.buy_token ELSE o.sell_token END
WHERE o.owner = $1
UNION ALL
-- Onchain placed orders (if sender differs from owner)
SELECT
o.uid,
CASE o.kind
WHEN 'sell' THEN t.buy_amount
WHEN 'buy' THEN t.sell_amount - t.fee_amount
END AS trade_amount,
CASE o.kind
WHEN 'sell' THEN (t.sell_amount - t.fee_amount) * o.buy_amount / o.sell_amount
WHEN 'buy' THEN t.buy_amount * o.sell_amount / o.buy_amount
END AS limit_amount,
o.kind,
ap.price AS surplus_token_native_price
FROM onchain_placed_orders opo
JOIN orders o ON o.uid = opo.uid AND o.owner != $1
JOIN trades t ON t.order_uid = o.uid
JOIN order_execution oe ON oe.order_uid = t.order_uid
LEFT JOIN auction_prices ap
ON ap.auction_id = oe.auction_id
AND ap.token = CASE o.kind WHEN 'sell' THEN o.buy_token ELSE o.sell_token END
WHERE opo.sender = $1
UNION ALL
-- JIT orders
SELECT
j.uid,
CASE j.kind
WHEN 'sell' THEN t.buy_amount
WHEN 'buy' THEN t.sell_amount - t.fee_amount
END AS trade_amount,
CASE j.kind
WHEN 'sell' THEN (t.sell_amount - t.fee_amount) * j.buy_amount / j.sell_amount
WHEN 'buy' THEN t.buy_amount * j.sell_amount / j.buy_amount
END AS limit_amount,
j.kind,
ap.price AS surplus_token_native_price
FROM jit_orders j
JOIN trades t ON t.order_uid = j.uid
JOIN order_execution oe ON oe.order_uid = t.order_uid
LEFT JOIN auction_prices ap
ON ap.auction_id = oe.auction_id
AND ap.token = CASE j.kind WHEN 'sell' THEN j.buy_token ELSE j.sell_token END
WHERE j.owner = $1
AND NOT EXISTS (SELECT 1 FROM orders o WHERE o.uid = j.uid)
)
SELECT
COALESCE(SUM(surplus_in_wei ORDER BY uid), 0) AS t…
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
No description provided.