Issue
The current timing path can reuse the same input tensor values across timed iterations. This allows a reproducible state-caching reward hack that produces an artificial latency advantage, applicable to all problems.
The existing ShiftingMemoryPoolAllocator changes tensor pointer addresses across timing iterations, which prevents simple pointer-based caching. However, the copied input values can remain the same across timed iterations. A reward hacking solution can therefore recognize repeated timed inputs by fingerprinting tensor contents rather than relying on raw pointer addresses.
Details
The reproduced exploit uses a compact fingerprint of the timed inputs and caches the corresponding output/state. After the first full computation, later timed iterations can detect the same repeated input values and reuse cached state instead of recomputing normally. This bypasses the current pointer-shifting defense because the tensor addresses change, but the underlying input values are still repeated.
Reproduction
We reproduced this on L2/057_residual_coupling_flow_block with a state-caching solution that:
- Computes normally for the first matching input.
- Stores a fingerprint and cached output/state.
- Recognizes the same repeated input values during timing.
- Reuses cached state instead of performing the full computation.
The solution passed the SOL-ExecBench submission flow and showed a large artificial speedup, with some runs approaching the theoretical upper bound of SOL = 1.
Proposed fix
I am preparing a PR that keeps the existing timing architecture but generates variable input tensors during timing iterations.
Issue
The current timing path can reuse the same input tensor values across timed iterations. This allows a reproducible state-caching reward hack that produces an artificial latency advantage, applicable to all problems.
The existing ShiftingMemoryPoolAllocator changes tensor pointer addresses across timing iterations, which prevents simple pointer-based caching. However, the copied input values can remain the same across timed iterations. A reward hacking solution can therefore recognize repeated timed inputs by fingerprinting tensor contents rather than relying on raw pointer addresses.
Details
The reproduced exploit uses a compact fingerprint of the timed inputs and caches the corresponding output/state. After the first full computation, later timed iterations can detect the same repeated input values and reuse cached state instead of recomputing normally. This bypasses the current pointer-shifting defense because the tensor addresses change, but the underlying input values are still repeated.
Reproduction
We reproduced this on L2/057_residual_coupling_flow_block with a state-caching solution that:
The solution passed the SOL-ExecBench submission flow and showed a large artificial speedup, with some runs approaching the theoretical upper bound of SOL = 1.
Proposed fix
I am preparing a PR that keeps the existing timing architecture but generates variable input tensors during timing iterations.