Time limit sensitivity should go both ways

[Matistjati]

Currently, if a submission is in time_limit_exceeded and gets runtime TL-epsilon, we get an error
ERROR TLE submission gemini.py (Python 3 w/Pypy) got AC (60) [CPU: 6.04s @ testcase secret/group2/029-g2-all_unique]
This is extremely un-nice, as when changing systems, it's unpredictable whether we will land above or below the time limit, giving us no good submission folder to place it in.

On the other hand, if a solution is in [TL, TL* safety margin], we only get a warning
WARNING TLE submission gemini2.cpp (C++) sensitive to time limit: limit of 6.0 secs -> TLE [CPU: 6.82s @ testcase secret/group1/020-g1-all_unique], limit of 12.0 secs -> AC (60) [CPU: 12.94s @ testcase secret/group2/031-g2-cheese2]

I would suggest that if a submission lands in [TL/safety margin, TL], then it's a warning, not an error. And we only error for [0, TL/safety margin).

[gkreitz]

A TLE case below the time limit is definitely an error in my mind (and how I read the spec, both legacy and 2025-09), so I definitely do not want to turn that into a warning rather than an error.

In fact, the current warning is likely too soft compared to the spec language. The intent, as I understand it, has always been that TLE submissions should run in > TL * time_safety_margin (or time_limit_to_tle` in the new version). I guess (without digging too much into history) the error was made a warning to make things close to TL * time_safety_margin not flip-flop between error and OK. I don't think the intent was ever to treat that warning as being OK to regularly hit.

[gkreitz]

Thinking a bit more about this - I wonder if we should improve the current check by making it harsher.

The current code just has a generic "sensitive to time limit" warning, which tells you if you get different results between TL and TL * safety_margin. I think it may be valuable to have a special case for TLE in that check, to make the messaging more clear.

Two ideas (I think both are good, but happy to take opinions):

• Change the "sensitive to time limit" message for submissions which should be TLE. Let the message be something to the effect that TLE submissions should have a runtime > time limit * safety margin. Word smithing suggestions very welcome, it's hard to phrase stuff like this in a one-liner which is comprehensible.
• Move the spot where we make this warning into an error, perhaps 50% into the safety margin. Regardless of where we put the breakpoint, if people are near it, they'll flip-flop. I think the current behavior of not flagging an error directly at time_limit * saftey margin is good (even if strict standard adherence probably would make this an error) to reduce flip-flopping.