feat(bench): use nanosecond resolution for benchmarks#3406
feat(bench): use nanosecond resolution for benchmarks#3406bikallem wants to merge 1 commit intomoonbitlang:mainfrom
Conversation
|
It's just a format issue? I see no strong reason to change ms to ns. |
not at all. It is the measurement resolution itself, i.e if the function/method being benchmarked measures in nanoseconds, the benchmark always gives
|
Update the bench library to work with nanoseconds instead of microseconds, enabling accurate measurement of sub-microsecond operations in `moon bench`. Changes: - monotonic_clock_wasm.mbt: multiply elapsed seconds by 1e9 (was 1e6) - monotonic_clock_js.mbt: multiply elapsed ms by 1e6 (was 1e3) - bench.mbt: rename iter_n_microseconds -> iter_n_nanoseconds, update warmup threshold from 100_000 (100ms in µs) to 100_000_000 (100ms in ns) Note: the native backend (`monotonic_clock_native.mbt`) needs no change since it directly calls `moonbit_monotonic_clock_stop` via extern "C", which is updated in the runtime to return nanoseconds. Coordinated with: - moonbitlang/moonbit-native-runtime: moonbit_monotonic_clock_stop returns ns - moonbitlang/moon: display code uses ns as base unit
08facf5 to
e81566b
Compare
| fn iter_count(name? : String, inner : () -> Unit, count : UInt) -> Summary { | ||
| let count = count.land(0x7FFFFFFF).reinterpret_as_int() | ||
| let threshold = 100000.0 // 100 ms | ||
| let threshold = 100000000.0 // 100 ms |
There was a problem hiding this comment.
🔴 Native backend monotonic_clock_end not updated for microsecond-to-nanosecond conversion
The PR converts benchmark timing from microseconds to nanoseconds for JS and WASM backends, but the native/llvm backend (bench/monotonic_clock_native.mbt:23) was not updated. The moonbit_monotonic_clock_stop C extern presumably still returns microseconds (matching the old contract that JS and WASM also returned microseconds). Since bench/bench.mbt:28 now sets threshold = 100000000.0 (100ms in nanoseconds) but native's clock still returns microseconds, the calibration loop in iter_count will interpret μs values as ns — making the threshold effectively 100 seconds instead of 100ms. This causes the warm-up phase to run ~1000× longer than intended on native/llvm targets, and all reported timing statistics will be in microseconds while the code expects nanoseconds.
Prompt for agents
The PR updates the JS backend (monotonic_clock_js.mbt, multiplier 1000.0 -> 1000000.0) and WASM backend (monotonic_clock_wasm.mbt, multiplier 1000000.0 -> 1000000000.0) to return nanoseconds instead of microseconds, but the native/llvm backend in bench/monotonic_clock_native.mbt was not touched. The extern C function moonbit_monotonic_clock_stop presumably still returns microseconds. Either the MoonBit wrapper in monotonic_clock_native.mbt needs to multiply the result by 1000.0 to convert from microseconds to nanoseconds, or the C runtime function itself needs to be updated. Without this fix, benchmarks on native/llvm targets will have the calibration threshold off by 1000x (interpreted as 100 seconds instead of 100ms) and all timing values will be in the wrong unit.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
Update the bench library to work with nanoseconds instead of microseconds, enabling
moon benchto accurately measure sub-microsecond operations.Changes
monotonic_clock_wasm.mbt: multiply elapsed seconds by1e9(was1e6)monotonic_clock_js.mbt: multiply elapsed milliseconds by1e6(was1e3)bench.mbt: renameiter_n_microseconds→iter_n_nanoseconds, update warmup threshold from100_000(100ms in µs) to100_000_000(100ms in ns)The native backend (
monotonic_clock_native.mbt) needs no change — it directly callsmoonbit_monotonic_clock_stopviaextern "C", which is updated in the runtime PR to return nanoseconds.Merge order
These three PRs must land in this order:
nstier toauto_select_unitBefore / After
Before (µs floor):
After (ns precision):
Test plan
moon benchon native target shows ns-resolution values for fast benchmarks (requires runtime#1)moon benchon wasm target shows correct values (wasm conversion factor updated)moon benchon js target shows correct values (js conversion factor updated)