Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
c969146
perf: store per-module constant prefix trees in oleans
Kha Jul 11, 2026
2abb3e2
perf: search trie children via per-node hash arrays
Kha Jul 11, 2026
e957cab
perf: select trie children via slot table and fingerprint bytes
Kha Jul 11, 2026
526deae
chore: keep `SMap`-compatible field names on `ConstMap`
Kha Jul 12, 2026
1673d79
chore: keep `Environment.const2ModIdx` as a derived accessor
Kha Jul 12, 2026
5142203
perf: cache imported-constant lookups in a process-global C++ table
Kha Jul 12, 2026
a2742bc
chore: extend `ImportedConsts` with `HashMap`-style accessors
Kha Jul 12, 2026
4aa7198
fix: make imported-constant cache safe under parallel elaboration
Kha Jul 12, 2026
58c655e
fix: bound imported-constant cache memory via open addressing
Kha Jul 12, 2026
772e7cc
perf: inline imported-constant cache accessors
Kha Jul 12, 2026
255e10e
perf: avoid per-child allocations in imported-constant tree merge
Kha Jul 13, 2026
40cb1ca
chore: add `ImportedConsts.foldM` for downstream compatibility
Kha Jul 13, 2026
e5ae1c9
fix: memoize `Environment.const2ModIdx` per import set
Kha Jul 13, 2026
de7e9f4
perf: merge the imported extra-constants view lazily
Kha Jul 15, 2026
5460173
perf: mark cached lookup values persistent via a thread-safe variant
Kha Jul 15, 2026
291f428
perf: avoid imported-constant lookups for names that cannot come from…
Kha Jul 15, 2026
9afc426
perf: look up code-generator names in the extra-constants view first
Kha Jul 15, 2026
ef2b08c
perf: keep lazy views and `const2ModIdx` out of compacted snapshots
Kha Jul 15, 2026
2409b4a
perf: size the imported-constant lookup cache to the import set
Kha Jul 15, 2026
7e710c4
perf: grow the imported-constant lookup cache with its working set
Kha Jul 16, 2026
5d27b34
perf: merge the imported constants view lazily
Kha Jul 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions .claude/skills/radar-results/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
name: radar-results
description: Retrieve Lean benchmark (radar/speedcenter) results for a commit or PR as JSON and rank per-module regressions. Use when asked to analyze bench CI results, find the most affected modules/benchmarks, or compare two commits' performance.
allowed-tools: Bash
---

# Retrieving radar benchmark results

Benchmark CI results live on https://radar.lean-lang.org. The web UI is an SPA, but there is a
plain JSON API under `/api`.

## Finding the comparison hashes

The `leanprover-radar` bot posts a PR comment like:

> [Benchmark results](https://radar.lean-lang.org/repos/lean4/commits/COMMIT?reference=REFERENCE) for COMMIT against REFERENCE are in.

Fetch it with `gh api repos/leanprover/lean4/issues/<PR>/comments --jq '.[] | select(.user.login == "leanprover-radar") | .body'`.
The two hashes are the benched commit (`COMMIT`, usually the PR head) and the baseline (`REFERENCE`, usually the merge-base).

## Waiting for a running bench: results arrive by comment EDIT

When a bench is submitted, the bot immediately posts a placeholder comment:

> Benchmarking COMMIT against REFERENCE ([preliminary results](...)).
> React with :eyes: to be notified when the results are in.

The final results are added by **editing this same comment in place** (body becomes "[Benchmark
results](...) for COMMIT against REFERENCE are in."), so polling for *new* comments misses them.
Note the placeholder's comment `id` and poll that comment until its body starts with
`[Benchmark results]`. Do NOT match on "are in" — the placeholder's footer ("...when the results
are in") already contains it.

```bash
while :; do
body=$(gh api repos/leanprover/lean4/issues/comments/<ID> -q .body)
case $body in "[Benchmark results]"*) break;; esac
sleep 300
done
```

Runs typically complete in ~15–60 minutes. The mathlib bench (`!bench mathlib`) posts a separate
comment for the `mathlib4-nightly-testing` repo, same edit-in-place mechanism.

## Fetching the comparison

```bash
curl -s "https://radar.lean-lang.org/api/compare/lean4/<REFERENCE>/<COMMIT>/" -o cmp.json
```

(first path segment after `compare/` is the repo name; then baseline, then new commit.)

Top-level structure: `{chashFirst, chashSecond, comparison}`, where `comparison` has
`significant`, `warnings`, `notes`, `newMetrics`, `largeChanges`, `mediumChanges`, `smallChanges`,
and the full `measurements` array (~18k entries). Each measurement:

```json
{"metric": "build/module/Init.Data.Sum//instructions",
"first": 0.49e9, "second": 1.21e9,
"firstSource": "runner-lean1", "secondSource": "runner-lean1",
"unit": null, "direction": -1}
```

`direction: -1` means lower is better. `first` = baseline value, `second` = new value.

## Useful metric families

- `build//instructions`, `build//cycles`, `build//wall-clock` — whole stdlib build totals.
- `build/module/<Module>//instructions|cycles` — per-module stdlib compile cost. Best signal for
"which modules are most affected".
- `build/module/<Module>//bytes .olean|.olean.private|.olean.server|.ilean` — per-module artifact sizes.
- `misc/import <Module>//instructions`, `misc/re-elab <Module>//...` — import/re-elaboration benchmarks.
- `lake/...`, `other//...` — Lake and micro benchmarks.

## Ranking regressions

```bash
python3 - <<'EOF'
import json
ms = json.load(open('cmp.json'))['comparison']['measurements']
instr = [m for m in ms if m['metric'].endswith('//instructions') and m['first'] and m['second']]
for key, label in [(lambda m: m['second']/m['first'], 'ratio'),
(lambda m: m['second']-m['first'], 'absolute')]:
instr.sort(key=key, reverse=True)
print(f"top 15 by {label}:")
for m in instr[:15]:
print(f" {m['metric']:70} {m['first']/1e9:10.2f}G -> {m['second']/1e9:10.2f}G {m['second']/m['first']:.2f}x")
EOF
```

A near-constant absolute offset across many small modules points at per-compile overhead (e.g.
import-time work); a multiplicative factor on elaboration-heavy modules points at per-operation
cost (e.g. environment lookups).
Loading
Loading