benchflow-ai · bingran-you · Jul 13, 2026 · Jul 13, 2026
diff --git a/README.md b/README.md
@@ -101,7 +101,7 @@ uv run python scripts/run_article_suite.py \
   --model gpt-5.6-sol
 ```
 
-Run all four canonical models and rebuild the aggregate leaderboard:
+Run all four canonical models and rebuild the 10 leaderboard artifacts:
 
 ```bash
 uv run python scripts/run_article_suite.py \
@@ -124,32 +124,15 @@ The first OpenCode sweep across all nine article-derived tasks:
 | 3 | GPT-5.6 Sol | 39.38 |
 | 4 | GPT-5.4 Mini | -29.72 |
 
-See [`leaderboard/ARTICLE_SUITE.md`](leaderboard/ARTICLE_SUITE.md) for every
-per-task score and `leaderboard/article_suite.json` for the machine-readable
-leaderboard.
+See [`leaderboard/ARTICLE_SUITE.md`](leaderboard/ARTICLE_SUITE.md) for the nine
+task-specific leaderboards followed by the final average leaderboard.
+`leaderboard/article_suite.json` contains the same 10-board structure in
+machine-readable form.
 
 Scores are unbounded normalized values: `0` matches the public starter and
 `100` matches the trusted article-level reference. Negative scores are genuine
 regressions; scores above `100` exceed the reference.
 
-## Legacy Ant-Only Leaderboard
-
-![GenesisBench Simulation Heuristics Ant v1 leaderboard](leaderboard/simulation_heuristics_ant_v1_leaderboard.png)
-
-The table below is the historical Ant-only OpenHands sweep. It remains for
-provenance; new GenesisBench leaderboard runs use OpenCode and the nine-task
-article suite.
-
-| Rank | Model | Hidden-suite score |
-| ---: | --- | ---: |
-| 1 | GPT-5.6 Sol | 3417.86 |
-| 2 | GPT-5.5 | 2382.23 |
-| 3 | GPT-5.4 Mini | 2369.61 |
-| 4 | Claude Opus 4.8 | 2235.71 |
-
-These are single-run research results, not multi-trial estimates of model
-quality. See `leaderboard/REPORT.md` for setup details and limitations.
-
 ## Contribute a Task
 
 Create a scaffold:

diff --git a/docs/learning-beyond-gradients-suite.md b/docs/learning-beyond-gradients-suite.md
@@ -61,15 +61,25 @@ chat-completions gateway cannot faithfully transform Azure GPT-5.6 Sol tool
 calls. BenchFlow continues to own task staging, Daytona/Docker isolation, ACP
 trajectory capture, timing, and verifier execution.
 
-## Aggregate score
+## Leaderboard outputs
 
-The article-suite leaderboard reports every normalized task score and their
-unweighted arithmetic mean:
+The offline article-suite report contains 10 independent leaderboards in a
+fixed order:
+
+1. one leaderboard for each of the nine article-derived tasks;
+2. the final averaged leaderboard.
+
+The final score is the unweighted arithmetic mean:
 
 ```text
 average = sum(nine normalized task scores) / 9
 ```
 
+The repository README intentionally shows only the final averaged leaderboard.
+Detailed task rankings and score-artifact links live in
+`leaderboard/ARTICLE_SUITE.md`; the matching machine-readable structure lives
+in `leaderboard/article_suite.json`.
+
 The runner and resumable leaderboard builder live in:
 
 - `scripts/run_article_suite.py`

diff --git a/experiments/article_suite/README.md b/experiments/article_suite/README.md
@@ -77,12 +77,19 @@ is disabled for this suite because BenchFlow 0.6.5's chat-completions gateway
 cannot faithfully transform GPT-5.6 Sol tool calls; the run metadata records
 this limitation explicitly.
 
-## Build the aggregate leaderboard
+## Build all 10 leaderboards
 
 ```bash
 uv run python scripts/build_article_suite_leaderboard.py
 ```
 
-The aggregate score is the arithmetic mean of the nine normalized task scores.
+The builder writes:
+
+- `leaderboard/ARTICLE_SUITE.md`: nine task-specific leaderboards followed by
+  the final average leaderboard;
+- `leaderboard/article_suite.json`: the same 10 leaderboards plus the
+  model-centric score records used for reproducibility.
+
+The final aggregate is the arithmetic mean of the nine normalized task scores.
 Each task maps its starter policy to `0` and its trusted article-level reference
-to `100`.
+to `100`. The top-level repository README shows only this final aggregate.
diff --git a/leaderboard/ARTICLE_SUITE.md b/leaderboard/ARTICLE_SUITE.md
@@ -1,10 +1,115 @@
 # GenesisBench Learning Beyond Gradients Article Suite
 
-The aggregate score is the arithmetic mean of nine normalized task scores. Starter policies map to 0 and trusted article-level references map to 100.
-
-| Rank | Model | Harness | Effort | Average | Ant | Pong | Breakout RAM | Breakout RGB | HalfCheetah | Doom D1 | Doom D3 | Atari57 | Montezuma |
-| ---: | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
-| 1 | GPT-5.5 | opencode | xhigh | 43.19 | -16.89 | 45.83 | 95.60 | 70.76 | 2.74 | 85.20 | 5.48 | 0.00 | 100.00 |
-| 2 | Claude Opus 4.8 | opencode | max | 39.82 | 14.05 | 100.00 | 0.00 | 0.00 | 26.99 | 95.06 | 122.26 | 0.00 | 0.00 |
-| 3 | GPT-5.6 Sol | opencode | max | 39.38 | 39.20 | 52.92 | 100.00 | 70.76 | 15.87 | 49.41 | 26.24 | 0.00 | 0.00 |
-| 4 | GPT-5.4 Mini | opencode | xhigh | -29.72 | -31.36 | -75.00 | 2.52 | 15.52 | -8.17 | -161.12 | -9.86 | 0.00 | 0.00 |
+This offline report contains 10 independent leaderboards: one for each of the nine article-derived tasks, followed by the final nine-task average.
+
+Task scores are unbounded normalized values. The public starter maps to 0 and the trusted article-level reference maps to 100.
+
+## 1. Ant
+
+Task: `simulation_heuristics_ant_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | GPT-5.6 Sol | opencode | max | 39.20 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_ant_v1/score.json) |
+| 2 | Claude Opus 4.8 | opencode | max | 14.05 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_ant_v1/score.json) |
+| 3 | GPT-5.5 | opencode | xhigh | -16.89 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_ant_v1/score.json) |
+| 4 | GPT-5.4 Mini | opencode | xhigh | -31.36 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_ant_v1/score.json) |
+
+## 2. Pong
+
+Task: `simulation_heuristics_pong_ram_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | Claude Opus 4.8 | opencode | max | 100.00 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_pong_ram_v1/score.json) |
+| 2 | GPT-5.6 Sol | opencode | max | 52.92 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_pong_ram_v1/score.json) |
+| 3 | GPT-5.5 | opencode | xhigh | 45.83 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_pong_ram_v1/score.json) |
+| 4 | GPT-5.4 Mini | opencode | xhigh | -75.00 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_pong_ram_v1/score.json) |
+
+## 3. Breakout RAM
+
+Task: `simulation_heuristics_breakout_ram_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | GPT-5.6 Sol | opencode | max | 100.00 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_breakout_ram_v1/score.json) |
+| 2 | GPT-5.5 | opencode | xhigh | 95.60 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_breakout_ram_v1/score.json) |
+| 3 | GPT-5.4 Mini | opencode | xhigh | 2.52 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_breakout_ram_v1/score.json) |
+| 4 | Claude Opus 4.8 | opencode | max | 0.00 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_breakout_ram_v1/score.json) |
+
+## 4. Breakout RGB
+
+Task: `simulation_heuristics_breakout_rgb_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | GPT-5.5 | opencode | xhigh | 70.76 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_breakout_rgb_v1/score.json) |
+| 1 | GPT-5.6 Sol | opencode | max | 70.76 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_breakout_rgb_v1/score.json) |
+| 3 | GPT-5.4 Mini | opencode | xhigh | 15.52 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_breakout_rgb_v1/score.json) |
+| 4 | Claude Opus 4.8 | opencode | max | 0.00 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_breakout_rgb_v1/score.json) |
+
+## 5. HalfCheetah
+
+Task: `simulation_heuristics_halfcheetah_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | Claude Opus 4.8 | opencode | max | 26.99 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_halfcheetah_v1/score.json) |
+| 2 | GPT-5.6 Sol | opencode | max | 15.87 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_halfcheetah_v1/score.json) |
+| 3 | GPT-5.5 | opencode | xhigh | 2.74 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_halfcheetah_v1/score.json) |
+| 4 | GPT-5.4 Mini | opencode | xhigh | -8.17 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_halfcheetah_v1/score.json) |
+
+## 6. VizDoom D1
+
+Task: `simulation_heuristics_vizdoom_d1_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | Claude Opus 4.8 | opencode | max | 95.06 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_vizdoom_d1_v1/score.json) |
+| 2 | GPT-5.5 | opencode | xhigh | 85.20 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_vizdoom_d1_v1/score.json) |
+| 3 | GPT-5.6 Sol | opencode | max | 49.41 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_vizdoom_d1_v1/score.json) |
+| 4 | GPT-5.4 Mini | opencode | xhigh | -161.12 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_vizdoom_d1_v1/score.json) |
+
+## 7. VizDoom D3
+
+Task: `simulation_heuristics_vizdoom_d3_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | Claude Opus 4.8 | opencode | max | 122.26 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_vizdoom_d3_v1/score.json) |
+| 2 | GPT-5.6 Sol | opencode | max | 26.24 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_vizdoom_d3_v1/score.json) |
+| 3 | GPT-5.5 | opencode | xhigh | 5.48 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_vizdoom_d3_v1/score.json) |
+| 4 | GPT-5.4 Mini | opencode | xhigh | -9.86 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_vizdoom_d3_v1/score.json) |
+
+## 8. Atari57
+
+Task: `simulation_heuristics_atari57_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | Claude Opus 4.8 | opencode | max | 0.00 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_atari57_v1/score.json) |
+| 1 | GPT-5.4 Mini | opencode | xhigh | 0.00 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_atari57_v1/score.json) |
+| 1 | GPT-5.5 | opencode | xhigh | 0.00 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_atari57_v1/score.json) |
+| 1 | GPT-5.6 Sol | opencode | max | 0.00 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_atari57_v1/score.json) |
+
+## 9. Montezuma's Revenge
+
+Task: `simulation_heuristics_montezuma_v1`
+
+| Rank | Model | Harness | Effort | Normalized score | Score details |
+| ---: | --- | --- | --- | ---: | --- |
+| 1 | GPT-5.5 | opencode | xhigh | 100.00 | [JSON](article_suite_submissions/gpt-5.5/simulation_heuristics_montezuma_v1/score.json) |
+| 2 | Claude Opus 4.8 | opencode | max | 0.00 | [JSON](article_suite_submissions/claude-opus-4.8/simulation_heuristics_montezuma_v1/score.json) |
+| 2 | GPT-5.4 Mini | opencode | xhigh | 0.00 | [JSON](article_suite_submissions/gpt-5.4-mini/simulation_heuristics_montezuma_v1/score.json) |
+| 2 | GPT-5.6 Sol | opencode | max | 0.00 | [JSON](article_suite_submissions/gpt-5.6-sol/simulation_heuristics_montezuma_v1/score.json) |
+
+## 10. Nine-task average
+
+Arithmetic mean of the nine normalized task scores.
+
+| Rank | Model | Harness | Effort | Nine-task average |
+| ---: | --- | --- | --- | ---: |
+| 1 | GPT-5.5 | opencode | xhigh | 43.19 |
+| 2 | Claude Opus 4.8 | opencode | max | 39.82 |
+| 3 | GPT-5.6 Sol | opencode | max | 39.38 |
+| 4 | GPT-5.4 Mini | opencode | xhigh | -29.72 |
diff --git a/leaderboard/README.md b/leaderboard/README.md
@@ -9,7 +9,8 @@
 | 3 | GPT-5.6 Sol | 39.38 |
 | 4 | GPT-5.4 Mini | -29.72 |
 
-See [`ARTICLE_SUITE.md`](ARTICLE_SUITE.md) for all nine task columns.
+See [`ARTICLE_SUITE.md`](ARTICLE_SUITE.md) for 10 independent leaderboards:
+nine task-specific rankings followed by the final average ranking.
 
 ## Legacy Simulation Heuristics Ant v1