Skip to content

Revise article scoring and publish HTML leaderboards#19

Merged
bingran-you merged 1 commit into
mainfrom
bry/iqm-leaderboard-images
Jul 13, 2026
Merged

Revise article scoring and publish HTML leaderboards#19
bingran-you merged 1 commit into
mainfrom
bry/iqm-leaderboard-images

Conversation

@bingran-you

Copy link
Copy Markdown
Collaborator

Summary

  • replace the arithmetic-mean primary rank with IQM while retaining mean and median diagnostics
  • publish native raw-score task leaderboards and a positive IQM + 100 display index for the final plot
  • render all website leaderboards as responsive HTML/CSS from synchronized JSON rather than raster images
  • document exact model routes and provider-specific reasoning settings (max / xhigh)
  • preserve the latest main hero artwork and nine-task navigator

Inference settings

Model Route Reasoning setting
GPT-5.6 Sol azure/gpt-5.6-sol max
GPT-5.5 azure/gpt-5.5 xhigh
Claude Opus 4.8 anthropic/claude-opus-4-8 via Claude OAuth max
GPT-5.4 Mini azure/gpt-5.4-mini xhigh

These are provider-specific categorical labels, not a shared numeric compute scale.

Validation

  • uv run pytest -q — 92 passed, 18 skipped
  • uv run ruff check .
  • uv build
  • uv run python scripts/validate_tasks.py — all nine tasks valid
  • desktop and mobile browser checks: 9 task charts, 4 final rows, no chart images, no overflow or console errors

@vercel

vercel Bot commented Jul 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
genesis-bench Ready Ready Preview, Comment Jul 13, 2026 10:28pm

Request Review

@bingran-you bingran-you merged commit 53e16c0 into main Jul 13, 2026
3 checks passed
@bingran-you bingran-you deleted the bry/iqm-leaderboard-images branch July 13, 2026 22:32

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e032bfc483

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +355 to +356
"positive_display_score": row["final_normalized_score"]
+ FINAL_DISPLAY_OFFSET,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle final display scores below zero

For any future model whose IQM is <= -100, which is allowed because the normalized scores are explicitly unbounded, this + 100 transform produces a non-positive positive_display_score. The new HTML and PNG final leaderboards both treat this field as a bar width/value from a zero lower bound, so a valid very-low-scoring model would be clipped or omitted instead of showing its meaningful negative gap; use a plotting domain that can include negative display values or derive an offset that is guaranteed for the current cohort.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant