Skip to content

Rename database benchmark to AXArena-Database - #216

Open
chenmingtang830 wants to merge 3 commits into
mainfrom
codex/axeval-database-naming
Open

Rename database benchmark to AXArena-Database#216
chenmingtang830 wants to merge 3 commits into
mainfrom
codex/axeval-database-naming

Conversation

@chenmingtang830

@chenmingtang830 chenmingtang830 commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • rename the public database benchmark from DAEB / AXArena Database to AXArena-Database
  • move the canonical benchmark path to ax-arena/benchmark/axarena-database and rename CLI commands to axarena-database-*
  • define stable publication identity benchmark: axarena-database plus display_name: AXArena-Database
  • retain legacy DAEB command aliases and read-path compatibility during migration

Migration safety

  • no frozen pack or approval bytes changed
  • suite_version remains 1
  • internal frozen IDs such as daeb-1-v1 remain provenance identifiers
  • this should merge before the first fresh production execution; downstream AXArena and axarena-internal PRs switch at the same boundary

Validation

  • npm run typecheck
  • npm run build
  • package-content checks for core and arena
  • focused CLI/release-gate tests: 45 passed
  • full core suite: 491 passed
  • full arena run on this machine remains affected by process/filesystem timeouts; after committing the canonical path, focused publication tests report only timeouts and no naming/path assertion failures (CI is authoritative)

@github-actions

Copy link
Copy Markdown

Normalized records diff

Overall is a macro-average of participating surface scores. Agents are never averaged together.

Overall by agent

Product Agent Base pass@1 Head pass@1 Δ Base pass³ Head pass³ Surfaces
fixture-db claude-code 92.9% 92.9% +0.0 pp 78.6% (11/14) 78.6% (11/14) 2
fixture-db codex 78.6% 78.6% +0.0 pp 64.3% (9/14) 64.3% (9/14) 2

Surface cells

Product Agent Surface Base pass@1 Head pass@1 Δ Base pass³ Head pass³ Latency ms Cost USD Tokens Harness version
fixture-db claude-code api 85.7% 85.7% +0.0 pp 71.4% (5/7) 71.4% (5/7) 1100 → 1100 0.4200 → 0.4200 3350 → 3350 2.1.5 → 2.1.5
fixture-db claude-code cli 100.0% 100.0% +0.0 pp 85.7% (6/7) 85.7% (6/7) 1300 → 1300 0.5100 → 0.5100 3720 → 3720 2.1.5 → 2.1.5
fixture-db codex api 85.7% 85.7% +0.0 pp 71.4% (5/7) 71.4% (5/7) 1200 → 1200 — → — 3600 → 3600 0.121.0 → 0.121.0
fixture-db codex cli 71.4% 71.4% +0.0 pp 57.1% (4/7) 57.1% (4/7) 1500 → 1500 — → — 4100 → 4100 0.121.0 → 0.121.0

Operational metrics are context only and never affect ranking.

@chenmingtang830 chenmingtang830 changed the title Rename database benchmark to AXeval-Database Rename database benchmark to AXArena-Database Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant