The open arena for post-training: contribute agentic RL environments, then measure what the resulting model generalizes to unseen tasks.
Website · Authoring spec · Training pipeline · Architecture status · Contributing · Discord
Important
PostTrain Arena is a proposed NeurIPS 2026 competition. The checked-in organizer recipe now targets Qwen/Qwen3.5-9B, uses Qwen/Qwen3.5-397B-A17B teacher rollouts, and runs one-epoch LoRA SFT followed by LoRA GRPO. Both the older Qwen3-4B path and a Qwen3.5 Data Agent canary have end-to-end execution evidence; neither has yet demonstrated held-out model-quality lift.
Most competitions fix the environment and ask teams to submit an agent. PostTrain Arena inverts that contract: teams contribute environment corpora, and the organizers hold the post-training recipe and evaluation suite fixed.
team task corpus
→ fixed SFT + GRPO recipe
→ trained team checkpoint
→ sealed held-out evaluation
→ Δ over a fixed reference checkpoint
The headline track rewards environments that teach capabilities which transfer beyond their own training tasks—not environments that only improve in-domain performance.
| Track | What a team submits | Per-entry scale | Evaluation |
|---|---|---|---|
| Track 2 — Environment Submission (headline) | Containerized task packages: task.md + environment/ + verifier/ + oracle/ |
50 minimum / 100 recommended / 200 maximum | Managed SFT→GRPO, then held-out generalization delta |
| Track 1 — Skill Learning | Modular SKILL.md packages |
20 minimum / 50 recommended / 100 maximum | Pass@1 of a frozen reference agent |
- Scoring. Track 2 uses
Δ = pass rate(team checkpoint) − pass rate(reference checkpoint)on a sealed 100-task suite, with paired bootstrap confidence intervals. A 20-task public sample is reserved for sanity checks. - Phases. Phase 0 is a public-sample warm-up, Phase 1 provides development feedback, and Phase 2 freezes submissions for private evaluation. Teams may enter both tracks as separate entries.
- Open release. Under the draft rules, accepted environments, teacher data, and trained checkpoints are released openly while authors retain credit.
The checked-in implementation now includes the full public-data organizer recipe plus a live Qwen3.5 eight-train/three-eval canary. Competition-scale execution, measured lift, and the sealed evaluation remain pending.
| Surface | Current public status |
|---|---|
| Participant task format and local validation | Implemented — eight worked examples, structural checks, Docker oracle replay, and empty-trial rejection |
| BenchFlow task-list training and evaluation | Implemented — pinned snapshots, one verified teacher rollout per training task, one-epoch LoRA SFT, LoRA GRPO over the training set, held-out evaluation, and score reports |
| OpenCode agent harness | Implemented end to end — teacher collection, baseline/gate/final eval, benchmark matrices, and TRL custom GRPO rollouts use OpenCode; TRL synchronizes the pinned base and each trained policy to the shared vLLM endpoint |
| Public data | Available — 2,238 training tasks and 366 held-out evaluation tasks in native task.md format |
| OpenEnv protocol path | Implemented — served adapter, typed client, lifecycle tests, Docker parity validation, and a native-dataset end-to-end smoke |
| HF Jobs execution | Implemented — portable UV job bundles, pinned code refs, named-secret boundaries, status inspection, and Hub publishing; live scheduler allocation currently awaits HF credits |
| Continuous leaderboard | Implemented — atomic Hub dataset records and a deployable Gradio Space |
| Multi-benchmark evaluation | Implemented — one base/final checkpoint pair can be scored across pinned Data Agent and SkillsBench suites |
| Qwen3.5-9B organizer recipe | Implemented; corrected live rerun in progress — full 2,238-train/366-eval config is checked in; the 8x3 run proved orchestration but exposed a GRPO prompt-ID mismatch that this branch fixes |
| Demonstrated model-quality lift | Not yet — completed smokes validated system mechanics, not learning gains |
Note
On July 10, 2026, a real one-train/one-held-out run completed snapshotting, baseline evaluation, verifier-approved teacher collection, LoRA SFT, a forced GRPO step, final evaluation, and artifact publication through the earlier OpenEnv/TRL evaluation path. Scores remained 0.0 → 0.0, so this is evidence of end-to-end operability—not quality improvement and not validation of the newer OpenCode evaluation path. See the native-dataset OpenEnv smoke report.
On July 14, 2026, an eight-training-task/three-held-out-task Qwen3.5-9B
canary completed the current Docker + OpenCode path, including four verified
teacher trajectories, one-epoch LoRA SFT, 16 OpenCode GRPO rollouts,
checkpoint synchronization, and final held-out evaluation. Baseline, SFT,
and final held-out pass rate each measured 1/3, so delta remained 0.0.
A post-run audit found that the historical GRPO parser retokenized prompts
differently from the serving endpoint, so this run is not valid GRPO
learning evidence. The corrected path uses exact served IDs and policy
attestation.
See the
Qwen3.5 Data Agent canary.
For compatibility details and evidence boundaries, use Architecture and implementation status as the source of truth.
| Path | Purpose |
|---|---|
starting-kit/ |
Task template and organizer-authored examples |
submissions/ |
Team entries and submission.yaml contract |
scripts/ |
Self-contained structural checks and local Docker harness |
pipelines/benchflow-task-posttrain/ |
Public BenchFlow + OpenEnv + TRL training implementation |
docs/ |
Architecture, operator guide, and validation evidence |
The examples under starting-kit/ are reference material, not competition entries.
Local task authoring requires Python 3 and Docker. It does not require BenchFlow, a GPU, or provider API keys.
git clone https://github.com/benchflow-ai/posttrainarena.git
cd posttrainarena
mkdir -p submissions/your-team/envs
cp -R starting-kit/template submissions/your-team/envs/your-env-name
# Edit the task package and add submissions/your-team/submission.yaml.
python3 scripts/check_task.py submissions/your-team/envs
python3 scripts/check_submission.py
# The oracle must score 1.0.
scripts/run_local.sh submissions/your-team/envs/your-env-name
# A do-nothing trial must fail.
scripts/run_local.sh submissions/your-team/envs/your-env-name --skip-oracleSee CONTRIBUTING.md for the submission workflow, validation ladder, and reviewer checklist. Organizers and researchers should start with the training pipeline guide and HF Jobs handoff.
- Task authoring specification
- Architecture and implementation status
- Training pipeline operator guide
- OpenCode GRPO rollout contract
- OpenCode SFT-to-GRPO smoke
- Qwen3.5 OpenCode teacher canary
- Qwen3.5 Data Agent SFT-to-GRPO canary
- OpenCode evaluation canary
- Hugging Face Jobs and leaderboard handoff
- HF handoff validation report
- Native-dataset OpenEnv smoke report
- Starting-kit guide
- Team submission guide
- Security · Support · Code of conduct
Questions are welcome on Discord or by email at labs@benchflow.ai.
Repository contents are licensed under AGPL-3.0 unless noted otherwise. Under the draft competition rules, submissions use CC-BY-4.0 for text and data and Apache-2.0 for code.