Skip to content

add pr2069 best 8xh100 submission package#2125

Open
tenet-diver wants to merge 1 commit into
openai:mainfrom
tenet-diver:submission/pr2069-best-8xh100-20260501T153233Z
Open

add pr2069 best 8xh100 submission package#2125
tenet-diver wants to merge 1 commit into
openai:mainfrom
tenet-diver:submission/pr2069-best-8xh100-20260501T153233Z

Conversation

@tenet-diver
Copy link
Copy Markdown

@tenet-diver tenet-diver commented May 1, 2026

Post-deadline negative rerun of PR #2069 on 8xH100

This PR preserves an 8xH100 target-hardware rerun of the same candidate I submitted in PR #2069:

#2069

PR #2069 packaged the best completed deadline-search result from a 4xH100 machine as a non-record / unlimited-compute 16MB submission because I did not get access to an 8xH100 box before the deadline. I am opening this follow-up after the deadline because I finally got access to an 8xH100 machine and wanted to preserve the direct reproduction evidence for the same candidate, including the fact that the 8xH100 rerun did not beat the leaderboard baseline.

Candidate

Background

The original deadline search promoted the strongest completed 1xH100 candidate into a 4xH100 batch. The best completed promoted candidate was:

Candidate PR #2069 val_bpb Notes
best4x_ttt_disabled_qk525 1.26066159 QK gain 5.25, TTT disabled

This PR reruns that same candidate on the intended 8xH100 hardware.

Configuration

The rerun uses the same candidate configuration:

  • CANDIDATE_IMPL=autoregressive_gpt
  • QK_GAIN_INIT=5.25
  • TTT_ENABLED=0
  • TRAIN_BATCH_TOKENS=2097152
  • VOCAB_SIZE=1024
  • MAX_WALLCLOCK_SECONDS=600

Results

  • val_bpb: 1.23485583
  • val_loss: 2.08500235
  • artifact bytes: 15843310
  • model bytes: 15738419
  • code bytes: 104891
  • train seconds: 702.656
  • seed: 1337
  • packaged records directory: records/track_10min_16mb/2026-05-01_pr2069-best-8xh100_20260501T153233Z

Leaderboard Context

This is a negative result relative to the current 10min_16mb leaderboard. The upstream README lists the naive baseline at 1.2244, so this rerun is worse by 0.01045583 bpb. It is included only to document the target-hardware reproduction attempt for PR #2069.

Validation

  • python3 -m json.tool records/track_10min_16mb/2026-05-01_pr2069-best-8xh100_20260501T153233Z/submission.json >/dev/null
  • python3 -m py_compile records/track_10min_16mb/2026-05-01_pr2069-best-8xh100_20260501T153233Z/train_gpt.py

Compliance Notes

This is not meant to retroactively change the deadline status of PR #2069.

The package is under the 16MB artifact limit, but the single rerun took 702.656s and includes one seed. The generated submission.json therefore records:

  • artifact_under_16mb: true
  • train_under_600s: false
  • three_seeds: false

I am submitting this as post-deadline reproduction evidence for the same candidate, not as a compliant 10-minute record claim or leaderboard improvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant