Skip to content

Prep #24: Add reproducible Blackwell experiment run pack #37

Description

@inaniloquentee

Background

Prerequisite for #24, assuming the Phase 1-7 roadmap foundation in #6 is already available.

#24 should run Blackwell/B-series post-training baseline and candidate A/B experiments. Before those runs, vime needs a small, reproducible experiment pack that fixes workload, hardware target, launch mode, artifact locations, and mismatch-knob order while reusing the existing Phase 1-7 audit, provenance, profiling, CI, and report mechanisms.

This issue owns #24-specific run preparation only. It does not create new framework telemetry, provenance, operator-comparison, distributed-contract, CI, or release-report machinery.

Scope

  • Add a checked-in [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 experiment/runbook entry for P0 smoke, P1 stable baseline, and P2 production-like split.
  • Provide config templates or launch snippets for the recommended Qwen3-30B-A3B Megatron + vLLM setup.
  • Specify required run inputs: checkpoint paths, data path, model family, Docker/image ID, commit hash, hardware type, node/GPU count, colocate vs non-colocate mode, and RL-Kernel candidate toggle when present.
  • Specify where Phase 1-7 artifacts should be collected for each run: config snapshot, logs, wandb links, audit/report outputs, profiler outputs, runtime provenance, and summary tables.
  • Add a dry-run/config-validation path where possible so command generation can be tested without owning B-card hardware.
  • Document the exact [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 mismatch knob order: near-aligned sanity, FP8 rollout, sampling filters, combined mismatch, colocate/non-colocate, then candidate operator path.

Non-Goals

Acceptance Criteria

  • A user can identify the exact command/config set for P0/P1/P2 [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 runs.
  • The run pack records where existing Phase 1-7 artifacts are expected, without introducing a second report format.
  • The dry-run/config-validation path can run on a developer machine or clearly reports which hardware-backed fields are unavailable.
  • The issue explicitly separates setup artifacts from final Blackwell/B-card experimental results.

Hardware Notes

  • Most of this issue can be developed on CPU or a single-card machine.
  • Final validation of P0/P1/P2 configs requires B200/GB200 hardware because the target workload and FP8 rollout path are hardware-specific.

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    consistencyRollout-training consistencyenhancementNew feature or requestrl-kernelRL-Kernel integrationtelemetryStructured telemetry and observability

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions