Generators: FP8 emission + expert unfusing + download tool

[woolcoxm]

Summary

Three changes to the test-model generators:

1. FP8 e4m3 emission — generators can now write weights as FP8 + 128×128 block scales in the exact layout of the real GLM-5.2-FP8 checkpoint, so the converter's FP8→int4 dequant path can be tested locally without the 379 GB download.
2. Expert weight unfusing — splits HF's fused 3-D gate_up_proj into the per-expert 2-D layout the real checkpoint, converter, and C engine all expect. Fixes a silent data-loss bug.
3. FP8 download tool — download_fp8.py with ModelScope+HF dual-source, parallel shards, resume.

Closes #238, closes #239.

Problem 1: No local FP8 fixture

The converter's FP8→int4 dequant path could only be exercised by downloading the 379 GB checkpoint. There was no way to test it locally, meaning dequant bugs would only surface during the multi-hour production conversion.

Fix: New glm_fp8_emit.py helper with fp8_block_quantize / fp8_block_dequantize / save_fp8_safetensors. Both generators gained --fp8 flag. The oracle round-trips weights through FP8 before computing the reference.

Problem 2: Fused expert weights (bug)

HF's GlmMoeDsaForCausalLM fuses gate_proj + up_proj into a 3-D gate_up_proj [E, 2M, I]. But the real checkpoint stores experts unfused as per-expert 2-D tensors:

model.layers.10.mlp.experts.0.gate_proj.weight       (2048, 6144)  FP8 e4m3
model.layers.10.mlp.experts.0.gate_proj.weight_scale_inv  (16, 48)  F32
model.layers.10.mlp.experts.0.up_proj.weight          (2048, 6144)  FP8 e4m3
...

The converter's ndim != 2 guard silently skipped the 3-D fused tensors. The engine then crashed with missing tensor: model.layers.3.mlp.experts.5.gate_proj.weight.

Fix: unfuse_experts() in glm_fp8_emit.py splits gate_up_proj [E, 2M, I] → per-expert gate_proj [M, I] + up_proj [M, I], and down_proj [E, I, M] → per-expert down_proj. Called after reference generation but before saving, in both generators.

Also fixed: make_glm_oracle.py FP8 round-trip guard p.dim() < 2 → p.dim() != 2 (3-D fused experts crashed fp8_block_quantize).

Problem 3: Download tool

The 379 GB FP8 download is the pipeline bottleneck. HuggingFace per-stream throttling drops throughput after ~50 GB. download_fp8.py tries ModelScope first (no throttling), falls back to HF, with parallel shards + resume.

Changes

File	Change
c/tools/glm_fp8_emit.py	NEW: FP8 block quantize/dequantize, unfuse_experts, save_fp8_safetensors
c/tools/make_glm_oracle.py	--fp8 flag, FP8 round-trip, unfuse, dim guard fix
c/tools/make_glm_bench_model.py	--fp8 flag, unfuse, manual config.json in FP8 path
c/download_fp8.py	NEW: dual-source parallel download with resume

Validation

generator --fp8 → 570 e4m3 tensors + 629 scale_inv (was 90 when fused)
converter --group-size 0   → per-row int4 (fmt=2), engine loads clean
converter --group-size 128 → grouped int4 (fmt=4), engine loads clean
dequant error: grouped 1.14-1.22x lower than per-row

Test plan

• Generator --fp8 produces valid FP8 checkpoint with per-expert 2-D tensors
• Converter ingests FP8 fixture, produces int4 output
• Engine loads converted model with 0 missing-tensor errors
• FP8 round-trip: dequant matches quant (maxdiff 0.0)
• Download tool resumes (79/141 → in progress)

[JustVugg]

Closing — like #243, this is already on dev, arrived via your own earlier commits.

I checked each file rather than guessing:

• c/download_fp8.py — identical to dev (came in with 69f65d5)
• c/tools/glm_fp8_emit.py — identical to dev (71c262c and 2c6946c)
• c/tools/make_glm_bench_model.py — identical to dev
• c/tools/make_glm_oracle.py — the only file that differs, and it differs because dev is ahead of this branch: it has fix: make_glm_oracle.py creates glm_tiny/ before saving (fresh-checkout failure) #318's glm_tiny/ mkdir and oracle: hard-fail on transformers < 5.11.0 (interleaved-RoPE floor, #281) #322's transformers version gate on top of your --fp8 work. That's why this PR now shows as conflicting: not because it's wrong, because it's behind itself.

Zero unique content left. The --fp8 path, the ref-after-round-trip design (computing ref_glm.json after the FP8 round-trip so the reference matches what the converter ingests — that's a careful piece of thinking, and exactly the trap #281 fell into from the other direction) and unfuse_experts are all live on dev.

Also closing #238 (FP8 emission + unfusing) and #239 (dual-source download tool) — I verified download_fp8.py has ModelScope + HF, resume, and parallel workers, so both are delivered.

Nothing lost here except the bookkeeping.