Skip to content

FP8 download tool: dual-source (ModelScope + HF), parallel, resume — for the 379 GB GLM-5.2-FP8 checkpoint #239

Description

@woolcoxm

TL;DR

New download_fp8.py tool for downloading the GLM-5.2-FP8 checkpoint (379 GB, 141 shards) with dual-source fallback (ModelScope primary, HuggingFace fallback), parallel shard download, stall recovery, and clean resume. Designed for the bandwidth-constrained, throttle-prone reality of downloading a 379 GB model.

Branch: windows-dev (commit 91b1035)


Why

The production int4 conversion requires the FP8 source checkpoint. At 379 GB across 141 shards, the download is the multi-day bottleneck of the entire pipeline. Two problems with the default HuggingFace download path:

  1. HuggingFace per-stream throttling: HF rate-limits individual streams after ~50 GB, dropping throughput to a crawl. There's no documented workaround for unauthenticated bulk downloads.
  2. No clean resume: hf_hub_download resumes by file, but a mid-shard interruption leaves a partial file that may or may not be revalidated correctly, and there's no visibility into which shards are complete.

ModelScope mirrors the same checkpoint (ZhipuAI/GLM-5.2-FP8) on a CDN that doesn't apply the same per-stream throttling, giving 3-5x higher sustained throughput in practice.

What it does

  • Dual-source: tries ModelScope first (ZhipuAI/GLM-5.2-FP8); if the API is unreachable, falls back to HuggingFace (zai-org/GLM-5.2-FP8). Force a source with --source ms or --source hf.
  • Parallel download: N worker threads (default 3) pull shards concurrently from a shared work queue.
  • Resume by size: skips any shard whose local file size matches the expected size. Partial shards are re-downloaded from scratch (shards are 5 GB each, so re-downloading one is cheap).
  • Stall recovery: 3 retry attempts per shard with exponential backoff; curl fallback (-C - for byte-range resume) if the Python download path fails.
  • Progress display: per-shard completion with running percentage, elapsed time, and ETA.

Usage

# Auto (ModelScope first, HF fallback), 3 parallel
python download_fp8.py

# 4 parallel, force ModelScope
python download_fp8.py --parallel 4 --source ms

# Force HuggingFace
python download_fp8.py --source hf

Re-running resumes — completed shards are skipped by size, only missing/partial ones are re-downloaded.

Impact on the program

Status

Download is in progress (79/141 shards complete as of this writing). Re-running the tool resumes from where it left off.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions