Skip to content

Feat: improve downloads of models under unstable connections - #549

Open
stdevPavelmc wants to merge 1 commit into
ROCm:mainfrom
stdevPavelmc:retry_downloads_issue-520
Open

Feat: improve downloads of models under unstable connections#549
stdevPavelmc wants to merge 1 commit into
ROCm:mainfrom
stdevPavelmc:retry_downloads_issue-520

Conversation

@stdevPavelmc

Copy link
Copy Markdown

This fixes #520 with some improvements

  • Add CURL resume support for partial downloads
  • Track progress correctly when resuming existing files
  • Increase retry attempts and use exponential backoff
  • Add remote file size lookup for resume logic
  • Refactor pull flow so partial and missing files are handled together
  • Remove redundant pre-download checks from main flow

Example of a session (this session was started and interrupted before several times to test and make an example of the features, it has 1 missing file and 2 partial downloads:

pavel@agatha:~$ flm pull qwen3.5:9b
[FLM]  Model: qwen3.5:9b
[FLM]  Name: Qwen3.5-9B-NPU2
[FLM]  Missing files (1):
  - chat_template.jinja
[FLM]  Present files (5):
  - config.json
  - model.q4nx
  - tokenizer.json
  - tokenizer_config.json
  - vision_weight.q4nx
[FLM]  Checking all files for missing or partial downloads...
[FLM]  Partial download detected: model.q4nx (374 MB / 7534 MB)
[FLM]  Partial download detected: vision_weight.q4nx (512 MB / 980 MB)
[FLM]  Downloading 3 file(s)...
[FLM]  Files to download (7627.10 MB): 
  - model.q4nx (7159.44 MB)
  - vision_weight.q4nx (467.66 MB)
  - chat_template.jinja (0.01 MB)
[FLM]  Downloading 1/3: model.q4nx
[FLM]  Resuming from 374.641 MB
[FLM]  Downloading: 21.4% (1611.6MB / 7534.1MB)CURL error: Transferred a partial file
[FLM]  Download failed (attempt 1/10)
[FLM]  Retrying in 1s...
[FLM]  Resuming from 1611.89 MB
[FLM]  Downloading: 37.2% (2801.0MB / 7534.1MB)CURL error: Failure when receiving data from the peer
[FLM]  Download failed (attempt 2/10)
[FLM]  Retrying in 2s...
[FLM]  Resuming from 2801.03 MB
[FLM]  Downloading: 54.7% (4121.2MB / 7534.1MB)CURL error: Failure when receiving data from the peer
[FLM]  Download failed (attempt 3/10)
[FLM]  Retrying in 4s...
[FLM]  Resuming from 4121.51 MB
[FLM]  Downloading: 54.8% (4125.7MB / 7534.1MB)CURL error: Failure when receiving data from the peer
[FLM]  Download failed (attempt 4/10)
[FLM]  Retrying in 8s...
[FLM]  Resuming from 4125.72 MB
[FLM]  Downloading: 54.8% (4130.8MB / 7534.1MB)CURL error: Failure when receiving data from the peer
[FLM]  Download failed (attempt 5/10)
[FLM]  Retrying in 16s...
[FLM]  Resuming from 4131.05 MB
[FLM]  Downloading: 54.8% (4131.7MB / 7534.1MB)CURL error: Failure when receiving data from the peer
[FLM]  Download failed (attempt 6/10)
[FLM]  Retrying in 30s...
[FLM]  Resuming from 4131.85 MB
[FLM]  Downloading: 100.0% (7533.7MB / 7534.1MB)
[FLM]  Checking Hash...
[FLM]  Download completed: /home/pavel/.config/flm/models/Qwen3.5-9B-NPU2/model.q4nx
[FLM]  Overall progress:  1/3 files
[FLM]  Downloading 2/3: vision_weight.q4nx
[FLM]  Resuming from 512.469 MB
[FLM]  Downloading: 99.9% (979.6MB / 980.1MB)
[FLM]  Checking Hash...
[FLM]  Download completed: /home/pavel/.config/flm/models/Qwen3.5-9B-NPU2/vision_weight.q4nx
[FLM]  Overall progress:  2/3 files
[FLM]  Downloading 3/3: chat_template.jinja
[FLM]  Downloading: 100.0% (0.0MB / 0.0MB)
[FLM]  Checking Hash...
[FLM]  Download completed: /home/pavel/.config/flm/models/Qwen3.5-9B-NPU2/chat_template.jinja
[FLM]  Overall progress:  3/3 files
[FLM]  All downloads completed successfully!
[FLM]  Model downloaded successfully!
[FLM]  All files verified successfully.

- Track progress correctly when resuming existing files
- Increase retry attempts and use exponential backoff
- Add remote file size lookup for resume logic
- Refactor pull flow so partial and missing files are handled together
- Remove redundant pre-download checks from main flow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Download timeout

1 participant