Skip to content

new: try loading models from cache before making any network calls #577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
joein merged 1 commit into from
Nov 25, 2025
+26 −11
Merged

new: try loading models from cache before making any network calls #577

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 26 additions & 11 deletions fastembed/common/model_management.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import json
import shutil
import tarfile
from copy import deepcopy
from pathlib import Path
from typing import Any, Optional, Union, TypeVar, Generic

Expand Down Expand Up @@ -224,11 +225,6 @@ def _save_file_metadata(
logger.warning(
"Local file sizes do not match the metadata."
) # do not raise, still make an attempt to load the model
else:
logger.warning(
"Metadata file not found. Proceeding without checking local files."
) # if users have downloaded models from hf manually, or they're updating from previous versions of
# fastembed
result = snapshot_download(
repo_id=hf_source_repo,
allow_patterns=allow_patterns,
Expand Down Expand Up @@ -408,14 +404,32 @@ def download_model(cls, model: T, cache_dir: str, retries: int = 3, **kwargs: An
hf_source = model.sources.hf
url_source = model.sources.url

extra_patterns = [model.model_file]
extra_patterns.extend(model.additional_files)

if hf_source:
try:
cache_kwargs = deepcopy(kwargs)
cache_kwargs["local_files_only"] = True
return Path(
cls.download_files_from_huggingface(
hf_source,
cache_dir=cache_dir,
extra_patterns=extra_patterns,
**cache_kwargs,
)
)
except Exception:
pass
finally:
enable_progress_bars()
Comment on lines +410 to +425

@coderabbitai coderabbitai Bot Nov 24, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Improve exception handling in the early cache check.

The cache-first approach is excellent, but the bare except Exception: pass on lines 422-423 silently swallows all errors, which could hide bugs or unexpected failures.

Based on the exception handling in the retry loop (line 442), consider catching specific exceptions:

         if hf_source:
             try:
                 cache_kwargs = deepcopy(kwargs)
                 cache_kwargs["local_files_only"] = True
                 return Path(
                     cls.download_files_from_huggingface(
                         hf_source,
                         cache_dir=cache_dir,
                         extra_patterns=extra_patterns,
                         **cache_kwargs,
                     )
                 )
-            except Exception:
-                pass
+            except (EnvironmentError, RepositoryNotFoundError, ValueError):
+                # Model not in cache, will attempt download
+                pass
             finally:
                 enable_progress_bars()

Alternatively, log at debug level to aid troubleshooting while still falling through to the download path.

As per static analysis hints.

🧰 Tools
🪛 Ruff (0.14.5)

422-423: try-except-pass detected, consider logging the exception

(S110)

422-422: Do not catch blind exception: Exception

(BLE001)

🤖 Prompt for AI Agents 
In fastembed/common/model_management.py around lines 410-425, the early
cache-check block currently swallows all exceptions with a bare except, which
hides errors; change that to catch only expected exceptions (e.g., OSError,
FileNotFoundError, or the specific HuggingFace download/cache exceptions used
elsewhere) or at minimum log the exception at debug level instead of passing
silently (use logger.debug("cache check failed", exc_info=True) or similar) and
then fall through to the download path; keep the finally enable_progress_bars()
as-is.


sleep = 3.0
while retries > 0:
retries -= 1

if hf_source:
extra_patterns = [model.model_file]
extra_patterns.extend(model.additional_files)

if hf_source and not local_files_only:
# we have already tried loading with `local_files_only=True` via hf and we failed
try:
return Path(
cls.download_files_from_huggingface(
Expand Down Expand Up @@ -448,11 +462,12 @@ def download_model(cls, model: T, cache_dir: str, retries: int = 3, **kwargs: An

if local_files_only:
logger.error("Could not find model in cache_dir")
break
else:
logger.error(
f"Could not download model from either source, sleeping for {sleep} seconds, {retries} retries left."
)
time.sleep(sleep)
sleep *= 3
time.sleep(sleep)
sleep *= 3

raise ValueError(f"Could not load model {model.model} from any source.")
Loading