fix(onnx): pass externalData to session_options when using customCache in Node.js#1600
Open
s-zx wants to merge 2 commits intohuggingface:mainfrom
Open
fix(onnx): pass externalData to session_options when using customCache in Node.js#1600s-zx wants to merge 2 commits intohuggingface:mainfrom
s-zx wants to merge 2 commits intohuggingface:mainfrom
Conversation
…he in Node.js
When a model uses an external data file (e.g. `model.onnx_data`) and the
user supplies a `customCache` (which returns a `Uint8Array` / `Response`
rather than a filesystem path), the ONNX runtime in Node.js fails to
locate the external data file with an error like:
file_size: system cannot find the specified path: "model.onnx_data"
Root cause: `session.js` only sets `session_options.externalData` when
`\!IS_NODE_ENV`. In Node.js with `useFSCache` this is correct — the ONNX
runtime finds the `.onnx_data` files automatically relative to the model
path on disk. But when `customCache` is used the data arrives as an
in-memory `Uint8Array` (not a file path), so the runtime has no way to
discover the sidecar files.
Fix: also set `session_options.externalData` in Node.js whenever any of
the resolved external-data entries is not a plain string (i.e. it is a
`{path, data}` object carrying in-memory bytes from a custom cache).
Fixes huggingface#1408
Collaborator
|
Hi there 👋 thanks for the PR. Any reason for doing |
More specific type check as suggested by maintainer. Since custom cache returns Uint8Array (binary data) vs string (file paths), checking for the exact type is clearer and more correct.
Author
|
@xenova Good call — |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a model uses external data (e.g.
model.onnx_data) and the user provides acustomCache(which returns aUint8Array/Responserather than a filesystem path), loading fails in Node.js with:Root Cause
session.jsonly setssession_options.externalDatawhen!IS_NODE_ENV.In Node.js with
useFSCachethis is intentional — the ONNX runtime locates.onnx_datafiles automatically relative to the on-disk model path.But when
customCacheis used the data arrives as an in-memoryUint8Array(not a file path), so the runtime has no way to discover the sidecar files on disk.Fix
Check whether any resolved external-data entry is a
{path, data}object (i.e. in-memory bytes from a custom cache). If so, passexternalDatatosession_optionseven in Node.js:Fixes #1408