fix(inference): include model field in nvidia-nim API requests (#2784)#2824
fix(inference): include model field in nvidia-nim API requests (#2784)#2824sanil-23 wants to merge 1 commit into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
oxoxDev
left a comment
There was a problem hiding this comment.
Author: @sanil-23 (
MEMBER— core team)
make_cloud_provider_by_slug in src/openhuman/inference/provider/factory.rs), identical title. #2791 was opened 11 days earlier and already has coderabbit approval. Issue #2784 is assigned to @graycyrus.
Surfacing as COMMENT (not blocking with CHANGES_REQUESTED) so you can coordinate with @graycyrus on which approach to land:
Comparison
| Aspect | #2791 (graycyrus) | #2824 (this PR) |
|---|---|---|
| Files | factory.rs + config_rejection.rs + factory_test.rs |
factory.rs only |
| Fallback chain | entry.default_model → bail with actionable error |
entry.default_model → config.default_model → DEFAULT_MODEL constant (silent) |
| Sentry guard | Adds "model field is required" to config_rejection classifier (suppresses TAURI-RUST-4NM) |
None |
| Tests | 3 unit tests | None |
AuthStyle::OpenhumanJwt exemption |
Yes | No |
| Failure mode | Loud, surfaces config bug | Silent — masks misconfig |
Concerns (independent of duplicate question)
-
factory.rs:779— silentDEFAULT_MODELfallback masks misconfiguration.reasoning-quick-v1is openhuman's abstract tier (Kimi K2.6 Turbo on Fireworks), NOT an nvidia-nim-native model. When the chain hits the third arm withslug=nvidia-nim, the request hits nvidia-nim's/v1/chat/completionswithmodel=reasoning-quick-v1— model nvidia-nim does not host → 404. Replaces a 400 with a different cryptic 4xx; user has zero signal their config is wrong. -
No regression tests. Coverage gate (≥80% on changed lines) and #2784's AC ("Unit test added to verify model field is present in nvidia-nim request payloads") — neither met.
-
Missing
AuthStyle::OpenhumanJwtguard.make_openhuman_backendderives its model fromconfig.default_modelindependently — the third-armDEFAULT_MODELrescue is dead code for openhuman entries and masks an honest empty for misconfigured auth_style cases. #2791 gates this withif entry.auth_style != AuthStyle::OpenhumanJwt && effective_model.trim().is_empty(). -
No
config_rejection.rsupdate. TAURI-RUST-4NM Sentry events from older clients with bad configs will keep firing post-deploy until install base rotates. #2791 adds"model field is required"tois_provider_config_rejection_messageprecisely to silence this.
Recommendation
Close #2824 in favour of #2791 (more complete, has tests, fails loudly, by ticket assignee). If you want the global-DEFAULT layer for some other reason, add it onto #2791's tail. Per [[feedback_check_open_prs_before_implement]] — checking open PRs against #2784 would have caught the overlap before opening.
Happy to flip to APPROVE / CHANGES_REQUESTED once you decide. Not blocking with formal review state yet.
Verified / looks good
.filter(|m| !m.trim().is_empty())guards correctly handle whitespace-onlydefault_modelstrings — actual hardening over main'sunwrap_or_default()which only handlesNone.crate::openhuman::config::DEFAULT_MODELpath resolves (src/openhuman/config/schema/types.rs:37).- CI green (2 e2e + 1 coverage-gate pending), no formatting/clippy issues.
|
Thanks for the side-by-side @oxoxDev — agreed, #2791 is the right landing (loud Closing in favor of @graycyrus's #2791. Should have checked open PRs against issue #2784 before opening — apologies for the noise. |
Closes #2784
Root cause
In
make_cloud_provider_by_slug(factory.rs), when a provider string like"nvidia-nim:"was used (slug with empty model after the colon), the effective model resolution fell through toentry.default_model.clone().unwrap_or_default(). If the cloud provider entry had nodefault_modelconfigured, this produced an empty string"", which was then sent as themodelfield in the OpenAI-compatible/v1/chat/completionsrequest body, causing a 400 error ("model field is required").Fix
Added a two-level fallback chain in
make_cloud_provider_by_slug:config.default_model(the user's top-level default model setting)DEFAULT_MODELconstant (reasoning-quick-v1)This mirrors the same fallback pattern already used by
cloud_entry_provider_string(the sibling function that builds provider strings for the primary cloud route).Verified
cargo check -p openhuman— compiles cleanly