fix(weave): map Bedrock Nova region prefix ap to apac - #7677
Open
feiiiiii5 wants to merge 1 commit into
Open
Conversation
For Nova models on Bedrock, the model name is prefixed with the
cross-region inference geography derived from the AWS region name
(aws_region_name.split('-')[0]). Asia-Pacific regions start with 'ap'
(e.g. ap-southeast-1), but the Bedrock inference profile geography is
'apac', so Nova calls from ap-* regions were emitted as
bedrock/ap.amazon.nova-* and never matched the cost table.
Map the 'ap' prefix to 'apac'; us/eu and other geographies are
unchanged. Add regression tests covering us/eu/ap regions and a
non-Nova model.
Closes wandb#7325
Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com>
Contributor
|
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅ |
feiiiiii5
changed the title
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root Cause
In
_setup_provider_credentials_and_model, Nova models on Bedrock get the cross-region inference geography prefixed onto the model name:Asia-Pacific regions start with
ap(e.g.ap-southeast-1), but Bedrock cross-region inference profile IDs use theapacgeography prefix. So Nova calls from anyap-*region becamebedrock/ap.amazon.nova-*and never matched the cost table (which only carriesapac.amazon.nova-*keys), silently losing token/cost tracking (issue #7325).us-*/eu-*worked by coincidence because their first segment already equals their geography prefix.Fix
Map the region's first segment to the geography prefix; only
apneeds remapping toapac(US/EU already match):Test
New
tests/trace_server/test_llm_completion_bedrock.py:us-east-1→bedrock/us.amazon.nova-lite-v1:0eu-west-1→bedrock/eu.amazon.nova-lite-v1:0ap-southeast-1/ap-northeast-1→bedrock/apac.amazon.nova-lite-v1:0Verified locally against the real
llm_completion.pycode withget_bedrock_credentialspatched: 5 passed.Diff scope
2 files, +33/-1:
weave/trace_server/llm_completion.py,tests/trace_server/test_llm_completion_bedrock.py.AI Disclosure
AI-assisted root-cause analysis, initial draft, and test scaffolding; human review of the geography mapping and cost-table keys.
Not PR-related failures
None; the new tests pass locally (litellm emits an unrelated cost-map fetch timeout warning during import, falling back to its local backup).