Integrate portkey.ai as multi-provider gateway with tool use support

[fede-kamel]

🚀 Portkey Integration for Strands SDK

Summary

This PR adds first-class support for Portkey, a gateway platform for LLM providers, into the Strands SDK. It enables a unified, configurable, and extensible way to interact with multiple AI models (OpenAI, Anthropic via Bedrock, Gemini, and more) through a single abstraction.

The integration emphasizes tool use and streaming completions, and lays foundational support for future capabilities like multi-modal interactions (images, audio — coming in a future PR).

---

🔍 What is Portkey?

Portkey is a powerful LLM gateway that provides:

• Unified API access to multiple model providers
• Smart routing based on performance, reliability, or cost
• Built-in observability and analytics (latency, usage)
• Retry, fallback, and caching strategies
• Key and model management for dynamic overrides

Portkey is becoming the go-to model abstraction layer for companies scaling LLM usage — including Notion, Descript, Ramp, and others — because it makes managing model complexity simple, modular, and scalable.

---

✅ Supported Model Providers

This integration supports and has been tested with:

• OpenAI (GPT-4, GPT-3.5)
• Anthropic, via Amazon Bedrock
• Amazon Bedrock-native models
• Google Gemini
• Other providers supported by Portkey configuration

🔧 Focus for This PR:

• Tool Use capabilities for:
  • OpenAI (native function-calling support)
  • Anthropic (via Bedrock with partial support)

---

🎯 Why This Matters

Adding Portkey unlocks significant benefits:

Benefit	Description
Model Agility	Swap providers without changing SDK logic
Unified Tooling	Consistent handling of tool_calls across models
Resilience	Add retries, rate limits, and observability
Flexibility	Customize parameters like temperature, tokens, and stream
Cost Optimization	Route to cheaper models or fallback options

---

🌍 Industry Adoption & Strategic Importance

Portkey is not just a convenience — it is a requirement for SDK adoption in many real-world AI systems.

Why We Must Support Portkey:

Leading companies already standardize on Portkey. Without first-class support, these companies cannot use the SDK, as their LLM access, observability, and governance are fully routed through Portkey.

Companies Using Portkey in Production:

• Notion – internal AI copilots with dynamic routing
• Ramp – cost-based model fallback and observability
• Descript – creative workflows across providers
• Puzzle, DeepOpinion, and other AI-first orgs

📎 Reference: Portkey Customers

Supporting Portkey makes the Strands SDK immediately compatible with the infrastructure already powering major AI systems — greatly reducing integration effort.

---

🧩 Strands SDK Context

This implementation extends the Model interface within the Strands SDK, Amazon’s agent platform for intelligent task automation.

Key Capabilities Enabled:

• ✅ Streaming responses
• ✅ Tool calling (tool_calls / tool_use)
• ✅ System prompt injection
• ✅ Token configuration per model
• ✅ Model-agnostic formatting of inputs/outputs

🧠 Coming Soon: In a follow-up PR, we will add multi-modal capabilities (images and audio input), further expanding the power of Strands agents to handle rich sensory data.

---

🔍 High-Level Implementation

This PR introduces a new model provider class:

• PortkeyModel, implementing the full Model interface
• Handles provider-specific tool use behavior (OpenAI vs Bedrock)
• Formats and streams messages with structured deltas
• Manages tool call life cycle: initiation, input, completion
• Defensive checks for context window and malformed payloads
• Full support for ToolSpec mapping and schema encoding

All logic is structured to allow future expansion to other Portkey-supported capabilities (e.g., caching, retry policies, routing rules).

---

🧪 Testing Scope

This feature was validated across the following dimensions:

• ✔️ Basic chat completions
• ✔️ Streaming token responses
• ✔️ Tool calling with input parameters
• ✔️ Tool result handoff
• ✔️ Multiple model backends:
  • OpenAI GPT-4 (tools)
  • Anthropic Claude (via Bedrock)
  • Gemini (streaming + simple use)
  • Bedrock-native (Jurassic, Titan)

---

🛠️ Next Steps

• Add multi-modal support (images, audio)
• Integrate Portkey usage metrics into internal observability stack
• Add routing and fallback policies via Portkey configuration
• Enable A/B experimentation between providers at runtime

---

Related Issues

N/A

Documentation PR

📚 References

• Portkey.ai Documentation
• Portkey Customer Base
• Amazon Bedrock Overview
• OpenAI Function Calling
• Strands SDK

Type of Change

• New feature
• Bug fix
• Breaking change
• Documentation update
• Other (please describe):

Adds a new model gateway provider (PortkeyModel) to enable multi-provider LLM integration and tool use streaming in the Strands SDK.

Testing

This change has been tested with:

• ✅ hatch fmt --linter
• ✅ hatch fmt --formatter
• ✅ hatch test --all
• ✅ Manual verification of:
  • OpenAI GPT-4 tool calling and streaming
  • Anthropic Claude via Bedrock (tool calling and streaming)
  • Gemini (streaming)
  • Bedrock-native models (Titan, Jurassic)
• ✅ Confirmed zero regression/warnings across:
  • agents-docs
  • agents-tools
  • agents-cli

Checklist

• I have read the CONTRIBUTING document
• I have added tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature
• My changes generate no new warnings
• Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.