Skip to content

Expand tool calling test coverage across providers and error paths #222

Description

@Kamilbenkirane

Priority: High
Related: #199

Current test coverage has significant gaps:

Provider coverage: Only 4/13 providers tested (Anthropic, OpenAI, Google, xAI). Groq, Mistral, Moonshot, DeepSeek, HuggingFace, Ollama have zero tool tests.

Missing test categories:

  • Unit tests for tool parameter mappers (schema transformation correctness)
  • Error paths: unsupported tool on provider, invalid tool schema, tool on non-supporting model
  • Raw passthrough tools
  • Streaming tool call aggregation (mock SSE events)
  • Multiple concurrent tool calls in one response
  • tools=[] and tools=None behavior
  • WebSearch configuration options (allowed_domains, blocked_domains, max_uses)
  • Multi-turn with multiple tool calls

Test quality: Integration tests are mostly "does it not crash" — no semantic assertions on tool call arguments or result content.

Files:

  • tests/integration_tests/text/test_tools.py
  • tests/unit_tests/test_client.py
  • tests/unit_tests/test_constraints.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions