diff --git a/fern/assistants/model-intelligence/metrics-methodology.mdx b/fern/assistants/model-intelligence/metrics-methodology.mdx new file mode 100644 index 000000000..71a3057c3 --- /dev/null +++ b/fern/assistants/model-intelligence/metrics-methodology.mdx @@ -0,0 +1,93 @@ +--- +title: Performance metrics and methodology reference +subtitle: See how Vapi sources and calculates latency, cost, and quality metrics. +description: The performance metrics reference explains how Vapi sources and calculates latency, cost, and quality data for each supported transcriber, model, and voice. +slug: assistants/model-intelligence/metrics-methodology +--- + +Vapi measures latency from production calls. Cost metrics are calculated estimates. Quality metrics come from Vapi or third-party benchmarks. + +## Source summary + +| Metric | Component | What it measures | Source | +|---|---|---|---| +| **Latency** | Transcriber, model, voice | Typical response time | Median from Vapi production calls | +| **Cost** | Transcriber, model, voice | Estimated cost per minute | [Calculated estimate based on typical usage](/assistants/model-intelligence/understanding-cost) | +| **Word error rate (WER)** | Transcriber | The percentage of words the transcriber gets wrong | [Daily benchmark](https://www.daily.co/blog/benchmarking-stt-for-voice-agents/) | +| **Intelligence** | Model | Reasoning and task capability | [Artificial Analysis benchmark](https://artificialanalysis.ai/models) | +| **Humanness** | Voice | How human the voice sounds | [Vapi Humanness Index](https://humannessindex.vapi.ai/) | + +## Latency + +Vapi reports the median response time for each component based on production calls. + +| Component | Latency measurement | +|---|---| +| Transcriber | Time to convert speech to text | +| Model | Time to first token | +| Voice | Time to first audio from the text-to-speech provider | + +The displayed total is simply the sum of the three component medians, so treat it as a rough estimate rather than an exact figure. + + +The total excludes [endpointing](/customization/speech-configuration) and transport time. + + +See [how latency works](/assistants/model-intelligence/understanding-latency) for help understanding latency and why more capable models tend to be slower. + +## Cost + +In order to estimate cost, we came up with a number of assumptions based on actual Vapi data. However, costs vary greatly depending on actual call performance and agent behavior. The displayed cost is an estimated rate per minute, use it to compare models rather than predict your exact bill. + + +For contracted customers, certain pricing may vary depending on your agreement. Please contact your account team for the most accurate estimate. + + +| Component | Estimation basis | +|---|---| +| Transcriber | Audio minutes for the caller and assistant | +| Model | Prompt size, tool definitions, prompt caching, and provider rates | +| Voice | Characters spoken based on typical speaking volume | + +LLM cost depends on the assistant's configuration. Vapi uses the assistant's prompt and tool definitions, so two assistants on the same model can show different estimates. The calculation also uses each provider's cached-input rate when prompt caching is supported. + + +The estimate assumes a moderate cache-hit rate, so actual costs for large prompts are often lower. It excludes growing conversation history, so actual costs for long calls are often higher. + + +See [how cost works](/assistants/model-intelligence/understanding-cost) for the formulas and assumptions behind each estimate. + +## Quality + +Each component has a quality metric that helps you compare performance. + +### Transcriber word error rate (WER) + +Word error rate (WER) measures the percentage of words a transcriber gets wrong. Lower values are better. A WER of 5% means about 1 in 20 words is transcribed incorrectly. + +Vapi sources WER from [Daily](https://www.daily.co/blog/benchmarking-stt-for-voice-agents/), a third-party speech-to-text benchmark. This provides a consistent comparison across providers instead of relying on self-reported accuracy. + +### Intelligence metric for models + +The **Intelligence** metric measures a model's reasoning and task capability. Vapi sources the score from the [Artificial Analysis](https://artificialanalysis.ai/models) Intelligence Index. This third-party benchmark scores models from 0 to 100 across reasoning and knowledge tasks. + +Voice agents run LLMs with reasoning turned off to keep latency low. The displayed scores reflect reasoning-off performance. The smartest voice agents run at around 20-30 intelligence score, be sure to compare models to each other for a relative benchmark. + +### Humanness metric for voices + +The **Humanness** metric measures how natural a voice sounds. Vapi measures it with the first-party [**Vapi Humanness Index**](https://humannessindex.vapi.ai/). The index uses a blind listening test to score voices from 1 to 100. A higher score means the voice is harder to distinguish from a human. + +## Data refresh schedule + +Vapi refreshes performance metrics weekly as models change and more call data becomes available. The metrics reflect each model at its last update rather than a continuous live feed. + +## Related + + + + The full formulas and assumptions behind the cost estimate. + + + What voice-to-voice latency measures and why it varies by model. + + diff --git a/fern/assistants/model-intelligence/overview.mdx b/fern/assistants/model-intelligence/overview.mdx new file mode 100644 index 000000000..81e8fc0e1 --- /dev/null +++ b/fern/assistants/model-intelligence/overview.mdx @@ -0,0 +1,41 @@ +--- +title: Model Intelligence for transcribers, models, and voices +subtitle: Choose assistant components with curated Model Presets and Performance Metrics. +description: Model Intelligence helps you choose transcribers, models, and voices with curated presets and weekly refreshed performance metrics for latency, cost, and quality. +slug: assistants/model-intelligence/overview +--- + +Model Intelligence is a Vapi feature that helps you choose the transcriber, model, and voice for your assistant. Choose a preset for a strong default combination or compare every option using weekly refreshed performance metrics. + +## Key concepts + +| Term | Definition | +|---|---| +| **Model Presets** | Curated configurations that bundle a transcriber, model, and voice tuned for one goal. | +| **Performance metrics** | The latency, cost, and quality metrics shown on every component, so you can compare options directly. | +| **Customized** | The state an assistant moves to when you change any single component away from its preset. | + +## How it works + +When building a voice agent, Vapi orchestrates the transcriber, model, and voice. You can swap each component for any supported provider and model, which gives you flexibility when building an assistant. + +**Model Presets** bundle the three components into a curated combination. Choose **Balanced**, **High Intelligence**, **Ultra Fast**, or **Cost Saver** based on your goal. Presets provide a dependable setup without requiring you to tune each component. + +**Performance metrics** show latency, cost, and quality data for every transcriber, model, and voice. The data appears on component panels and the dropdown menu for each component. Use it to compare options and build the combination your assistant needs. See the [performance metrics reference](/assistants/model-intelligence/metrics-methodology) for how each metric is sourced and calculated. + +If you change one component from its preset, the assistant moves to **Customized**. Every other preset setting remains unchanged, and you can continue editing any component. + +## Start with presets and optimize with performance metrics + +[Model Presets](/assistants/model-intelligence/presets) give you a dependable starting point when you do not know which component to choose. [Performance metrics](/assistants/model-intelligence/metrics-methodology) help you compare options and optimize your configuration with data instead of guessing. + +## Related + + + + See all four presets, when to use each, and how to apply or customize one. + + + How every latency, cost, and quality metric is sourced. + + diff --git a/fern/assistants/model-intelligence/presets.mdx b/fern/assistants/model-intelligence/presets.mdx new file mode 100644 index 000000000..5fc27b594 --- /dev/null +++ b/fern/assistants/model-intelligence/presets.mdx @@ -0,0 +1,119 @@ +--- +title: Model Presets for transcribers, models, and voices +subtitle: Configure an assistant's transcriber, model, and voice with one preset. +description: Model presets optimize for balanced performance, reasoning, low latency, or low cost. Learn when to use each preset and how to apply or customize its configuration. +slug: assistants/model-intelligence/presets +--- + +**Model Presets** are curated configurations that bundle a transcriber, model, and voice for a specific goal. Use a preset for a dependable setup or as a starting point for further customization. + +## How it works + +New assistants use **Balanced** as the default. When you start from one of Vapi’s use-case templates, the assistant uses the preset that best fits the template. + +Presets are a starting point rather than a lock - you can fine-tune or swap out any component to better fit your use case. + +## Presets at a glance + +| Preset | Optimizes for | Best for | +|---|---|---| +| **Balanced** | A strong all-round mix of quality, speed, and cost | Most assistants; start here if you're unsure | +| **High Intelligence** | Reasoning and conversation quality | Complex, nuanced, or high-stakes conversations | +| **Ultra Fast** | Lowest latency | Fast, responsive, high-volume conversations | +| **Cost Saver** | Lowest cost per minute | Simple, high-volume calls where cost matters most | + +## Choose a preset + +Choose the preset that best matches your use case. Start with **Balanced** if you are unsure. + +### Balanced + +**Balanced** is the default for new assistants and the best starting point for most use cases. It provides a strong mix of quality, responsiveness, and cost. + +Choose **Balanced** when any of the following apply. + +- You're building a new assistant and aren't sure which preset fits. +- Your use case covers support, scheduling, FAQs, or qualification. +- You want an all-around dependable setup out of the box. + + +If you later need more reasoning, faster responses, or lower cost, switch to the preset built for that. + + +### High Intelligence + +**High Intelligence** prioritizes reasoning and conversation quality. Use it when the assistant needs to handle nuance, follow multi-step logic, or reason reliably across tools and context. + +Choose **High Intelligence** when any of the following apply. + +- Conversations are complex, open-ended, or high-stakes. +- The assistant needs to reason through multi-step problems or use tools reliably. +- Accuracy and quality matter more than speed or cost. + + +More capable models respond a little slower and cost more per minute than **Balanced**. See [how latency works](/assistants/model-intelligence/understanding-latency) for why more capable models take longer to respond. + + +### Ultra Fast + +**Ultra Fast** prioritizes low latency and responsive conversations. Use it when response speed matters more than complex reasoning. + +Choose **Ultra Fast** when any of the following apply. + +- Responsiveness is the priority and replies should feel fast. +- Calls are high-volume and relatively straightforward. +- The flow is scripted or transactional rather than open-ended. + + +The fastest models are smaller and less capable, so **Ultra Fast** may not suit complex reasoning or heavy tool use. Choose **Balanced** or **High Intelligence** if difficult tasks produce quality issues. + + +### Cost Saver + +**Cost Saver** prioritizes the lowest cost per minute. Use it for high-volume calls when cost is the main constraint. + +Choose **Cost Saver** when any of the following apply. + +- Call volume is high and you're optimizing spend. +- Interactions are simple or scripted. +- A small quality trade-off is acceptable in exchange for lower cost. + + +The lowest-cost models are less capable, so this preset fits simpler interactions best. See [how cost works](/assistants/model-intelligence/understanding-cost) for what drives cost per minute. + + +## The Customized state + +Swapping a model out of any component moves your assistant to **Customized** and the **Performance Metrics** displayed will update to match the new configuration. + +## Apply a Model Preset + + + + Open your assistant in the Vapi Dashboard, then open **Assistant**. + + + Choose an option under **Model Presets**. The transcriber, model, voice, and displayed totals update to match the preset. + + + Click the pencil icon on the transcriber, model, or voice panel. Use the dropdown menu in the settings panel to switch out the model or provider. Your assistant moves to **Customized**. **Performance Metrics** update to reflect your choices. + + + Click **Publish** to apply your changes. + + + +## Verify the configuration + +The option you chose is highlighted under **Model Presets**. Confirm the component panels and displayed totals match the preset or your custom configuration. + +## Related + + + + Learn how presets and performance metrics help you choose components. + + + How every latency, cost, and quality metric is sourced. + + diff --git a/fern/assistants/model-intelligence/understanding-cost.mdx b/fern/assistants/model-intelligence/understanding-cost.mdx new file mode 100644 index 000000000..ab0fd8a79 --- /dev/null +++ b/fern/assistants/model-intelligence/understanding-cost.mdx @@ -0,0 +1,97 @@ +--- +title: How voice agent component costs and estimates work +subtitle: Learn what drives costs across the transcriber, model, and voice. +description: Understand how Vapi estimates transcriber, model, and voice costs per minute, including prompt caching assumptions and the effects of conversation history. +slug: assistants/model-intelligence/understanding-cost +--- + +The cost to run a voice agent mainly depends on the costs of the transcriber, LLM, and voice. Each component uses a different billing unit, so we've made some assumptions in order to provide a way for you to compare estimated cost per minute. + + +The Cost number within **Performance Metrics** provide estimates for comparison rather than exact quotes. Use them to compare models consistently and understand what affects cost. See [Vapi pricing](/pricing) for current pricing. For contracted customers, certain pricing may depend on your agreement, please contact your account team for contract-specific rates. + + +## Key concepts + +| Term | Definition | +|---|---| +| Input tokens per minute | Tokens sent to the model each minute based on your prompt size, tool-definition size, and a typical request rate. | +| Effective input rate | The per-token input rate blended between standard and cached rates when the model supports prompt caching. | +| Output tokens per minute | Tokens the model generates each minute based on typical Vapi call data. | + +## Component billing units + +Each component uses a different billing unit. + +| Component | Billing unit | +|---|---| +| Transcriber | Minutes of audio | +| Model | 1 million input and output tokens | +| Voice | Characters of spoken text | + +### Transcriber cost + +Transcription is billed per minute of audio. Vapi transcribes audio from both the caller and the assistant. The estimate therefore includes two audio channels. This cost scales with call duration rather than your prompt or configuration. + +### Voice cost + +The Voice model is billed per character of spoken text. The estimate uses the number of characters spoken in a typical minute. Cost increases when the assistant speaks more. + +### LLM cost + +Model cost varies more than transcriber or voice cost because it heavily depends on the assistant's configuration. Vapi calculates the estimate from the assistant's actual prompt and tool definitions, so two assistants using the same LLM model can show different cost estimates. + +The estimate uses each provider's input and output token rates. + +| Calculation | Formula | +|---|---| +| **Input cost** | Input tokens per minute × effective input rate | +| **Output cost** | Output tokens per minute × output rate | +| **Estimated model cost per minute** | Input cost + output cost | + +#### Input tokens per minute + +**Input tokens per minute** is the largest cost factor you can control. Vapi calculates it from your [system prompt](/prompting-guide), [tool definitions](/tools), and a typical number of model requests per minute. Longer prompts and more tools increase the estimate, while leaner configurations reduce it. + +Vapi estimates 5 requests per turn. We use number of requests rather than conversational turns, since a single turn can trigger multiple requests, and providers bill each request. + +#### Effective input rate + +**Effective input rate** accounts for prompt caching. Cached input is stable prompt content that providers reuse across requests. Many providers offer a discounted rate for this input. + +When a model supports caching, Vapi blends its standard and cached rates based on an assumed 50% cache hit rate. Otherwise, Vapi uses the standard rate. The calculation uses the actual rates for each model and provider. + +#### Output tokens per minute + +Vapi estimates 150 **output tokens per minute** based on typical generated speech in production call data. It prices those tokens at the model’s output rate. + +## Estimate limitations + +Two assumptions can cause actual model cost to differ from the estimate. + +| Scenario | Typical effect | Reason | +|---|---|---| +| Large prompts with caching supported | Actual cost is often lower | Actual cache use is usually higher than the estimate assumes | +| Long calls | Actual cost is often higher | Growing conversation history is sent again with each request | + +These deliberate simplifications keep estimates comparable across models. Vapi may refine them over time. + +## Reduce cost + +You can reduce cost in four ways. + +- Shorten your system prompt and tool definitions. This often has the largest effect on model cost. +- Use models that support prompt caching for large, stable prompts. Providers charge less for prompt content reused from cache, which lowers model cost. +- Choose a lower-cost model when the use case allows. The [Cost Saver preset](/assistants/model-intelligence/presets#cost-saver) optimizes components for the lowest cost per minute. +- Keep calls focused. Shorter, more focused conversations cost less and tend to provide a better experience. + +## Related + + + + Compare presets or apply Cost Saver. + + + See how Vapi calculates model cost estimates. + + diff --git a/fern/assistants/model-intelligence/understanding-latency.mdx b/fern/assistants/model-intelligence/understanding-latency.mdx new file mode 100644 index 000000000..42d9af059 --- /dev/null +++ b/fern/assistants/model-intelligence/understanding-latency.mdx @@ -0,0 +1,59 @@ +--- +title: How voice-to-voice latency works in voice agents +subtitle: Learn what voice-to-voice latency measures and why capable models respond more slowly. +description: Understand voice-to-voice latency, the components included in Vapi performance metrics, and how model capability, endpointing, and transport affect response time. +slug: assistants/model-intelligence/understanding-latency +--- + +Voice-to-voice latency is the gap between when a caller finishes speaking and when your assistant starts responding. It is a major factor in whether a conversation feels natural or frustrating. This page explains how Vapi measures latency and how to compare models. + +## Voice-to-voice latency sequence + +Voice-to-voice latency combines five stages that run in sequence. Each stage begins after the previous one. + +1. **Transport:** The caller's audio travels over the network for processing. +2. **Endpointing:** Vapi detects that the caller has finished speaking. +3. **Transcription:** The transcriber converts the caller's speech to text. +4. **Model response:** The model reads the transcript and begins generating a reply. Vapi measures this stage as time to first token. +5. **Speech synthesis:** The voice provider turns the reply into audio. Vapi measures this stage as time to first audio. + +A caller's total voice-to-voice latency is roughly the sum of all five stages. + +## What the displayed latency measures + +**Performance metrics** show latency for each component. The displayed total adds the median latency for the transcriber, model, and voice. This should be treated as a rough estimate. + +Vapi calculates each median from production calls, which reflect a typical call - half of calls are faster and half are slower. Individual calls can vary. + +The displayed total excludes [endpointing](/customization/speech-configuration) and transport time. Both contribute to the delay a caller experiences. Endpointing can account for a meaningful share of that delay. Improving endpointing can make an assistant feel more responsive even though it is not included in the displayed total. + +## Model capability and latency + +More capable models usually respond more slowly. A model's time to first token is often the largest component of voice-to-voice latency. More capable models tend to be larger and take longer to begin generating a response. + +[**Ultra Fast**](/assistants/model-intelligence/presets#ultra-fast) uses speed-optimized models for lower latency. [**High Intelligence**](/assistants/model-intelligence/presets#high-intelligence) accepts higher latency for greater capability. Neither preset is always better than the other; choose based on whether your use case needs faster responses or stronger reasoning. + +## How to judge latency + +Use the displayed latency to compare models and then test the assistant in a real conversation. + +- Lower latency improves the experience only until callers stop noticing the difference. +- People naturally pause between turns. A response can feel natural without being instant. +- Endpointing, turn-taking, and silence handling all affect how latency feels. + +Testing the assistant in a real conversation provides the most reliable evaluation. A model with slightly higher latency can still feel natural. Choosing the lowest number may sacrifice capability that your use case needs. + + +Other latency figures may use ideal lab conditions or measure only one component, while Vapi reports medians from actual production calls. Compare figures only when their scope and conditions match. + + +## Related + + + + Compare presets or apply Ultra Fast. + + + See how Vapi calculates latency. + + diff --git a/fern/assistants/quickstart.mdx b/fern/assistants/quickstart.mdx index e7c42ba04..6eb87d7e1 100644 --- a/fern/assistants/quickstart.mdx +++ b/fern/assistants/quickstart.mdx @@ -19,6 +19,10 @@ Create a voice assistant with a simple prompt, attach a phone number, and make y ## 1) Create an assistant + +By default, when you create an assistant, the transcriber, model, and voice are set to the [Balanced preset](/assistants/model-intelligence/presets#balanced). + + @@ -92,6 +96,7 @@ Create a voice assistant with a simple prompt, attach a phone number, and make y - **Add tools**: [Custom tools](/tools/custom-tools) - **Tune speech**: [Speech configuration](/customization/speech-configuration) +- **Use a preset**: [Model Intelligence presets](/assistants/model-intelligence/presets) bundle a transcriber, model, and voice tuned for common use cases, so you can configure your assistant in one click. - **Structure data**: [Structured outputs](/assistants/structured-outputs) - **Move to multi-assistant**: [Squads](/squads) diff --git a/fern/docs.yml b/fern/docs.yml index 9abfca42d..9597b077b 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -175,6 +175,19 @@ navigation: path: openai-realtime.mdx - page: Provider keys path: customization/provider-keys.mdx + - section: Model Intelligence + icon: fa-light fa-sliders + contents: + - page: Overview + path: assistants/model-intelligence/overview.mdx + - page: Model Presets + path: assistants/model-intelligence/presets.mdx + - page: Performance metrics + path: assistants/model-intelligence/metrics-methodology.mdx + - page: Understanding cost + path: assistants/model-intelligence/understanding-cost.mdx + - page: Understanding latency + path: assistants/model-intelligence/understanding-latency.mdx - section: Tools path: tools/introduction.mdx collapsed: true diff --git a/fern/fern.config.json b/fern/fern.config.json index 11fe60d5f..ec0e9bbd2 100644 --- a/fern/fern.config.json +++ b/fern/fern.config.json @@ -1,4 +1,4 @@ { "organization": "vapi", - "version": "5.72.1" + "version": "5.75.6" } diff --git a/fern/providers/voice/overview.mdx b/fern/providers/voice/overview.mdx index effd0504f..76cdeebfb 100644 --- a/fern/providers/voice/overview.mdx +++ b/fern/providers/voice/overview.mdx @@ -50,6 +50,10 @@ voice's ID to use on an assistant. Some providers also let you use your own custom voice. See [Custom voices](/customization/custom-voices/custom-voice) for setup. + +[Model Intelligence presets](/assistants/model-intelligence/presets) bundle a transcriber, model, and voice tuned for common use cases, so you can configure your assistant in one click. + + ## Supported voice providers diff --git a/fern/quickstart/introduction.mdx b/fern/quickstart/introduction.mdx index 159e309ae..be6e660ff 100644 --- a/fern/quickstart/introduction.mdx +++ b/fern/quickstart/introduction.mdx @@ -32,6 +32,10 @@ Every Vapi assistant combines three core technologies: You have full control over each component, with dozens of providers and models to choose from; OpenAI, Anthropic, Google, Gladia, Deepgram, ElevenLabs, and many, many more. + +[Model Intelligence presets](/assistants/model-intelligence/presets) bundle a transcriber, model, and voice tuned for common use cases, so you can configure your assistant in one click. + + ## Two ways to build voice agents Vapi offers two main primitives, designed for different use cases: diff --git a/fern/quickstart/phone.mdx b/fern/quickstart/phone.mdx index b30171c43..e0e09fc70 100644 --- a/fern/quickstart/phone.mdx +++ b/fern/quickstart/phone.mdx @@ -341,6 +341,7 @@ Now that you have a working voice assistant: - **Customize the conversation:** Update the system prompt to match your use case - **Add tools:** Connect your assistant to external APIs and databases - **Configure models:** Try different speech and language models for better performance +- **Use a preset:** [Model Intelligence presets](/assistants/model-intelligence/presets) bundle a transcriber, model, and voice tuned for common use cases, so you can configure your assistant in one click. - **Scale with APIs:** Use Vapi's REST API to create assistants programmatically diff --git a/fern/quickstart/web.mdx b/fern/quickstart/web.mdx index 078ec147a..42cee103a 100644 --- a/fern/quickstart/web.mdx +++ b/fern/quickstart/web.mdx @@ -1305,6 +1305,7 @@ Now that you understand both client and server SDK capabilities: - **Explore use cases:** Check out our [examples section](/assistants/examples/inbound-support) for complete implementations - **Add tools:** Connect your voice agents to external APIs and databases with [custom tools](/tools/custom-tools) - **Configure models:** Try different [speech and language models](/assistants/speech-configuration) for better performance +- **Use a preset:** [Model Intelligence presets](/assistants/model-intelligence/presets) bundle a transcriber, model, and voice tuned for common use cases, so you can configure your assistant in one click. - **Scale with squads:** Use [Squads](/squads) for multi-assistant setups and complex processes ## Resources