diff --git a/openhands/usage/settings/llm-settings.mdx b/openhands/usage/settings/llm-settings.mdx index e767960bf..b6121c4e6 100644 --- a/openhands/usage/settings/llm-settings.mdx +++ b/openhands/usage/settings/llm-settings.mdx @@ -1,6 +1,6 @@ --- title: Language Model (LLM) Settings -description: This page goes over how to set the LLM to use in OpenHands. As well as some additional LLM settings. +description: This page goes over how to set the LLM to use in OpenHands, including LLM profiles for switching models during conversations. --- ## Overview @@ -43,3 +43,96 @@ conversations. - `Enable memory condensation` - Turn on this setting to activate this feature. - `Memory condenser max history size` - The condenser will summarize the history after this many events. + +## LLM Profiles + +LLM profiles allow you to save multiple LLM configurations and switch between them, even during an active conversation. +This is useful when you want to use different models for different tasks—for example, a faster model for simple tasks +and a more powerful model for complex reasoning. + +### Creating an LLM Profile + +Profiles are automatically created when you save a configuration on the LLM settings page. To create a new profile: + +1. Navigate to `Settings > LLM`. +2. Configure your desired LLM provider, model, and API key. +3. Click `Save Changes`. + +A new profile will be created with your configuration. The most recently saved profile becomes the active profile +for new conversations. + +Alternatively, you can click the `Add LLM Profile` button in the Available Profiles section to create a new profile +directly. + +### Managing LLM Profiles + +You can manage your saved profiles in the `Available Profiles` section of the LLM settings page. Each profile shows: + +- **Profile name**: A unique identifier for the configuration +- **Model**: The LLM model associated with the profile +- **Active badge**: Indicates which profile is currently active + +Click the menu icon (three dots) on any profile to access these actions: + +- **Edit**: Modify the profile's LLM configuration +- **Rename**: Change the profile name +- **Set as Active**: Make this profile the default for new conversations +- **Delete**: Remove the profile + + +You can save up to 10 LLM profiles per account. Delete unused profiles if you need to create new ones. + + +### Switching Profiles During a Conversation + +One of the most powerful features of LLM profiles is the ability to switch models mid-conversation without losing context. +This allows you to: + +- Start with a fast, cost-effective model for initial exploration +- Switch to a more powerful model when the task requires deeper reasoning +- Use specialized models for specific types of tasks + +To switch profiles during an active conversation: + +1. Look for the **profile selector button** in the chat input area. It displays the name of the currently active profile. +2. Click the button to open the profile menu. +3. Select the profile you want to switch to. + +The conversation will continue with the new model, maintaining all previous context and history. The switch takes effect +immediately for subsequent messages. + + +The profile selector shows a checkmark next to the currently active profile. If no profile matches the running model, +the button will show "Select a model" as a placeholder. + + +### Switching Profiles with the `/model` Slash Command + +You can also list and switch profiles directly from the chat input using the `/model` slash command: + +- `/model` — Lists your saved LLM profiles, including which one is active and whether each profile has an API key set. +- `/model ` — Switches the conversation to the profile with the given name. + +This is equivalent to using the profile selector button and works without leaving the chat. + +### How Profile Switching Works + +When you switch profiles during a conversation: + +1. The new LLM configuration is loaded from your saved profile +2. The conversation context (all previous messages and actions) is preserved +3. Future messages are processed using the new model +4. The conversation metadata is updated to reflect the new model + +This seamless switching allows you to leverage different models' strengths without starting a new conversation or +losing your progress. + +### Best Practices for Using LLM Profiles + +- **Name profiles descriptively**: Use names like "Claude Sonnet - Fast" or "GPT-4 - Complex Tasks" to easily + identify which profile to use. +- **Create task-specific profiles**: Set up profiles optimized for different workflows, such as code review, + documentation, or debugging. +- **Keep API keys updated**: Ensure each profile has a valid API key. +- **Test before critical work**: When switching profiles mid-conversation, send a simple test message to confirm + the new model is responding correctly.