From ae1fffc577190cc962de9f892a44274db4ca1ac7 Mon Sep 17 00:00:00 2001 From: zainhas Date: Tue, 14 Jul 2026 10:51:59 +0000 Subject: [PATCH] together-dedicated-endpoints: list LoRA-enabled base models and target modules Add a LoRA-Enabled Base Models table to references/api-reference.md listing each supported base model, its supported LoRA target modules, and the max loaded adapters per endpoint. Add a Quick Routing bullet in SKILL.md pointing to the new section. Synced from mintlify-docs#1215. --- skills/together-dedicated-endpoints/SKILL.md | 2 ++ .../references/api-reference.md | 32 +++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/skills/together-dedicated-endpoints/SKILL.md b/skills/together-dedicated-endpoints/SKILL.md index 22115f2..785ce29 100644 --- a/skills/together-dedicated-endpoints/SKILL.md +++ b/skills/together-dedicated-endpoints/SKILL.md @@ -46,6 +46,8 @@ Typical fits: - Read [references/dedicated-models.md](references/dedicated-models.md) - **Hardware and sizing choices** - Read [references/hardware-options.md](references/hardware-options.md) +- **LoRA-enabled base model support (target modules, max loaded adapters)** + - Read the LoRA-Enabled Base Models section in [references/api-reference.md](references/api-reference.md) ## Workflow diff --git a/skills/together-dedicated-endpoints/references/api-reference.md b/skills/together-dedicated-endpoints/references/api-reference.md index 8ec838d..ff8a97e 100644 --- a/skills/together-dedicated-endpoints/references/api-reference.md +++ b/skills/together-dedicated-endpoints/references/api-reference.md @@ -13,6 +13,7 @@ - [Upload Model](#upload-model) - [List Models](#list-models) - [Using the Endpoint](#using-the-endpoint) +- [LoRA-Enabled Base Models](#lora-enabled-base-models) - [Auto-Shutdown](#auto-shutdown) - [Speculative Decoding](#speculative-decoding) - [Prompt Caching](#prompt-caching) @@ -460,6 +461,37 @@ curl -X POST "https://api.together.xyz/v1/chat/completions" \ }' ``` +## LoRA-Enabled Base Models + +LoRA-enabled dedicated endpoints are available for the base models below. Each adapter uploaded or +attached to the endpoint must target the same base model and train only from that model's supported +target modules. `Max loaded adapters` is the maximum number of adapters that can stay attached to a +single endpoint for that base model. + +| Model | Supported target modules | Max loaded adapters | +|-------|--------------------------|---------------------| +| `google/gemma-3-270m-it-lora` | `k_proj`, `up_proj`, `o_proj`, `q_proj`, `down_proj`, `v_proj`, `gate_proj` | 16 | +| `google/gemma-3-27b-it-lora` | `k_proj`, `up_proj`, `o_proj`, `q_proj`, `down_proj`, `v_proj`, `gate_proj` | 16 | +| `google/gemma-4-31B-it-lora` | `k_proj`, `up_proj`, `o_proj`, `q_proj`, `down_proj`, `v_proj`, `gate_proj` | 16 | +| `mistralai/Mixtral-8x7B-Instruct-v0.1-FP8-Lora` | `k_proj`, `o_proj`, `q_proj`, `v_proj` | 16 | +| `meta-llama/Llama-3.3-70B-Instruct-FP8-Lora` | `k_proj`, `up_proj`, `o_proj`, `q_proj`, `down_proj`, `v_proj`, `gate_proj` | 16 | +| `meta-llama/Llama-4-Scout-17B-16E-Instruct-FP8-Lora` | `k_proj`, `o_proj`, `q_proj`, `v_proj`, `shared_expert.gate_proj`, `shared_expert.up_proj`, `shared_expert.down_proj`, `feed_forward.gate_proj`, `feed_forward.up_proj`, `feed_forward.down_proj` | 16 | +| `Qwen/Qwen3.5-0.8B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-2B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-4B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-9B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-27B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-35B-A3B-Base-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-35B-A3B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-122B-A10B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.5-397B-A17B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.6-27B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | +| `Qwen/Qwen3.6-35B-A3B-Lora` | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` † | 5 | + +† For the Qwen3.5 and Qwen3.6 models, adapters target the standard attention (`q_proj`, `k_proj`, +`v_proj`, `o_proj`) and MLP (`gate_proj`, `up_proj`, `down_proj`) projections. Adapters that target +other layers, such as the vision encoder, may not load for serving. + ## Auto-Shutdown Endpoints auto-stop after 1 hour of inactivity by default. Set `inactive_timeout` in minutes to