diff --git a/skills/together-dedicated-endpoints/SKILL.md b/skills/together-dedicated-endpoints/SKILL.md index 22115f2..63931ff 100644 --- a/skills/together-dedicated-endpoints/SKILL.md +++ b/skills/together-dedicated-endpoints/SKILL.md @@ -46,6 +46,8 @@ Typical fits: - Read [references/dedicated-models.md](references/dedicated-models.md) - **Hardware and sizing choices** - Read [references/hardware-options.md](references/hardware-options.md) +- **Serve multiple LoRA adapters on one endpoint** + - Read the Multi-LoRA Adapters section in [references/api-reference.md](references/api-reference.md) ## Workflow @@ -62,6 +64,7 @@ Typical fits: - Endpoint management uses endpoint IDs, while inference usually uses the endpoint name as `model`. - Autoscaling, auto-shutdown, prompt caching, and speculative decoding materially affect operations and cost. - For custom or fine-tuned models, do not skip the intermediate verification steps before deployment. +- One LoRA-enabled endpoint can serve multiple adapters that share its base model (beta): attach/list/remove them on the running endpoint via `client.endpoints.adapters` and select by model name, instead of deploying separate hardware per adapter. ## Resource Map diff --git a/skills/together-dedicated-endpoints/references/api-reference.md b/skills/together-dedicated-endpoints/references/api-reference.md index 8ec838d..826c26b 100644 --- a/skills/together-dedicated-endpoints/references/api-reference.md +++ b/skills/together-dedicated-endpoints/references/api-reference.md @@ -13,6 +13,7 @@ - [Upload Model](#upload-model) - [List Models](#list-models) - [Using the Endpoint](#using-the-endpoint) +- [Multi-LoRA Adapters](#multi-lora-adapters) - [Auto-Shutdown](#auto-shutdown) - [Speculative Decoding](#speculative-decoding) - [Prompt Caching](#prompt-caching) @@ -460,6 +461,59 @@ curl -X POST "https://api.together.xyz/v1/chat/completions" \ }' ``` +## Multi-LoRA Adapters + +A single LoRA-enabled dedicated endpoint can serve multiple LoRA adapters that share the same base +model. Attach, list, and remove adapters on a running endpoint (beta) without redeploying, then +select among them by model name at inference time. + +Requirements: the endpoint must be a private dedicated endpoint with LoRA enabled, running a base +model compatible with the adapter, and the adapter and endpoint must be owned by the same account. + +| Method | Path | Description | +|--------|------|-------------| +| `POST /endpoints/{id}/adapters` | Add adapter | Bind a LoRA adapter to the endpoint | +| `GET /endpoints/{id}/adapters` | List adapters | List adapters bound to the endpoint | +| `DELETE /endpoints/{id}/adapters` | Remove adapter | Unbind an adapter from the endpoint | + +Each adapter is identified by a combined `model_id` of the form `endpoint_name:adapter_model_name`, +where `endpoint_name` must match the endpoint resolved from the endpoint ID and `adapter_model_name` +is the adapter's uploaded `model_name`. + +```python +from together import Together + +client = Together() + +# Attach +result = client.endpoints.adapters.add( + "endpoint-abc123", + model_id="my-endpoint-name:my-adapter-model", +) + +# List +adapters = client.endpoints.adapters.list("endpoint-abc123") +for adapter in adapters.data or []: + print(adapter.api_model_id, adapter.adapter_name, adapter.endpoint_name) + +# Remove +client.endpoints.adapters.remove( + "endpoint-abc123", + model_id="my-endpoint-name:my-adapter-model", +) +``` + +```shell +together endpoints adapters add : +together endpoints adapters list +together endpoints adapters remove : +``` + +Once attached, send inference requests using the adapter model name (or the full +`endpoint_name:adapter_model_name`) as the `model` parameter; requests route to the endpoint +automatically. Note: the Python SDK exposes the combined identifier as `api_model_id` on the +response object, while the raw API and TypeScript SDK use `model_id`. + ## Auto-Shutdown Endpoints auto-stop after 1 hour of inactivity by default. Set `inactive_timeout` in minutes to