0.18.23

Gateway is optional

Previously, running any service required setting up a gateway. With this update, a gateway is no longer needed to run a service for development purposes.

Service endpoint

• If no gateway is created, the service’s endpoint will be accessible at <dstack server URL>/proxy/services/<project name>/<run name>/.
• If a service has a model mapping, the model will be accessible via the OpenAI-compatible endpoint at <dstack server URL>/proxy/models/<project name>/.

Note

While this change makes it much easier to use services for development, you will still need a gateway if you want to use a custom domain, enable HTTPS, or use auto-scaling.

Gateway property

If a gateway is created but isn’t needed for a service, set the gateway property to false. If you have multiple gateways, you can choose one by setting gateway to the name of the gateway.

Model mapping

If the model is in OpenAI format, you can now use a shorter syntax for model mapping—simply set the model property to the model's name.

type: service

image: ollama/ollama
commands:
  - ollama serve &
  - sleep 3
  - ollama pull llama3.1
  - fg
port: 11434

model: llama3.1

The longer syntax with more settings remains available.

Updating running services

Previously, updating a service’s configuration required restarting it. Now, you can update the replicas and scaling properties in place. Just run dstack apply, and the changes will take effect. New replicas will be created while the old ones continue running.

What's changed

• [dind] Update start-dockerd script by @un-def in #1928
• Add /proxy prefix to dstack-proxy endpoints by @jvstme in #1939
• [shim] Unmount volumes when run exits by @un-def in #1937
• Return error when instance added to multiple fleets(#1699) by @swsvc in #1938
• unify project administration by @olgenn in #1946
• [shim] Change NVIDIA GPU detection method by @un-def in #1945
• Support service scaling via in-place updates by @r4victor in #1958
• [Docs] Document resources.gpu.vendor property by @un-def in #1957
• Fix SSH fleet hosts validation by @un-def in #1955
• Support chat models in dstack-proxy by @jvstme in #1953
• Add user tag to AWS and GCP volumes by @james-boydell in #1948
• Fix dstack-proxy dependencies by @jvstme in #1959
• Support DSTACK_SERVER_ADMIN_TOKEN env by @r4victor in #1960
• Fix migration 82b32a135ea2 by @un-def in #1962
• Fix dstack apply runs with new names by @r4victor in #1964
• [Blog] Introducing instance volumes to persist data on instances by @peterschmidt85 in #1965
• [UI]: Support in-server model proxy by @olgenn in #1966
• Short model mapping syntax by @jvstme in #1967
• Fix VolumeModel.user not loaded for volume detach by @r4victor in #1970
• Drop the PROXY feature flag by @jvstme in #1971
• Allow specifying gateway in service configurations by @jvstme in #1972
• Improve error handling in model proxy by @jvstme in #1973

New contributors

• @james-boydell made their first contribution in #1948

Full changelog: 0.18.22...0.18.23