diff --git a/docs/docs/concepts/services.md b/docs/docs/concepts/services.md index 5dd92f19c3..b4b7288bca 100644 --- a/docs/docs/concepts/services.md +++ b/docs/docs/concepts/services.md @@ -679,6 +679,61 @@ utilization_policy: [`max_price`](../reference/dstack.yml/service.md#max_price), and among [others](../reference/dstack.yml/service.md). +## Rolling deployment + +To deploy a new version of a service that is already `running`, use `dstack apply`. `dstack` will automatically detect changes and suggest a rolling deployment update. + +
+ +```shell +$ dstack apply -f my-service.dstack.yml + +Active run my-service already exists. Detected changes that can be updated in-place: +- Repo state (branch, commit, or other) +- File archives +- Configuration properties: + - env + - files + +Update the run? [y/n]: +``` + +
+ +If approved, `dstack` gradually updates the service replicas. To update a replica, `dstack` starts a new replica, waits for it to become `running`, then terminates the old replica. This process is repeated for each replica, one at a time. + +You can track the progress of rolling deployment in both `dstack apply` or `dstack ps`. +Older replicas have lower `deployment` numbers; newer ones have higher. + + + +```shell +$ dstack apply -f my-service.dstack.yml + +⠋ Launching my-service... + NAME BACKEND PRICE STATUS SUBMITTED + my-service deployment=1 running 11 mins ago + replica=0 job=0 deployment=0 aws (us-west-2) $0.0026 terminating 11 mins ago + replica=1 job=0 deployment=1 aws (us-west-2) $0.0026 running 1 min ago +``` + +The rolling deployment stops when all replicas are updated or when a new deployment is submitted. + +??? info "Supported properties" + + + Rolling deployment supports changes to the following properties: `resources`, `volumes`, `docker`, `files`, `image`, `user`, `privileged`, `entrypoint`, `working_dir`, `python`, `nvcc`, `single_branch`, `env`, `shell`, `commands`, as well as changes to [repo](repos.md) or [file](#files) contents. + + Changes to `replicas` and `scaling` can be applied without redeploying replicas. + + Changes to other properties require a full service restart. + + To trigger a rolling deployment when no properties have changed (e.g., after updating [secrets](secrets.md) or to restart all replicas), + make a minor config change, such as adding a dummy [environment variable](#environment-variables). + --8<-- "docs/concepts/snippets/manage-runs.ext" !!! info "What's next?" diff --git a/src/dstack/_internal/server/services/runs.py b/src/dstack/_internal/server/services/runs.py index ce8325ce72..629f9b2ecc 100644 --- a/src/dstack/_internal/server/services/runs.py +++ b/src/dstack/_internal/server/services/runs.py @@ -930,6 +930,7 @@ def _validate_run_spec_and_set_defaults(run_spec: RunSpec): "replicas", "scaling", # rolling deployment + # NOTE: keep this list in sync with the "Rolling deployment" section in services.md "resources", "volumes", "docker",