|
| 1 | +--- |
| 2 | +title: "Rolling deployment, Secrets, Files, Tenstorrent, and more" |
| 3 | +date: 2025-07-10 |
| 4 | +description: "TBA" |
| 5 | +slug: changelog-07-25 |
| 6 | +image: https://dstack.ai/static-assets/static-assets/images/changelog-07-25.png |
| 7 | +categories: |
| 8 | + - Changelog |
| 9 | +--- |
| 10 | + |
| 11 | +# Rolling deployment, Secrets, Files, Tenstorrent, and more |
| 12 | + |
| 13 | +Thanks to feedback from the community, `dstack` continues to evolve. Here’s a look at what’s new. |
| 14 | + |
| 15 | +#### Rolling deployments |
| 16 | + |
| 17 | +Previously, updating running services could cause downtime. The latest release fixes this with [rolling deployments](../../docs/concepts/services.md/#rolling-deployment). Replicas are now updated one by one, allowing uninterrupted traffic during redeployments. |
| 18 | + |
| 19 | +<div class="termy"> |
| 20 | + |
| 21 | +```shell |
| 22 | +$ dstack apply -f .dstack.yml |
| 23 | + |
| 24 | +Active run my-service already exists. Detected changes that can be updated in-place: |
| 25 | +- Repo state (branch, commit, or other) |
| 26 | +- File archives |
| 27 | +- Configuration properties: |
| 28 | + - env |
| 29 | + - files |
| 30 | + |
| 31 | +Update the run? [y/n]: y |
| 32 | +⠋ Launching my-service... |
| 33 | + |
| 34 | + NAME BACKEND PRICE STATUS SUBMITTED |
| 35 | + my-service deployment=1 running 11 mins ago |
| 36 | + replica=0 deployment=0 aws (us-west-2) $0.0026 terminating 11 mins ago |
| 37 | + replica=1 deployment=1 aws (us-west-2) $0.0026 running 1 min ago |
| 38 | +``` |
| 39 | + |
| 40 | +</div> |
| 41 | + |
| 42 | +<!-- more --> |
| 43 | + |
| 44 | +#### Secrets |
| 45 | + |
| 46 | +Secrets let you centrally manage sensitive data like API keys and credentials. They’re scoped to a project, managed by project admins, and can be [securely referenced](../../docs/concepts/secrets.md) in run configurations. |
| 47 | + |
| 48 | +<div editor-title=".dstack.yml"> |
| 49 | + |
| 50 | +```yaml hl_lines="7" |
| 51 | +type: task |
| 52 | +name: train |
| 53 | + |
| 54 | +image: nvcr.io/nvidia/pytorch:25.05-py3 |
| 55 | +registry_auth: |
| 56 | + username: $oauthtoken |
| 57 | + password: ${{ secrets.ngc_api_key }} |
| 58 | + |
| 59 | +commands: |
| 60 | + - git clone https://github.com/pytorch/examples.git pytorch-examples |
| 61 | + - cd pytorch-examples/distributed/ddp-tutorial-series |
| 62 | + - pip install -r requirements.txt |
| 63 | + - | |
| 64 | + torchrun \ |
| 65 | + --nproc-per-node=$DSTACK_GPUS_PER_NODE \ |
| 66 | + --nnodes=$DSTACK_NODES_NUM \ |
| 67 | + multinode.py 50 10 |
| 68 | +
|
| 69 | +resources: |
| 70 | + gpu: H100:1..2 |
| 71 | + shm_size: 24GB |
| 72 | +``` |
| 73 | +
|
| 74 | +</div> |
| 75 | +
|
| 76 | +#### Files |
| 77 | +
|
| 78 | +By default, `dstack` mounts the repo directory (where you ran `dstack init`) to all runs. |
| 79 | + |
| 80 | +If the directory is large or you need files outside of it, use the new [files](../../docs/concepts/dev-environments/#files) property to map specific local paths into the container. |
| 81 | + |
| 82 | +<div editor-title=".dstack.yml"> |
| 83 | + |
| 84 | +```yaml |
| 85 | +type: task |
| 86 | +name: trl-sft |
| 87 | +
|
| 88 | +files: |
| 89 | + - .:examples # Maps the directory where `.dstack.yml` to `/workflow/examples` |
| 90 | + - ~/.ssh/id_rsa:/root/.ssh/id_rsa # Maps `~/.ssh/id_rsa` to `/root/.ssh/id_rs |
| 91 | + |
| 92 | +python: 3.12 |
| 93 | + |
| 94 | +env: |
| 95 | + - HF_TOKEN |
| 96 | + - HF_HUB_ENABLE_HF_TRANSFER=1 |
| 97 | + - MODEL=Qwen/Qwen2.5-0.5B |
| 98 | + - DATASET=stanfordnlp/imdb |
| 99 | + |
| 100 | +commands: |
| 101 | + - uv pip install trl |
| 102 | + - | |
| 103 | + trl sft \ |
| 104 | + --model_name_or_path $MODEL --dataset_name $DATASET |
| 105 | + --num_processes $DSTACK_GPUS_PER_NODE |
| 106 | +
|
| 107 | +resources: |
| 108 | + gpu: H100:1 |
| 109 | +``` |
| 110 | +
|
| 111 | +</div> |
| 112 | +
|
| 113 | +#### Tenstorrent |
| 114 | +
|
| 115 | +`dstack` remains committed to supporting multiple GPU vendors—including NVIDIA, AMD, TPUs, and more recently, [Tenstorrent :material-arrow-top-right-thin:{ .external }](https://tenstorrent.com/){:target="_blank"}. The latest release improves Tenstorrent support by handling hosts with multiple N300 cards and adds Docker-in-Docker support. |
| 116 | + |
| 117 | +<img src="https://dstack.ai/static-assets/static-assets/images/dstack-tenstorrent-n300.png" width="630"/> |
| 118 | + |
| 119 | +Huge thanks to the Tenstorrent community for testing these improvements! |
| 120 | + |
| 121 | +#### Docker in Docker |
| 122 | + |
| 123 | +Using Docker inside `dstack` run configurations is now even simpler. Just set `docker` to `true` to [enable the use of Docker CLI](../../docs/concepts/tasks.md#docker-in-docker) in your runs—allowing you to build images, run containers, use Docker Compose, and more. |
| 124 | + |
| 125 | +<div editor-title=".dstack.yml"> |
| 126 | + |
| 127 | +```yaml |
| 128 | +type: task |
| 129 | +name: docker-nvidia-smi |
| 130 | +
|
| 131 | +docker: true |
| 132 | +
|
| 133 | +commands: |
| 134 | + - | |
| 135 | + docker run --gpus all \ |
| 136 | + nvidia/cuda:12.3.0-base-ubuntu22.04 \ |
| 137 | + nvidia-smi |
| 138 | +
|
| 139 | +resources: |
| 140 | + gpu: H100:1 |
| 141 | +``` |
| 142 | + |
| 143 | +</div> |
| 144 | + |
| 145 | +#### AWS EFA |
| 146 | + |
| 147 | +EFA is a network interface for EC2 that enables low-latency, high-bandwidth communication between nodes—crucial for scaling distributed deep learning. With `dstack`, EFA is automatically enabled when using supported instance types in fleets. Check out our [example](../../examples/clusters/efa/index.md) |
| 148 | + |
| 149 | +#### Default Docker images |
| 150 | + |
| 151 | +If no `image` is specified, `dstack` uses a base Docker image that now comes pre-configured with `uv`, `python`, `pip`, essential CUDA drivers, InfiniBand, and NCCL tests (located at `/opt/nccl-tests/build`). |
| 152 | + |
| 153 | +<div editor-title="examples/clusters/nccl-tests/.dstack.yml"> |
| 154 | + |
| 155 | +```yaml |
| 156 | +type: task |
| 157 | +name: nccl-tests |
| 158 | +
|
| 159 | +nodes: 2 |
| 160 | +
|
| 161 | +startup_order: workers-first |
| 162 | +stop_criteria: master-done |
| 163 | +
|
| 164 | +env: |
| 165 | + - NCCL_DEBUG=INFO |
| 166 | +commands: |
| 167 | + - | |
| 168 | + if [ $DSTACK_NODE_RANK -eq 0 ]; then |
| 169 | + mpirun \ |
| 170 | + --allow-run-as-root \ |
| 171 | + --hostfile $DSTACK_MPI_HOSTFILE \ |
| 172 | + -n $DSTACK_GPUS_NUM \ |
| 173 | + -N $DSTACK_GPUS_PER_NODE \ |
| 174 | + --bind-to none \ |
| 175 | + /opt/nccl-tests/build/all_reduce_perf -b 8 -e 8G -f 2 -g 1 |
| 176 | + else |
| 177 | + sleep infinity |
| 178 | + fi |
| 179 | +
|
| 180 | +resources: |
| 181 | + gpu: nvidia:1..8 |
| 182 | + shm_size: 16GB |
| 183 | +``` |
| 184 | + |
| 185 | +</div> |
| 186 | + |
| 187 | +These images are optimized for common use cases and kept lightweight—ideal for everyday development, training, and inference. |
| 188 | + |
| 189 | +#### Server performance |
| 190 | + |
| 191 | +Server-side performance has been improved. With optimized handling and background processing, each server replica can now handle more runs. |
| 192 | + |
| 193 | +#### Google SSO |
| 194 | + |
| 195 | +Alongside the open-source version, `dstack` also offers [dstack Enterprise :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack-enterprise){:target="_blank"} — which adds dedicated support and extra integrations like Single Sign-On (SSO). The latest release introduces support for configuring your company’s Google account for authentication. |
| 196 | + |
| 197 | +<img src="https://dstack.ai/static-assets/static-assets/images/dstack-enterprise-google-sso.png" width="630"/> |
| 198 | + |
| 199 | +If you’d like to learn more about `dstack` Enterprise, [let us know](https://calendly.com/dstackai/discovery-call). |
| 200 | + |
| 201 | +That’s all for now. |
| 202 | + |
| 203 | +!!! info "What's next?" |
| 204 | + Give dstack a try, and share your feedback—whether it’s [GitHub :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack){:target="_blank"} issues, PRs, or questions on [Discord :material-arrow-top-right-thin:{ .external }](https://discord.gg/u8SmfwPpMd){:target="_blank"}. We’re eager to hear from you! |
0 commit comments