Skip to content

Commit fd35fa5

Browse files
authored
[DOC][310P] Update Atlas 200I Pro container startup method (vllm-project#11179)
### What this PR does / why we need it? Documentation Update: Updated the 310P hardware tutorial to include specific deployment instructions for the Atlas 200I Pro acceleration module. Container Configuration: Added detailed Docker run commands for both Ubuntu 24.04 and openEuler 24.03, including necessary volume mounts for driver libraries and configuration files to ensure proper functionality of npu-smi. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@dc68bd8 --------- Signed-off-by: YangShuai52 <shuaiyang047@163.com>
1 parent b29e699 commit fd35fa5

1 file changed

Lines changed: 108 additions & 8 deletions

File tree

  • docs/source/tutorials/hardwares

docs/source/tutorials/hardwares/310p.md

Lines changed: 108 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
# Atlas 300I DUO
1+
# Atlas 300I DUO / Atlas 200I Pro
2+
3+
This tutorial covers vLLM deployment on Ascend 310P inference hardware, including
4+
Atlas 300I DUO and Atlas 200I Pro acceleration modules.
25

36
```{note}
47
Atlas 300I DUO does not support `triton` or `triton-ascend`.
@@ -27,6 +30,10 @@ The current release supports `FULL_DECODE_ONLY` graph mode on Atlas 300I DUO dev
2730

2831
### Deployment
2932

33+
Choose the startup command according to your hardware form factor.
34+
35+
#### Atlas 300I DUO
36+
3037
Run docker container:
3138

3239
```{code-block} bash
@@ -59,6 +66,93 @@ docker run --rm \
5966
-it $IMAGE bash
6067
```
6168

69+
#### Atlas 200I Pro Acceleration Module
70+
71+
When starting a container on Atlas 200I Pro, mount additional driver libraries and
72+
configuration files required by `npu-smi`. Without these mounts, `npu-smi` commands
73+
may fail inside the container. Use the command below for your container OS.
74+
75+
```{note}
76+
Atlas 200I Pro also uses Ascend 310P. Adjust `--device=/dev/davinci0` according to
77+
the NPU ID you want to use. You can query available devices with
78+
`ll /dev/ | grep davinci`.
79+
```
80+
81+
:::::{tab-set}
82+
83+
::::{tab-item} Ubuntu 24.04
84+
:selected:
85+
86+
```{code-block} bash
87+
:substitutions:
88+
89+
export IMAGE=quay.io/ascend/vllm-ascend:|vllm_ascend_version|-310p
90+
91+
docker run --rm \
92+
--privileged \
93+
--name vllm-ascend \
94+
--shm-size=10g \
95+
--device=/dev/davinci0:/dev/davinci0 \
96+
--device=/dev/davinci_manager \
97+
--device=/dev/ascend_manager \
98+
--device=/dev/user_config \
99+
-v /etc/sys_version.conf:/etc/sys_version.conf \
100+
-v /etc/ld.so.conf.d/mind_so.conf:/etc/ld.so.conf.d/mind_so.conf \
101+
-v /etc/hdcBasic.cfg:/etc/hdcBasic.cfg \
102+
-v /var/dmp_daemon:/var/dmp_daemon \
103+
-v /usr/lib64/libmmpa.so:/usr/lib64/libmmpa.so \
104+
-v /usr/lib64/libcrypto.so.1.1:/usr/lib64/libcrypto.so.1.1 \
105+
-v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
106+
-v /usr/lib64/libstackcore.so:/usr/lib64/libstackcore.so \
107+
-v /usr/lib/aarch64-linux-gnu/libyaml-0.so.2:/usr/lib64/libyaml-0.so.2 \
108+
-v /etc/slog.conf:/etc/slog.conf \
109+
-v /var/slogd:/var/slogd \
110+
-v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64 \
111+
-v /usr/lib64/libtensorflow.so:/usr/lib64/libtensorflow.so \
112+
-v /root/.cache:/root/.cache \
113+
-p 8080:8080 \
114+
-it $IMAGE bash
115+
```
116+
117+
::::
118+
119+
::::{tab-item} openEuler 24.03
120+
121+
```{code-block} bash
122+
:substitutions:
123+
124+
export IMAGE=quay.io/ascend/vllm-ascend:|vllm_ascend_version|-310p-openeuler
125+
126+
docker run --rm \
127+
--privileged \
128+
--name vllm-ascend \
129+
--shm-size=10g \
130+
--device=/dev/davinci0:/dev/davinci0 \
131+
--device=/dev/davinci_manager \
132+
--device=/dev/ascend_manager \
133+
--device=/dev/user_config \
134+
-v /etc/sys_version.conf:/etc/sys_version.conf \
135+
-v /etc/ld.so.conf.d/mind_so.conf:/etc/ld.so.conf.d/mind_so.conf \
136+
-v /etc/hdcBasic.cfg:/etc/hdcBasic.cfg \
137+
-v /var/dmp_daemon:/var/dmp_daemon \
138+
-v /usr/lib64/libsemanage.so.2:/usr/lib64/libsemanage.so.2 \
139+
-v /usr/lib64/libmmpa.so:/usr/lib64/libmmpa.so \
140+
-v /usr/lib64/libcrypto.so.1.1:/usr/lib64/libcrypto.so.1.1 \
141+
-v /usr/lib64/libyaml-0.so.2.0.9:/usr/lib64/libyaml-0.so.2 \
142+
-v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
143+
-v /usr/lib64/libstackcore.so:/usr/lib64/libstackcore.so \
144+
-v /etc/slog.conf:/etc/slog.conf \
145+
-v /var/slogd:/var/slogd \
146+
-v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64 \
147+
-v /usr/lib64/libtensorflow.so:/usr/lib64/libtensorflow.so \
148+
-v /root/.cache:/root/.cache \
149+
-p 8080:8080 \
150+
-it $IMAGE bash
151+
```
152+
153+
::::
154+
:::::
155+
62156
Set up environment variables:
63157

64158
```bash
@@ -106,7 +200,10 @@ Use the W8A8SC quantized weights from the Eco-Tech official ModelScope repositor
106200
- [Eco-Tech/Qwen3-32B-w8a8sc-310-vllm](https://www.modelscope.cn/models/Eco-Tech/Qwen3-32B-w8a8sc-310-vllm)
107201
```
108202

109-
#### Qwen3-8B-W8A8SC
203+
:::::{tab-set}
204+
205+
::::{tab-item} Qwen3-8B-W8A8SC
206+
:selected:
110207

111208
```{code-block} bash
112209
:substitutions:
@@ -127,7 +224,9 @@ vllm serve Eco-Tech/Qwen3-8B-w8a8sc-310-vllm/TP1/Qwen3-8B-w8a8sc-310-vllm-tp1 \
127224
--load_format sharded_state
128225
```
129226

130-
#### Qwen3-14B-W8A8SC
227+
::::
228+
229+
::::{tab-item} Qwen3-14B-W8A8SC
131230

132231
```{code-block} bash
133232
:substitutions:
@@ -148,7 +247,9 @@ vllm serve Eco-Tech/Qwen3-14B-w8a8sc-310-vllm/TP1/Qwen3-14B-w8a8sc-310-vllm-tp1
148247
--load_format sharded_state
149248
```
150249

151-
#### Qwen3-32B-W8A8SC
250+
::::
251+
252+
::::{tab-item} Qwen3-32B-W8A8SC
152253

153254
```{code-block} bash
154255
:substitutions:
@@ -171,6 +272,9 @@ vllm serve Eco-Tech/Qwen3-32B-w8a8sc-310-vllm/TP4/Qwen3-32B-w8a8sc-310-vllm-tp4
171272
--load_format sharded_state
172273
```
173274

275+
::::
276+
:::::
277+
174278
Once the server is started, you can query the model with input prompts:
175279

176280
```bash
@@ -190,11 +294,9 @@ If the script runs successfully, you can see the generated result.
190294
Run the following script, `example.py`, to execute offline inference on NPU.
191295

192296
:::::{tab-set}
193-
:sync-group: inference
194297

195298
::::{tab-item} Qwen3-8B-W8A8SC
196299
:selected:
197-
:sync: qwen3-8b
198300

199301
```{code-block} python
200302
:substitutions:
@@ -259,7 +361,6 @@ clean_up()
259361
::::
260362

261363
::::{tab-item} Qwen3-14B-W8A8SC
262-
:sync: qwen3-14b
263364

264365
```{code-block} python
265366
:substitutions:
@@ -324,7 +425,6 @@ clean_up()
324425
::::
325426

326427
::::{tab-item} Qwen3-32B-W8A8SC
327-
:sync: qwen3-32b
328428

329429
```{code-block} python
330430
:substitutions:

0 commit comments

Comments
 (0)