You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Doc][Misc] Standardize hardware specifications, fix typos, and correct links in model tutorials (vllm-project#13100)
### What this PR does / why we need it?
This pull request performs a comprehensive cleanup and standardization
of the model tutorial documentation. Specifically, it:
- Standardizes memory unit representations (e.g., changing `64G` to
`64GB` and `32G` to `32GB`).
- Standardizes context and token length casing (e.g., changing `128K` to
`128k`, `16K` to `16k`, etc.).
- Corrects various typos (e.g., `ominidocbench` to `omnidocbench`,
`multinodes` to `multi-node clusters`, and `Rope` to `RoPE`).
- Fixes broken or outdated documentation URLs and GitHub issue links.
- Improves grammar, spacing, and formatting across multiple markdown
files.
These changes ensure consistency, accuracy, and readability across all
model deployment guides.
### Does this PR introduce _any_ user-facing change?
No, this PR only updates documentation and does not introduce any
user-facing API or behavioral changes.
### How was this patch tested?
Documentation changes only; verified by reviewing the rendered markdown
files.
- vLLM version: v0.25.1
- vLLM main:
vllm-project/vllm@d02df74
Signed-off-by: x60087760 <xiechangxiu3@h-partners.com>
Co-authored-by: x60087760 <xiechangxiu3@h-partners.com>
Copy file name to clipboardExpand all lines: docs/source/tutorials/models/DeepSeek-R1.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -165,7 +165,7 @@ Key Parameter Descriptions:
165
165
166
166
- Setting the environment variable `VLLM_ASCEND_BALANCE_SCHEDULING=1` enables balance scheduling. This may help increase output throughput and reduce TPOT in v1 scheduler. However, TTFT may degrade in some scenarios. Furthermore, enabling this feature is not recommended in scenarios where PD is separated.
167
167
- For single-node deployment, we recommend using `dp4tp4` instead of `dp2tp8`.
168
-
-`--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5K and output length of 1.5K, a value of `16384` is sufficient, however, for precision testing, please set it to at least `35000`.
168
+
-`--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5k and output length of 1.5k, a value of `16384` is sufficient, however, for precision testing, please set it to at least `35000`.
169
169
-`--no-enable-prefix-caching` indicates that prefix caching is disabled. To enable it, remove this option.
170
170
- If you use the w4a8 weight, more memory will be allocated to kvcache, and you can try to increase system throughput to achieve greater throughput.
Copy file name to clipboardExpand all lines: docs/source/tutorials/models/DeepSeek-V3.1.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -129,7 +129,7 @@ If you want to deploy multi-node environment, you need to set up environment on
129
129
130
130
### 5.1 Single-Node Online Deployment
131
131
132
-
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V3.1-w8a8-mtp-QuaRot` can be deployed on 1 Atlas 800 A3 (64G × 16).
132
+
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V3.1-w8a8-mtp-QuaRot` can be deployed on 1 Atlas 800 A3 (64GB × 16).
133
133
134
134
Startup Command:
135
135
@@ -177,7 +177,7 @@ Key Parameter Descriptions:
177
177
178
178
- Setting the environment variable `VLLM_ASCEND_BALANCE_SCHEDULING=1` enables balance scheduling. This may help increase output throughput and reduce TPOT in v1 scheduler. However, TTFT may degrade in some scenarios. Furthermore, enabling this feature is not recommended in scenarios where PD is separated.
179
179
- For single-node deployment, we recommend using `dp4tp4` instead of `dp2tp8`.
180
-
-`--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5K and output length of 1.5K, a value of `16384` is sufficient, however, for precision testing, please set it at least `35000`.
180
+
-`--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5k and output length of 1.5k, a value of `16384` is sufficient, however, for precision testing, please set it at least `35000`.
181
181
-`--no-enable-prefix-caching` indicates that prefix caching is disabled. To enable it, remove this option.
182
182
- If you use the w4a8 weight, more memory will be allocated to kvcache, and you can try to increase system throughput to achieve greater throughput.
183
183
@@ -233,7 +233,7 @@ The service returns HTTP 200 OK with a JSON response containing the `choices` fi
233
233
234
234
### 5.2 Multi-Node Data Parallel Deployment
235
235
236
-
-`DeepSeek-V3.1-w8a8-mtp-QuaRot`: require at least 2 Atlas 800 A2 (64G × 8).
236
+
-`DeepSeek-V3.1-w8a8-mtp-QuaRot`: require at least 2 Atlas 800 A2 (64GB × 8).
237
237
238
238
Run the following scripts on two nodes respectively.
239
239
@@ -393,9 +393,9 @@ In the standard single-node deployment mode, Prefill (prompt processing) and Dec
393
393
394
394
This architecture is recommended for production deployments with concurrent multi-user workloads, where stable latency and high throughput are both required.
395
395
396
-
Take Atlas 800 A3 (64G × 16) for example, we recommend to deploy 2P1D (4 nodes) rather than 1P1D (2 nodes), because there is no enough NPU memory to serve high concurrency in 1P1D case.
396
+
Take Atlas 800 A3 (64GB × 16) for example, we recommend to deploy 2P1D (4 nodes) rather than 1P1D (2 nodes), because there is no enough NPU memory to serve high concurrency in 1P1D case.
To run the vllm-ascend `Prefill-Decode Disaggregation` service, you need to deploy a `launch_online_dp.py` script and a `run_dp_template.sh` script on each node and deploy a `proxy.sh` script on prefill master node to forward requests.
401
401
@@ -873,8 +873,8 @@ Here is one accuracy evaluation method.
It is recommended to download the model weight to the shared directory of multiple nodes, such as `/root/.cache/`.
29
29
@@ -137,7 +137,7 @@ If you want to deploy a multi-node environment, you need to set up the environme
137
137
138
138
### 5.1 Single-Node Online Deployment
139
139
140
-
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V4-Flash-w8a8-mtp` can be deployed on 1 Atlas 800 A3 (128G × 8) or 1 Atlas 800 A2 (64G × 8).
140
+
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V4-Flash-w8a8-mtp` can be deployed on 1 Atlas 800 A3 (128GB × 8) or 1 Atlas 800 A2 (64GB × 8).
141
141
142
142
=== "A2 series"
143
143
@@ -306,11 +306,11 @@ In the standard single-node deployment mode, Prefill (prompt processing) and Dec
306
306
307
307
PD (Prefill-Decode) separation addresses these issues by running Prefill and Decode on dedicated node groups, each configured independently. This architecture is recommended for production deployments with concurrent multi-user workloads, where stable latency and high throughput are both required.
308
308
309
-
The following sections describe PD separation deployment on both Atlas 800 A3 (128G × 8) and Atlas 800 A2 (64G × 8) multi-node environments.
309
+
The following sections describe PD separation deployment on both Atlas 800 A3 (128GB × 8) and Atlas 800 A2 (64GB × 8) multi-node environments.
310
310
311
311
#### 5.2.1 A3 Series PD Separation Deployment
312
312
313
-
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A3 (128G × 8) multi-node environment with 1P1D for better performance.
313
+
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A3 (128GB × 8) multi-node environment with 1P1D for better performance.
314
314
315
315
Before you start, please:
316
316
@@ -601,7 +601,7 @@ Before you start, please:
601
601
602
602
#### 5.2.2 A2 Series PD Separation Deployment
603
603
604
-
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A2 (64G × 8) multi-node environment with4\*1P 1\*4D for better performance.
604
+
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A2 (64GB × 8) multi-node environment with4\*1P 1\*4D for better performance.
605
605
606
606
Before you start, please:
607
607
@@ -916,8 +916,8 @@ Here are two accuracy evaluation methods.
@@ -961,7 +961,7 @@ Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/contributing/) for more
961
961
962
962
`max-model-len`and`max-num-seqs` need to be set according to the actual usage scenario. For other settings, please refer to the [Deployment](#5-online-service-deployment) chapter.
963
963
964
-
Currently, we support 4K prefix cache hit in an experimental manner. You only need to change the value of --block-size from128 to 32in the service.
964
+
Currently, we support 4k prefix cache hit in an experimental manner. You only need to change the value of --block-size from128 to 32in the service.
It is recommended to download the model weight to the shared directory of multiple nodes, such as `/root/.cache/`.
29
29
@@ -133,7 +133,7 @@ If you want to deploy a multi-node environment, you need to set up the environme
133
133
134
134
### 5.1 Multi-Node Online Deployment
135
135
136
-
The quantized model `DeepSeek-V4-Pro-w4a8-mtp` requires at least 2 Atlas 800 A3 (128G × 8) nodes or 4 Atlas 800 A2 (64G × 8) nodes. Run the following scripts on each node respectively.
136
+
The quantized model `DeepSeek-V4-Pro-w4a8-mtp` requires at least 2 Atlas 800 A3 (128GB × 8) nodes or 4 Atlas 800 A2 (64GB × 8) nodes. Run the following scripts on each node respectively.
137
137
138
138
=== "A2 series"
139
139
@@ -447,11 +447,11 @@ We recommend using Mooncake for deployment: [Mooncake](../features/pd_disaggrega
447
447
448
448
In the standard deployment mode, Prefill (prompt processing) and Decode (token generation) tasks run on the same set of NPUs. PD (Prefill-Decode) separation addresses this by running Prefill and Decode on dedicated node groups, each configured independently. This architecture is recommended for production deployments with concurrent multi-user workloads, where stable latency and high throughput are both required.
449
449
450
-
The following sections describe PD separation deployment on both Atlas 800 A3 (128G × 8) and Atlas 800 A2 (64G × 8) multi-node environments.
450
+
The following sections describe PD separation deployment on both Atlas 800 A3 (128GB × 8) and Atlas 800 A2 (64GB × 8) multi-node environments.
451
451
452
452
#### 5.2.1 A3 Series PD Separation Deployment
453
453
454
-
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A3 (128G × 8) multi-node environment with 1P1D for better performance.
454
+
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A3 (128GB × 8) multi-node environment with 1P1D for better performance.
455
455
456
456
Before you start, please:
457
457
@@ -826,7 +826,7 @@ Before you start, please:
826
826
827
827
#### 5.2.2 A2 Series PD Separation Deployment
828
828
829
-
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A2 (64G × 8) multi-node environment with1P1Dfor better performance.
829
+
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A2 (64GB × 8) multi-node environment with1P1Dfor better performance.
830
830
831
831
Before you start, please:
832
832
@@ -1202,8 +1202,8 @@ Here are two accuracy evaluation methods.
0 commit comments