Skip to content

Commit c8710f2

Browse files
ella1107x60087760
andauthored
[Doc][Misc] Standardize hardware specifications, fix typos, and correct links in model tutorials (vllm-project#13100)
### What this PR does / why we need it? This pull request performs a comprehensive cleanup and standardization of the model tutorial documentation. Specifically, it: - Standardizes memory unit representations (e.g., changing `64G` to `64GB` and `32G` to `32GB`). - Standardizes context and token length casing (e.g., changing `128K` to `128k`, `16K` to `16k`, etc.). - Corrects various typos (e.g., `ominidocbench` to `omnidocbench`, `multinodes` to `multi-node clusters`, and `Rope` to `RoPE`). - Fixes broken or outdated documentation URLs and GitHub issue links. - Improves grammar, spacing, and formatting across multiple markdown files. These changes ensure consistency, accuracy, and readability across all model deployment guides. ### Does this PR introduce _any_ user-facing change? No, this PR only updates documentation and does not introduce any user-facing API or behavioral changes. ### How was this patch tested? Documentation changes only; verified by reviewing the rendered markdown files. - vLLM version: v0.25.1 - vLLM main: vllm-project/vllm@d02df74 Signed-off-by: x60087760 <xiechangxiu3@h-partners.com> Co-authored-by: x60087760 <xiechangxiu3@h-partners.com>
1 parent b100615 commit c8710f2

24 files changed

Lines changed: 142 additions & 142 deletions

docs/source/tutorials/models/DeepSeek-R1.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ Key Parameter Descriptions:
165165

166166
- Setting the environment variable `VLLM_ASCEND_BALANCE_SCHEDULING=1` enables balance scheduling. This may help increase output throughput and reduce TPOT in v1 scheduler. However, TTFT may degrade in some scenarios. Furthermore, enabling this feature is not recommended in scenarios where PD is separated.
167167
- For single-node deployment, we recommend using `dp4tp4` instead of `dp2tp8`.
168-
- `--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5K and output length of 1.5K, a value of `16384` is sufficient, however, for precision testing, please set it to at least `35000`.
168+
- `--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5k and output length of 1.5k, a value of `16384` is sufficient, however, for precision testing, please set it to at least `35000`.
169169
- `--no-enable-prefix-caching` indicates that prefix caching is disabled. To enable it, remove this option.
170170
- If you use the w4a8 weight, more memory will be allocated to kvcache, and you can try to increase system throughput to achieve greater throughput.
171171

docs/source/tutorials/models/DeepSeek-V3.1.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ If you want to deploy multi-node environment, you need to set up environment on
129129

130130
### 5.1 Single-Node Online Deployment
131131

132-
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V3.1-w8a8-mtp-QuaRot` can be deployed on 1 Atlas 800 A3 (64G × 16).
132+
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V3.1-w8a8-mtp-QuaRot` can be deployed on 1 Atlas 800 A3 (64GB × 16).
133133

134134
Startup Command:
135135

@@ -177,7 +177,7 @@ Key Parameter Descriptions:
177177

178178
- Setting the environment variable `VLLM_ASCEND_BALANCE_SCHEDULING=1` enables balance scheduling. This may help increase output throughput and reduce TPOT in v1 scheduler. However, TTFT may degrade in some scenarios. Furthermore, enabling this feature is not recommended in scenarios where PD is separated.
179179
- For single-node deployment, we recommend using `dp4tp4` instead of `dp2tp8`.
180-
- `--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5K and output length of 1.5K, a value of `16384` is sufficient, however, for precision testing, please set it at least `35000`.
180+
- `--max-model-len` specifies the maximum context length - that is, the sum of input and output tokens for a single request. For performance testing with an input length of 3.5k and output length of 1.5k, a value of `16384` is sufficient, however, for precision testing, please set it at least `35000`.
181181
- `--no-enable-prefix-caching` indicates that prefix caching is disabled. To enable it, remove this option.
182182
- If you use the w4a8 weight, more memory will be allocated to kvcache, and you can try to increase system throughput to achieve greater throughput.
183183

@@ -233,7 +233,7 @@ The service returns HTTP 200 OK with a JSON response containing the `choices` fi
233233

234234
### 5.2 Multi-Node Data Parallel Deployment
235235

236-
- `DeepSeek-V3.1-w8a8-mtp-QuaRot`: require at least 2 Atlas 800 A2 (64G × 8).
236+
- `DeepSeek-V3.1-w8a8-mtp-QuaRot`: require at least 2 Atlas 800 A2 (64GB × 8).
237237

238238
Run the following scripts on two nodes respectively.
239239

@@ -393,9 +393,9 @@ In the standard single-node deployment mode, Prefill (prompt processing) and Dec
393393

394394
This architecture is recommended for production deployments with concurrent multi-user workloads, where stable latency and high throughput are both required.
395395

396-
Take Atlas 800 A3 (64G × 16) for example, we recommend to deploy 2P1D (4 nodes) rather than 1P1D (2 nodes), because there is no enough NPU memory to serve high concurrency in 1P1D case.
396+
Take Atlas 800 A3 (64GB × 16) for example, we recommend to deploy 2P1D (4 nodes) rather than 1P1D (2 nodes), because there is no enough NPU memory to serve high concurrency in 1P1D case.
397397

398-
- `DeepSeek-V3.1-w8a8-mtp-QuaRot 2P1D Layerwise` require 4 Atlas 800 A3 (64G × 16).
398+
- `DeepSeek-V3.1-w8a8-mtp-QuaRot 2P1D Layerwise` require 4 Atlas 800 A3 (64GB × 16).
399399

400400
To run the vllm-ascend `Prefill-Decode Disaggregation` service, you need to deploy a `launch_online_dp.py` script and a `run_dp_template.sh` script on each node and deploy a `proxy.sh` script on prefill master node to forward requests.
401401

@@ -873,8 +873,8 @@ Here is one accuracy evaluation method.
873873
874874
| dataset | version | metric | mode | vllm-api-general-chat | note |
875875
|----- | ----- | ----- | ----- | -----| ----- |
876-
| ceval | - | accuracy | gen | 90.94 | 1 Atlas 800 A3 (64G × 16) |
877-
| gsm8k | - | accuracy | gen | 96.28 | 1 Atlas 800 A3 (64G × 16) |
876+
| ceval | - | accuracy | gen | 90.94 | 1 Atlas 800 A3 (64GB × 16) |
877+
| gsm8k | - | accuracy | gen | 96.28 | 1 Atlas 800 A3 (64GB × 16) |
878878
879879
### Using Language Model Evaluation Harness
880880
@@ -924,7 +924,7 @@ After about several minutes, you can get the performance evaluation result.
924924
925925
#### Table 1: Scenario Overview
926926
927-
> `*Total NPUs` indicates the total number of NPUs used across all nodes. 1 node = 1 Atlas 800 A3 server (64G × 16 NPUs).
927+
> `*Total NPUs` indicates the total number of NPUs used across all nodes. 1 node = 1 Atlas 800 A3 server (64GB × 16 NPUs).
928928
929929
|Scenario|Deployment Mode|*Total NPUs|Weight Version|Key Considerations|
930930
|--------|---------------|-----------|--------------|------------------|

docs/source/tutorials/models/DeepSeek-V3.2.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -492,7 +492,7 @@ Parameter descriptions:
492492
export ASCEND_TRANSPORT_PRINT=1
493493
export ACL_OP_INIT_MODE=1
494494
export ASCEND_A3_ENABLE=1
495-
# Timeout (in seconds) for automatically releasing the prefillers KV cache for a particular request.
495+
# Timeout (in seconds) for automatically releasing the prefiller's KV cache for a particular request.
496496
export VLLM_MOONCAKE_ABORT_REQUEST_TIMEOUT=480
497497

498498
export ASCEND_RT_VISIBLE_DEVICES=$1

docs/source/tutorials/models/DeepSeek-V4-Flash.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Refer to [feature guide](../../user_guide/feature_guide/index.md) to get the fea
2323

2424
### 3.1 Model Weight
2525

26-
- `DeepSeek-V4-Flash-w8a8-mtp` (Quantized version): requires 1 Atlas 800 A3 (128G × 8) node or 1 Atlas 800 A2 (64G × 8) node. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/DeepSeek-V4-Flash-w8a8-mtp)
26+
- `DeepSeek-V4-Flash-w8a8-mtp` (Quantized version): requires 1 Atlas 800 A3 (128GB × 8) node or 1 Atlas 800 A2 (64GB × 8) node. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/DeepSeek-V4-Flash-w8a8-mtp)
2727

2828
It is recommended to download the model weight to the shared directory of multiple nodes, such as `/root/.cache/`.
2929

@@ -137,7 +137,7 @@ If you want to deploy a multi-node environment, you need to set up the environme
137137

138138
### 5.1 Single-Node Online Deployment
139139

140-
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V4-Flash-w8a8-mtp` can be deployed on 1 Atlas 800 A3 (128G × 8) or 1 Atlas 800 A2 (64G × 8).
140+
Single-node deployment completes both Prefill and Decode within the same node. The quantized model `DeepSeek-V4-Flash-w8a8-mtp` can be deployed on 1 Atlas 800 A3 (128GB × 8) or 1 Atlas 800 A2 (64GB × 8).
141141

142142
=== "A2 series"
143143

@@ -306,11 +306,11 @@ In the standard single-node deployment mode, Prefill (prompt processing) and Dec
306306

307307
PD (Prefill-Decode) separation addresses these issues by running Prefill and Decode on dedicated node groups, each configured independently. This architecture is recommended for production deployments with concurrent multi-user workloads, where stable latency and high throughput are both required.
308308

309-
The following sections describe PD separation deployment on both Atlas 800 A3 (128G × 8) and Atlas 800 A2 (64G × 8) multi-node environments.
309+
The following sections describe PD separation deployment on both Atlas 800 A3 (128GB × 8) and Atlas 800 A2 (64GB × 8) multi-node environments.
310310

311311
#### 5.2.1 A3 Series PD Separation Deployment
312312

313-
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A3 (128G × 8) multi-node environment with 1P1D for better performance.
313+
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A3 (128GB × 8) multi-node environment with 1P1D for better performance.
314314

315315
Before you start, please:
316316

@@ -601,7 +601,7 @@ Before you start, please:
601601

602602
#### 5.2.2 A2 Series PD Separation Deployment
603603

604-
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A2 (64G × 8) multi-node environment with 4\*1P 1\*4D for better performance.
604+
This section shows the deployment guide of DeepSeek-V4-Flash on Atlas 800 A2 (64GB × 8) multi-node environment with 4\*1P 1\*4D for better performance.
605605

606606
Before you start, please:
607607

@@ -916,8 +916,8 @@ Here are two accuracy evaluation methods.
916916

917917
| dataset | version | metric | mode | vllm-api-general-chat | note |
918918
| ----- | ----- | ----- | ----- | ----- | ----- |
919-
| GPQA | - | accuracy | gen | 88.17 | 1 Atlas 800 A3 (128G × 8) |
920-
| GSM8K | - | accuracy | gen | 96.30 | 1 Atlas 800 A3 (128G × 8) |
919+
| GPQA | - | accuracy | gen | 88.17 | 1 Atlas 800 A3 (128GB × 8) |
920+
| GSM8K | - | accuracy | gen | 96.30 | 1 Atlas 800 A3 (128GB × 8) |
921921

922922
## 8 Performance Evaluation
923923

@@ -961,7 +961,7 @@ Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/contributing/) for more
961961

962962
`max-model-len` and `max-num-seqs` need to be set according to the actual usage scenario. For other settings, please refer to the [Deployment](#5-online-service-deployment) chapter.
963963

964-
Currently, we support 4K prefix cache hit in an experimental manner. You only need to change the value of --block-size from 128 to 32 in the service.
964+
Currently, we support 4k prefix cache hit in an experimental manner. You only need to change the value of --block-size from 128 to 32 in the service.
965965

966966
### 9.2 Tuning Guidelines
967967

docs/source/tutorials/models/DeepSeek-V4-Pro.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Refer to [feature guide](../../user_guide/feature_guide/index.md) to get the fea
2323

2424
### 3.1 Model Weight
2525

26-
- `DeepSeek-V4-Pro-w4a8-mtp` (Quantized version): requires 2 Atlas 800 A3 (128G × 8) nodes or 4 Atlas 800 A2 (64G × 8) nodes. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/DeepSeek-V4-Pro-w4a8-mtp)
26+
- `DeepSeek-V4-Pro-w4a8-mtp` (Quantized version): requires 2 Atlas 800 A3 (128GB × 8) nodes or 4 Atlas 800 A2 (64GB × 8) nodes. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/DeepSeek-V4-Pro-w4a8-mtp)
2727

2828
It is recommended to download the model weight to the shared directory of multiple nodes, such as `/root/.cache/`.
2929

@@ -133,7 +133,7 @@ If you want to deploy a multi-node environment, you need to set up the environme
133133

134134
### 5.1 Multi-Node Online Deployment
135135

136-
The quantized model `DeepSeek-V4-Pro-w4a8-mtp` requires at least 2 Atlas 800 A3 (128G × 8) nodes or 4 Atlas 800 A2 (64G × 8) nodes. Run the following scripts on each node respectively.
136+
The quantized model `DeepSeek-V4-Pro-w4a8-mtp` requires at least 2 Atlas 800 A3 (128GB × 8) nodes or 4 Atlas 800 A2 (64GB × 8) nodes. Run the following scripts on each node respectively.
137137

138138
=== "A2 series"
139139

@@ -447,11 +447,11 @@ We recommend using Mooncake for deployment: [Mooncake](../features/pd_disaggrega
447447

448448
In the standard deployment mode, Prefill (prompt processing) and Decode (token generation) tasks run on the same set of NPUs. PD (Prefill-Decode) separation addresses this by running Prefill and Decode on dedicated node groups, each configured independently. This architecture is recommended for production deployments with concurrent multi-user workloads, where stable latency and high throughput are both required.
449449

450-
The following sections describe PD separation deployment on both Atlas 800 A3 (128G × 8) and Atlas 800 A2 (64G × 8) multi-node environments.
450+
The following sections describe PD separation deployment on both Atlas 800 A3 (128GB × 8) and Atlas 800 A2 (64GB × 8) multi-node environments.
451451

452452
#### 5.2.1 A3 Series PD Separation Deployment
453453

454-
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A3 (128G × 8) multi-node environment with 1P1D for better performance.
454+
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A3 (128GB × 8) multi-node environment with 1P1D for better performance.
455455

456456
Before you start, please:
457457

@@ -826,7 +826,7 @@ Before you start, please:
826826

827827
#### 5.2.2 A2 Series PD Separation Deployment
828828

829-
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A2 (64G × 8) multi-node environment with 1P1D for better performance.
829+
This section shows the deployment guide of DeepSeek-V4-Pro on Atlas 800 A2 (64GB × 8) multi-node environment with 1P1D for better performance.
830830

831831
Before you start, please:
832832

@@ -1202,8 +1202,8 @@ Here are two accuracy evaluation methods.
12021202

12031203
| dataset | version | metric | mode | vllm-api-general-chat | note |
12041204
| ----- | ----- | ----- | ----- | ----- | ----- |
1205-
| GPQA | - | accuracy | gen | 89.90 | 1 Atlas 800 A3 (128G × 8) |
1206-
| GSM8K | - | accuracy | gen | 96.21 | 1 Atlas 800 A3 (128G × 8) |
1205+
| GPQA | - | accuracy | gen | 89.90 | 1 Atlas 800 A3 (128GB × 8) |
1206+
| GSM8K | - | accuracy | gen | 96.21 | 1 Atlas 800 A3 (128GB × 8) |
12071207

12081208
## 8 Performance Evaluation
12091209

docs/source/tutorials/models/DeepSeekOCR2.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -196,7 +196,7 @@ curl http://<node0_ip>:<port>/v1/completions \
196196
| dataset | version | metric | mode | vllm-api-general-chat | note |
197197
|----- | ----- | ----- | ----- | -----| ----- |
198198
| textvqa | - | accuracy | gen | 50.28 | 1 Atlas 800 A2 |
199-
| ominidocbench | - | accuracy | gen | 66.86 | 1 Atlas 800 A2 |
199+
| omnidocbench | - | accuracy | gen | 66.86 | 1 Atlas 800 A2 |
200200

201201
## 8 Performance Evaluation
202202

@@ -210,7 +210,7 @@ The performance result is:
210210

211211
**Input/Output**: 1080P/256
212212

213-
**Performance**: TTFT = 2s, TPOT = 200ms, Average performance of each card is 864 TPS (Token Per Second).
213+
**Performance**: TTFT = 2s, TPOT = 200ms, Average performance of each card is 864TPS (Token Per Second).
214214

215215
## 9 Performance Tuning
216216

@@ -220,7 +220,7 @@ The performance result is:
220220
221221
#### Table 1: Scenario Overview
222222

223-
> `*Total NPUs` indicates the total number of NPUs used across all nodes. 1 node = 1 Atlas 800 A3 server (64G × 16 NPUs).
223+
> `*Total NPUs` indicates the total number of NPUs used across all nodes. 1 node = 1 Atlas 800 A3 server (64GB × 16 NPUs).
224224
225225
|Scenario|Deployment Mode|*Total NPUs|Weight Version|Key Considerations|
226226
|--------|---------------|-----------|--------------|------------------|

docs/source/tutorials/models/GLM4.x.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ You can use our official docker image to run `GLM-4.x` directly.
7676

7777
=== "A2 series"
7878

79-
Start the docker image on your each node.
79+
Start the docker image on each node.
8080

8181
```bash
8282

@@ -262,7 +262,7 @@ While the previous documentation advises against multi-node deployment on the At
262262

263263
### 5.3 Prefill-Decode Disaggregation
264264

265-
We'd like to show the deployment guide of `GLM-4.7` on multi-node environment with 2P1D for better performance.
265+
We'd like to show the deployment guide of `GLM-4.7` in a multi-node environment with 2P1D for better performance.
266266

267267
Before you start, please
268268

docs/source/tutorials/models/GLM5.2.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,9 @@ Refer to [feature guide](../../user_guide/feature_guide/index.md) to get the fea
1616

1717
### Model Weight
1818

19-
- `GLM-5.2`(BF16 version): requires 2 Atlas 800 A3 (128G × 8) node or 4 Atlas 800 A2 (64G × 8) node.[Download model weight](https://www.modelscope.cn/models/ZhipuAI/GLM-5.2).
20-
- `GLM-5.2-w8a8`: requires 1 Atlas 800 A3 (128G × 8) node or 2 Atlas 800 A2 (64G × 8) node.[Download model weight](https://www.modelscope.cn/models/Eco-Tech/GLM-5.2-w8a8).
21-
- `GLM-5.2-w4a8c8`: requires 1 Atlas 800 A3 (128G × 8) node or 2 Atlas 800 A2 (64G × 8) node.[Download model weight](https://modelscope.cn/models/Eco-Tech/GLM-5.2-w4a8c8).
19+
- `GLM-5.2`(BF16 version): requires 2 Atlas 800 A3 (128GB × 8) node or 4 Atlas 800 A2 (64GB × 8) node.[Download model weight](https://www.modelscope.cn/models/ZhipuAI/GLM-5.2).
20+
- `GLM-5.2-w8a8`: requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 8) node.[Download model weight](https://www.modelscope.cn/models/Eco-Tech/GLM-5.2-w8a8).
21+
- `GLM-5.2-w4a8c8`: requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 8) node.[Download model weight](https://modelscope.cn/models/Eco-Tech/GLM-5.2-w4a8c8).
2222
- You can use [msmodelslim](https://gitcode.com/Ascend/msmodelslim) to quantize the model directly.
2323

2424
It is recommended to download the model weight to the shared directory of multiple nodes, such as `/root/.cache/`
@@ -110,7 +110,7 @@ If you want to deploy multi-node environment, you need to set up environment on
110110

111111
### Single-node Deployment
112112

113-
- Quantized model `GLM-5.2-w4a8c8` can be deployed on 1 Atlas 800 A3 (64G × 16) .
113+
- Quantized model `GLM-5.2-w4a8c8` can be deployed on 1 Atlas 800 A3 (64GB × 16) .
114114

115115
Run the following script to execute online inference.
116116

@@ -157,7 +157,7 @@ If you want to deploy multi-node environment, you need to verify multi-node comm
157157

158158
=== "A3 series"
159159

160-
- `GLM-5.2-w4a8c8`: can be deployed on 2 Atlas 800 A3 (64G × 16).
160+
- `GLM-5.2-w4a8c8`: can be deployed on 2 Atlas 800 A3 (64GB × 16).
161161

162162
Run the following scripts on two nodes respectively.
163163

@@ -263,7 +263,7 @@ If you want to deploy multi-node environment, you need to verify multi-node comm
263263

264264
=== "A2 series"
265265

266-
- `GLM-5.2-w4a8c8`: can be deployed on 2 Atlas 800 A2 (64G × 32).
266+
- `GLM-5.2-w4a8c8`: can be deployed on 2 Atlas 800 A2 (64GB × 32).
267267

268268
**node 0**
269269

@@ -378,9 +378,9 @@ If you want to deploy multi-node environment, you need to verify multi-node comm
378378

379379
### Prefill-Decode Disaggregation
380380

381-
We'd like to show the deployment guide of `GLM-5` on multi-node environment with 1P1D for better performance.
381+
We'd like to show the deployment guide of `GLM-5.2` on multi-node environment with 1P1D for better performance.
382382

383-
Prefill-Decode disaggregation can be deployed on 4 Atlas 800 A3 (64G × 32).
383+
Prefill-Decode disaggregation can be deployed on 4 Atlas 800 A3 (64GB × 32).
384384

385385
Before you start, please
386386

@@ -1122,12 +1122,12 @@ Please refer to the following python file for further explanation and restrictio
11221122
11231123
### 1M Context Configuration
11241124
1125-
Recommended configurations for serving `GLM-5.2` with a 1M context window on Atlas 800 A3 (64G x 16) and quantized GLM-5.2(W4A8C8) weights:
1125+
Recommended configurations for serving `GLM-5.2` with a 1M context window on Atlas 800 A3 (64GB x 16) and quantized GLM-5.2(W4A8C8) weights:
11261126
11271127
| Mode | Hardware | Parallelism | Context |
11281128
| ---- | -------- | ----------- | ------- |
1129-
| Single-node co-located | 1 Atlas 800 A3 (64G x 16) | `DP1 PP1 TP16 PCP1 DCP16` | `1024000` |
1130-
| Dual-node co-located | 2 Atlas 800 A3 (64G x 16) | `DP4 PP1 TP8 PCP1 DCP8` | `1024000` |
1129+
| Single-node co-located | 1 Atlas 800 A3 (64GB x 16) | `DP1 PP1 TP16 PCP1 DCP16` | `1024000` |
1130+
| Dual-node co-located | 2 Atlas 800 A3 (64GB x 16) | `DP4 PP1 TP8 PCP1 DCP8` | `1024000` |
11311131
| 1P1D PD disaggregation | 1 prefiller with 2 A3 nodes + 1 decoder with 2 A3 nodes | Prefill `DP4 PP1 TP8 PCP1 DCP8`, Decode `DP4 PP1 TP8 PCP1 DCP8` | `1024000` |
11321132
11331133
#### Single-Node 1M Deployment

0 commit comments

Comments
 (0)