Skip to content

Commit 87c1ceb

Browse files
authored
[Doc][Misc] Update Ascend PyTorch Profiler guide (vllm-project#13156)
### What this PR does / why we need it? This PR updates the Ascend PyTorch Profiler guide to: - Remove deprecated/invalid `VLLM_PROMPT_SEQ_BUCKET_MAX` and `VLLM_PROMPT_SEQ_BUCKET_MIN` environment variables from the online service startup example. - Add a new section "Profiling in PD Disaggregation Scenarios" explaining how to configure and control profiling independently on Prefiller and Decoder nodes, including direct `curl` commands since the main PD load-balance proxy does not forward profiling requests. - Clarify that the EPD proxy does support broadcasting profiling commands. - Add a note that `/start_profile` and `/stop_profile` endpoints return 404 Not Found if `--profiler-config` is not set. ### Does this PR introduce _any_ user-facing change? No, this is a documentation-only update. ### How was this patch tested? Tested by verifying the documentation changes and checking the proxy server implementations (`load_balance_proxy_server_example.py` and `epd_load_balance_proxy_layerwise_server_example.py`) to ensure the documented behavior is accurate. - vLLM version: v0.25.1 - vLLM main: vllm-project/vllm@d02df74 Signed-off-by: Xu Chi <xuchi0808@gmail.com> Signed-off-by: xuchi <xuchicolson@163.com>
1 parent 8b39f17 commit 87c1ceb

1 file changed

Lines changed: 28 additions & 2 deletions

File tree

docs/source/developer_guide/performance_and_debug/service_profiling_guide.md

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,6 @@ No additional packages need to be installed; it can be enabled through command-l
3636
Start the online service and set the `--profiler-config` parameter to control the path for saving performance files. After the parameter is set, the collection function is enabled.
3737

3838
```bash
39-
export VLLM_PROMPT_SEQ_BUCKET_MAX=128
40-
export VLLM_PROMPT_SEQ_BUCKET_MIN=128
4139
python3 -m vllm.entrypoints.openai.api_server \
4240
--port 8080 \
4341
--model "facebook/opt-125m" \
@@ -52,6 +50,10 @@ python3 -m vllm.entrypoints.openai.api_server \
5250
5351
### 2. Start Collection
5452

53+
!!! note
54+
55+
The `/start_profile` and `/stop_profile` endpoints are **only registered** when the server is launched with `--profiler-config` set (profiler field non-empty). If you forget to set `--profiler-config`, these endpoints will not exist and `curl` will return **404 Not Found**.
56+
5557
Performance collection is controlled by sending API requests. You can start collection after stabilizing the actual business data and collect profiling for a few seconds before stopping; or you can start collection first, then send business requests, and finally stop.
5658

5759
Send the following request to start the profiling service:
@@ -112,6 +114,30 @@ After analysis, the `*ascend_pt` directory will contain many files, with the mai
112114

113115
- `trace_view.json`: Chrome tracing format data, can be opened with [MindStudio Insight](https://www.hiascend.com/document/detail/zh/mindstudio/81RC1/GUI_baseddevelopmenttool/msascendinsightug/Insight_userguide_0002.html)
114116

117+
### Supplementary: Profiling in PD Disaggregation Scenarios
118+
119+
In PD (Prefill-Decode) disaggregation deployments, the Prefiller and Decoder are **separate vLLM instances**, each with its own API server. Profiling must be configured and controlled **independently** on each node:
120+
121+
1. **Both P and D** must launch with `--profiler-config` (pointing to different directories to avoid overwriting).
122+
123+
2. The main PD load-balance proxy (`load_balance_proxy_server_example.py`) does **not** forward `/start_profile` or `/stop_profile`. You must curl **each node's HTTP port directly**:
124+
125+
```bash
126+
# Start profiling on both P and D
127+
curl -X POST http://<prefiller_ip>:<prefiller_port>/start_profile
128+
curl -X POST http://<decoder_ip>:<decoder_port>/start_profile
129+
130+
# Send business requests through the proxy...
131+
132+
# Stop profiling on both P and D
133+
curl -X POST http://<prefiller_ip>:<prefiller_port>/stop_profile
134+
curl -X POST http://<decoder_ip>:<decoder_port>/stop_profile
135+
```
136+
137+
!!! note
138+
139+
The EPD proxy (`epd_load_balance_proxy_layerwise_server_example.py`) **does** support broadcasting profiling commands to all E/P/D instances.
140+
115141
---
116142

117143
## MS Service Profiler

0 commit comments

Comments
 (0)