Skip to content

Commit 2b54a67

Browse files
Bihan  RanaBihan  Rana
authored andcommitted
Add concurrent services with sglang router
1 parent 8e1044f commit 2b54a67

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

src/dstack/_internal/proxy/gateway/resources/nginx/sglang_workers.jinja2

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{% for replica in replicas %}
22
# Worker {{ loop.index }}
3-
upstream sglang_worker_{{ loop.index }}_upstream {
3+
upstream sglang_worker_{{ ports[loop.index0] }}_upstream {
44
server unix:{{ replica.socket }};
55
}
66

@@ -12,7 +12,7 @@ server {
1212
proxy_send_timeout 300s;
1313

1414
location / {
15-
proxy_pass http://sglang_worker_{{ loop.index }}_upstream;
15+
proxy_pass http://sglang_worker_{{ ports[loop.index0] }}_upstream;
1616
proxy_http_version 1.1;
1717
proxy_set_header Host $host;
1818
proxy_set_header X-Real-IP $remote_addr;

0 commit comments

Comments
 (0)