Skip to content

DX-596-together-gpu-clusters: sync with mintlify-docs#1165 #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
neerajshah-ux merged 1 commit into from
Jul 9, 2026
Merged

DX-596-together-gpu-clusters: sync with mintlify-docs#1165 #41

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions skills/together-gpu-clusters/references/cluster-management.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,7 @@ During provisioning, nodes undergo automatic tests:
- **GPU Burn (5 min)** -- stress test for thermal/power issues
- **Single-Node NCCL** -- GPU-to-GPU communication within a node
- **Multi-Node NCCL** -- cross-node GPU communication
- **Storage Performance** -- sequential read/write throughput and data-integrity validation on attached storage volumes

Nodes showing "Tests Failed" are not added to the cluster until repaired.

Expand All @@ -350,6 +351,9 @@ Nodes showing "Tests Failed" are not added to the cluster until repaired.
**PCIe Performance:**
- NVBandwidth: CPU-to-GPU, GPU-to-CPU bandwidth, GPU-CPU latency

**Storage:**
- Storage Performance: `fio` sequential read/write throughput plus a checksummed write/read-back for data-integrity validation against shared and local volumes attached to the cluster. Results include a **Skipped** state (in addition to Passed/Failed) -- on Kubernetes clusters the shared-storage portion reports **Skipped** with message `Shared volume PVC not created` until a shared-volume PVC exists (see the [Storage](#storage) section).

### Node Repair

- **Quick Reprovision**: VM recreated on a random physical node (for software issues)
Expand Down
Loading