Summary
Add an efa-networking diagnostic-tool skill to aws-dev-toolkit. It walks a builder through isolating and fixing EFA/NCCL problems on GPU instances — the misconfigurations that silently drop distributed training onto the TCP fallback path (~100x slower, usually unnoticed until the bill). Same shape as aws-debug/aws-health-check: diagnostic workflow, command-driven ladder, Gotchas, structured output.
Motivation
GPU startups on P-family instances hit the same EFA issues repeatedly (SG rules, interfaces not attached at launch, cross-AZ nodes, container flags, stale NGC plugins). The job still runs — just ~100x slower on TCP fallback. No existing skill covers EFA/NCCL: networking stops at VPC/SG, mlops covers training architecture, but the EFA stack (SRD, Libfabric, aws-ofi-nccl, GPUDirect RDMA) and its diagnostics are absent.
Proposed Solution
New skill under solution-architecture/plugins/aws-dev-toolkit/skills/efa-networking/:
- SKILL.md — auto-activating; a Diagnostic Workflow, a 9-rung Diagnostic Ladder (check → pass condition → fix, cheapest-infra → software), NCCL verification (healthy vs. NET/Socket
fallback), Gotchas, and an Output Format. Read-only allowed-tools + awsknowledge MCP.
- references/diagnostic-scripts.md — copy-paste checks + AWS Support escalation template.
- references/instance-reference.md — P-family device counts + EFA installer reference.
Alternatives Considered
- Fold into networking — rejected; EFA is a distinct stack, would bloat and dilute both.
- Fold into mlops — rejected; that's training architecture, not fabric debugging.
- A doc page — rejected; value is in-IDE during an active debug, not a doc to find.
Open Questions
- Hardcode the installer version + device-count table, or always defer to awsknowledge to avoid drift?
- Naming: efa-networking vs efa?
Drawbacks
- Device table / installer version will drift; mitigated by in-skill instructions to verify against live docs.
Summary
Add an efa-networking diagnostic-tool skill to aws-dev-toolkit. It walks a builder through isolating and fixing EFA/NCCL problems on GPU instances — the misconfigurations that silently drop distributed training onto the TCP fallback path (~100x slower, usually unnoticed until the bill). Same shape as aws-debug/aws-health-check: diagnostic workflow, command-driven ladder, Gotchas, structured output.
Motivation
GPU startups on P-family instances hit the same EFA issues repeatedly (SG rules, interfaces not attached at launch, cross-AZ nodes, container flags, stale NGC plugins). The job still runs — just ~100x slower on TCP fallback. No existing skill covers EFA/NCCL: networking stops at VPC/SG, mlops covers training architecture, but the EFA stack (SRD, Libfabric, aws-ofi-nccl, GPUDirect RDMA) and its diagnostics are absent.
Proposed Solution
New skill under solution-architecture/plugins/aws-dev-toolkit/skills/efa-networking/:
fallback), Gotchas, and an Output Format. Read-only allowed-tools + awsknowledge MCP.
Alternatives Considered
Open Questions
Drawbacks