[Bug]: NPU AFD communication can fail with HCCL EI0006 without a documented connection timeout

[ShwStone]

Current environment

This has been observed across multiple Ascend NPU AFD test configurations rather
than one specific recipe or topology.

Platform: Ascend NPU
Execution: disaggregated Attention and FFN roles
Affected area: HCCL connection establishment, frequently around initial FFN communication
Exact software matrix: will be attached from the next clean reproduction

Reproduction

1. Ensure HCCL_CONNECT_TIMEOUT is not explicitly configured on either the
   Attention or FFN processes.
2. Start an Ascend NPU AFD deployment with separate Attention and FFN roles.
3. Wait for connector initialization or the first matching communication.
4. Some runs fail during HCCL socket connection establishment.

The confirmed error fingerprint is:

Communication_Error_Get_Socket(EI0006)

Increasing HCCL_CONNECT_TIMEOUT, for example:

export HCCL_CONNECT_TIMEOUT=900

appears to mitigate the failure in our testing. This still needs a controlled
comparison with complete logs from every rank.

Expected behavior

The repository should clearly document the HCCL connection timeout needed by
long-running, cross-process NPU AFD tests.

Users should be able to determine:

• what HCCL_CONNECT_TIMEOUT controls;
• when it should be increased;
• that the same value must be configured before launching every participating
  Attention and FFN process;
• which error fingerprints indicate a connection timeout;
• that it does not control execution timeout after the HCCL connection has
  already been established.

Relevant NPU recipes should either set an overrideable tested default or link to
the timeout guidance.

Actual behavior and logs

Communication_Error_Get_Socket(EI0006)

The current upstream repository does not mention or configure
HCCL_CONNECT_TIMEOUT. As a result, NPU tests can depend on the external HCCL
default without making that dependency visible to users.

Full HCCL logs and one complete environment dump will be attached from the next
clean reproduction.

AFD configuration

This issue is not limited to one AFD configuration or recipe. It has been
observed during multiple NPU AFD tests using separate Attention and FFN roles.

The common condition is that HCCL connection establishment occurs without an
explicitly configured HCCL_CONNECT_TIMEOUT.

Preliminary assessment

EI0006 and Communication_Error_Get_Socket indicate that the failure occurs
during HCCL socket connection establishment.

Increasing HCCL_CONNECT_TIMEOUT is therefore relevant to this failure.
However, it should not be presented as a fix for notify, collective execution,
device task, or idle operator timeouts.

Proposed repository changes

1. Add an NPU timeout section to the deployment documentation.

2. Document HCCL_CONNECT_TIMEOUT, including its connection-establishment
   scope, error fingerprints, cross-role consistency requirement, and its
   difference from HCCL or ACL execution timeouts.

3. Add an overrideable value to affected NPU recipe examples, subject to
   maintainers confirming the default:

   export HCCL_CONNECT_TIMEOUT="${HCCL_CONNECT_TIMEOUT:-900}"

4. Print the effective value during startup or include it in the recommended
   environment-dump command.

5. Preserve the earliest HCCL error from every rank in NPU test failure output.

Acceptance criteria

• NPU documentation explains when and how to configure
  HCCL_CONNECT_TIMEOUT.
• Relevant NPU recipes contain an overrideable value or link directly to the
  timeout documentation.
• Documentation does not confuse connection timeout with HCCL or ACL execution
  timeout.
• A controlled reproduction confirms whether 900 is an appropriate tested
  value.

Before submitting

• I searched existing issues for related reports.
• This has been observed on the current vLLM v0.26 NPU integration; a
  complete environment dump will be attached from the next reproduction.