Skip to content

[E2E]: Add CAMAsync connector coverage with DeepSeek-V2-Lite #232

Description

@jiangkuaixue123

Summary

Add an Ascend NPU end-to-end test for CAMAsyncAFDConnector after the async connector refactor merged in #203.

The test should follow the refactored E2E framework under tests/e2e/ instead of introducing a connector-specific launch or evaluation harness. Use DeepSeek-V2-Lite to keep the model-side resource requirement manageable. The device count is not fixed to three: select and document a representative topology supported by CAMAsync and the available hardware, with more devices allowed when required.

Scope

  • Add one stable E2E case, for example async-cam-eager, for CAMAsyncAFDConnector.
  • Run on Ascend NPU with DeepSeek-V2-Lite.
  • Make the Attention/FFN device lists and topology explicit and configurable through the refactored E2E framework; do not hard-code a 2 Attention + 1 FFN topology for this case.
  • If the chosen CAMAsync topology exceeds the current framework's three-device/2A1F assumptions, extend the common topology validation and command construction cleanly rather than adding a connector-specific harness.
  • Configure the async connector through the shared runner:
    • connector=CAMAsyncAFDConnector
    • async=true
    • compute_gate_on_attention=true
    • eager execution, since CAMAsync does not support graph execution
  • Reuse the shared service startup, readiness checks, GSM8K evaluation, signal handling, and process cleanup in tests/e2e/runner.py and tests/e2e/process_utils.py.
  • Add focused CPU-safe unit coverage for the generated commands, connector flags, launch order, configurable topology validation, and failure/cleanup paths.
  • Update tests/e2e/README.md and the E2E design/support matrix where needed.

Validation

The PR E2E case should:

  • evaluate the standard first 7 GSM8K samples with 8-shot prompting;
  • enforce the existing sample-count and accuracy gates;
  • fail on missing setup, service failure, incomplete/NaN results, or cleanup failure;
  • run without skip or xfail;
  • leave no Attention, FFN, or evaluator process behind;
  • record the exact Attention/FFN topology, device count, vLLM, vLLM-Ascend, AFD, CANN/CAM operator, model, duration, and accuracy evidence from a real NPU run.

Acceptance criteria

  • A stable async-cam-eager pytest node is added using the refactored E2E framework.
  • The case uses DeepSeek-V2-Lite.
  • The device count and Attention/FFN topology are configurable and reflect the actual CAMAsync validation setup; the case is not restricted to three NPUs.
  • Any required relaxation of the common framework's current three-device/2A1F assumptions is implemented and unit-tested in the shared framework.
  • The shared runner launches Attention before FFN for CAMAsyncAFDConnector and passes the required async configuration.
  • GSM8K validation and strict lifecycle/cleanup checks are reused from the common harness.
  • CPU-safe unit tests cover the new case configuration and command construction.
  • Real Ascend hardware evidence, including the exact card count and topology, is attached to the implementation PR.
  • E2E documentation is updated with the command, prerequisites, limitations, and expected result.

Dependency

Builds on the async connector refactor merged in #203.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions