From e5f000eb666a823a72b78e0bbad2aef5d7963de6 Mon Sep 17 00:00:00 2001 From: zzh Date: Thu, 13 Aug 2026 10:17:24 +0800 Subject: [PATCH] feat: add GPU runtime upgrade skill Add a version-neutral, exact-source workflow for planning, implementing, and validating AFD GPU vLLM upgrades. Include the evidence workbook, documentation refresh guide, historical v0.19.1-to-v0.26.0 lessons, and agent metadata required by Issue #224. Signed-off-by: zzh --- .agents/skills/upgrade-gpu-version/SKILL.md | 495 +++++++++++++++++ .../upgrade-gpu-version/agents/openai.yaml | 4 + .../references/documentation-refresh.md | 237 +++++++++ .../references/upgrade-workbook.md | 500 ++++++++++++++++++ .../references/v0191-to-v026-lessons.md | 192 +++++++ 5 files changed, 1428 insertions(+) create mode 100644 .agents/skills/upgrade-gpu-version/SKILL.md create mode 100644 .agents/skills/upgrade-gpu-version/agents/openai.yaml create mode 100644 .agents/skills/upgrade-gpu-version/references/documentation-refresh.md create mode 100644 .agents/skills/upgrade-gpu-version/references/upgrade-workbook.md create mode 100644 .agents/skills/upgrade-gpu-version/references/v0191-to-v026-lessons.md diff --git a/.agents/skills/upgrade-gpu-version/SKILL.md b/.agents/skills/upgrade-gpu-version/SKILL.md new file mode 100644 index 00000000..129109f3 --- /dev/null +++ b/.agents/skills/upgrade-gpu-version/SKILL.md @@ -0,0 +1,495 @@ +--- +name: upgrade-gpu-version +description: Upgrade and align the AFD Plugin GPU backend across exact pinned vLLM revisions and CUDA runtime/toolchain environments. Use when Codex must plan, audit, implement, review, or finally validate a GPU vLLM version upgrade; rebase compatibility patches; adapt GPU models, workers, model runners, connectors, CUDA Graph, DBO, TP, profiling, packaging, or isolation contracts; or resolve GPU regressions caused by an upstream version change. Do not use for NPU-only upgrades, model-only adaptation, ordinary GPU bugs unrelated to an upstream upgrade, or E2E-only execution. +--- + +# Upgrade the AFD GPU backend + +Run this workflow as a gated, version-neutral state machine. Resolve the exact +current and target vLLM revisions, inventory AFD behavior, and pass the +architecture gate before editing production code. Preserve supported AFD +behavior on the target upstream architecture; do not preserve an obsolete +implementation shape merely because an older release required it. + +Read the repository `AGENTS.md` before acting. Read +[`references/upgrade-workbook.md`](references/upgrade-workbook.md) completely +for evidence tables, source-diff commands, handoff records, and report +templates. List and read every `references/*lessons.md` file before upstream +analysis. Treat lessons as historical failure patterns, never as target-version +instructions; re-derive every applicable fact from the exact current and target +sources. Read +[`references/documentation-refresh.md`](references/documentation-refresh.md) +only after the required GPU validation gates pass and before refreshing public +support claims. + +## Task modes + +Choose exactly one mode from the request. Do not silently combine modes. + +- `planning-audit`: perform read-only identity, inventory, upstream diff, + architecture-gate, and staged-plan work. Do not edit AFD or allocate GPUs. +- `implementation`: implement the approved upgrade in bounded adaptations, + add focused tests, and pass the static/unit gate. Continue to GPU validation + only when the request includes it and the required environment is available. +- `final-validation`: freeze an already selected AFD revision and run read-only + native and AFD validation. Do not fix production code unless the user changes + the mode to implementation. + +Record the selected mode. If it is ambiguous and the stopping point changes +what may be edited or whether GPUs may be allocated, ask once before proceeding. + +## Hard boundaries + +- Treat current vLLM and target vLLM as immutable revisions. Resolve both refs + to full commit SHAs before diffing or editing. If either ref, SHA, or exact + source tree is unavailable, mark `FREEZE_IDENTITY` `BLOCKED` and every later + phase `SKIPPED`; do not substitute a historical lesson or installed package. +- Discover the current ref from repository evidence when possible. Require the + target vLLM ref and a local vLLM source repository. Never infer a target + revision from the newest installed package or an unpinned branch. +- Define the AFD base revision as the immutable pre-upgrade AFD commit named by + the issue/RFC or proven by branch ancestry; do not assume `origin/main` or the + current merge-base. Define the target AFD mirrored version as the package or + compatibility version the repository will publish for the target vLLM, not a + branch name. Cite the repository policy; use `NOT_APPLICABLE` when no mirrored + version policy exists. +- Do not modify, switch, or check out another revision in a user-provided vLLM + source directory. If its HEAD is the target SHA, use it read-only. Otherwise + use `git show` in read-only audits, or create a separate detached worktree + only when workspace writes are in scope. Read a missing current tree the same + way without disturbing the provided checkout. +- Record the intended target Python, PyTorch build CUDA and C++ ABI, vLLM + artifact/build identity, CUDA toolkit/runtime, NVIDIA driver, host compiler, + NCCL, device compute capability/count/topology, and model paths. Inventory + plugin-owned GPU native sources separately. Planning may mark unavailable + hardware evidence `UNKNOWN`; final validation must stop when the exact runtime + or required hardware cannot be established. +- Do not edit AFD production code before the architecture gate passes. If a + major architecture change is found, stop and report the required design + decision. Do not silently remove a supported feature or build a broad + compatibility layer around it. +- Preserve unrelated dirty changes. Record branch, HEAD, and status before + work. Stop if user changes overlap planned files. Never reset, discard, stage, + or commit unrelated paths. +- Never modify the vLLM source tree to make AFD pass. Keep target worktrees + read-only after creation. +- Prefer native vLLM extension points, inheritance, and composition. Do not add + a new public connector, runner, protocol, or runtime abstraction unless the + exact target contract proves existing interfaces insufficient; treat a broad + cross-component expansion as an architecture-gate decision. +- Follow `AGENTS.md` for every copied or patched function: exact target + signature and return type, source/reason/functionality comments, narrowly + marked AFD deltas, focused AFD and non-AFD tests, performance impact, and a + removal or upstream plan. +- Access known upstream members directly. Do not hide drift with broad + `getattr`, `hasattr`, exception swallowing, `Any`, or version-probing wrappers. +- In implementation mode, keep each coherent upstream cause and its focused + regression tests together. Create local signed-off commits only when commits + are in scope; never push, publish, merge, or open a PR unless separately + requested. +- Run every required non-accuracy GPU E2E case before final accuracy. Never use + a limited accuracy run as final qualification, and never hard-code a test + count in this skill. + +## Evidence authority + +Use this order when evidence conflicts: + +1. exact current and target vLLM source revisions; +2. current AFD source, tests, package pins, and normative repository contracts; +3. results from the frozen validation cell; +4. historical `*lessons.md` files as hypotheses to check, not facts to carry. + +An old workaround, file path, runner generation, test count, topology, or CUDA +behavior remains applicable only after target-source verification. + +## Subagent orchestration + +Use distinct subagents when available to separate analysis, implementation, +testing, and review. The primary agent owns phase state, architecture decisions, +the integrated diff, commits, and the final report. Subagent conclusions are +evidence, not automatic gate decisions. + +- Keep analysis agents read-only. Split by vLLM history or AFD surface and + require exact SHAs, paths, symbols, classifications, invariants, and tests. +- Dispatch implementation agents only after the architecture gate. Give each + one non-overlapping AFD file set and one upstream root cause. Prohibit branch + changes, staging, commits, pushes, and upstream edits. +- Keep test agents read-only for tracked files. Make GPU E2E agents read + `../run-e2e/SKILL.md` and `AGENTS.md`. Serialize GPU jobs unless reservations, + devices, ports, writable caches, logs, processes, and cleanup ownership are + explicitly isolated. +- Keep review agents independent and read-only. Require simplicity and + minimality review first, then exact upstream signatures, patch markers, + ownership, call order, performance risks, isolation, and test coverage. + +Record every handoff using the workbook. Never test or review a worktree while +an implementation agent is writing it. Inspect returned diffs and command +results directly before accepting them. + +Use this loop for each adaptation in implementation mode: + +```text +ANALYZE -> PRIMARY GATE -> IMPLEMENT -> FOCUSED TEST + -> INDEPENDENT REVIEW -> PRIMARY COMMIT -> REGRESSION GATE +``` + +## Required phase record + +At every phase boundary record: + +```text +phase: +status: PASS | FAIL | BLOCKED | STOPPED | SKIPPED +evidence: +blocker: +next_allowed_action: +``` + +Do not enter a later required phase when an earlier phase is `FAIL`, `BLOCKED`, +or `STOPPED`. + +```text +FREEZE_IDENTITY + -> INVENTORY_AFD_CONTRACTS + -> DIFF_VLLM + -> ARCHITECTURE_GATE + -> PRODUCE_STAGED_PLAN + -> IMPLEMENT_ATOMIC_ADAPTATIONS + -> STATIC_AND_UNIT_GATE + -> TARGET_NATIVE_GPU_CONTROL + -> GPU_BASIC_E2E_GATE + -> GPU_ACCURACY_GATE + -> DOCUMENTATION_REFRESH_GATE + -> FINAL_AUDIT + -> CAPTURE_LESSONS_AND_REPORT +``` + +## Phase 0 — Freeze identity + +Record the task mode; AFD checkout, branch, HEAD, immutable pre-upgrade base +revision and its evidence, dirty status, and Python environment; vLLM source +path, HEAD, branch, and dirty status; current and target refs plus resolved SHAs; +and any detached read-only worktrees. + +Discover the current vLLM ref in this order: + +1. `pyproject.toml`, lockfile, and package metadata; +2. compatibility constants, patch annotations, design matrices, README, GPU + guides, recipes, CI, and active upgrade RFC; +3. installed package versions and adjacent checkout state as corroboration only. + +Resolve annotated tags to commits. If authoritative sources disagree, mark the +identity phase `BLOCKED` rather than choosing the convenient value. + +Record the target AFD published package/compatibility version and validate it +against the repository's mirrored-version policy, or record `NOT_APPLICABLE` +with evidence. Record the intended target stack: Python; PyTorch version, build +CUDA, and C++11 ABI; vLLM source/package/native-artifact identity; CUDA toolkit +and runtime; NVIDIA driver; `nvcc`; host compiler; NCCL; GPU model and compute +capability; device count and interconnect/topology; runner generation; and +model/checkpoint path. In final validation, record the effective values and +reconcile them with the intended tuple. + +## Phase 1 — Inventory AFD contracts + +Build the inventory before reading the upstream diff. Use source search as the +authority and documentation as supporting evidence. Cover at least: + +- version pins, runtime checks, package build/install, plugin registration, + CPU-safe imports, and non-AFD isolation; +- every compatibility patch and copied upstream method, including copies outside + `afd_plugin/compat/patches/**`; +- GPU Attention and FFN workers, model runners, EngineCore integration, KV-cache + and scheduler ownership, startup, failure propagation, and shutdown; +- model registration, role-aware construction, native lifecycle, forward paths, + parameter ownership, quantization, and weight filtering/loading; +- GPU connectors, payloads, custom ops, process groups, rank/device/topology + mapping, synchronization, ordering, errors, and cleanup; +- CUDA Graph capture/replay, compile behavior, dummy/warmup runs, DBO/uBatch, + TP/DP/EP/PP, profiling, sampling, request output, and memory accounting; +- GPU native sources and build interfaces in AFD and target vLLM, including + artifact ownership, build flags, supported compute capabilities, and ABI; +- focused tests, complete GPU E2E scenarios, recipes, documentation, CI, and + release metadata; +- shared files whose change may require NPU regression evidence without making + an NPU support claim. + +For every patched or copied function, record AFD target, current upstream source +and symbol, exact signature, marker blocks, preserved invariant, state/lifecycle +owner, direct consumers, focused AFD and non-AFD tests, and removal/upstream +plan. + +Freeze the current supported feature matrix. Discover all represented runner +generations, models, connectors, eager/graph modes, DBO/uBatch cells, gate +placements, TP/DP/EP/PP topologies, profiling modes, quantization/EPLB paths, +accuracy cases, and negative/isolation cases. An omitted feature is `UNKNOWN`, +not implicitly out of scope. + +For `planning-audit`, the minimum sufficient inventory is every AFD dependency +on the changed upstream surface, every copied or patched upstream method, and +every currently supported feature cell affected by that surface. Prove an area +unaffected with source-search evidence. If a user-imposed path or time bound +leaves an affected surface `UNKNOWN`, block the architecture gate rather than +calling the inventory complete. + +## Phase 2 — Diff vLLM + +Compare current vLLM SHA to target vLLM SHA. Begin with rename-aware name +status and stats, first-parent history, and focused function-context diffs. Map +every relevant upstream change to AFD consumers and classify it: + +- `MECHANICAL`: import move, rename, or exact signature change with equivalent + ownership and behavior; +- `BEHAVIORAL`: defaults, schema, state lifetime, call order, device mapping, + graph behavior, or ownership changed while the AFD invariant still maps; +- `ARCHITECTURAL`: an execution generation, stable seam, lifecycle owner, + feature, protocol, or cross-component boundary was removed or conceptually + replaced. + +For copied upstream logic, reconstruct method by method: + +1. use the exact target upstream function as the new skeleton; +2. use current AFD markers and focused tests as the local-difference ledger; +3. use current supported AFD behavior as the semantic contract; +4. replay only still-required AFD differences into the target skeleton; +5. verify decorators, signature, return type, call order, state ownership, + resource lifetime, hot-path cost, graph/compile behavior, and shutdown. + +Do not mechanically apply an upstream old-to-new diff onto an AFD copy. Check +whether target vLLM added an extension point that lets the patch be removed or +replaced by delegation. + +## Phase 3 — Architecture gate + +Stop without production edits when any of these is true: + +- the target vLLM revision and requested CUDA/PyTorch/toolchain tuple are not a + reproducible supported environment; +- the GPU worker/model-runner generation or lifecycle is replaced and AFD needs + redesign rather than local adaptation; +- a required patch seam disappears without an equivalent stable target, or it + can no longer be expressed as target upstream logic plus small AFD markers; +- upstream removes or disables a supported graph, DBO, parallelism, model, + connector, or profiling contract and continuing would require silent feature + deletion or validation bypass; +- model split, expert/gate ownership, scheduler/KV ownership, connector payload, + topology, process group, sampling/output, or cleanup contracts change across + multiple subsystems without a one-to-one invariant mapping; +- preserving behavior requires a new public protocol, broad core rewrite, + extensive upstream copies, reflection-heavy compatibility, or hidden fallback; +- the selected PyTorch/vLLM native artifacts, CUDA runtime, C++ ABI, or target + compute capability cannot form one executable target cell; +- the required CUDA/driver/compiler/NCCL transition cannot be built and tested + as one exact target environment. + +A moved implementation is not automatically architectural. Continue only when +inputs, outputs, ordering, ownership, and lifetime map cleanly and the adaptation +remains narrow. On stop, use the workbook's architecture report and leave AFD +production files unchanged. + +## Phase 4 — Produce a staged plan + +Create an upstream-to-AFD impact matrix and a reviewable implementation plan. +For each stage record upstream cause, expected files, preserved invariant, +patch disposition, focused and regression tests, graph/performance/distributed +risks, documentation impact, and removal or upstream contribution plan. + +Classify each compatibility seam as `remove`, `delegate/inherit`, `port as an +exact-version adapter`, `propose upstream`, or `stop`. Do not use a scoring +system instead of making the architectural judgment. + +In `planning-audit` mode, emit the plan, patch audit, feature matrix, exclusions, +and evidence gaps, then mark later phases `SKIPPED` and stop without writes. If +an earlier gate is blocked, emit only a blocker-resolution plan naming the exact +missing evidence and next allowed actions; label the implementation impact +matrix and staged plan unavailable rather than filling them with hypotheses. + +## Phase 5 — Implement atomic adaptations + +In implementation mode, create or use a dedicated upgrade branch only when +branch changes are in scope; otherwise record and preserve the current branch. +Before each edit batch record files, upstream cause, invariant, tests, risks, +and patch removal/upstream plan. + +Implement in dependency order unless the impact graph proves another order: + +1. version/package contract and CPU-safe imports; +2. compatibility patches and plugin initialization order; +3. model construction, native lifecycle, role ownership, and weight policy; +4. Attention/FFN workers and model runners, EngineCore, KV/scheduler ownership; +5. connectors, payloads, distributed topology, and device mapping; +6. CUDA Graph, DBO/uBatch, parallelism, profiling, warmup, and cleanup; +7. focused tests and repository-owned E2E scenarios needed for qualification. + +Use existing connector and runner interfaces unless the architecture gate +explicitly approved a new boundary. Preserve native model forward, loading, and +runtime behavior wherever the target provides a usable seam. Do not change +shared runtime code to conceal an incomplete GPU adaptation. + +For each retained patch, copy the target function, match its exact signature, +mark only AFD differences, test selected and non-AFD branches, and document +performance and removal/upstream plans. Remove an obsolete patch only after +proving target upstream absorbed the behavior. + +After each coherent adaptation passes focused checks and independent review, +inspect the staged diff and create a signed-off local commit when commits are in +scope. Keep source changes and focused regression tests together. + +## Phase 6 — Static and unit gate + +Before allocating GPUs, inspect the repository's current CI and test entry +points and run their actual commands. Cover: + +- diff, lint, format, compile, import, and package build/install checks; +- exact target runtime import and plugin registration; +- version pin/runtime check consistency; +- target vLLM native-module provenance/import plus PyTorch build-CUDA and C++11 + ABI consistency; +- a clean AFD package build and import for every plugin-owned compiled GPU + extension that exists in the target tree; +- exact patch signatures, decorators, return types, comments, and markers; +- affected focused tests, then the complete CPU/unit suite; +- model construction and weight policy, worker/runner contracts, connector + ordering/topology, graph/DBO helpers, cleanup, and non-AFD isolation; +- dependency consistency such as `python -m pip check` in the target runtime. + +Treat collection or import failure as a version/environment failure until the +identity tuple is proven. Import, construction, dummy execution, or server +startup alone never proves GPU support. + +If AFD has no plugin-owned compiled GPU source, record `NOT_APPLICABLE` with the +inspected build files instead of inventing a build requirement. Do not rebuild +vLLM merely for preflight when the repository requires an installed runtime; +verify that runtime's native artifact and reserve source builds for an approved +target-build qualification. + +In implementation mode without requested hardware validation, report all GPU +phases `SKIPPED` and readiness `implemented, GPU-unverified`. + +## Phase 7 — Target native GPU control + +Before AFD E2E, prove the exact target vLLM runtime can load the selected model +and serve at least one real eager GPU request with AFD disabled. Keep model, +checkpoint, runner generation, quantization, prompt, sampling, and comparable +parallelism aligned with the first AFD cell. Record startup, weight loading, +readiness, output/correctness oracle, resource use, and cleanup. + +On the reserved target GPU, execute the smallest target-native kernel or +operator smoke that proves the loaded artifact runs on the recorded compute +capability. When AFD owns a compiled GPU extension, run its smallest real +operator smoke before the request; otherwise record `NOT_APPLICABLE`. + +Follow `AGENTS.md` for remote access and GPU scheduler ownership. Never treat a +free-looking device as authorization. If native construction, loading, +readiness, or request execution fails, preserve the first useful traceback and +stop AFD parity validation. A fallback setting creates a separate validation +cell and must not silently replace the requested target. + +## Phase 8 — GPU basic E2E gate + +Read and follow `../run-e2e/SKILL.md` for hardware detection, provisioning, +pytest selection, live output, skip reporting, and cleanup. Do not duplicate or +override its backend-selection logic in this skill. `AGENTS.md` remains the +authority for scheduler use and remote test artifacts. + +Run the complete marker-based GPU non-accuracy suite before accuracy: + +1. feature scenarios; +2. model scenarios; +3. any repository-owned GPU upgrade scenario not covered by those categories. + +Use enough reserved GPUs to avoid capacity skips when claiming a full upgrade. +Every skip must map to a documented unsupported or explicitly excluded cell; a +new, unexplained, or hardware-capacity skip is not a full pass. Record actual +collected and executed counts from pytest instead of assuming a fixed suite +size. + +Verify behavior, not only launch: real responses, eager/graph comparisons, +actual CUDA Graph capture/replay, actual DBO stages, TP/DP/EP collectives, +profiler traces, error propagation, and cleanup as represented by repository +tests. After every run verify the exact experiment PID tree, ports, GPU +processes, scheduler state, and IPC resources are clean. + +For each failure, preserve the cell, command, environment, first useful root +traceback, logs, resource state, and cleanup. In final-validation mode classify +and report without source edits. In implementation mode trace one root cause, +add the narrowest regression test, implement one minimal fix, repeat focused +review, then rerun the failed case and complete basic gate. + +## Phase 9 — GPU accuracy gate + +Enter only after the complete basic E2E gate passes. Delegate all GPU accuracy +execution to `run-e2e`. Run every repository-owned GPU accuracy case, including +eager and graph variants, with no dataset limit for final qualification. A +limited run is diagnostic evidence only. + +On failure, preserve metrics and artifacts and compare the exact native and AFD +cells. Never lower thresholds or widen tolerance to make an upgrade pass unless +the user explicitly approves a separately justified policy change. After a code +fix, rerun the complete basic gate before full accuracy. + +## Phase 10 — Documentation refresh gate + +Enter only after required basic and accuracy gates pass. Read +[`references/documentation-refresh.md`](references/documentation-refresh.md) +completely before editing upgrade documentation. Refresh exact runtime claims, +GPU guides, recipes, design/patch inventories, contributor templates, and +release metadata from the validation ledger. + +Distinguish `hardware validated`, `unit validated`, `implemented but +unvalidated`, `unsupported`, and `shared-code only`. Preserve intentionally +historical versions and measurements. Never infer NPU support from a GPU result +or relabel an old recipe as target evidence. + +In final-validation mode without documentation-write scope, audit and report +required changes instead of editing them. + +## Phase 11 — Final audit + +Audit version pins, source annotations, patch inventory, support matrices, +package/release metadata, recipes, GPU documentation, CI, test discovery, and +historical branch notes. Confirm every retained patch has focused AFD and +non-AFD coverage plus a removal/upstream plan. Reconcile every public support +claim with an exact validation cell and list all exclusions. + +Report CPU-safe and GPU evidence separately. Shared-code unit evidence is not a +GPU hardware claim; GPU evidence is not an NPU claim. Performance claims require +matching measured evidence rather than successful functional E2E alone. + +## Phase 12 — Capture lessons and report + +After a completed implementation upgrade, create a new version-specific file +under `references/` using the workbook template. Name it +`--to--lessons.md`; sanitize refs and use a +short SHA when no immutable tag exists. Never overwrite an earlier lesson. + +Record exact identities, upstream changes, adaptations, failures, misleading +attempts, useful validation, documentation conflicts, remaining debt, and the +next-upgrade checks. Separate reusable principles from version-specific facts. +Link raw evidence instead of copying large logs. Have an independent read-only +review when available, then commit the lesson only when repository writes and +commits are in scope. + +In planning or validation-only mode, do not create a lesson file. Include the +same fields in the final report as proposed lessons instead. + +Use the workbook completion template to report: + +- mode, all refs and full SHAs, target AFD version, and exact CUDA stack; +- architecture decision, impact matrix, staged plan, patch dispositions, and + feature matrix; +- changed components and each local commit/root cause; +- static/unit, native build/ABI, native control, basic E2E, and full accuracy + commands/results; +- actual pass/fail/error/skip counts, metrics, logs, cleanup, and exclusions; +- documentation consistency and intentionally preserved history; +- remaining private seams, performance observations, release blockers, and + branch/publish status; +- readiness separated into GPU-validated, shared-code, and unverified claims. + +Declare the requested scope complete only when every required earlier phase +passes. A full GPU upgrade requires immutable compatible identity, architecture +approval, focused and complete CPU gates, native GPU control, complete basic GPU +E2E with no unexplained skips, full accuracy, documentation consistency, clean +resource teardown, and an evidence-backed final report. diff --git a/.agents/skills/upgrade-gpu-version/agents/openai.yaml b/.agents/skills/upgrade-gpu-version/agents/openai.yaml new file mode 100644 index 00000000..b2350ce4 --- /dev/null +++ b/.agents/skills/upgrade-gpu-version/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "GPU Runtime Upgrade" + short_description: "Upgrade AFD across pinned vLLM and CUDA revisions" + default_prompt: "Use $upgrade-gpu-version to plan, implement, or validate an AFD GPU runtime upgrade." diff --git a/.agents/skills/upgrade-gpu-version/references/documentation-refresh.md b/.agents/skills/upgrade-gpu-version/references/documentation-refresh.md new file mode 100644 index 00000000..60c1c174 --- /dev/null +++ b/.agents/skills/upgrade-gpu-version/references/documentation-refresh.md @@ -0,0 +1,237 @@ +# GPU Upgrade Documentation Refresh + +Use this guide only after the target GPU runtime has passed the required +validation gates. Documentation is part of the upgrade contract: every claim +must describe the exact target revision and the evidence actually collected. + +## Contents + +1. Evidence ledger +2. Claim levels +3. Root documentation +4. GPU connector and runtime guides +5. Recipes and launch scripts +6. Design and contributor documentation +7. Generated and release metadata +8. Historical artifacts +9. Consistency audit +10. Completion record + +## 1. Evidence Ledger + +Create one ledger before editing documentation. Record: + +- AFD revision and package version; +- current and target vLLM revisions; +- Python, PyTorch, PyTorch build CUDA and C++11 ABI, vLLM native-artifact + provenance, CUDA runtime/toolkit, driver, compiler, and NCCL used for + validation; +- GPU model, count, memory, and topology; +- GPU compute capability, target-native kernel smoke, and any AFD compiled GPU + extension build/import/operator smoke or `NOT_APPLICABLE` evidence; +- model and dataset paths; +- execution mode, connector, TP size, CUDA Graph state, DBO or uBatch state, + and profiling state for each validated cell; +- exact commands, results, exclusions, and cleanup evidence. + +When a value is unknown, write `not verified`. Do not infer a target environment +from a container tag, dependency constraint, or historical run. + +## 2. Claim Levels + +Label support using the strongest evidence available: + +- **hardware validated**: exercised on the stated target GPU stack; +- **target-native control validated**: target vLLM ran without AFD for the same + model and relevant environment; +- **CPU validated**: static, import, unit, packaging, or collection checks only; +- **implemented, GPU validation pending**: code exists but no target-hardware + result is available; +- **shared-code only**: a common path changed, but this backend was not + exercised; +- **unsupported**: deliberately excluded with a reason; +- **not evaluated**: outside the completed validation matrix. + +Do not turn CPU-only evidence into a GPU support claim. Do not describe an +option as validated merely because it was accepted by the command line: confirm +that logs, counters, traces, or execution behavior show the path was used. + +## 3. Root Documentation + +Review the root `README.md` and any linked installation page for: + +- the exact supported vLLM revision or range; +- the target Python, PyTorch, CUDA, driver, compiler, and NCCL requirements; +- supported GPU architectures and known hardware limits; +- installation commands and package extras; +- supported models, connectors, parallel modes, and execution modes; +- CUDA Graph, DBO or uBatch, TP, profiling, and isolation support; +- a minimal launch example with current arguments and environment variables; +- links to current recipes, tests, design notes, and troubleshooting pages; +- explicit unsupported or unvalidated combinations. + +Avoid broad phrases such as “GPU fully supported” when only selected models or +cells were validated. Prefer a concise matrix or a link to the validation +record. + +## 4. GPU Connector and Runtime Guides + +Review `afd_plugin/connectors/README.md` first. Keep its connector status matrix +consistent with the root README, GPU guide, recipes, design runtime matrix, and +E2E evidence. State eager, CUDA Graph, DBO/uBatch, TP, asymmetric topology, and +profiling evidence separately where their support differs. + +Review `docs/gpu/NCCL_P2P_CONNECTOR_USER_GUIDE.md` against the target source and +runtime. Confirm: + +- process and rank topology; +- producer, router, and consumer responsibilities; +- tensor, metadata, token, and expert-routing ownership; +- send/receive ordering and synchronization assumptions; +- environment variables and configuration keys; +- launch order, health checks, shutdown, and cleanup; +- CUDA Graph and DBO or uBatch interactions; +- TP behavior and asymmetric topology behavior; +- profiling and observability behavior; +- version-specific limitations and failure signatures. + +If the implementation delegates to a target-vLLM public API, document the +public contract rather than preserving the shape of an old copied patch. + +## 5. Recipes and Launch Scripts + +Review `recipe/README.md`, every target recipe under `recipe/gpu/**`, and each +referenced launch script. For each recipe, verify: + +- model path and model family; +- number of processes and GPUs; +- TP and other parallel settings; +- connector and transport selection; +- CUDA Graph, DBO or uBatch, and profiling settings; +- ports, hostnames, environment variables, and required directories; +- target-runtime arguments and removed or renamed flags; +- expected output and shutdown procedure; +- whether the exact recipe was run, partially run, or only inspected. + +Do not silently modernize a historical recipe and continue presenting it as the +original result. Preserve historical evidence and add a current recipe or a +clear version note. + +## 6. Design and Contributor Documentation + +Review every current design source of truth, not only documents whose names +appear related to the changed code: + +| Document | Required GPU review | +|---|---| +| `docs/design/module/index.md` | target runtime, upstream refs, validation entry points, ownership, and links | +| `docs/design/module/compatibility_and_patches.md` | exact patch symbols/signatures, reasons, tests, dispositions, and removal/upstream plans | +| `docs/design/module/execution_platforms.md` | GPU worker/runner initialization, device selection, CUDA Graph, DBO/uBatch, profiling, and runtime matrix | +| `docs/design/module/connector_contracts.md` | NCCL/P2P configuration, payload/metadata ownership, ordering, topology, lifecycle, and cleanup | +| `docs/design/module/attention_runtime.md` | Attention lifecycle, KV/scheduler ownership, forward context, connector handoff, graph, and ubatching | +| `docs/design/module/ffn_runtime.md` | FFN daemon/EngineCore lifecycle, MoE execution, graph dispatch, failure propagation, and shutdown | +| `docs/design/module/model_integration.md` | registration, role-aware construction/loading, gate/expert ownership, quantization, and weights | +| `docs/design/module/plugin_boundary.md` | registration/patch order, worker classes, aliases, environment variables, and upstream boundary | + +For every reviewed design document, update `upstream_refs`, runtime/validation +references, and review metadata only after checking the corresponding content. +Rebuild the patch inventory from source search and exact target vLLM source. + +Review native-build contracts in `csrc/gpu/README.md`, `setup.py`, +`pyproject.toml`, and package manifests. If AFD still has no compiled GPU +extension, keep that status explicit rather than documenting a build step. If +one exists in the target tree, document its prerequisites, build flags, +supported compute capabilities, ABI, import verification, and operator smoke. + +Review the contributor workflow when the target runtime or evidence contract +changes: + +- `.github/ISSUE_TEMPLATE/100-bug-report.yml`: request vLLM identity, PyTorch, + CUDA/driver/compiler/NCCL, GPU/compute capability, connector, topology, + graph/DBO, and reproduction evidence; +- `.github/ISSUE_TEMPLATE/200-feature-request.yml`: remove obsolete runtime or + extension-point assumptions; +- `.github/PULL_REQUEST_TEMPLATE.md`: require backend-scoped compatibility, + exact upstream refs, native build/ABI status, GPU validation, skips, cleanup, + and documentation impact. + +Contributor instructions must keep the exact-current-versus-exact-target +comparison rule and the repository patch policy visible. Remove examples that +teach an obsolete private API or stale copied implementation. + +## 7. Generated and Release Metadata + +Inspect metadata that can make a correct implementation appear stale or can +publish the wrong compatibility contract: + +- package version and dependency constraints; +- `pyproject.toml`, `setup.py`, lockfiles, wheel/build metadata, and native + artifact declarations; +- generated model or plugin registration tables; +- issue and pull-request templates; +- changelog or release notes; +- CI matrices and container references; +- comments or labels that state an old runtime version. + +Update generated files through their documented generator when one exists. +Record the generator command and resulting diff. Do not hand-edit generated +output unless the repository explicitly permits it. + +## 8. Historical Artifacts + +Treat old RFCs, benchmark reports, logs, and upgrade lessons as historical +evidence. Preserve their original version context. If they are still useful: + +- add a visible historical label; +- link to the current support statement; +- distinguish reusable principles from version-specific facts; +- do not rewrite old results to imply they were produced on the new target. + +New lessons belong in a new version-pair file. Do not replace an earlier lesson +file with the latest upgrade story. + +## 9. Consistency Audit + +Search the full repository for the old version and changed public names. Start +with targeted searches such as: + +```bash +rg -n '||' \ + README.md docs recipe tests .github afd_plugin/connectors/README.md \ + csrc/gpu/README.md pyproject.toml setup.py uv.lock +rg -n '||' \ + README.md docs recipe tests .github afd_plugin csrc/gpu setup.py +rg -n 'supported|unsupported|validated|not verified|CUDA Graph|DBO|uBatch|TP' \ + README.md docs recipe afd_plugin/connectors/README.md +``` + +Classify each hit as one of: + +- current and correct; +- historical and clearly labeled; +- stale and must be updated; +- generated and must be regenerated; +- unrelated to the GPU upgrade. + +Then check that README, connector README, GPU guide, recipes, design modules, +tests, CI, package/build metadata, contributor templates, and release notes +agree on the same target identity, native-build status, and support scope. + +## 10. Completion Record + +Finish with a documentation record containing: + +- files reviewed; +- files changed; +- exact evidence supporting each changed claim; +- historical files intentionally preserved; +- generated artifacts regenerated and commands used; +- native build/ABI and `NOT_APPLICABLE` claims reconciled with build files and + target-GPU smoke evidence; +- stale references removed; +- known documentation gaps and owners; +- the final consistency-search commands and outcomes. + +Documentation refresh is complete only when a new contributor can identify the +target environment, launch a supported configuration, understand the proven +support boundary, and find the evidence without consulting the upgrade author. diff --git a/.agents/skills/upgrade-gpu-version/references/upgrade-workbook.md b/.agents/skills/upgrade-gpu-version/references/upgrade-workbook.md new file mode 100644 index 00000000..0741e4a1 --- /dev/null +++ b/.agents/skills/upgrade-gpu-version/references/upgrade-workbook.md @@ -0,0 +1,500 @@ +# GPU upgrade workbook + +Use these ledgers during any GPU vLLM upgrade. Keep filled working tables and +completion reports in task notes or designated evidence storage; do not add +generated reports to the repository unless the user requests it. Historical +lesson files are evidence records, not target-version recipes. + +## Contents + +1. Identity ledger +2. Source resolution and diff commands +3. Native build and ABI evidence +4. Subagent handoff ledger +5. Patch inventory +6. Feature inventory +7. Upstream-to-AFD impact matrix +8. Architecture stop report +9. Staged upgrade plan +10. Commit discipline +11. Validation ledger +12. Upgrade lesson template +13. Completion report + +## Identity ledger + +```text +Task: + mode: planning-audit | implementation | final-validation + active RFC or issue: + requested stopping point: + +AFD: + path: + branch: + HEAD: + immutable pre-upgrade base revision: + base revision evidence: issue/RFC or proven branch ancestry + dirty paths: + target published package/compatibility version: + mirrored-version policy or NOT_APPLICABLE evidence: + +vLLM: + source directory: path, HEAD, branch, dirty status + current ref: + current resolved full SHA: + current identity evidence: + target ref: + target resolved full SHA: + target worktree: + +Target software stack: + Python: + PyTorch: + PyTorch build CUDA: + PyTorch C++11 ABI: + vLLM package/build identity: + vLLM native module/artifact path: + CUDA toolkit/runtime: + NVIDIA driver: + nvcc: + host compiler: + NCCL: + other native/communication packages: + +Target validation environment: + host/image: + GPU model and compute capability: + native artifact support for target compute capability: + GPU count: + interconnect/network topology: + scheduler reservation requirement: + model/checkpoint and quantization: + model paths: + runner generation: +``` + +Record where every value came from and resolve conflicts explicitly. Package +pins and exact source revisions outrank installed-package guesses. An installed +version is corroboration until it is tied to the intended source/build. + +## Source resolution and diff commands + +Replace placeholders with verified values from the identity ledger. Do not +check out or edit refs in user-provided vLLM directories. Reuse a checkout only +when its HEAD is the needed immutable SHA. Prefer `git show` in a read-only +planning audit; add a detached worktree only when workspace writes are in +scope. + +```bash +git -C rev-parse --verify '^{commit}' +git -C rev-parse HEAD +git -C status --short --branch +git -C worktree add --detach +git -C show ':' +git -C diff --find-renames --name-status .. +git -C diff --find-renames --stat .. +git -C log --first-parent --reverse --oneline .. +git -C diff --function-context .. -- +``` + +Use repository discovery before assuming the affected surface: + +```bash +rg -n '^(from|import) vllm' afd_plugin tests +rg -n 'PATCH START|PATCH END|Upstream source|Upstream:' afd_plugin tests +rg -n 'vllm|TARGET_VLLM_VERSION|CUDA|NCCL' \ + pyproject.toml uv.lock README.md docs recipe .github afd_plugin tests +rg -n 'GPUModelRunner|gpu_worker|EngineCore|CUDA Graph|cudagraph|DBO|ubatch' \ + afd_plugin tests docs recipe +rg -n 'TP|DP|EP|PP|process.group|PyNccl|profiler|weight|load_weights' \ + afd_plugin tests docs recipe +``` + +Adapt searches to renamed target symbols. For every copied method, inspect the +full current AFD method and both upstream definitions. Compare decorator, +method type, parameters, ordering, defaults, return annotation, body order, +state ownership, resource lifetime, and every AFD marker block. + +For a final-validation environment, record output from the available canonical +tools rather than assuming intended settings were effective. Typical evidence +includes Python and package versions, `nvidia-smi`, topology output, PyTorch +CUDA/NCCL information, `nvcc`, and the host compiler. Follow `AGENTS.md` for the +actual machine and scheduler commands. + +## Native build and ABI evidence + +Inspect `setup.py`, `pyproject.toml`, `csrc/gpu/**`, package manifests, and the +exact target vLLM build configuration before deciding whether AFD owns a +compiled GPU extension. Record: + +```text +PyTorch version/build CUDA/C++11 ABI: +vLLM install provenance and native module path: +CUDA runtime/toolkit, driver, nvcc, and host compiler: +target GPU model and compute capability: +vLLM artifact-declared compute-capability support or UNKNOWN: +target-native GPU kernel-smoke result: +AFD plugin-owned compiled GPU sources: paths | NOT_APPLICABLE with evidence +AFD clean package build and extension import: result | NOT_APPLICABLE +AFD compiled GPU operator smoke: result | NOT_APPLICABLE +``` + +Use the target source to discover the current native module name and build +metadata; do not assume a permanent module path. Final validation always +requires a real target-native kernel smoke on the recorded GPU. If published +metadata does not enumerate compiled architectures, record that field `UNKNOWN`; +the smoke proves execution without binary fingerprinting or speculative +inspection. Do not rebuild vLLM when `AGENTS.md` requires the installed runtime; +perform a source build only when the approved upgrade scope explicitly includes +build qualification. + +When AFD has plugin-owned compiled GPU sources, build the package cleanly in the +target stack, import the produced extension from that environment, and execute +its smallest real operator on a reserved target GPU. When it has none, record +the inspected paths and `NOT_APPLICABLE`; do not treat Python-registered custom +ops as compiled AFD extensions. + +## Subagent handoff ledger + +Record each delegated task before dispatch. Do not assign overlapping files to +concurrent implementation agents. + +| Agent/task | Role | Objective/root cause | Allowed files/commands | Dependencies | Status/evidence | +| --- | --- | --- | --- | --- | --- | +| | analysis / implementation / test / review | | | | | + +Use this template when the task needs a detailed handoff: + +```text +Role: +Objective or root cause: +Task mode: +Current and target vLLM refs/full SHAs: +vLLM source/worktree paths: +AFD checkout/worktree and fixed base HEAD: +Starting diff: +Allowed AFD files: +Allowed commands: +Required invariants and focused tests: +Primary review checks: +Prohibited actions: +Dependencies: +Exclusive GPU reservation/device IDs: +Exclusive port range: +Read-only model path: +Exclusive cache/temp/log/artifact paths: +Owned process/service instances: +Cleanup owner and command: +Required output: +``` + +Default prohibited actions are staging, committing, pushing, changing branches, +editing upstream trees, and resetting user changes. Analysis and review agents +are read-only. Test agents must not edit tracked files. Implementation agents +may edit only assigned AFD files and their focused tests. + +Serialize implementation in a shared checkout. For safe parallel work, use +primary-created fixed-base worktrees with non-overlapping ownership and integrate +returned raw diffs one at a time. Serialize GPU tests unless every exclusive +resource field is isolated. + +## Patch inventory + +Inventory all copied and patched upstream behavior, not only files under +`compat/patches`. + +| AFD file/symbol | Upstream path/symbol | Current SHA/signature | Target SHA/signature | AFD invariant/markers | Owner/lifetime | Tests | Disposition | +| --- | --- | --- | --- | --- | --- | --- | --- | +| | | | | | | | remove / delegate-inherit / exact adapter / propose upstream / stop | + +Disposition rules: + +- `remove`: prove target upstream absorbed the AFD behavior and delete + registration plus now-obsolete tests only after replacement coverage exists; +- `delegate-inherit`: identify the target extension point and prove ownership, + ordering, resources, performance, and non-AFD behavior remain correct; +- `exact adapter`: copy the exact target skeleton, replay only required marked + AFD deltas, and record a removal/upstream plan; +- `propose upstream`: retain only the smallest temporary exact adapter and state + the concrete upstream extension needed; +- `stop`: use when no narrow invariant-preserving adaptation exists. + +## Feature inventory + +Expand this seed table from repository source, tests, recipes, and docs. Do not +assume every future release has the same features or test count. + +| Feature cell | Current evidence | Target upstream impact | Planned adaptation | Required test/evidence | Status | +| --- | --- | --- | --- | --- | --- | +| package/version/plugin isolation | | | | | | +| GPU runner generation | | | | | | +| Attention worker/model runner | | | | | | +| FFN worker/model runner/daemon | | | | | | +| model construction and loading | | | | | | +| gate/expert ownership | | | | | | +| connector/payload/ordering | | | | | | +| eager / CUDA Graph | | | | | | +| DBO / uBatch | | | | | | +| TP / DP / EP / PP topology | | | | | | +| profiling / warmup / dummy runs | | | | | | +| quantization / EPLB | | | | | | +| serving / request output / sampling | | | | | | +| accuracy | | | | | | +| startup / failure / shutdown / cleanup | | | | | | +| shared-code NPU impact | | | | | | + +An omitted feature is `UNKNOWN`. Record unsupported or unvalidated cells +explicitly instead of dropping them from the upgrade scope. + +## Upstream-to-AFD impact matrix + +| Upstream SHA/path/symbol | Old contract | Target contract | Class | AFD consumers | Preserved invariant | Adaptation | Test | Risk | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| | | | mechanical / behavioral / architectural | | | | | | + +For hot or distributed methods, record call order and resource ownership. Check +ForwardContext installation, scheduler/request state, KV-cache ownership, +graph capture/replay, DBO stage metadata, process groups, physical device +mapping, P2P ordering, sampling/output ownership, error propagation, and +shutdown where applicable. + +## Architecture stop report + +```text +Decision: STOPPED — major architecture change + +Identity: +- AFD HEAD/base: +- current vLLM ref/full SHA: +- target vLLM ref/full SHA: +- requested CUDA/PyTorch/toolchain tuple: + +Change: +- old architecture/interface: +- target architecture/interface: +- exact upstream evidence: + +AFD impact: +- affected files/features: +- invariant that no longer maps: +- why target-upstream-plus-small-AFD-diff is impossible: +- why feature deletion, validation bypass, reflection, or broad copying is unsafe: + +Decision required: +- smallest maintainer/design choice: +- candidate directions, without implementation: + +Worktree: +- production edits made: none +- unrelated dirty paths preserved: +``` + +## Staged upgrade plan + +Use this table only after the architecture gate passes. When an earlier gate is +blocked, replace it with a short blocker-resolution plan listing the missing +identity/source/environment evidence and next allowed actions; do not populate +implementation rows from historical lessons. + +| Stage | Upstream cause | Expected AFD files | Preserved invariant | Patch disposition | Tests | Performance/distributed risk | Docs/removal plan | +| --- | --- | --- | --- | --- | --- | --- | --- | +| | | | | | | | | + +Order stages by actual dependencies. Keep mechanical target-source refreshes +reviewable separately from AFD behavior changes when practical. Each stage must +name evidence that would prove it complete. + +## Commit discipline + +Create local commits only in implementation mode when commits are in scope. +Keep one upstream cause or validated failure root cause with its focused tests. +Stage explicit paths, inspect the staged diff, and never include unrelated work. + +Suggested form after checking repository history: + +```bash +git commit -s -m 'fix(gpu-upgrade): adapt to ' \ + -m 'Upgrade requirement or failure: +- + +Upstream cause: +- vLLM ..: + +Why this change: +- +- + +Validation: +- +- ' +``` + +Use the appropriate `feat`, `fix`, `refactor`, `test`, `docs`, or `chore` type. +Do not commit speculative code that still fails its focused gate, and do not +amend unrelated earlier commits to hide later failures. + +## Validation ledger + +```text +Validation cell: + purpose: + AFD commit/tree: + vLLM ref/full SHA/package build: + Python/PyTorch/build CUDA/C++11 ABI: + vLLM native artifact path/provenance: + CUDA runtime/toolkit/driver/compiler/NCCL: + host/image and GPU topology: + target compute capability and native kernel smoke: + AFD compiled GPU extension build/import/smoke or NOT_APPLICABLE: + scheduler reservation: + model/checkpoint/quantization: + runner generation: + connector and Attention/FFN topology: + TP/DP/EP/PP: + eager/graph/DBO/uBatch/profiling: + prompt/sampling/accuracy inputs: + command and relevant environment: + +Result: + status: PASS | FAIL | BLOCKED | SKIPPED + collected/passed/failed/error/skipped counts: + metrics/oracle: + first useful root traceback: + log/artifact paths: + skip reasons: + process/port/GPU/IPC/scheduler cleanup: +``` + +Use this sequence: + +1. static, format, compile, import, package, native build/ABI, and exact patch + checks; +2. affected focused tests, then complete CPU/unit suite; +3. exact target native vLLM GPU control; +4. complete GPU feature E2E; +5. complete GPU model E2E; +6. every full GPU accuracy case with no final dataset limit; +7. documentation refresh and claim reconciliation; +8. final version/source/docs/recipe/metadata audit; +9. version-specific lesson when implementation completion is in scope. + +After a code failure, preserve the failing artifacts, rerun the narrow +reproduction after the fix, then rerun the complete earlier gates. A passing +retry does not erase the original failed cell. + +## Upgrade lesson template + +After a completed implementation upgrade, create +`references/--to--lessons.md`. Sanitize refs, use a +short resolved SHA when no immutable tag exists, and add a suffix instead of +overwriting an existing file. + +```markdown +# Lessons from the to GPU upgrade + +Use this file as historical evidence, not as a recipe. Re-derive every fact from +the exact revisions of the next upgrade. + +## Runtime identity and scope + +- AFD before/after commits: +- current and target vLLM refs/full SHAs: +- Python/PyTorch/CUDA/driver/compiler/NCCL/GPU topology: +- models, connectors, modes, parallelism, and exclusions: + +## Upstream changes that mattered + +- changed contract and evidence: +- affected AFD invariant: +- mechanical, behavioral, or architectural classification: + +## Adaptations and patch lifecycle + +- smallest correct adaptation seam: +- patches retained, delegated, removed, ported, or proposed upstream: +- commits and focused tests: + +## Failures and root causes + +- symptom and first useful traceback: +- upstream or AFD root cause: +- misleading attempts or assumptions: +- final fix and proof: + +## Validation and environment lessons + +- checks that exposed real issues: +- CUDA, scheduler, topology, model, port, cache, or cleanup pitfalls: +- gaps between unit, native, E2E, accuracy, and performance evidence: + +## Documentation and process lessons + +- stale or conflicting claims: +- workflow, review, or commit practices that helped or failed: + +## Reusable principles + +- lessons likely to apply beyond this version pair: + +## Version-specific facts + +- facts that must not be generalized without revalidation: + +## Remaining debt and next-upgrade checklist + +- unsupported or unvalidated cells: +- temporary adapters and removal triggers: +- symbols, commands, and invariants to inspect first next time: +``` + +Keep lessons concise and evidence-backed. Link commits, tests, logs, or source +symbols rather than embedding large outputs. Exclude credentials, private +endpoints, and machine-specific secrets. Review the frozen lesson diff before a +signed-off documentation commit. + +## Completion report + +```text +Task mode and requested stopping point: + +Runtime identity: +- AFD path/branch/HEAD/base/target version: +- current and target vLLM refs/full SHAs: +- target source/worktree paths: + +Target environment: +- Python/PyTorch/CUDA/driver/compiler/NCCL: +- host/image/GPU topology/scheduler: +- model paths and validation scope: + +Architecture gate: +Impact summary and staged plan: +Patch inventory and removal/upstream plans: +Feature matrix: + +Changes and commits: +- — <upstream cause/root cause> + +CPU-safe validation: +- commands/results/skips: + +GPU validation: +- native control: +- features: +- models: +- full accuracy: +- actual counts, metrics, logs, skips, and cleanup: + +Documentation refresh and claim reconciliation: +Intentionally preserved historical references: +Final version/source/docs audit: +Upgrade lesson path/commit or proposed lesson notes: + +Unsupported/unvalidated/excluded: +Remaining private seams and performance observations: +Release blockers: +Readiness: GPU-validated | implemented-GPU-unverified | planning-only | blocked +Shared-code and NPU claim status: +Branch/publish status: +``` diff --git a/.agents/skills/upgrade-gpu-version/references/v0191-to-v026-lessons.md b/.agents/skills/upgrade-gpu-version/references/v0191-to-v026-lessons.md new file mode 100644 index 00000000..cf13919b --- /dev/null +++ b/.agents/skills/upgrade-gpu-version/references/v0191-to-v026-lessons.md @@ -0,0 +1,192 @@ +# Lessons from the v0.19.1 to v0.26.0 GPU upgrade + +Use this history as a source of failure patterns, not as a target-version +recipe. Re-derive every applicable fact from the exact current and target vLLM +revisions, AFD contracts, tests, and validation environment of the next upgrade. + +## Scope and evidence + +The transition was broader than a version-pin edit: + +- [RFC #167](https://github.com/vllm-project/afd-plugin/issues/167) defined the + GPU migration and release-gate expectations. +- [PR #176](https://github.com/vllm-project/afd-plugin/pull/176) developed and + simplified the DeepSeek remote-experts and GPU runtime adaptation. +- [PR #182](https://github.com/vllm-project/afd-plugin/pull/182) carried the + reviewed GPU slice onto the v0.26 integration branch and recorded exact-head + CPU and GPU validation. +- [PR #186](https://github.com/vllm-project/afd-plugin/pull/186) integrated GPU + and NPU work, refreshed package/docs contracts, and exposed final review + issues around signatures, topology, inventory, and support wording. + +The target recorded by that work was vLLM `0.26.0`. GPU hardware evidence used +NVIDIA L20X and DeepSeek-V2-Lite with ModelRunner V1. These are historical +version facts, not defaults for a future upgrade. + +## Reusable failure patterns + +### A version pin is only the identity change + +The real migration crossed model construction and loading, Attention and FFN +workers/model runners, EngineCore lifecycle, P2P metadata, graph capture, DBO, +parallelism, profiling, package isolation, recipes, and documentation. Build an +affected-surface and feature inventory before editing; otherwise a passing +import can hide unported runtime paths. + +### Prove a new public interface is necessary + +The early PR #176 design added experts-specific connector methods, headers, +transfer IDs, grouped operations, and a second protocol. Review showed the +existing Attention-to-FFN and FFN-to-Attention data path could carry the needed +ordered router logits. Removing the extra abstractions substantially reduced +the patch while preserving the native MoE boundary. + +For future upgrades, treat new public connector, protocol, coordinator, or +runner abstractions as an architecture-gate decision. First prove that the +target native contract and existing AFD interface cannot express the required +invariant. + +### Preserve native lifecycle instead of copying churn + +The successful model boundary retained native model, decoder-layer, and MoE +forward behavior while AFD owned role-aware construction, remote expert +handoff, and weight filtering. Routed/shared experts remained on FFN and +request/KV/sampling ownership remained on Attention. + +When target upstream changes ordinary forward, loading, compile, graph, or +parallel behavior, prefer delegation through the new native lifecycle. Copy +only the constructor or method for which no stable injection point exists. + +### Ordered transport includes metadata semantics + +Adding an optional tensor to an existing P2P path is not merely a send/receive +change. The upgrade had to preserve operation order, graph replay, DBO stage +identity, gate placement, shape/dtype ownership, and FFN-side consumption. +Review the producer, wire allocation, consumer, control plane, dummy/warmup +path, and cleanup as one contract. + +### Patch inventory extends beyond the patch directory + +Patched or copied upstream methods also lived in model and worker modules. Final +review found that even one return annotation different from target upstream +violated the exact-signature requirement. Compare decorators, method type, +parameters, defaults, and return annotation, not just callable behavior. + +Keep design patch inventories synchronized with actual assignments and state +owners. A stale inventory causes the next upgrade to audit the wrong symbol. + +### Equal-rank topology can hide aggregation bugs + +Final review identified an FFN DP metadata issue for asymmetric Attention:FFN +layouts: P2P fan-in aggregated token tensors while native MoE could still see +raw per-Attention counts. Symmetric 1A1F or 2A2F tests did not prove 4A2F-style +subgroup aggregation. + +Inventory topology formulas and test at least one asymmetric valid layout when +the supported contract allows it. Distinguish wire-allocation counts from the +metadata consumed by target native compute. + +### Focused tests do not complete the exact-head gate + +PR #176 had focused eager, graph, parallel, DBO, and limited accuracy evidence, +but explicitly had not rerun the complete repository GPU matrix on its final +head. That head was therefore an implementation candidate, not final release +evidence. PR #182 later recorded complete marker-based execution for the frozen +integration revision. + +Tie every result to the exact AFD tree and runtime identity. Evidence from an +earlier commit remains historical after a behaviorally relevant change until +the affected and required regression gates are rerun. + +### Configuration is not proof of execution + +Graph and DBO support required evidence that graph capture/replay occurred and +that both ubatch stages executed, not only that flags were present. Parallelism +requires collective and topology behavior, not server startup. Profiling +requires usable traces from both roles where claimed. + +Record observable execution evidence for the feature being claimed. Do not +promote launch success into graph, DBO, TP/DP/EP, accuracy, or performance +support. + +### Accuracy scope must remain explicit + +The recorded GPU integration used limited GSM8K samples for eager and graph. +That was useful diagnostic evidence, but it was not a full-dataset final +qualification. Report dataset size, threshold, metric, mode, and exact cell. +Never let a limited run become the generic accuracy claim for the release. + +### Native control separates runtime failure from AFD failure + +A target model, wheel, CUDA stack, or kernel may fail before AFD participates. +Establish an exact native eager control before interpreting AFD failures. A +fallback mode is a separate cell; do not silently turn it into the requested +target or patch AFD around a native runtime defect. + +### Cleanup is part of correctness evidence + +The useful GPU evidence included process, port, and selected-device cleanup. +Distributed success followed by leaked workers, communicators, IPC state, or +scheduler reservations is not a clean pass. Preserve cleanup output with the +run and investigate post-response shutdown errors separately from request +correctness. + +### Documentation claims require their own audit + +The integration review found stale generated metadata, patch descriptions, and +support wording. Current documentation and intentionally historical recipes +needed different treatment: active target claims were updated, while old +branch-specific material stayed historical. + +Search broadly, inspect each result in context, and distinguish GPU-validated, +shared-code, unit-only, unvalidated, and unsupported claims. A GPU upgrade does +not prove NPU support merely because shared code changed. + +## Historical validation sequence + +The useful sequence was: + +1. diff, lint, format, compile/import, package, and focused contract tests; +2. complete CPU/unit coverage with expected backend skips identified; +3. focused GPU eager cases for both gate placements; +4. graph, TP/EP, DP/EP, and representative DBO cells with execution evidence; +5. the complete repository GPU marker suite on the frozen integration head; +6. limited eager/graph GSM8K reported as limited scope; +7. process, port, device, and scheduler cleanup; +8. integration review of signatures, topology, docs, and support claims. + +Future upgrades must discover the current repository tests and run the full +accuracy scope required by the current skill. Do not preserve the historical +test count or limited dataset as a permanent gate. + +## Reusable principles + +- Derive architecture from exact source before implementation. +- Prefer target native behavior plus small AFD differences. +- Keep public interface expansion behind an explicit architecture decision. +- Freeze supported feature and topology matrices before editing. +- Revalidate behaviorally affected evidence on the exact final tree. +- Separate functional, accuracy, performance, cleanup, and documentation claims. +- Preserve historical evidence without turning it into current support. + +## Version-specific facts + +The following facts belong only to this historical transition and must be +revalidated for another target: + +- target vLLM was `0.26.0`; +- the selected GPU path used ModelRunner V1; +- hardware evidence cited NVIDIA L20X and DeepSeek-V2-Lite; +- CUDA Graph claims were scoped to the modes implemented at that revision; +- the repository GPU suite had a particular collected count at that time; +- final GPU accuracy evidence in the integration record was limited-sample, not + a permanent full-qualification policy; +- quantized GPU MoE, EPLB, and ModelRunner V2 were excluded from that slice. + +## Historical process lesson + +The work benefited from a draft for architecture simplification, a separate +exact-source GPU integration commit, and a final combined integration review. +Future upgrades need not reproduce those branch names or PR boundaries, but +should keep architectural decisions, coherent adaptations, exact-head hardware +evidence, and final claim reconciliation independently reviewable.