[Feature] Split c8 to li c8 and sfa c8 (vllm-project#12470)

### What this PR does / why we need it?

port vllm-project#12351 to main branch.

### Does this PR introduce _any_ user-facing change?

### How was this patch tested?


- vLLM version: v0.25.1
- vLLM main:
vllm-project/vllm@fe784ff

---------

Signed-off-by: ZYang6263 <50876451+ZYang6263@users.noreply.github.com>
Signed-off-by: Angazenn <supperccell@163.com>
Co-authored-by: ZYang6263 <50876451+ZYang6263@users.noreply.github.com>

Author: Angazenn

docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po

@@ -287,21 +287,37 @@ msgstr ""
 "布局。此选项仅对使用 MLA 的模型（例如 DeepSeek）生效。                                |"
 
 msgid ""
-"| `enable_sparse_c8`                  | bool | `False` | Whether to enable "
-"KV cache C8 in DSA models (e.g., DeepSeek V3.2 and GLM5). Not supported on "
-"Ascend 950 devices now |"
+"| `enable_sparse_sfa_c8`              | bool | `False` | Whether to enable "
+"the packed C8 KV cache for Sparse Flash Attention in DSA models (e.g., "
+"DeepSeek V3.2 and GLM5). This option is independent of "
+"`enable_sparse_li_c8`. SFA prefill context parallelism and Ascend 950 DCP "
+"are not supported. |"
 msgstr ""
-"| `enable_sparse_c8`                  | bool | `False` | 是否在 DSA 模型（例如 "
-"DeepSeek V3.2 和 GLM5）中启用 KV 缓存 C8。目前不支持 Ascend 950 设备。              |"
+"| `enable_sparse_sfa_c8`              | bool | `False` | 是否在 DSA 模型（例如 "
+"DeepSeek V3.2 和 GLM5）中启用用于稀疏闪存注意力的打包 C8 KV 缓存。此选项独立于 "
+"`enable_sparse_li_c8`。目前不支持 SFA 预填充上下文并行和 Ascend 950 DCP。 |"
+
+msgid ""
+"| `enable_sparse_li_c8`               | bool | `False` | Whether to enable "
+"the C8 key and scale caches for LightningIndexer in DSA models. This option "
+"is independent of `enable_sparse_sfa_c8` and only applies to eligible "
+"indexer layers from the model quantization config. SFA prefill context "
+"parallelism and Ascend 950 DCP are not supported. |"
+msgstr ""
+"| `enable_sparse_li_c8`               | bool | `False` | 是否在 DSA 模型中为 "
+"LightningIndexer 启用 C8 key 和 scale 缓存。此选项独立于 "
+"`enable_sparse_sfa_c8`，并且仅适用于模型量化配置中的合格 indexer 层。目前不支持 "
+"SFA 预填充上下文并行和 Ascend 950 DCP。 |"
 
 msgid ""
 "| `c8_enable_reshape_optim`           | bool | `False` | Whether to enable "
-"StoreKVBlock operator achieves acceleration under the C8 feature (this means"
-" that enable_sparse_c8 needs to be enabled). In the PD separation scenario, "
+"the StoreKVBlock operator to accelerate LightningIndexer C8 cache writes. "
+"`enable_sparse_li_c8` must also be enabled. In the PD separation scenario, "
 "only the P node is enabled. |"
 msgstr ""
 "| `c8_enable_reshape_optim`           | bool | `False` | 是否启用 StoreKVBlock "
-"算子在 C8 特性下实现加速（这意味着需要启用 enable_sparse_c8）。在 PD 分离场景中，仅 P 节点启用。 |"
+"算子加速 LightningIndexer C8 缓存写入。还必须启用 `enable_sparse_li_c8`。在 PD "
+"分离场景中，仅 P 节点启用。 |"
 
 msgid ""
 "| `enable_mc2_hierarchy_comm`         | bool | `False` | Enable "

docs/source/user_guide/configuration/additional_config.md

@@ -78,8 +78,9 @@ The following table lists additional configuration options available in vLLM Asc
 | `enable_sleep_mode_extra_cleanup`   | bool | `False` | Enables extra sleep-mode cleanup for RL workloads, including HCCL process-group release and ACL graph workspace cleanup. Disabled by default because wakeup may need to restore HCCL and recapture ACL graphs. |
 | `pa_shape_list`                     | list | `[]`    | The custom shape list of page attention ops.                                                              |
 | `enable_kv_nz`                      | bool | `False` | Whether to enable KV cache NZ layout. This option only takes effects on models using MLA (e.g., DeepSeek).                                      |
-| `enable_sparse_c8`                  | bool | `False` | Whether to enable KV cache C8 in DSA models (e.g., DeepSeek V3.2 and GLM5). Not supported on Ascend 950 devices now |
-| `c8_enable_reshape_optim`           | bool | `False` | Whether to enable StoreKVBlock operator achieves acceleration under the C8 feature (this means that enable_sparse_c8 needs to be enabled). In the PD separation scenario, only the P node is enabled. |
+| `enable_sparse_sfa_c8`              | bool | `False` | Whether to enable the packed C8 KV cache for Sparse Flash Attention in DSA models (e.g., DeepSeek V3.2 and GLM5). This option is independent of `enable_sparse_li_c8`. SFA prefill context parallelism and Ascend 950 DCP are not supported. |
+| `enable_sparse_li_c8`               | bool | `False` | Whether to enable the C8 key and scale caches for LightningIndexer in DSA models. This option is independent of `enable_sparse_sfa_c8` and only applies to eligible indexer layers from the model quantization config. SFA prefill context parallelism and Ascend 950 DCP are not supported. |
+| `c8_enable_reshape_optim`           | bool | `False` | Whether to use the StoreKVBlock operator to accelerate LightningIndexer C8 cache writes. `enable_sparse_li_c8` must also be enabled. In the PD separation scenario, only the P node is enabled. |
 | `enable_mc2_hierarchy_comm`         | bool | `False` | Enable dispatch/combine op inter-node communication by ROCE. |
 | `enable_prefill_mc2`                | bool | `False` | Whether to reserve mc2_token_capacity for prefill batches. When enabled, `max_num_batched_tokens` is used to calculate the mc2_token_capacity instead of the decode-only capacity. In this scenario, the recommended maximum value of `max_num_batched_tokens` is `tp_size * 512`. This is a temporary switch; once MC2 operators are complete for all scenarios, this switch will be removed and MC2 will be enabled by default. |
 | `mega_moe_max_tokens`               | int  | `65536` | Per-rank token capacity after dispatch in the mega moe (dispatch_ffn_combine) fused operator. When load imbalance causes a rank to receive more tokens than this limit, the excess tokens are dropped and skipped from computation, degrading accuracy. Do not set this too large: workspace memory scales linearly with this value. |

tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml

@@ -63,7 +63,7 @@ test_cases:
       - "--compilation-config"
       - '{"cudagraph_mode": "FULL_DECODE_ONLY", "cudagraph_capture_sizes":[4, 16, 64, 128]}'
       - "--additional-config"
-      - '{"enable_dsa_cp": true, "ascend_compilation_config":{"enable_npugraph_ex": true, "enable_static_kernel": false}, "multistream_overlap_shared_expert": true, "enable_mc2_hierarchy_comm": false, "enable_sparse_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": false}'
+      - '{"enable_dsa_cp": true, "ascend_compilation_config":{"enable_npugraph_ex": true, "enable_static_kernel": false}, "multistream_overlap_shared_expert": true, "enable_mc2_hierarchy_comm": false, "enable_sparse_sfa_c8": true, "enable_sparse_li_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": false}'
       - "--speculative-config"
       - '{"num_speculative_tokens": 3, "method": "deepseek_mtp"}'
     test_content: []

tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py

@@ -141,7 +141,8 @@ def match_outputs_with_goldens(outputs: list[tuple[list[int], str]], goldens: Se
             "additional_config": {
                 "enable_flashcomm1": True,
                 "enable_dsa_cp": True,
-                "enable_sparse_c8": True,
+                "enable_sparse_sfa_c8": True,
+                "enable_sparse_li_c8": True,
             },
             "speculative_config": {
                 "method": "mtp",

tests/ut/attention/a2/test_sfa_v1.py

@@ -24,7 +24,7 @@
     custom_kv_rmsnorm_rope,
 )
 from vllm_ascend.attention.utils import get_sfa_qsfa_packed_head_dim
-from vllm_ascend.device.device_op import DeviceOperator
+from vllm_ascend.device.device_op import BaseDeviceAdaptor, DeviceOperator
 from vllm_ascend.quantization.methods import (
     AscendW8A8DynamicLinearMethod,
     AscendW8A8LinearMethod,
@@ -194,6 +194,104 @@ def test_execute_sparse_flash_attention_c8_returns_softmax_components(self):
         self.assertTrue(call_kwargs["return_softmax_lse"])
 
 
+class TestAscendSFACacheComposition(TestBase):
+    def test_compose_independent_sfa_and_li_c8_layouts(self):
+        for enable_sfa_c8, enable_li_c8 in (
+            (False, False),
+            (True, False),
+            (False, True),
+            (True, True),
+        ):
+            with self.subTest(
+                enable_sfa_c8=enable_sfa_c8,
+                enable_li_c8=enable_li_c8,
+            ):
+                impl = AscendSFAImpl.__new__(AscendSFAImpl)
+                impl.layer_name = "model.layers.0.self_attn.attn"
+                impl.has_indexer = True
+                impl.enable_sparse_sfa_c8 = enable_sfa_c8
+                impl.enable_sparse_li_c8 = enable_li_c8
+
+                main_cache = tuple(torch.empty(1) for _ in range(1 if enable_sfa_c8 else 2))
+                indexer_cache = tuple(torch.empty(1) for _ in range(2 if enable_li_c8 else 1))
+                impl.indexer = SimpleNamespace(k_cache=SimpleNamespace(kv_cache=indexer_cache))
+
+                composed = impl._compose_sfa_kv_cache(main_cache)
+
+                expected = (*main_cache, *indexer_cache)
+                self.assertIsNotNone(composed)
+                assert composed is not None
+                self.assertEqual(len(composed), len(expected))
+                for actual_tensor, expected_tensor in zip(composed, expected):
+                    self.assertIs(actual_tensor, expected_tensor)
+
+    @patch("vllm_ascend.attention.sfa_v1.get_ascend_config")
+    def test_li_c8_reshape_optim_requires_layer_li_c8(self, mock_get_ascend_config):
+        impl = AscendSFAImpl.__new__(AscendSFAImpl)
+        mock_get_ascend_config.return_value.c8_enable_reshape_optim = True
+
+        impl.enable_sparse_li_c8 = False
+        self.assertFalse(impl._use_li_c8_reshape_optim())
+
+        impl.enable_sparse_li_c8 = True
+        self.assertTrue(impl._use_li_c8_reshape_optim())
+
+        mock_get_ascend_config.return_value.c8_enable_reshape_optim = False
+        self.assertFalse(impl._use_li_c8_reshape_optim())
+
+    @patch(
+        "vllm_ascend.device.device_op.torch.ops._C_ascend.npu_lightning_indexer_quant",
+        create=True,
+    )
+    def test_li_c8_indexer_uses_cache_slots_after_main_cache(self, mock_indexer):
+        expected_topk = torch.zeros(2, 1, 4, dtype=torch.int32)
+        mock_indexer.return_value = expected_topk
+        q_li = torch.zeros(2, 1, 128, dtype=torch.int8)
+        q_li_scale = torch.ones(2, 1, dtype=torch.float16)
+        weights = torch.ones(2, 1, dtype=torch.bfloat16)
+        attn_metadata = SimpleNamespace(block_table=torch.zeros(1, 2, dtype=torch.int32))
+
+        for enable_sfa_c8 in (False, True):
+            with self.subTest(enable_sfa_c8=enable_sfa_c8):
+                main_cache = (
+                    (torch.empty(2, 16, 1, 656, dtype=torch.int8),)
+                    if enable_sfa_c8
+                    else (
+                        torch.empty(2, 16, 1, 512, dtype=torch.bfloat16),
+                        torch.empty(2, 16, 1, 64, dtype=torch.bfloat16),
+                    )
+                )
+                indexer_k_cache = torch.empty(2, 16, 1, 128, dtype=torch.int8)
+                indexer_scale_cache = torch.empty(2, 16, 1, 1, dtype=torch.float16)
+                kv_cache = (*main_cache, indexer_k_cache, indexer_scale_cache)
+                impl = AscendSFAImpl.__new__(AscendSFAImpl)
+                impl.enable_sparse_sfa_c8 = enable_sfa_c8
+                impl.use_torch_npu_lightning_indexer = False
+                mock_indexer.reset_mock()
+
+                result = BaseDeviceAdaptor.indexer_select_post_process(
+                    impl,
+                    q_li,
+                    q_li_scale,
+                    q_li.shape,
+                    weights,
+                    kv_cache,
+                    attn_metadata,
+                    torch.tensor([2], dtype=torch.int32),
+                    torch.tensor([2], dtype=torch.int32),
+                    True,
+                    False,
+                )
+
+                self.assertIs(result, expected_topk)
+                call_kwargs = mock_indexer.call_args.kwargs
+                self.assertIs(call_kwargs["key"], indexer_k_cache)
+                self.assertEqual(
+                    call_kwargs["key_dequant_scale"].data_ptr(),
+                    indexer_scale_cache.data_ptr(),
+                )
+
+
 class TestAscendSFAKVQuantSparseAttention(TestBase):
     @patch("vllm_ascend.attention.sfa_v1.torch_npu.npu_dynamic_block_quant")
     @patch("vllm_ascend.attention.sfa_v1.torch_npu.npu_interleave_rope")
@@ -224,7 +322,7 @@ def test_pack_prefill_kv_cache(self, mock_rms_norm, mock_rope, mock_block_quant)
 
     def test_execute_kv_quant_sparse_flash_attention(self):
         impl = AscendSFAImpl.__new__(AscendSFAImpl)
-        impl.use_sparse_c8_sfa = True
+        impl.enable_sparse_sfa_c8 = True
         impl.scale = 0.125
         impl.sfa_qsfa_tile_size = 128
         impl.qk_rope_head_dim = 16
@@ -270,7 +368,7 @@ def test_execute_kv_quant_sparse_flash_attention(self):
     def test_prolog_v3_enables_packed_int8_kv_cache(self):
         impl = AscendSFAImpl.__new__(AscendSFAImpl)
         impl._quant_type = AscendW8A8DynamicLinearMethod
-        impl.use_sparse_c8_sfa = True
+        impl.enable_sparse_sfa_c8 = True
         impl.has_indexer = True
         impl.sfa_qsfa_tile_size = 128
         impl.sfa_qsfa_k_nope_clip_alpha = torch.ones(1)
@@ -678,9 +776,10 @@ def setUp(
         # Default ascend config (non-MLAPO, non-C8)
         mock_ascend_config = MagicMock()
         mock_ascend_config.enable_mlapo = False
-        mock_ascend_config.enable_sparse_c8 = False
+        mock_ascend_config.enable_sparse_sfa_c8 = False
+        mock_ascend_config.enable_sparse_li_c8 = False
         mock_ascend_config.enable_shared_expert_dp = False
-        mock_ascend_config.is_sparse_c8_layer.return_value = False
+        mock_ascend_config.is_sparse_li_c8_layer.return_value = False
         mock_get_ascend_config.return_value = mock_ascend_config
         self.mock_ascend_config = mock_ascend_config
 
@@ -827,8 +926,8 @@ def test_exec_kv_sparse_c8_uses_custom(
         mock_npu_kv_rmsnorm_rope_cache,
         mock_custom_kv_rmsnorm_rope,
     ):
-        """exec_kv with use_sparse_c8_sfa → delegates to custom_kv_rmsnorm_rope."""
-        self.impl.use_sparse_c8_sfa = True
+        """exec_kv with enable_sparse_sfa_c8 delegates to custom_kv_rmsnorm_rope."""
+        self.impl.enable_sparse_sfa_c8 = True
         self.impl.c8_k_cache_dtype = torch.int8
         self.impl.enable_dsa_cp = False
         self.impl.kv_a_layernorm = MagicMock()
@@ -885,7 +984,7 @@ def test_resolve_path_w8a8dynamic_c8_goes_prolog_v3(self):
         """W8A8Dynamic + C8 + PD consumer → PROLOG_V3."""
         self._set_quant(AscendW8A8DynamicLinearMethod)
         self.impl.is_kv_consumer = True
-        self.impl.use_sparse_c8_sfa = True
+        self.impl.enable_sparse_sfa_c8 = True
 
         path = self.impl._resolve_preprocess_type(torch.bfloat16)
         self.assertEqual(path, PreprocessType.PROLOG_V3)
@@ -902,7 +1001,7 @@ def test_resolve_path_mxfp_c8_goes_prolog_v3(self):
         """MXFP + is_kv_consumer + C8 → PROLOG_V3."""
         self._set_quant(AscendW8A8MXFP8DynamicLinearMethod)
         self.impl.is_kv_consumer = True
-        self.impl.use_sparse_c8_sfa = True
+        self.impl.enable_sparse_sfa_c8 = True
 
         path = self.impl._resolve_preprocess_type(torch.bfloat16)
         self.assertEqual(path, PreprocessType.PROLOG_V3)
@@ -911,7 +1010,7 @@ def test_resolve_path_unquantized_c8_goes_prolog_v3(self):
         """Unquantized + is_kv_consumer + C8 → PROLOG_V3 (blocked by reasons)."""
         self._set_quant(None)
         self.impl.is_kv_consumer = True
-        self.impl.use_sparse_c8_sfa = True
+        self.impl.enable_sparse_sfa_c8 = True
 
         path = self.impl._resolve_preprocess_type(torch.bfloat16)
         # Enters candidate but blocked by _get_fused_type_unsupported_reasons
@@ -930,7 +1029,7 @@ def test_resolve_path_w8a8dynamic_c8_no_mlapo_still_prolog_v3(self):
         """W8A8Dynamic+C8 enters PROLOG_V3 even when enable_mlapo=False."""
         self._set_quant(AscendW8A8DynamicLinearMethod)
         self.impl.is_kv_consumer = True
-        self.impl.use_sparse_c8_sfa = True
+        self.impl.enable_sparse_sfa_c8 = True
 
         path = self.impl._resolve_preprocess_type(torch.bfloat16)
         self.assertEqual(path, PreprocessType.PROLOG_V3)
@@ -965,15 +1064,15 @@ def test_reasons_kv_producer_blocked(self):
     def test_reasons_unquantized_c8_blocked(self):
         self._setup_prolog_v3_state()
         self.impl._quant_type = None
-        self.impl.use_sparse_c8_sfa = True
+        self.impl.enable_sparse_sfa_c8 = True
 
         reasons = self.impl._get_fused_type_unsupported_reasons(PreprocessType.PROLOG_V3)
         self.assertTrue(any("C8 sparse requires quantized" in r for r in reasons))
 
     def test_reasons_mlapo_c8_blocked(self):
         self._setup_prolog_v3_state()
         self.impl.preprocess_type = PreprocessType.MLAPO
-        self.impl.use_sparse_c8_sfa = True
+        self.impl.enable_sparse_sfa_c8 = True
 
         reasons = self.impl._get_fused_type_unsupported_reasons(PreprocessType.MLAPO)
         self.assertTrue(any("sparse C8" in r for r in reasons))
@@ -984,7 +1083,7 @@ def test_sfa_preprocess_prolog_v3_mxfp(self):
         """MXFP branch: npu_dynamic_mx_quant + q_c scale wrapping."""
         impl = AscendSFAImpl.__new__(AscendSFAImpl)
         impl._quant_type = AscendW8A8MXFP8DynamicLinearMethod
-        impl.use_sparse_c8_sfa = False
+        impl.enable_sparse_sfa_c8 = False
         impl.local_num_heads = 2
         impl.num_heads = 2
         impl.kv_lora_rank = 128

tests/ut/test_ascend_config.py

@@ -22,6 +22,7 @@
 
 from tests.ut.base import TestBase
 from vllm_ascend.ascend_config import (
+    AscendConfig,
     SchedulerConfig,
     ShortRequestFirstConfig,
     clear_ascend_config,
@@ -59,6 +60,46 @@ def _make_model_config(
             get_total_num_kv_heads=lambda: total_num_kv_heads,
         )
 
+    @staticmethod
+    def _make_sparse_li_c8_config(quant_description):
+        quant_config = SimpleNamespace(quant_description=quant_description)
+        config = AscendConfig.__new__(AscendConfig)
+        config.enable_sparse_li_c8 = True
+        (
+            config._sparse_li_c8_layer_ids,
+            config._sparse_li_c8_layer_names,
+        ) = AscendConfig._parse_sparse_li_c8_layers_from_quant_config(quant_config)
+        config._sparse_li_c8_layer_filter_enabled = AscendConfig._has_sparse_li_c8_layer_config(quant_config)
+        return config
+
+    def test_sparse_li_c8_layer_filter_uses_indexer_quant_type(self):
+        config = self._make_sparse_li_c8_config(
+            {
+                "model.layers.1.self_attn.indexer.quant_type": "INT8_DYNAMIC",
+                "model.layers.2.self_attn.indexer.quant_type": "BF16",
+            }
+        )
+
+        self.assertTrue(config.is_sparse_li_c8_layer("model.layers.1.self_attn.indexer.k_cache"))
+        self.assertFalse(config.is_sparse_li_c8_layer("model.layers.2.self_attn.indexer.k_cache"))
+
+    def test_sparse_li_c8_layer_filter_uses_indexer_wq_b_weight(self):
+        config = self._make_sparse_li_c8_config(
+            {
+                "model.layers.3.self_attn.indexer.wq_b_weight": "W8A8_MXFP8",
+                "model.layers.4.self_attn.indexer.wq_b_weight": "W8A8_DYNAMIC",
+            }
+        )
+
+        self.assertTrue(config.is_sparse_li_c8_layer("model.layers.3.self_attn.indexer.k_cache"))
+        self.assertFalse(config.is_sparse_li_c8_layer("model.layers.4.self_attn.indexer.k_cache"))
+
+    def test_sparse_li_c8_without_layer_metadata_applies_to_all_indexers(self):
+        config = self._make_sparse_li_c8_config({"indexer_quant_type": "INT8_DYNAMIC"})
+
+        self.assertTrue(config.is_sparse_li_c8_layer("model.layers.1.self_attn.indexer.k_cache"))
+        self.assertTrue(config.is_sparse_li_c8_layer("model.layers.2.self_attn.indexer.k_cache"))
+
     @_clean_up_ascend_config
     @patch("vllm_ascend.platform.NPUPlatform.check_and_update_config")
     def test_init_ascend_config_without_additional_config(self, mock_fix_incompatible_config):