Add fused GatedDeltaNet decode Triton kernel

[Gasoonjia]

Fuse Q/K/V split, L2 normalization, head repeat, gating computation, and delta-rule recurrent state update into a single Triton kernel for decode (T=1). Replaces ~6 small AOTI-generated kernels with one, reducing GatedDeltaNet kernel time by ~62% and improving end-to-end decode throughput by ~2% (106 -> 108.5 tok/s on A100).

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18865

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Rolling out OSDC (ARC) runners on pull workflow for PyTorch trunk commits

❌ 6 New Failures, 102 Cancelled Jobs, 51 Unrelated Failures

As of commit c19d43e with merge base 2eaa16c ():

NEW FAILURES - The following jobs have failed:

• MLX / test-mlx / test-mlx (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• MLX / test-mlx-qwen35-moe / test-mlx-qwen35-moe (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 2
• Test CUDA Builds / test-model-cuda-e2e (google, gemma-3-4b-it, quantized-int4-tile-packed) / linux-job (gh)
  RuntimeError: Command docker exec -t 48b03597d082ac7bcf4bbcb310186b2beeeff06639573429f2dd525d1f32c2a5 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (mistralai, Voxtral-Mini-3B-2507, non-quantized) / linux-job (gh)
  RuntimeError: Command docker exec -t 03ac1f79094f1c4417d164acb81d67087460a663f3133b59d21e32fdd2f529f6 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (nvidia, parakeet-tdt, non-quantized) / linux-job (gh)
  RuntimeError: Command docker exec -t bfc01f554d879e4f583edf2d0d4ad00f4f5e7b1ab2ca69f1dbf3d9e692fdc378 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (nvidia, parakeet-tdt, quantized-int4-tile-packed) / linux-job (gh)
  RuntimeError: Command docker exec -t 7e60a7f41e215069fb736953d3ff73f73beeb83a60c3f267d297178172dccf8f /exec failed with exit code 1

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• trunk / test-arm-backend-zephyr (ethos-u55) / linux-job (gh)
• trunk / test-coreml-delegate / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-cortex-m-e2e (mv2) / linux-job (gh)
• trunk / test-cortex-m-e2e (mv3) / linux-job (gh)
• trunk / test-custom-ops-macos (cmake) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-macos (bert|coreml_fp32_gpu|--quantize) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-macos (distilbert|coreml_fp32_gpu|--quantize) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-macos (llama3.2-1b|coreml_fp32_gpu|--quantize) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-macos (olmo-1b|coreml_fp32_gpu|--quantize) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-macos (qwen3-0.6b|coreml_fp32_gpu|--quantize) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-macos (smollm2-135m|coreml_fp32_gpu|--quantize) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-xnnpack (gemma3-1b|xnnpack|--quantize) / linux-job (gh)
• trunk / test-huggingface-transformers-xnnpack (llama3.2-1b|xnnpack|--quantize) / linux-job (gh)
• trunk / test-huggingface-transformers-xnnpack (qwen3-1.7b|xnnpack|--quantize) / linux-job (gh)
• trunk / test-huggingface-transformers-xnnpack (smollm3-3b|xnnpack|--quantize) / linux-job (gh)
• trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
• trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
• trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh)
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
• trunk / test-llama-torchao-lowbit / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (add_mul, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (add_mul, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (add, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (add, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (emformer_join, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (emformer_join, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (emformer_transcribe, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (emformer_transcribe, xnnpack-quantization-delegation, linux.arm64.2xla... / linux-job (gh)
• trunk / test-models-linux-aarch64 (ic3, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (ic3, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (ic4, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (ic4, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (linear, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (linear, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (llama3_2_vision_encoder, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (lstm, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (mobilebert, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (mobilebert, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (mul, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (mv2, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (mv2, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (mv3, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (mv3, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (qwen2_5_1_5b, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (resnet18, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (resnet18, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (resnet50, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (resnet50, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (softmax, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (vit, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (vit, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (w2l, portable, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-linux-aarch64 (w2l, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh)
• trunk / test-models-macos-coreml (dl3) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (edsr) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (efficient_sam) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (emformer_join) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (emformer_transcribe) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (ic3) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (ic4) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (mobilebert) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (mv2) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (mv3) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (resnet50) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (vit) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-coreml (w2l) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (efficient_sam, portable) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (emformer_join, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (ic4, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (llama, portable) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (llama2, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (llama3_2_vision_encoder, portable) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (mobilebert, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (mv3, portable) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (mv3, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (resnet50, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (vit, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (w2l, xnnpack-quantization-delegation) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-mps / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-windows (emformer_transcribe, portable) / windows-job (gh)
• trunk / test-models-windows (emformer_transcribe, xnnpack-q8) / windows-job (gh)
• trunk / test-models-windows (mobilebert, portable) / windows-job (gh)
• trunk / test-models-windows (mv3, portable) / windows-job (gh)
• trunk / test-models-windows (mv3, xnnpack-q8) / windows-job (gh)
• trunk / test-models-windows (resnet50, portable) / windows-job (gh)
• trunk / test-models-windows (vit, portable) / windows-job (gh)
• trunk / test-multimodal-macos (gemma3-4b) / macos-job (gh)
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
• trunk / test-selective-build-macos (cmake) / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-static-llama-ane / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-torchao-huggingface-checkpoints (lfm2_5_1_2b, linux.arm64.2xlarge, executorch-ubuntu-22.04-g... / linux-job (gh)
• trunk / test-torchao-huggingface-checkpoints (phi_4_mini, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
• trunk / test-torchao-huggingface-checkpoints (qwen3_4b, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc1... / linux-job (gh)
• trunk / unittest-release / linux / linux-job (gh)
• trunk / unittest-release / macos / macos-job (gh)
  ##[error]The operation was canceled.
• Windows MSVC Build / build-windows-msvc / windows-job (gh)
  ##[error]The operation was canceled.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• trunk / test-arm-backend-ethos-u (test_memory_allocation) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-ethos-u (test_pytest_models_ethos_u55) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-ethos-u (test_pytest_models_ethos_u85) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-ethos-u (test_pytest_ops_ethos_u55) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-ethos-u (test_pytest_ops_ethos_u85) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-ethos-u (test_run_ethos_u55) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-ethos-u (test_run_ethos_u85) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-ethos-u (test_smaller_stories_llama) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-vkml (test_pytest_ops_vkml) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-zephyr (cortex-m55) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-backend-zephyr (ethos-u85) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-ootb-linux (run_deit_e2e_ethos_u) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-ootb-linux (run_ootb_tests_ethos_u) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-arm-ootb-linux (run_ootb_tests_tosa) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-xnnpack (qwen3-0.6b|xnnpack|--quantize) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-huggingface-transformers-xnnpack (smollm2-135m|xnnpack|--quantize) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-llama-runner-linux (bf16, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-llama-runner-linux (fp32, portable, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (phi_4_mini, portable, linux.arm64.m7g.4xlarge) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-models-windows (mobilebert, xnnpack-q8) / windows-job (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• trunk / test-models-windows (resnet50, xnnpack-q8) / windows-job (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• trunk / test-models-windows (vit, xnnpack-q8) / windows-job (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, conv_former) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, vit) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, cvt) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, dit) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, efficientnet) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, focalnet) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, mobilevit_v1) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, mobilevit_v2) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, pvt) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, roberta) / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-qnn-optimum-model (fp32, swin) / linux-job (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• trunk / test-torchao-huggingface-checkpoints (lfm2_5_1_2b, linux.2xlarge, executorch-ubuntu-22.04-clang12... / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-torchao-huggingface-checkpoints (phi_4_mini, linux.2xlarge, executorch-ubuntu-22.04-clang12,... / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.
• trunk / test-torchao-huggingface-checkpoints (qwen3_4b, linux.2xlarge, executorch-ubuntu-22.04-clang12, x... / linux-job (gh) (similar failure)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• pull / unittest-editable / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• Test CUDA Builds / unittest-cuda / linux-job (gh) (trunk failure)
  backends/cuda/tests/test_fused_moe.py::TestFusedMoE::test_e2e_cpp_runner
• trunk / unittest-release / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.