Skip to content
#

wgmma

Star

Here is 1 public repository matching this topic...

qflen / nsa-from-scratch

Sponsor
Star 5

From-scratch reimplementation of DeepSeek's Native Sparse Attention (arXiv:2502.11089) in Triton + CUDA Hopper WGMMA. 7.4x faster than FlashAttention-3 at 64k context. Five-model training fleet, perplexity sweep, LongBench v2, MoBA comparison.

cuda pytorch transformer triton hopper attention-mechanism nsa gpu-kernels sparse-attention llm long-context flash-attention deepseek native-sparse-attention wgmma
  • Updated May 12, 2026
  • Python

Improve this page

Add a description, image, and links to the wgmma topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the wgmma topic, visit your repo's landing page and select "manage topics."

Learn more