Phase 3 — nnz-bucketing SpMM + streaming large-sparse + NEFF cache

[scttfrdmn]

Roadmap phase tracker

This issue tracks trnsparse's work on Phase 3 of the trnsci roadmap.

See the suite-level roadmap
for the full phase matrix and cross-project dependencies. A reader-friendly
version lives at trnsci.dev/roadmap/.

What this phase means for trnsparse:
The Phase 1 SpMM assumes uniform nnz per row. Real sparse matrices
(Fock matrices, scientific PDE discretizations) have skewed
distributions. Phase 3 buckets rows by nnz and dispatches per bucket.

Done means:

• Row-bucketing by nnz: assign rows to fixed-size buckets, tile per bucket, pad only within a bucket.
• Streaming SpMM for matrices that don't fit in SBUF.
• Benchmarks on canonical sparse matrices (e.g. SuiteSparse) at trnsci.dev/trnsparse/benchmarks/.
• NEFF cache reuse verified.

Coordination

• Label: phase-3-perf — matches the same label in every sub-project.
• Cross-project dependencies, if any, are called out in the "Done means"
  list above. Link any PRs / child issues here for tracking.

Close this issue when all "Done means" items are satisfied.

[scttfrdmn]

Reframing this under the architectural-exploitation lens (see memory: feedback_architecture_over_benchmarks.md).

The original scope — row-bucketing CSR + gather-matmul-scatter + streaming — was written under the "port cuSPARSE to NKI" framing. Under the architectural frame that trnsparse is now committed to (see v0.3.0 architecture.md), BSR is the Trainium-native sparse format: every 128×128 block maps to a Tensor-Engine tile with zero gather overhead. CSR fallback is served by the torch.sparse_csr_tensor PyTorch path (v0.1.3) which is already within 2× of scipy.

Reasons to demote CSR row-bucketing from the v0.3.0 headline:

• DMA indirect-gather primitive is not exposed in NKI 2.24 per the sibling survey. So the core of this issue is blocked on Neuron SDK work we don't control.
• Row-bucketing only wins for CSR path. If users have block-structured matrices (Fock, FEM, GNN, block-sparse attention), BSR is strictly better — no bucketing needed, no padding within buckets.
• For truly irregular CSR, a BSR conversion with a tolerance parameter (pad sparse rows into blocks, like scipy's bsr_matrix(tol=...)) is simpler than row-bucketing and gets you onto the fast path.

Keeping this issue open as backlog — revisit if Neuron SDK exposes indirect DMA or if users explicitly ask for arbitrary-sparsity NKI dispatch beyond what BSR covers. v0.3.0 lands BSR (see #18); follow-up ideas are in #19 (fused screen+matmul), #20 (on-chip iterative solvers), #21 (block-sparse attention).