perf(fit): the batched LM engine is slower on Apple-MPS than on the CPU

[CSSFrancis]

The batched LM engine now runs on Apple-MPS (#TBD / feat/mps-fitting-ebsd),
and it is correct there — float32 reproduces HyperSpy's multifit to ~1e-5
and MPS tracks CPU-float32 to the last digit. It is just not faster.

The measurement

M1 MacBook Air (4+4 CPU cores, 8-core GPU), torch 2.13, float32 both sides,
after the per-iteration sync removal already landed:

P (positions)	C	CPU	MPS	MPS/CPU
64	1024	171 ms	173 ms	0.99x
1024	1024	1.72 s	2.48 s	0.69x
16384	1024	28.4 s	41.6 s	0.68x

Against a raw float32 GEMM ceiling of 2.0x on the same box (MPS 1747 vs CPU
868 GFLOP/s). So the hardware has roughly 2x to give and the engine is
returning 0.68x — a ~3x gap to close before MPS is worth defaulting to.

For contrast, EBSD dictionary indexing on the same laptop reaches 1.06-1.16x
and rises with problem size. That path is one big E @ D'. The difference is
the workload, not the backend.

Why it is slow (diagnosis, not speculation)

The engine is not GEMM-bound. Its whole design premise is that the free
parameter count n is tiny (<= ~20), which is what makes the normal equations
a batch of (P, n, n) solves. That is a virtue for memory and a problem for
Metal: an LM iteration is a long sequence of small kernels (column norms,
diag_embed, cholesky_ex, cholesky_solve, several wheres, the Adam-ish
lambda update), each paying launch overhead against very little arithmetic.

This is exactly the shape CLAUDE.md's GPU Computing section warns about --
"when a GPU step is slow, the cause is almost always a Python loop launching
small kernels or a blown-up intermediate tensor, not the arithmetic."

Already done, so don't redo it: the two device->host syncs per iteration
(solvable.any() and the converged.all() early exit) are gone -- one .any()
costs 367 us on MPS against 30 us left on-device, and removing them was worth
4.3x on a small fit (1303 ms -> 304 ms). What remains is kernel count, not
synchronisation.

Leads, roughly in order of expected value

1. Fuse the per-iteration tail. The step from J to delta is ~8 small
   ops on (B, n, n) / (B, n) tensors. torch.compile (or a hand-fused
   torch.func composition) could collapse most of it. Worth trying first
   because it is cheap and needs no maths.
2. Solve n x n directly instead of via cholesky. At n <= 20, an
   explicit batched solve -- or even a fixed-size unrolled Cholesky -- may beat
   linalg.cholesky_ex + cholesky_solve, which are general-purpose and
   launch several kernels each.
3. Check whether jacfwd is the cost. _fit_chunk already prefers
   analytic gradients when every component supplies them
   (has_analytic_grad); confirm the MPS runs are taking that branch and are
   not silently falling back to vmap(jacfwd). On CUDA that was measured at
   51.6 ms vs 3.4 ms per residual evaluation.
4. Retune the chunk size for Metal. _JACOBIAN_ELEMENT_CAP (2**26) was
   tuned on CUDA, where a bigger batch was strictly better. A GPU with a much
   smaller working set may want a different point, and the cap interacts with
   how many iterations run concurrently.
5. Reconsider the iteration budget. With convergence now detected properly
   in float32, many positions finish well before max_iter=60; the batch still
   runs until all of them converge. Compacting the batch to drop finished
   positions would cut real work, on every backend.

Definition of done

fit_batched at P>=4096 is at least at parity with the CPU on an M1 Air, with
the numbers added to benchmarks.md -- and ideally measured on a Pro/Max part
too, since that is where the ratio should be most favourable and nobody has run
it there yet.

Parity is non-negotiable throughout: test_fitting_engine.py::TestMPSParity
and TestFloat32Convergence must stay green, since the point of the engine is
that it reproduces multifit.

Context

• Benchmarks + method (including the warmup trap that made MPS look 50x worse
  than it is): benchmarks.md, "Apple-MPS for the 0.3.0 compute paths".
• Current behaviour: both paths prefer the accelerator, overridable with
  SPYDE_FIT_DEVICE / SPYDE_EBSD_DEVICE. If this turns out to be hard, the
  cheap interim fix is to default the fitting engine to CPU under MPS while
  leaving EBSD on the GPU.