Example: Levenberg-Marquadt Algorithm

[max-vassili3v]

Per Fitzgibbon's paper: the optimisations of DualArrays.jl provide a way to speed up the Levenberg-Marquadt algorithm (LM algorithm), particularly useful in fitting small to medium neural networks. This is because the LM algorithm combines the speed of convergence of the Gauss-Newton method with the stability of gradient descent (Yu and Wilamowski, 2010). An example of this on a neural network to showcase this could be good

[dlfivefifty]

Sounds like a great example. I’d suggest doing both a “classical” case (perhaps involving an ODE to get banded jacobians) as well as the NN case