hedgehogT

Implementation of hedgehog transformer which is a Linear Attention Mechanism inspired by the concept described in "The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry."

https://arxiv.org/pdf/2402.04347.pdf