Using logfloat with ad

[adscib]

I was wondering whether it would be possible to use logfloat with the ad library. In machine learning we commonly need to compute a gradient of the log-likelihood. I don't imagine this is currently feasible since I can't see any way to differentiate through logarithms in logFloat etc.

I can think of two potential solutions - either to abstract all the logfloat functions using suitable typeclasses and then write instances of those for ad types or to abstract the LogFloat over the underlying numeric type. I don't know how feasible is the former, but I suspect the latter could be done in principle although in practice may be problematic since logfloat performs some low-level operations specific to Double.

In any case, do you think there's some way to make the two libraries work together? And if so how much effort would it take?

[wrengr]

I think using the ad library will require making things polymorphic rather than using Double as the underlying representation. That's not fundamentally a showstopper, but it would require a bunch of work. My particular concern would be not introducing any performance regressions along the way. For the time being, you may want to see if log-domain works for your needs.

[adscib]

Cool, thanks for the pointer.