The Simple Essence of Automatic Differentiation - Conal Elliott

AD is just a Cartesian lens of Jacobians and total derivatives in the category of smooth functions, what's the problem? ▶ this video

I'm yet to give reverse mode a second try, but my attempts at forward mode (linked in parent post) were on my own without the paper's help. There's a talk that Conal gives that makes the paper a bit more accessible - ▶ this video - but only a bit more. I know it'll take effort to wrap my head around the implementation details ... but the paper certainly helped clarify things I was stumbling around about.

The most interesting thing I've seen on AD is "The simple essence of automatic differentiation" (2018) [1]. See past discussion [2], and talk [3]. I think the main idea is that by compiling to categories and pairing up a function with its derivative, the pair becomes trivially composable in forward mode, and the whole structure is easily converted to reverse mode afterwards. [1]: link [2]: link [3]: Talk at Microsoft Research: ▶ this video Other presentations listed here: link

He also gave nice presentation on AD: ▶ this video (surprised that you didn't already mention it, considering the venue ;-))

Another paper related to AD was posted last month: The Simple Essence of Automatic Differentiation [0]. Video presentation [1]. I found it pretty awesome, just in case you hadn't seen it. [0]: https//news.ycombinator.com/item?id=18306860 [1]: ▶ this video

This paper won a best paper award at ICFP 2018. Conal Elliott recently gave a longer talk about it at Microsoft Research: ▶ this video