The Simple Essence of Automatic Differentiation - Conal Elliott
The Simple Essence of Automatic Differentiation - Conal Elliott on HackerTube: recommended by 3 Hacker News commenters across 4 separate months, 2018–2023.
Conal Elliott presents his paper on automatic differentiation, deriving a simple, generalized AD algorithm from a natural specification. Varying the representation of derivatives yields forward and reverse mode algorithms that avoid the complexity and statefulness of backpropagation.
What HN commenters said
6 Hacker News comments link this video. Each one links back to its thread.
infogulchApr 2022Strongly recommends
The most interesting thing I've seen on AD is "The simple essence of automatic differentiation" (2018) [1]. See past discussion [2], and talk [3]. I think the main idea is that by compiling to categories and pairing up a function with its derivative, the pair becomes trivially composable in forward mode, and the whole structure is easily converted to reverse mode afterwards. [1]: [link] [2]: [link] [3]: Talk at Microsoft Research: ▶ this video Other presentations listed here: [link]
in “Differentiable Programming – A Simple Introduction” · View on HN
srikuSep 2022Recommends
I'm yet to give reverse mode a second try, but my attempts at forward mode (linked in parent post) were on my own without the paper's help. There's a talk that Conal gives that makes the paper a bit more accessible - ▶ this video - but only a bit more. I know it'll take effort to wrap my head around the implementation details ... but the paper certainly helped clarify things I was stumbling around about.
in “Automatic Differentiation in 38 lines of Haskell” · View on HN
noblethrasherNov 2019Recommends
He also gave nice presentation on AD: ▶ this video (surprised that you didn't already mention it, considering the venue ;-))
in “Differentiable Programming Manifesto” · View on HN
infogulchNov 2018Recommends
Another paper related to AD was posted last month: The Simple Essence of Automatic Differentiation [0]. Video presentation [1]. I found it pretty awesome, just in case you hadn't seen it. [0]: https//news.ycombinator.com/item?id=18306860 [1]: ▶ this video
in “Automatic Differentiation in Machine Learning: A Survey [pdf]” · View on HN
bmc7505Oct 2018Cites it
This paper won a best paper award at ICFP 2018. Conal Elliott recently gave a longer talk about it at Microsoft Research: ▶ this video
in “The Simple Essence of Automatic Differentiation [pdf]” · View on HN
eli_gottliebAug 2023Joke
AD is just a Cartesian lens of Jacobians and total derivatives in the category of smooth functions, what's the problem? ▶ this video
in “Understanding Automatic Differentiation in 30 lines of Python” · View on HN