[1412.6980] Adam: A Method for Stochastic Optimization
6
Public whispers
6
Contributors
2026-07-26 10:05:24
First whispered
Public whispers on this page
Text Highlight2026-07-26 16:29:24
Original Highlight Excerpt
"straightforward to implement"
Whisper Note
Finally a paper that doesn't make me wrestle with the code just to try it out.
Text Highlight2026-07-26 13:26:24
Original Highlight Excerpt
"based on adaptive estimates of lower-order moments"
Whisper Note
Adaptive moments sound great but I still tune the lr manually sometimes.
Text Highlight2026-07-26 13:17:24
Original Highlight Excerpt
"based on adaptive estimates of lower-order moments"
Whisper Note
So basically it's just fancy momentum with adaptive learning rates, right?
Text Highlight2026-07-26 10:23:24
Original Highlight Excerpt
"an algorithm for first-order gradient-based optimization of stochastic objective functions"
Whisper Note
Sounds good but I still wonder how it handles non-convex stuff in practice.
Text Highlight2026-07-26 10:14:24
Original Highlight Excerpt
"an algorithm for first-order gradient-based optimization of stochastic objective functions"
Whisper Note
I've tried it on sparse gradients and honestly the difference is night and day.
Text Highlight2026-07-26 10:05:24
Original Highlight Excerpt
"an algorithm for first-order gradient-based optimization of stochastic objective functions"
Whisper Note
Finally an optimizer that just works without babysitting the learning rate.
Share this page's whispers
Short link
https://domwhisper.com/s/e41b8a2c1c77Embed snippet
<iframe src="https://domwhisper.com/embed/e41b8a2c1c77" width="100%" height="480" style="border:0;border-radius:16px" loading="lazy"></iframe>See what people are discussing on arxiv.org
Install DomWhisper to view live whispers as you browse, and join the discussion.
Get the extension