[1412.6980] Adam: A Method for Stochastic Optimization
6
公开标注数
6
参与人数
2026-07-26 10:05:24
首次 Whisper
本页的公开 Whisper
划选高亮2026-07-26 16:29:24
原文高亮摘录
“straightforward to implement”
Whisper 随想笔记
Finally a paper that doesn't make me wrestle with the code just to try it out.
划选高亮2026-07-26 13:26:24
原文高亮摘录
“based on adaptive estimates of lower-order moments”
Whisper 随想笔记
Adaptive moments sound great but I still tune the lr manually sometimes.
划选高亮2026-07-26 13:17:24
原文高亮摘录
“based on adaptive estimates of lower-order moments”
Whisper 随想笔记
So basically it's just fancy momentum with adaptive learning rates, right?
划选高亮2026-07-26 10:23:24
原文高亮摘录
“an algorithm for first-order gradient-based optimization of stochastic objective functions”
Whisper 随想笔记
Sounds good but I still wonder how it handles non-convex stuff in practice.
划选高亮2026-07-26 10:14:24
原文高亮摘录
“an algorithm for first-order gradient-based optimization of stochastic objective functions”
Whisper 随想笔记
I've tried it on sparse gradients and honestly the difference is night and day.
划选高亮2026-07-26 10:05:24
原文高亮摘录
“an algorithm for first-order gradient-based optimization of stochastic objective functions”
Whisper 随想笔记
Finally an optimizer that just works without babysitting the learning rate.
分享本页 Whisper
短链接
https://domwhisper.com/s/e41b8a2c1c77嵌入代码
<iframe src="https://domwhisper.com/embed/e41b8a2c1c77" width="100%" height="480" style="border:0;border-radius:16px" loading="lazy"></iframe>