Gradient descent - Wikipedia
8
公开标注数
5
参与人数
2026-07-15 09:16:34
首次 Whisper
本页的公开 Whisper
划选高亮2026-07-25 13:02:36
原文高亮摘录
“serves as the most basic algorithm used for training most deep networks today.”
Whisper 随想笔记
Stochastic part is key though—full batch would be way too slow for modern models.
划选高亮2026-07-25 12:53:36
原文高亮摘录
“serves as the most basic algorithm used for training most deep networks today.”
Whisper 随想笔记
Yeah but it took decades for hardware to make it actually practical for deep nets.
划选高亮2026-07-25 09:59:36
原文高亮摘录
“take repeated steps in the opposite direction of the gradient”
Whisper 随想笔记
Ah the classic 'follow your nose' algorithm, works until it doesn't.
划选高亮2026-07-25 09:50:36
原文高亮摘录
“take repeated steps in the opposite direction of the gradient”
Whisper 随想笔记
But what if the function is flat somewhere, does it just stop there?
划选高亮2026-07-25 09:41:36
原文高亮摘录
“take repeated steps in the opposite direction of the gradient”
Whisper 随想笔记
So basically you just keep walking downhill until you hit a valley, right?
划选高亮2026-07-15 09:34:34
原文高亮摘录
“first-order iterative algorithm for minimizing a differentiable multivariate function”
Whisper 随想笔记
I used this in my ML class, it's basically how you teach a model to learn.
划选高亮2026-07-15 09:25:34
原文高亮摘录
“first-order iterative algorithm for minimizing a differentiable multivariate function”
Whisper 随想笔记
Doesn't that mean it can get stuck in local minima easily?
划选高亮2026-07-15 09:16:34
原文高亮摘录
“first-order iterative algorithm for minimizing a differentiable multivariate function”
Whisper 随想笔记
So it only needs the gradient, not second derivatives—that's why it's called first-order.
分享本页 Whisper
短链接
https://domwhisper.com/s/7921e01a7738嵌入代码
<iframe src="https://domwhisper.com/embed/7921e01a7738" width="100%" height="480" style="border:0;border-radius:16px" loading="lazy"></iframe>