[2211.17192] Fast Inference from Transformers via Speculative Decoding
5
公开标注数
5
参与人数
2026-07-23 09:17:19
首次 Whisper
本页的公开 Whisper
划选高亮2026-07-23 12:38:19
原文高亮摘录
“an algorithm to sample from autoregressive models faster”
Whisper 随想笔记
Sounds cool but does it really keep the exact same outputs?
划选高亮2026-07-23 12:29:19
原文高亮摘录
“an algorithm to sample from autoregressive models faster”
Whisper 随想笔记
Saving this for later, my inference costs are killing me.
划选高亮2026-07-23 09:35:19
原文高亮摘录
“decoding K tokens takes K serial runs of the model”
Whisper 随想笔记
My training runs spend more time decoding than actually training sometimes.
划选高亮2026-07-23 09:26:19
原文高亮摘录
“decoding K tokens takes K serial runs of the model”
Whisper 随想笔记
Makes sense, but how do they keep the outputs identical if you're guessing ahead?
划选高亮2026-07-23 09:17:19
原文高亮摘录
“decoding K tokens takes K serial runs of the model”
Whisper 随想笔记
That's the whole bottleneck right there, every token is a serial dependency.
分享本页 Whisper
短链接
https://domwhisper.com/s/951958eafe57嵌入代码
<iframe src="https://domwhisper.com/embed/951958eafe57" width="100%" height="480" style="border:0;border-radius:16px" loading="lazy"></iframe>