[2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
6
公开标注数
6
参与人数
2026-07-20 10:06:07
首次 Whisper
本页的公开 Whisper
划选高亮2026-07-20 16:30:07
原文高亮摘录
“arXivLabs: experimental projects with community collaborators”
Whisper 随想笔记
Oh cool, so arXiv is basically a playground for devs now, that's kinda neat.
划选高亮2026-07-20 13:27:07
原文高亮摘录
“time and memory complexity of self-attention are quadratic in sequence length”
Whisper 随想笔记
Honestly, the real bottleneck is memory bandwidth, not just compute.
划选高亮2026-07-20 13:18:07
原文高亮摘录
“time and memory complexity of self-attention are quadratic in sequence length”
Whisper 随想笔记
This quadratic blowup is why my Transformer training always dies on long docs.
划选高亮2026-07-20 10:24:07
原文高亮摘录
“Transformers are slow and memory-hungry on long sequences”
Whisper 随想笔记
I wonder if this also helps with memory on consumer GPUs, or just for the big clusters.
划选高亮2026-07-20 10:15:07
原文高亮摘录
“Transformers are slow and memory-hungry on long sequences”
Whisper 随想笔记
Quadratic complexity is brutal, but that IO-aware trick sounds like the real fix.
划选高亮2026-07-20 10:06:07
原文高亮摘录
“Transformers are slow and memory-hungry on long sequences”
Whisper 随想笔记
Yeah no kidding, my GPU starts crying the moment I go past 1k tokens.
分享本页 Whisper
短链接
https://domwhisper.com/s/5727fed78bdd嵌入代码
<iframe src="https://domwhisper.com/embed/5727fed78bdd" width="100%" height="480" style="border:0;border-radius:16px" loading="lazy"></iframe>