标签：llm
带有 llm 标签的公开 Whisper
划选高亮2026-08-12 13:35:25
原文高亮摘录
“capable of recalling and reasoning over fine-grained information”
Whisper 随想笔记
Recalling fine-grained stuff is cool, but I'd rather see it not hallucinate the details first.
划选高亮2026-08-12 13:26:25
原文高亮摘录
“capable of recalling and reasoning over fine-grained information”
Whisper 随想笔记
10M tokens is wild, but can it actually find that one specific line in a 500-page PDF?
划选高亮2026-08-12 10:32:25
原文高亮摘录
“next generation of highly compute-efficient multimodal models”
Whisper 随想笔记
Still waiting for the open source model that does this without breaking the bank.
划选高亮2026-08-12 10:23:25
原文高亮摘录
“next generation of highly compute-efficient multimodal models”
Whisper 随想笔记
Million token context is wild, finally can throw whole codebases at it.
划选高亮2026-08-11 18:38:31
原文高亮摘录
“QLoRA backprop”
Whisper 随想笔记
My 48GB card finally feels useful for something other than gaming.
划选高亮2026-08-10 14:32:38
原文高亮摘录
“a herd of language models that natively support multilingua”
Whisper 随想笔记
multilinguality is cool but tool usage is where it's really at
划选高亮2026-08-10 14:23:38
原文高亮摘录
“a herd of language models that natively support multilingua”
Whisper 随想笔记
405B params is insane, wonder how much compute that took
划选高亮2026-08-10 14:14:38
原文高亮摘录
“a herd of language models that natively support multilingua”
Whisper 随想笔记
finally a model that actually gets multiple languages without breaking a sweat
划选高亮2026-08-01 03:09:18
原文高亮摘录
“standard interface for models, embeddings, vector stores, and more.”
Whisper 随想笔记
Thats the core value prop right there, a standard interface for everything LLM.
划选高亮2026-07-28 16:29:34
原文高亮摘录
“reduce the training time”
Whisper 随想笔记
Finally a normalization that works for RNNs without all that batch size fuss.
划选高亮2026-07-28 13:17:34
原文高亮摘录
“batch norma”
Whisper 随想笔记
So layer norm is basically batch norm but for single examples, makes sense for RNNs.
划选高亮2026-07-28 10:23:34
原文高亮摘录
“normalize the activities of the neurons”
Whisper 随想笔记
tried it on my lstm and yeah, training felt way smoother
划选高亮2026-07-28 10:19:28
原文高亮摘录
“The main bottleneck for training very large neural network models is the intense demand for a large amount of GPU memory”
Whisper 随想笔记
Yep, my last model OOM'd even after offloading to CPU. Brutal.
划选高亮2026-07-28 10:11:53
原文高亮摘录
“A typical generative task is as follows. At each step, a datapoint is sampled from the dataset, and part of the data is removed, and the model must infer the removed part.”
Whisper 随想笔记
That's how they train those language models? Seems almost too simple to work that well.
划选高亮2026-07-28 10:10:28
原文高亮摘录
“The main bottleneck for training very large neural network models is the intense demand for a large amount of GPU memory”
Whisper 随想笔记
Couldn't afford those GPUs anyway, so I just rent and pray.
划选高亮2026-07-28 10:02:53
原文高亮摘录
“A typical generative task is as follows. At each step, a datapoint is sampled from the dataset, and part of the data is removed, and the model must infer the removed part.”
Whisper 随想笔记
So it's basically like playing a game of peek-a-boo with data, hiding a piece and making the model guess.
划选高亮2026-07-28 10:01:28
原文高亮摘录
“The main bottleneck for training very large neural network models is the intense demand for a large amount of GPU memory”
Whisper 随想笔记
True, but the real pain is the optimizer states, they quietly eat everything.
划选高亮2026-07-27 13:04:16
原文高亮摘录
“excellent few-shot learners with task-specific exemplars”
Whisper 随想笔记
Makes me wonder if all those fancy few-shot prompts were just overkill the whole time.
划选高亮2026-07-27 13:01:28
原文高亮摘录
“Large language models (LLMs), currently their most advanced form as of 2026, are predominantly based on transformers”
Whisper 随想笔记
n-grams were simpler, but you can't argue with scale
划选高亮2026-07-27 12:55:16
原文高亮摘录
“excellent few-shot learners with task-specific exemplars”
Whisper 随想笔记
It's funny how we always assumed they needed examples, but just saying 'think step by step' does the trick.