“what's called dense models. So they don't have the mixture…” — Practical AI at 41:28
“what's called dense models. So they don't have the mixture of experts element. They run all of their parameters all the time and they're distilled models. So what DeepSeek did is they took their flagship, DeepSeek R1, they created their flagship model, and then they use the process of knowledge distillation to create…”
0:00
50:47