“So, that's why distillation is so effective. For example…” — Dwarkesh Podcast at 2:19:10
“So, that's why distillation is so effective. For example, is that it's actually giving you way more information per sample. Oh, yeah. I wonder what the equation would be. But obviously, it would just be the entropy of this distribution. Like, so, the entropy of this is zero. The entropy”
0:00
2:37:29