“like your pass rate, your prior probability of having said…” — Dwarkesh Podcast at 2:15:02
“like your pass rate, your prior probability of having said blue. And as a function of that, like, in supervised learning through cross entropy loss, you would learn negative log p, p being pass rate bits once you get this label. Whereas in RL, you're just randomly guessing shit and seeing it”
0:00
2:37:29