“these models often are not, you know, at the scale where…” — Dwarkesh Podcast at 1:57:31
“these models often are not, you know, at the scale where every flop counts and they need to like kind of squeeze out the performance of every single flop as the dominating decision deciding factor in pre-training, right? Instead their focus is more like we want a certain capability to show up.”
0:00
2:37:29