“So, multiply accumulate, happens at every single step of a…” — Dwarkesh Podcast at 2:43
“So, multiply accumulate, happens at every single step of a matrix multiply. It makes sense. Then the other observation is that the precision will almost always be higher in the accumulation step than in the multiplication step.”
0:00
1:20:30