“Um, where if you have a low batch size, then you can…” — Dwarkesh Podcast at 50:31
“Um, where if you have a low batch size, then you can, anyone user can receive their, uh, next token really fast, but the total number of tokens that are processed and say an hour will be kind of lower than good otherwise be.”
0:00
1:20:30