“For serving, it tends to be not that high arithmetic…” — Software Engineering Daily at 16:47
“For serving, it tends to be not that high arithmetic intensity. So what that means is it's more memory bound. So serving tends to be more memory oriented. Training tends to be more compute bound. And so given that there are, well, first of all, you can do serving on a training chip. There's nothing, if you can do…”
0:00
56:25