“because there's also a lot of interesting stuff that…” — Practical AI at 14:52
“because there's also a lot of interesting stuff that happens with the large language models, specifically because it's generating one token at a time, very latency bound. And that means that it's a lot of memory access to load those weights. So if you can quantize those and then get rid of half of them, you're…”
0:00
38:28