“And it has way cheaper inference costs as well at long…” — Latent Space: The AI Engineer Podcast at 17:55
“And it has way cheaper inference costs as well at long context. What's the cap? So you have 16K context. Could it go to 32, 64K? Like how much can you expend the context size? We actually have not found a limit. It just comes down to resources. Like each subsequent training costs more than previous. So this costs…”
0:00
32:59