“can be scaled up or down based on the difficulty of the…” — Latent Space: The AI Engineer Podcast at 4:37
“can be scaled up or down based on the difficulty of the problem and the compute budget. So in this regard, our architecture has a state, like a memory, that is very similar to how we oftentimes think about parameters in other models. Just like for a really big training run, you want to use a ton of parameters.…”
0:00
32:59