The scaling paradox: how larger models develop unexpected emergent abilities

Large neural networks display emergent abilities — capabilities such as reasoning, code generation, abstract thinking, and in-context learning that are absent or near-random in smaller models but appear suddenly and reliably once scale in parameters, data, and compute crosses certain thresholds. The term "emergent abilities" gained traction through research by Jason Wei et al.