Equivalence of Context and Parameter Updates in Modern Transformer Blocks

This research explores how modern Large Language Models adapt to new information during inference by framing in-context learning as a series of implicit weight updates. The authors demonstrate that the influence of a prompt can be mathematically mapped to specific, rank-1 patches on a model's existing parameters, effectively "reprogramming" the network without formal retraining. By establishing a framework of input and output controllability, the study proves this phenomenon applies to complex architectures like Gemma, Llama, and Mixture of Experts. Their experiments on Gemma 3 validate that a model with modified weights and no context produces the same outputs as the original model with a prompt. This work provides a mechanistic foundation for understanding how static pre-trained transformers dynamiclly transmute contextual cues into effective internal parameters.