Learning Representations by Back-Propagating Errors (Rumelhart, Hinton & Williams)

Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Series: Weekly AI Papers · Historical classic Rumelhart, Hinton, and Williams explain how backpropagation trains hidden units to learn useful internal representations. A forward pass computes the network's outputs; a backward pass uses the chain rule to calculate how each weight contributes to the error. The weights are adjusted to reduce squared output error, with momentum helping the learning process converge. Experiments on mirror symmetry and family relationships show hidden units discovering structure that was not supplied as a training target. The paper extends the method to recurrent networks by unfolding their computation across time. This 1986 Nature paper also discusses local minima and distinguishes its procedure from a biologically plausible model of learning. Source: Learning representations by back-propagating errors · David E. Rumelhart, Geoffrey E. Hinton and Ronald J. Williams Explore: AI & Computer Science · AI Papers Related episodes: Machines That Think Like People (Lake et al.) · V-JEPA (LeCun)