NoProp: Learning Neural Networks Without Backpropagation

The provided research introduces NoProp, a novel method for training neural networks that deviates from traditional backpropagation by eliminating both forward and backward passes. Instead, it draws inspiration from diffusion models, where each layer learns independently to denoise a noisy version of the target. This approach trains networks without hierarchical representation learning in the conventional sense, achieving competitive or superior accuracy and efficiency compared to existing backpropagation-free techniques on image classification tasks. The work explores discrete-time and continuous-time variations of NoProp, including connections to flow matching, and examines its performance and memory usage relative to backpropagation and other alternative optimization methods.