Efficient Processing of Deep Neural Networks: A Comprehensive Survey

This tutorial and survey explores the development and efficient processing of Deep Neural Networks (DNNs), which currently underpin modern artificial intelligence. While these brain-inspired models achieve human-level accuracy in tasks like image recognition and robotics, their superior performance requires immense computational complexity and energy. The authors provide a comprehensive history of the field, tracing the evolution from early models like LeNet to modern breakthroughs like AlexNet and ResNet. The text explains the fundamental mechanics of convolutions, training through backpropagation, and inference performed on hardware. By analyzing the trade-offs between throughput, power consumption, and hardware costs, the sources offer a framework for evaluating specialized accelerators. Ultimately, the survey highlights the necessity of joint hardware and algorithm co-design to enable sophisticated AI on resource-constrained embedded devices.