Why Self-Driving Trucks Are So Hard! Data, Sensors, AI & Real-World Driving

What does it actually take to build self-driving trucks that can interpret the real world and react faster than humans?In this episode of the An Hour of Innovation podcast, Vit Lyoshin sits down with Achyut Boggaram to explore the engineering behind autonomous trucks and the massive AI infrastructure that powers them.This conversation breaks down how autonomous trucks perceive the world using cameras, lidar, radar, and advanced sensors. Achyut explains how machine learning models process enormous volumes of driving data, how motion planning helps vehicles decide what to do in complex road scenarios, and why real-world driving is far harder than most people imagine. The episode also dives into how AI models are trained using synthetic data, simulation, and large-scale machine learning infrastructure. Along the way, listeners get a behind-the-scenes look at the real challenges of deploying autonomous vehicles on public highways.Achyut Boggaram is an AI and machine learning engineer working on autonomous driving technology at Torc Robotics. His work focuses on building the machine learning infrastructure and data pipelines that power self-driving truck models at scale. After leading ML platform development, he moved into applied research, where he contributes directly to developing frontier AI models for autonomous vehicles. His experience offers a rare inside perspective on how modern robotics engineering and AI systems come together to power real-world autonomous driving software.Takeaways* A single 20-minute autonomous truck test run can generate about 100 terabytes of raw sensor data, showing how data-intensive self-driving systems really are.* Autonomous trucks rely on sensor fusion from cameras, lidar, radar, GPS, and IMU sensors to build a real-time understanding of the road.* Self-driving systems are structured in layers: perception models understand the environment, while planning and behavior models decide what the vehicle should do next.* Machine learning models must generalize from training examples.* When the AI becomes uncertain, the system can execute a minimal-risk maneuver, such as slowing down and pulling off the road.* Training autonomous vehicle models requires diverse real-world data across conditions like night driving, fog, rain, and heavy traffic.* Engineers often use synthetic data and neural rendering to simulate rare scenarios that are difficult or dangerous to capture in real life.* Autonomous driving systems must be designed to resist adversarial attacks, where small visual changes can trick AI into misinterpreting road signs.* AI-powered perception systems can sometimes detect objects hundreds of meters away and even see through fog.* Reinforcement learning and large-scale simulation allow engineers to train driving behaviors without putting humans at risk on real roads.* The biggest barrier to widespread deployment is the “long tail” of rare driving scenarios, where unpredictable real-world situations challenge even the best AI systemsTimestamps00:00 Introduction03:46 Understanding Autonomous Vehicles06:56 Components of Autonomous Vehicle Technology14:41 Generalization in Machine Learning Models19:11 Data Collection and Processing for Training Models24:47 Adversarial Attacks and Model Robustness27:00 Advanced Sensor Technologies for Hazardous Conditions29:41 Reinforcement Learning in Autonomous Vehicles32:45 Challenges in Self-Driving Technology Adoption37:13 Future of Logistics and Autonomous Vehicles40:37 Innovation Q&ASupport the PodcastTo support our work, please check out our sponsors and get discounts: https://www.anhourofinnovation.com/sponsors/For inquiries about sponsoring An Hour of Innovation, email [email protected] Connect with Achyut* Website: https://torc.ai/ * LinkedIn: https://www.linkedin.com/in/achyutsarma/ Connect with Vit* Substuck: https://anhourofinnovation.substack.com/ * LinkedIn: https://www.linkedin.com/in/vit-lyoshin/