How Data Scientists Use Knowledge Distillation to Compress Models

What this episode covers

In this episode, Lucas and Luna explore how knowledge distillation allows data scientists to compress large neural networks into smaller, faster models without catastrophic accuracy loss. They break down the teacher-student training paradigm, using real examples from Google's DistilBERT — which shrank BERT by 40% while retaining 97% of its language understanding — and NVIDIA's work compressing vision models for autonomous vehicles. Lucas explains the role of temperature scaling in softening probabilities, and Luna questions when distillation outperforms pruning or quantization. They also discuss practical trade-offs: when a distilled model is good enough for production versus when you need the full ensemble. This episode gives you one concrete technique to reduce inference cost and latency in your own ML pipeline. #KnowledgeDistillation #ModelCompression #TeacherStudent #DistilBERT #NVIDIA #BERT #DeepLearning #InferenceOptimization #MachineLearning #DataScience #Technology #FexingoBusiness #BusinessPodcast #NeuralNetworks #EdgeAI #Pruning #Quantization #TemperatureScaling Keep every episode free: buymeacoffee.com/fexingo

PodParley-generated summary based on available episode metadata and transcript content.