Roboflow: A Multimodal AI Pipeline for NBA Player ID
from Intellectually Curious Hosted by Mike Breault
Join us as we unpack a dense, multimodal AI stack designed to detect, track, and identify players in chaotic basketball footage. We explore RFDETR-based detection, SAM2 with a temporal memory bank for re-identification after occlusions, and team clustering via SigLep, UMA, and K-means. Then we dive into jersey-number extraction—comparing a fine-tuned VLM2 approach with a specialized ResNet—plus an iOS-overlap verification and frame-stability heuristic to lock IDs across frames. We also discuss current speed bottlenecks (SAM2 as the bottleneck, around 1–2 FPS on a T4) and what it would take to reach real-time 60 FPS for on-court analytics and decision-making. Source: https://blog.roboflow.com/identify-basketball-players/ Note: This podcast was AI-generated, and sometimes AI can make mistakes. Please double-check any critical information. Sponsored by Embersilk LLC
Embed this episode
What this episode covers
Join us as we unpack a dense, multimodal AI stack designed to detect, track, and identify players in chaotic basketball footage. We explore RFDETR-based detection, SAM2 with a temporal memory bank for re-identification after occlusions, and team clustering via SigLep, UMA, and K-means. Then we dive into jersey-number extraction—comparing a fine-tuned VLM2 approach with a specialized ResNet—plus an iOS-overlap verification and frame-stability heuristic to lock IDs across frames. We also discus...
No transcript yet.