PODCAST · education
AI Illuminated
by The AI Illuminators
A new way to keep up with AI research. Delivered to your ears. Illuminated by AI.Part of the GenAI4Good initiative.
Episodes
-
MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos
0:00 Introduction 0:20 Limitations of traditional SfM and SLAM techniques. 0:57 Shortcomings of existing neural network methods. 1:07 MegaSaM's approach: balance of accuracy, speed, and robustness. 1:31 Differentiable bundle adjustment (BA) layer. 2:03 Integration of monocular depth priors and...
-
SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
[00:00] SVD-Quant: 4-bit diffusion model quantization [00:27] Challenge: Outlier sensitivity in 4-bit quantization [00:59] Solution: Smoothing + SVD approach [01:37] Technical: SVD's role in low-rank approximation [02:08] Nunchuku: New inference engine with kernel fusion [02:35] Comparison: INT4…
more -
D3RoMa: Disparity Diffusion-based Depth Sensing for Material-Agnostic Robotic Manipulation
[00:00] Intro [00:18] Current limitations in depth-sensing technology [00:56] D3RoMa's diffusion model approach to depth estimation [01:47] Integration of geometric constraints in the model [02:27] HiSS: New dataset for transparent/specular objects [03:18] Benchmark results showing major accuracy...
-
Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers
[00:00] Intro [00:21] Key problem: Poor generalization in robotic learning [00:51] HPT: New transformer architecture for robotics [00:59] Core components of HPT architecture [01:44] Scale analysis: Data and model size impacts [02:16] Training data: Real robots, simulations, human videos [02:54]...
-
HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
[00:00] Introduction to Hover: Neural Whole Body Controller for Humanoids [00:15] Problem: Current controllers lack versatility across tasks [00:50] Human motion imitation as a unified control approach [01:23] Policy distillation: Learning from an oracle policy [02:01] Command space: Kinematic,...
-
Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
[00:00] Intro [00:24] Tackles RL challenges using a visual backbone, efficient RL, and human feedback. [01:20] Pretrained backbone boosts stability and exploration efficiency. [02:06] RLPD combines offline data and human corrections effectively. [02:57] Human-guided interventions reduce errors,...
-
Local Policies Enable Zero-shot Long Horizon Manipulation
[00:00] Paper intro: Zero-shot robotic manipulation via local policies [00:26] Key challenges: Limited generalization and sim-to-real transfer [01:03] Local policies: Task decomposition through localized focus regions [01:38] Foundation models: VLMs for task understanding [02:07] Training…
more -
MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning
[00:00] Introduction to Mentor system for visual RL [00:29] Problem: Sample inefficiency in robotic learning [00:59] Innovation: Mixture of Experts (MoE) architecture [01:55] Results: MoE achieves 100% success in multi-task testing [02:33] Feature: Task-oriented perturbation for exploration…
more -
SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
[00:00] SkillGen: AI system for robotic learning and automation. [00:19] Core Innovation: Automated dataset generation from minimal human input. [01:12] Skill Segmentation: Smart system for breaking down and adapting complex tasks. [01:59] Hybrid Skill Policy: Framework for controlling robot...
-
LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias
[00:00] Intro to LVSM: Novel transformer for view synthesis [00:14] Problems with existing 3D synthesis methods [00:59] LVSM architecture: encoder-decoder vs decoder-only [01:41] Performance trade-offs between architectures [02:13] Using Pluecker rays for implicit 3D geometry [02:49] Zero-shot...
-
Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling
[00:00] Introduction to 3D Gaussian tracking for robotic manipulation [00:26] Limitations of current video prediction methods [01:11] Advantages of 3D Gaussian representation [02:04] Graph Neural Networks for modeling object dynamics [02:54] Control particle implementation and computation…
more -
SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
[00:00] Introduction [00:20] Core limitations in robot manipulation: challenges with RL and IL [01:08] SPIRE's hybrid approach: combining task planning with learning methods [01:44] TAMP-gated learning: selective application of learned policies [02:20] Training innovations: warm-starting RL and...
-
VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation
[00:00] VILA-U: A unified visual AI model [00:29] Problem: Inefficiency of separate visual modules [01:11] Vision tower: Novel quantization approach [02:09] Training strategy: CLIP-based staged learning [03:03] RVQ technique: Enhanced visual representation [03:47] Multi-modal training:...
-
CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics
[00:00] Intro [00:31] Challenge: Limited multi-humanoid training data [00:55] CooHOI's two-phase learning framework [01:49] Object dynamics as implicit agent communication [02:25] Bounding box strategy for long objects [03:07] Results: Superior performance vs baselines [03:42] Ablation study...
-
SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraints
[00:00] Introduction to SynFlowNet [00:29] Problem: AI-generated molecules often can't be synthesized [01:17] Solution: SynFlowNet - uses real chemical reactions [02:03] GFlowNets: Enables diverse molecule generation [02:47] Scalability: Morgan fingerprints handle 200K+ compounds [03:14]…
more -
L3DG: Latent 3D Gaussian Diffusion
[00:00] Intro to L3DG for 3D modeling [00:32] Solving room-sized 3D scene complexity [01:36] VQ-VAE compresses 3D Gaussian representation [02:41] Generative sparse transpose convolution [03:20] Latent diffusion for scene generation [04:30] Visual improvements over baselines [05:14] Scalability...
-
The Ingredients for Robotic Diffusion Transformers
[00:00] Intro [00:33] Combining transformers & diffusion models [01:12] Key design: Scalable attention blocks (AdaLN) [02:30] Efficient observation tokenization [03:45] DiT Block policy architecture overview [04:20] BiPlay dataset introduction [04:53] Performance improvements over baselines…
more -
Estimating Body and Hand Motion in an Ego-sensed World
[00:00] Introduction to EgoAllo system [00:38] Challenges in egocentric motion estimation [01:20] Importance of spatial/temporal invariance [02:11] Comparison of conditioning parameterizations [02:57] Integration of hand observations [03:50] Global alignment phase [04:28] Guidance losses in...
-
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
[00:00] Intro [00:28] Limitation of existing unified models [00:57] Janus's decoupled visual encoding solution [01:18] Advantages of decoupling [02:03] Janus architecture [02:50] Three-stage training [03:41] Ablation studies [04:23] Extensions for Janus [05:10] Performance gains [05:47] Current...
-
One Step Diffusion via Shortcut Models
[00:00] Introduction [00:23] Computational cost of traditional diffusion models [00:59] Reducing iterations in image generation [01:06] Shortcut models [01:39] Training process and self-consistency property [02:22] Advantages over other methods [03:05] Results on image generation benchmarks…
more -
Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
[00:00] Intro: Research on multi-task learning in LLMs [00:38] Balancing safety and performance in multilingual settings [01:17] Model merging techniques explored [02:16] Model merging outperforms data mixing [02:49] Merging monolingual models improves multilingual capabilities [03:28] Key…
more -
CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos
[00:00] Intro: CoTracker 3 paper [00:17] Challenge: Point tracker training [00:51] Existing methods' limitations [01:05] CoTracker 3 innovations [01:41] Novel semi-supervised training [02:24] Online vs offline versions [03:02] Performance improvements [03:47] Ablation study insights [04:29]...
-
nGPT: Normalized Transformer with Representation Learning on the Hypersphere
[00:00] Introduction [00:30] Consistent unit norm normalization in NGPT [01:08] Mathematical mechanism behind faster convergence [01:52] Elimination of weight decay in NGPT [02:21] Role of learnable eigen learning rates in optimization [03:04] Discussion on training speedup vs. per-step…
more -
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
[00:00] Intro [00:20] Challenges in bimanual manipulation models [01:00] Robotics Diffusion Transformer (RDT) approach [01:34] RDT architecture design [02:22] Data scarcity and unified action space [03:07] Multi-task bimanual dataset [03:49] RDT's experimental results [04:30] Benefits of...
-
Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
[00:00] Continuous-time consistency models (CTMs) [00:20] Limitations of existing CTMs [00:51] TrigFlow: New CTM formulation [01:42] CTM training instability [02:21] Training objective modifications [02:55] Scaling CTMs to 1.5 billion parameters [03:37] Comparison with state-of-the-art models...
We're indexing this podcast's transcripts for the first time — this can take a minute or two. We'll show results as soon as they're ready.
No matches for "" in this podcast's transcripts.
No topics indexed yet for this podcast.
Loading reviews...
Loading similar podcasts...
Frequently Asked Questions
How many episodes does AI Illuminated have?
AI Illuminated currently has 25 episodes available on PodParley. New episodes are automatically indexed when they're published to the podcast feed.
What is AI Illuminated about?
A new way to keep up with AI research. Delivered to your ears. Illuminated by AI.Part of the GenAI4Good initiative.
How often does AI Illuminated release new episodes?
AI Illuminated has 25 episodes. Check the episode list to see recent publication dates and frequency.
Where can I listen to AI Illuminated?
You can listen to AI Illuminated on PodParley by clicking any episode. We provide an embedded audio player for direct listening, and you can also subscribe via your preferred podcast app using the RSS feed.
Who hosts AI Illuminated?
AI Illuminated is created and hosted by The AI Illuminators.
URL copied to clipboard!