PODCAST · technology
RoboPapers
by Chris Paxton and Michael Cho
Chris Paxton & Michael Cho geek out over robotic papers with paper authors. robopapers.substack.com
Episodes
-
Ep#111: SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning
Reinforcement learning is often seen as key to achieving scalable, highly reliable real-world robot performance, yet it requires good reward functions that can supervise the wide range of tasks that need to be performed. The obvious thing to try would be a large vision-language model (VLM), but...
-
Ep#110: ENPIRE: Agentic Robot PolicySelf-Improvement in the Real World
In many fields, automated research has become increasingly plausible with the improving capabilities of frontier language models, and the field robotics may be no exception. In ENPIRE, Wenli Xiao, Jia Xie, and Tonghe Zhang build a harness framework that captures physical feedback and allows for...
-
Ep#109: Enigma - Connecting 100 Robots To The Internet
What happens if you connect a hundred AI-powered robots to the internet and just let people give them any instructions they want? Enigma founder Jonathan Jacobi joins us to talk about his experience deploying a fleet of robots on the internet, keeping them operational, and watching people truly...
-
Ep#108: VLS: Steering Pretrained Robot Policies via Vision–Language Models
Pretrained vision-language-action models for robots often fail when a task is performed in a slightly different environment. With Vision-Language Steering, you can get better generalization performance out of a frozen robot policy, by steering the sampling process of your pretrained diffusion or...
-
Ep#107: Do As I Do: Dexterous Manipulation Data from Everyday Human Videos
Achieving human-like dexterity is the next frontier for robotics, and yet dexterity data is often subtly hard to scale. Real-world dexterity data, including things like finger-pose estimates, is often slightly off, making it physically invalid and hard to execute on real hardware and hard to…
more -
Ep#106: Flex-π: A Multi-Stream World-Action Model with Compute Flexibility
World Action Models are becoming more popular in robotics, as they learn to predict the world jointly with learning how to act on it. However, these world predictions are usually purely based on reconstructing color images from video. This is limiting, because color is far from the most important...
-
Ep#105: Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning
Imitation learning, especially with interventions, has driven so much recent robotics progress. However, improving a policy via targeted interventions until it reaches a useful and deployable success rate is a time and labor intensive process. Instead, wouldn’t it be great if policies could…
more -
Ep#104: Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation
One of the key advantages of legged robots like humanoids should be how effectively they can move across a wide variety of terrain types to accomplish their task. But Light-Loco-Parkour from the team at Light Origins aims to change that: using only onboard sensing, they show a policy which can...
-
Ep#103: Freeform Preference Learning for Robotic Manipulation
Specifying reward functions for robots is one of the hardest things about reinforcement learning. Robot rewards often need to be very detailed; metrics like progress can be ill-defined and hard to estimate. This leads to most robot learning defaulting to sparse rewards or simple preference...
-
Ep#102: EgoSuite-Open100K from Lightwheel
Robotics has a data problem, and egocentric data is a compelling way to solve it because egocentric data is (1) scalable — it’s cheap to collect — and (2) easily captures the breadth and diversity of real human tasks, while (3) being true to real physics. Lightwheel recently released a hundred...
-
Ep#101: Coupled Local and Global World Models for Efficient First Order RL
Reinforcement learning has unlocked tremendous gains in broader AI and humanoid robot whole body control, but still has yet to deliver strong results for robot manipulation due to the difficulty of capturing tasks in an accurate simulation; one way to fix this is to create complex environments…
more -
Ep#100: 100 Episodes of Embodied AI
Embodied AI is exploding, and it’s been amazing to learn about it with you all. We have now recorded over a hundred discussions with the best researchers in the world on embodied artificial intelligence and robotics, covering topics like scaling robot learning, humanoid whole body control…
more -
Ep#99: DYNA-2: A 1 Million Hour Scaling Law for World-Action Models
Excitement about world-action models and robot learning has never been higher — they promise a way to use human egocentric data to train massive robotics models which can provide the “GPT” moment for robotics and unlock general-purpose embodied intelligence. And yet there’s been little concrete...
-
Ep#97: Cross-Embodiment Robot Manipulation via a Unified Hand Action Space
Representing actions for robot hands is actually very challenging. Robot hands have different sizes and shapes, and different degrees of freedom. Representing their actions purely as joint states — as we naively would — leads to less transfer across the wide variety of available hands. Instead,...
-
Ep#96: EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration
How can roboticists scale human manipulation data in the wild? EgoHumanoid builds a comprehensive pipeline which unlocks mobile manipulation training for robots — meaning that you can walk around and collect data for your robot anywhere you go. There are two key components to their recipe: view...
-
Ep#95: Action-to-Action Flow Matching
Diffusion Policy was one of the big breakthroughs that has enabled an explosion in real-world robot learning. However, it’s always had a weakness, which is that it works by computing a final action trajectory from random noise, which leads to high latency when predicting a final action sequence....
-
Ep#94: Learning a Thousand Tasks in a Day
For general-purpose robots to be useful, we must be able to teach them new tasks quickly and repurpose them to new roles. But most techniques for teaching robots — even from human demonstrations — take a great deal of new data. But it turns out that it is possible, in part through decomposing…
more -
Ep#93: CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy
Many tasks cannot be completed by one robot alone. But coordinating multiple robots performing complex manipulation tasks is very difficult. Many solutions rely on complicated centralized control, which tends towards bespoke methods that do not scale well with team size. Enter CHORUS by Ria…
more -
Ep#92: Human Universal Grasping
How can we learn robot grasping from egocentric human video alone? General-purpose dexterous manipulation learning will require a lot of data, and yet robot data is hard to find at scale. Better leveraging human data, then, will be key to general-purpose dexterous manipulation. Kevin Wu, Irmak...
-
Ep#91: RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
Memory is one of the most important problems in robotics. Long horizon memory is key for a variety of robot manipulation problems. However, there exist no good benchmarks for understanding progress in how well generalist robot policies can understand language. Yinpei Dai and Yuejiang Liu made...
-
Ep#90: From Capable Controllers to Deployable Humanoid Systems
We want humanoid robots to be able to perform complex, long-horizon tasks in the real world — putting away the groceries or cleaning a room, for example. This requires diverse loco-manipulation skills, which can be easily parameterized to handle object affordances and interact with the world…
more -
Ep#89: Contact Grounded Policy
Contact-rich manipulation is still very challenging for robotics. Problems like opening a jar, or in-hand reorientation of an object, require making repeated contact with different parts of a robot’s hand, and this is hard to do with pure vision. Instead, research is moving towards using tactile...
-
Ep#88: DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation
Human skin plays an important role in how we interact with the world and robustly manipulate objects. It’s not just important when we can’t see things with out eyes, but when we want to pick up something heavy, or apply a very specific amount of force. So, it makes sense to want to give robots...
-
Ep#87: MolmoAct 2: An open foundation for robots that work in the real world
There are few truly open models in the world, including both weights and data. However, these models are crucial for research and development of new systems — they help us learn which data is important and help develop new capabilities for deploying robots in the real world. MolmoAct2 provides a...
-
Ep#86: RISE: Self-Improving Robot Policy with Compositional World Model
Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and...
-
Ep#85: Tutor Intelligence
Collecting robot data at scale is key to deploying working manipulation policies, and the team from Tutor Intelligence is here to tell us about how to accomplish it. Their new announcement: a massive, 100-robot “data factory,” with a behind-the-scenes look at how to build a teleoperation platform...
-
Ep#84: Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons
Learning robust, general-purpose reward functions for robotics unlocks many potential applications, like on-robot reinforcement learning or dataset validation. However, there’s a question of how to actually train such reward functions. Training success/failure prediction leads to ambiguous…
more -
Ep#83: PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation
Spatial understanding is important to moving around in complex environments and is a huge part of the challenge of generalizing to new scenes. Most world models, however, largely ignore this spatial dimension, focusing on 2D images. Not PointWorld, though. PointWorld is a 3D world model trained...
-
Ep#82: SimTooReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation
Humans use tools to perform almost all of the physical work that we do from day to day. However, tools come in many different sizes and shapes, and it’s very difficult to collect human data for them in general. What about training in simulation? SimTooReal aims to address this through,...
-
Ep#81: mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs
Robotics fundamentally involves understanding the dynamics of how things change in the world in response to action and force. This is impossible to learn from static images; instead, it’s far more effective and more data-efficient to learn from video. Elvis Nava joins us to talk about mimic-video...
-
Ep#80: LATENT: Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data
Sports like tennis are great examples of the sort of dynamic whole-body interaction that’s possible with humanoid robots. But capturing examples of fast, dynamic interactions from humans is really difficult. Enter LATENT, which uses lower-quality human data plus reinforcement learning to teach a...
-
Ep#79: Rhoda AI - Causal Video Models Are Data-Efficient Robot Policy Learners
Training robot foundation models faces two key hurdles: how to get enough data to train an effective model, and how to make sure that new skills can be acquired quickly. The team at Rhoda AI believes that the answer is training Direct Video Action models from web data. Web data is plentiful, to…
more -
Ep#78: Three Eras of Robot Learning
Robotics has changed dramatically over the last eight years. Ted has been involved in the cutting edge of robot learning through this period, spending those eight years at Google Brain/Google Deepmind. And he’s identified three eras of robot learning. These eras are: * The Era of Existence Proofs…
more -
Ep#77: DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
World models have many different uses, from evaluation to training data generation to robot planning. DreamDojo is a new foundation world model that allows for impressively general and long-horizon interaction, generating coherent videos for interaction sequences over a minute long. It works in a...
-
Ep#76: OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control
We’ve seen lots of incredible videos of humanoid robots dancing, doing martial arts, running up walls — but these extreme behaviors are usually from individual, highly specialized policies. But now OmniXtreme shows us how to achieve incredible behaviors that push the limits of humanoid motion, by...
-
Ep#75: TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics
Reinforcement on robots is highly limited by our ability to design good reward functions; this means that designing strong, generalizable reward functions is a key enabler to progress on real-world reinforcement learning. But we already have a very general class of models: VLMs. Wouldn’t it be...
-
Ep#74: Weave Robotics
Do you want to never fold clothes again? Weave is a robotics startup founded in early 2024, aiming to build useful home robots as a product. We talked with co-founder Kaan Doğrusöz, and learned about his journey building a home robotics startup. We covered building products out of end-to-end...
-
Ep#73: VideoManip: Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstruction
Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve…
more -
Ep#72: SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
How can we build a general-purpose “foundation model” for robot motion? Zhengyi Luo joitns us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a...
-
Ep#71: Build Your Own Robot
Robots, unfortunately, tend to be expensive. And finding a robot that’s both capable of performing a wide variety of mobile manipulation tasks, and is affordable and “hackable”, is extremely difficult. Many different problems need to be addressed, from arm control to navigation to integrating…
more -
Ep#70: A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulation
Co-training has become a key part of the recipe for training large robotics models; it means that you mix some proportion of real robot data with other data sources, like simulation or egocentric human video data. This is especially important because robotics data tends to lack diversity which…
more -
Ep#69: MolmoSpaces, an Open Ecosystem for Embodied AI
Benchmarking, evaluating, and developing robotics code is difficult, and part of this is because no simulator really reflects the diversity and scale of real embodiments. Enter MolmoSpaces from AI2: a massive open ecosystem with a range of 230,000 handcrafted and procedurally-generated home...
-
Ep#68: DreamZero: World Action Models are Zero-Shot Policies
Achieving generalizable manipulation is the north star for robotics learning, and while we’ve in the past seen incredible results on specific tasks using fine-tuned VLAs, this north star has remained elusive. Perhaps what is needed is a different approach. DreamZero proposes World Action models...
-
Ep#67: Asimov - Open Source Humanoid
Robotics research is moving fast, and being able to modify and improve upon hardware is crucial to maintaining velocity. That’s why Menlo Research has started working on their own open-source humanoid project, Asimov. And they are moving fast. It’s been roughly six months since they started the...
-
Ep#66: Ordered Action Tokenization
How should we represent robot actions for autoregressive transformers? Most robot policies use diffusion or flow to generate continuous action sequences, but this isn’t how large language models work; they predict output tokens, which has many advantages. But coming up with a set of useful action...
-
Ep#65: VLM4VLA: Revisiting Vision-Language Models in Vision-Language-Action Models
Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance?...
-
Ep#64: Project Instinct
Human motion is instinctual. We know how to interact with the world around us, almost without thinking about it at all. Ziwen and Shaoting joined us on RoboPapers to talk about their ambitious Project Instinct: which provides the tools, algorithms, and environments necessary to build humanoid...
-
Ep#63: NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos
The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an...
-
Ep#62: PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies
Evaluating robot policies is hard. Ideally, instead of testing every new policy on a real robot, you could test in simulation; but simulations rarely correlate well with real-world performance. In order to make good, useful simulations, you need to spend a great deal of time and effort. That’s...
-
Ep#61: 1x World Model
Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot manipulation of a wide range of tasks — which is a real challenge for robotics, since so many cutting-edge approaches require expert fine tuning on a small set of in-domain data. Humanoid...
We're indexing this podcast's transcripts for the first time — this can take a minute or two. We'll show results as soon as they're ready.
No matches for "" in this podcast's transcripts.
No topics indexed yet for this podcast.
Loading reviews...
ABOUT THIS SHOW
Chris Paxton & Michael Cho geek out over robotic papers with paper authors.
robopapers.substack.com
HOSTED BY
Chris Paxton and Michael Cho
CATEGORIES
Loading similar podcasts...
Frequently Asked Questions
How many episodes does RoboPapers have?
RoboPapers currently has 110 episodes available on PodParley. New episodes are automatically indexed when they're published to the podcast feed.
What is RoboPapers about?
Chris Paxton & Michael Cho geek out over robotic papers with paper authors. robopapers.substack.com
How often does RoboPapers release new episodes?
RoboPapers publishes regularly with 110 episodes available. Check the episode list for the typical release schedule.
Where can I listen to RoboPapers?
You can listen to RoboPapers on PodParley by clicking any episode. We provide an embedded audio player for direct listening, and you can also subscribe via your preferred podcast app using the RSS feed.
Who hosts RoboPapers?
RoboPapers is created and hosted by Chris Paxton and Michael Cho.
URL copied to clipboard!