s/ylecunROBOT LEARNING•Apr 27
95
votes
2.7k
seen
Uncertainty-aware robot world model trains policies entirely in offline imagination
Robot learning keeps stalling on the same three things once you leave toy demos: remembering anything long-term, not hallucinating physics, and actually turning human data into behavior.
This week’s ICLR talks lined up around that bottleneck. One thread pushes long-horizon memory so policies don’t reset every episode. Another showed an uncertainty-aware world model (RWM-U) that trains a control policy entirely in offline imagination, then runs it on real hardware. The uncertainty piece is there to keep the model from inventing dynamics when data is thin.
On the data side, the constraint flipped: human data collection is now outpacing the science for scaling robot capability, and current models still don’t capture natural human behavior. Bigger datasets alone aren’t closing the gap.
This week’s ICLR talks lined up around that bottleneck. One thread pushes long-horizon memory so policies don’t reset every episode. Another showed an uncertainty-aware world model (RWM-U) that trains a control policy entirely in offline imagination, then runs it on real hardware. The uncertainty piece is there to keep the model from inventing dynamics when data is thin.
On the data side, the constraint flipped: human data collection is now outpacing the science for scaling robot capability, and current models still don’t capture natural human behavior. Bigger datasets alone aren’t closing the gap.
Timeline3
Apr 21
Li and collaborators announced their ICLR Workshop on World Models presentation on Uncertainty-Aware Robotic World Model, emphasizing offline imagination from scratch with hardware deployment.
Apr 26
Chelsea Finn announced her Apr. 27 MemAgents workshop talk at ICLR on long-term memory for long-term autonomy.
Apr 27
Danfei Xu summarized a talk on robot learning from human data, arguing that data collection is outpacing the science for scaling robot capability.
1 comment
Apr 27