DayDreamer

DayDreamer is a world-model reinforcement learning method applied to training physical robots directly in the real world, without simulators. It builds on the Dreamer algorithm (DreamerV2 line): it learns a latent world model (Recurrent State-Space Model) and then trains an actor-critic policy by planning in imagination, substantially reducing the amount of real-world trial and error required.

In the headline experiment, a Unitree A1 quadruped learned to roll off its back, stand up, and walk from scratch in about 1 hour of interaction, without resets, and adapted to perturbations within roughly 10 minutes. The method was also validated on robotic arms (UR5, XArm) for picking and placing objects from camera images, and on a wheeled robot for visual navigation.

The paper "DayDreamer: World Models for Physical Robot Learning" was produced at the University of California, Berkeley (with co-author Danijar Hafner) and presented at CoRL 2022.