Dreamer 4

Dreamer 4 is a scalable model-based reinforcement learning agent developed by Danijar Hafner, Wilson Yan and Timothy Lillicrap (Google DeepMind). It was released as an arXiv preprint (2509.24527) on 29 September 2025 and is the next generation of the Dreamer line after DreamerV3.

The agent learns behaviors by reinforcement learning inside of a world model — in imagination — without interacting with the real environment. The world model uses a causal video tokenizer (a masked autoencoder that compresses frames into continuous representations) and an efficient transformer architecture with a new “shortcut forcing” objective, enabling real-time interactive inference on a single GPU.

Results

In Minecraft, the world model accurately predicts object interactions and game mechanics, outperforming previous world models by a large margin. The authors pose the challenge of obtaining diamonds in Minecraft from only offline data — a task that requires choosing sequences of over 20,000 mouse and keyboard actions from raw pixels. By learning behaviors in imagination, Dreamer 4 is the first agent to obtain diamonds in Minecraft purely from offline data, without environment interaction.

In the offline setting Dreamer 4 significantly outperforms OpenAI’s VPT offline agent while using 100 times less data, as well as behavioral cloning approaches based on vision-language models. The world model learns action conditioning from only a small amount of labeled data, extracting most of its knowledge from diverse unlabeled videos.

The work is motivated by practical applications such as robotics, where learning from environment interaction can be unsafe and slow. The model has 2B parameters (400M tokenizer, 1.6B dynamics model) and was trained on 256–1024 TPU-v5p chips.