DreamerV3

DreamerV3 is a general-purpose model-based reinforcement learning algorithm developed by Danijar Hafner, Jurgis Pasukonis, Jimmy Ba and Timothy Lillicrap. It was first released as an arXiv preprint on 10 January 2023 (arXiv:2301.04104) and published in Nature in 2025.

The agent learns a representation of the environment from raw observations (such as images) using a Recurrent State-Space Model (RSSM) with discrete latent representations. An actor-critic policy is trained on trajectories rolled out in imagination by the world model, without executing those actions in the real environment.

Results

With a single fixed set of hyperparameters, DreamerV3 outperforms specialised methods on more than 150 tasks across many domains (DMLab, Atari 100k/200M, Crafter, ProcGen, Minecraft, BSuite, continuous control benchmarks). It is the first algorithm to collect diamonds in Minecraft from scratch from pixels and sparse rewards, without human data or curricula.

Scaling

The paper demonstrates favourable scaling behaviour: larger models (from roughly 12M to 400M parameters) consistently improve both final performance and sample efficiency. Increasing the number of gradient steps further improves data efficiency.