DreamerV2

DreamerV2 is a model-based reinforcement learning agent developed by Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi and Jimmy Ba. It was released as an arXiv preprint on 5 October 2020 (arXiv:2010.02193, "Mastering Atari with Discrete World Models") and presented at ICLR 2021.

The agent learns a compact representation of the environment from raw observations using a Recurrent State-Space Model (RSSM) with discrete (categorical) latent representations. An actor-critic policy is trained purely on trajectories rolled out in imagination by the world model, without executing those actions in the real environment.

Results

DreamerV2 is the first agent to achieve human-level performance on the Atari benchmark (55 games, 200M frames) by learning behaviors from a separately trained world model. Within the same compute budget and wall-clock time it surpasses the top single-GPU agents IQN and Rainbow. It also demonstrates continuous control, learning to control a humanoid robot from pixel-only inputs.

Availability

The reference implementation in TensorFlow 2 is publicly available on GitHub (danijar/dreamerv2) under the MIT license and runs on a single GPU. DreamerV2 is part of the Dreamer/PlaNet world-model lineage and was later succeeded by DreamerV3.