A typical sim-to-real pipeline: (1) build a physics and rendering simulator mirroring the robot and task; (2) train the policy in simulation, usually with reinforcement learning across many parallel instances; (3) close the reality gap with one or more methods: domain randomization (randomizing textures, masses, friction, latencies), domain adaptation (tuning representations toward real data), system identification (calibrating simulator parameters to measurements from the real robot); (4) deploy and evaluate on the physical robot; (5) optionally fine-tune on a small amount of real data. The automatic domain randomization (ADR) variant gradually increases difficulty as the policy improves.
Training robots directly on physical hardware is slow, costly and risky (wear, failures, safety). Simulation solves this by providing unlimited data, but policies learned in simulation usually fail on a real robot because of the reality gap. Sim-to-Real closes that gap, letting skills transfer from simulation to hardware.
Randomly varying simulation parameters (textures, lighting, masses, friction, latencies) so that reality appears as just another training variation.
Tuning the representation or policy so that the feature distribution from simulation approaches that of real-world data.
Calibrating simulator parameters (dynamics, friction, latencies) to measurements from the real robot so the simulation better matches hardware.
A physics and rendering engine (often massively parallel on GPU) generating training data that mirrors the robot and task.
Unmodeled physics, latencies and sensor noise cause a policy that excels in simulation to fail on hardware.
Too-wide randomization yields overly conservative, suboptimal policies in the real world.
Sadeghi and Levine demonstrate collision-free flight learned purely from randomly rendered simulation images, without a single real training image.
Tobin et al. formalise domain randomization, enabling networks trained purely on synthetic data to transfer to real hardware.
OpenAI trains an in-hand manipulation policy in simulation with domain randomization and transfers it to a physical Shadow robotic hand.
OpenAI solves the Rubik's cube one-handed using automatic domain randomization (ADR) of increasing difficulty.
The survey by Zhao, Peña Queralta and Westerlund organises the main approaches: domain randomization and adaptation, imitation learning, meta-learning and knowledge distillation.
NVIDIA Isaac Gym and locomotion work (e.g. the ANYmal quadruped) demonstrate training thousands of parallel environments on GPU and transferring walking policies to real robots in minutes.
