ACT
How it works
1) Action chunking: instead of one action, the policy predicts a block of k consecutive actions (k=90 in the original paper, controlling joint positions at 50 Hz), reducing the task's effective horizon by about a factor of k. 2) CVAE: during training a CVAE encoder compresses the action sequence and joint observation into a style variable z; the decoder (the actual policy) reconstructs the action sequence conditioned on multi-camera images, joint positions, and z. The encoder is discarded at test time (z set to the prior mean). 3) Architecture: ResNet image encoders provide visual features that a transformer encoder and decoder process to predict target joint positions. 4) Loss: reconstruction is trained with an L1 loss (instead of the more common L2) for more precise sequence modeling, plus a KL term for the CVAE. 5) Temporal ensembling: at run time the policy is queried more frequently than the chunk length, and overlapping action predictions are averaged (exponentially decaying weights), smoothing motion and improving stability.
Problem solved
It addresses the compounding-error problem in imitation learning: when a policy predicts actions one at a time, small errors drift the robot off the training data distribution and subsequent errors accumulate over a long horizon. It also tackles the non-stationarity of human demonstrations (the same observation may be labeled with different actions). By predicting actions in chunks and using a generative model, ACT shortens the effective horizon and smooths execution without expensive, high-precision hardware.
Components
Predicting a block of k consecutive actions instead of a single action; shortens the effective horizon and limits compounding error.
Generative model: the encoder compresses the action sequence and observation into a style variable z, the decoder (policy) reconstructs actions. The encoder is discarded at test time.
Integrates features from multiple cameras and joint positions and decodes a sequence of target joint positions.
Extract visual features from multi-viewpoint images, fed into the transformer.
Trains action-sequence reconstruction with an L1 loss instead of L2 — the authors observed more precise action modeling.
At run time the policy is queried more often than the chunk length and overlapping predictions are averaged with exponentially decaying weights, smoothing motion.
Implementation
Too small a chunk fails to reduce compounding error; too large makes the policy less reactive and harder to learn.
Aggressive averaging of overlapping chunks smooths motion but can delay reaction to environment changes.
ACT learns only from demonstrations; non-stationary or inconsistent teleoperation data degrade the policy.
Evolution
Zhao, Kumar, Levine, and Finn publish ACT in the low-cost ALOHA paper (RSS 2023), reaching 80-90% success on fine bimanual tasks from ~10 min of demonstrations.
ACT becomes one of the reference imitation-learning policies in the open-source LeRobot library, alongside Diffusion Policy, popularizing it across the robotics community.
Hyperparameters (configurable axes)
Number of consecutive actions predicted at once. k=90 in the original paper. Larger k shortens the horizon more but is harder to learn and less reactive.
Whether and how aggressively to average overlapping chunks at run time (exponential weighting), trading off smoothness against reactivity.
Dimension of the CVAE latent style variable modeling demonstration multimodality.
Rate at which actions are executed (e.g., 50 Hz for joint positions in the original paper).
Weight of the KL regularization in the CVAE; controls how strongly z is constrained toward the prior.
Execution paradigm
Dense transformer encoder-decoder; all paths active on every forward pass. No routing mechanism.
Hardware requirements
Training and inference of a transformer with ResNet encoders run most efficiently on GPUs with tensor cores.
The compact ACT policy can also run on more modest onboard accelerators for real-time robot control.