ROBOTIS published a comparison of three manipulation policies on the AI Worker robot on October 18. The task — separating two nested paper cups — was solved by the vision-only policy in 0 percent of attempts. The architecture that predicts future tactile state reached 40 percent.
Key takeaways
- Baseline ACT on RGB and joint states alone: 0% success
- Tactile ACT with raw pressure readings passed through an MLP: 10%
- ViTacFormer, which predicts future tactile states: 40%
- Hardware: AI Worker, two 7-DoF arms, two HX5-D20 hands, a ZED camera
- 90 taxels across the pair of hands, a 54-dimensional action space
Why a camera is not enough here
Separating two nested cups needs grip force inside a narrow band. Too little and the top cup will not come out. Too much and it crushes, or drags the bottom cup along.
From the camera's point of view, gripping one cup looks almost identical to gripping two. The fingers also occlude the object as the hand closes, so the head camera feed stops carrying contact information.
Three policies, one task
| Policy | What it receives as input | Success |
|---|---|---|
| Baseline ACT | RGB images + joint states | 0% |
| Tactile ACT | + raw fingertip pressure through an MLP as extra tokens | 10% |
| ViTacFormer | predict future touch, then condition the action decoder | 40% |
On the AI Worker, Baseline ACT could not regulate contact force — it knocked the cups over or failed to separate them at all. Tactile ACT stabilised the grasp, but the robot usually lifted both cups together.
ViTacFormer treats touch differently. This two-stage architecture first predicts future tactile states from visual, proprioceptive and tactile history, then conditions the action decoder on those predicted tokens. The robot peeled off the top cup, stacked it and finished the stack.
Comparison video published by ROBOTIS alongside the technical documentation.
What the hardware measures
Each HX5-D20 hand has five fingertip sensors, and each returns a 3×3 pressure array. A single cell of that array is a taxel.
The learning from demonstration data was gathered by Meta Quest 3 teleoperation. Each trajectory holds camera images, robot state, tactile pressure and the operator's actions. Each taxel's unloaded baseline is subtracted before training and inference.
Why it matters
The conclusion is counterintuitive for anyone about to buy sensors. Sensing without a matching architecture stays largely unexploited, and the cost lands on hardware rather than on results.
Contact-rich tasks with deformable objects are exactly where vision-only policies break down fastest. A camera sees shape, not force.
What's next
- 40 percent is still far from deployment, and the authors flag the remaining headroom themselves
- The code, technical documentation and comparison video are publicly available in the ROBOTIS docs
Sources
- ROS Discourse — Teaching Robots to Feel with AI Worker Tactile Imitation Learning
- ROBOTIS Docs — HX5-D20: ACT with Tactile Sensors
