A soft elastic 'skin' (elastomer) deforms on contact with an object. A camera behind it captures this deformation, and multi-color/multi-angle LED lighting lets the system recover local surface slopes (photometric stereo) and reconstruct the 3D shape of the imprint. Motion of markers printed on the membrane allows estimating shear forces and slip. The resulting maps (depth, forces) feed control algorithms or neural networks that learn manipulation policies.
Classic tactile sensors (taxel arrays) have low spatial resolution. Vision-based tactile sensing provides dense, high-resolution contact information critical for dexterous manipulation.
A soft, deformable contact layer, often with a reflective membrane.
Official
Captures the elastomer's deformation from the inside.
Official
Multi-angle/multi-color light enabling photometric stereo.
Official
Turns the image into depth and force maps (classically or via a network).
Official
The soft surface degrades under heavy contact.
The camera + elastomer are thicker than thin taxel arrays.
Image reconstruction adds latency to the control loop.
Edward Adelson's lab demonstrates a high-resolution vision-based tactile sensor.
Meta AI releases a low-cost, compact vision-based tactile sensor.
Vision-based tactile sensors increasingly feed manipulation policies (VLA, robot learning).
