A soft elastic 'skin' (elastomer) deforms on contact with an object. A camera behind it captures this deformation from the inside, and multi-color/multi-angle LED lighting lets the system recover local surface slopes via photometric stereo and reconstruct the 3D shape of the imprint. Markers printed on the membrane, tracked between frames, enable estimation of shear forces and slip detection. The resulting maps (depth, forces, marker flow) are then fed either to classical control algorithms or to neural networks that learn manipulation policies — increasingly as an extra modality alongside vision in Vision-Language-Action models.
Classic tactile sensors (taxel arrays) have low spatial resolution and poorly capture texture, micro-slip or precise force distribution. Visuotactile sensing solves this by turning contact into a high-resolution image — providing the dense information about shape and forces required for dexterous manipulation and reliable grasping.
A soft, deformable contact layer, usually a transparent gel with a reflective membrane.
Official
A miniature camera capturing the elastomer's deformation from the inside.
Official
Multi-color/multi-angle light enabling photometric stereo.
Official
Printed dots whose motion allows estimating shear forces and slip.
Official
Turns the image into depth and force maps — classically (photometrically) or via a neural network.
Official
The soft surface degrades under heavy contact.
The camera+gel module is thicker than thin taxel arrays.
Image reconstruction adds latency to the control loop.
Johnson and Adelson (MIT) demonstrate a high-resolution camera- and elastomer-based tactile sensor — the birth of visuotactile sensing.
A biomimetic optical tactile sensor tracking the motion of internal papillae/markers — an alternative visuotactile sensor family.
Meta AI releases a low-cost, compact vision-based tactile sensor for in-hand manipulation, popularizing the technology in robot learning.
Visuotactile sensors increasingly feed manipulation policies and VLA models; multimodal fingertips (e.g. DIGIT 360) emerge, combining touch with other modalities.
How the deformation image is converted into geometry and forces.
Presence of membrane markers determines shear-force and slip measurement.
Trade-off between sensitivity (soft gel) and durability (harder gel).
Neural reconstruction of the deformation image benefits from GPU acceleration.
Classic photometric reconstruction can run on CPU.
