Robots Atlas>ROBOTS ATLAS
Robotics

Vision-based tactile sensing

2009ActiveUpdated: 19 August 2026Published
Key innovation
Sensing touch via a camera observing the deformation of a soft elastomer — turning an information-rich image into a high-resolution map of contact geometry and forces.
Category
Robotics
Abstraction level
Pattern
Operation level
Robot controlData
Use cases
Dexterous robotic manipulation and graspingIn-hand object and texture recognitionSlip detection and grip-force controlLearning manipulation policies (robot learning)Quality control and surface metrology

How it works

A soft elastic 'skin' (elastomer) deforms on contact with an object. A camera behind it captures this deformation, and multi-color/multi-angle LED lighting lets the system recover local surface slopes (photometric stereo) and reconstruct the 3D shape of the imprint. Motion of markers printed on the membrane allows estimating shear forces and slip. The resulting maps (depth, forces) feed control algorithms or neural networks that learn manipulation policies.

Problem solved

Classic tactile sensors (taxel arrays) have low spatial resolution. Vision-based tactile sensing provides dense, high-resolution contact information critical for dexterous manipulation.

Key mechanisms

A soft elastomer as the contact surface
A camera capturing deformation from the inside
LED lighting and photometric stereo (slope map)
Markers for estimating shear forces and slip
Geometry/force reconstruction via classical methods or neural networks

Strengths & limitations

Strengths
✓Very high spatial resolution of touch
✓Rich information: shape, texture, forces, slip
✓A cheap sensor based on a camera and elastomer
✓Works well with machine learning
Limitations
✗Sensor size/thickness (camera + elastomer)
✗Elastomer wear and damage
✗Latency and cost of image processing
✗Limited contact area and lighting dependence

Components

Elastomer (skin)Contact surface

A soft, deformable contact layer, often with a reflective membrane.

Official

CameraImage acquisition

Captures the elastomer's deformation from the inside.

Official

LED illuminationIllumination

Multi-angle/multi-color light enabling photometric stereo.

Official

Reconstruction algorithmProcessing

Turns the image into depth and force maps (classically or via a network).

Official

Implementation

Reference implementations
GelSight (MIT / GelSight Inc.)
— · GelSight
Official
DIGIT (Meta AI)
— · Meta AI
Official
Implementation pitfalls
Elastomer wear and damageMedium

The soft surface degrades under heavy contact.

Fix:Replaceable pads, tougher materials, surface protection.
Size and integrationMedium

The camera + elastomer are thicker than thin taxel arrays.

Fix:Optics miniaturization, mirrors, smaller modules.
Processing latencyMedium

Image reconstruction adds latency to the control loop.

Fix:Lightweight models, edge acceleration.

Evolution

2009
GelSight (MIT)
Inflection point

Edward Adelson's lab demonstrates a high-resolution vision-based tactile sensor.

2020
DIGIT and low-cost sensors for robotics

Meta AI releases a low-cost, compact vision-based tactile sensor.

2024
Integration with manipulation learning

Vision-based tactile sensors increasingly feed manipulation policies (VLA, robot learning).

Hyperparameters (configurable axes)

Reconstruction methodHigh
fotometrycznaClassic photometric stereo with calibration.
sieć neuronowaLearning an image→depth/force mapping.
MarkersMedium
z markeramiEnable estimating shear forces and slip.
bez markerówSimpler image, less force information.