US startup Encord is testing an unusual data source for training robots — the brain waves of its own operators. Working with German neurotech company Zander Labs, it is checking whether EEG and muscle signals improve the quality of datasets for physical AI. The pilot runs in a warehouse in San Leandro, California, as TechCrunch reported on July 26, 2026.
Key takeaways
- Encord and Zander Labs record operators' brain waves while they manipulate objects.
- EEG sensors are meant to detect mental states — the moment of error, intent and surprise.
- Frontier physical AI needs, per Encord, roughly five times the entire YouTube video corpus.
- Densely annotated data is worth about 100× more than raw egocentric footage, but costs about 20× more.
- The rig also includes forearm sensors that reconstruct hand position in 3D.
The problem: training data for robots simply does not exist
Language models were trained on internet text, and video generators on billions of clips. For robots, no such resource exists.
Vineeth Velmurugan, Head of Robot Learning at Encord and a former member of OpenAI's robotics lab, puts it plainly.
The data simply does not exist.
— Vineeth Velmurugan, Head of Robot Learning, Encord.
The scale of the need is enormous. According to Encord estimates cited by TechCrunch, training frontier physical AI requires a dataset on the order of five times YouTube's entire video corpus.
Yet raw camera footage is not enough — the model must know exactly what the hands are doing and why. That is what sets robotics apart from language models, which had the ready-made internet to learn from.
What the San Leandro pilot looks like
In the San Leandro warehouse, Encord operators perform seemingly trivial tasks: building a Jenga tower, pouring coffee, stacking poker chips, and plugging and unplugging ethernet cables.
Every action is captured by several cameras, including an egocentric one?Egocentric camera: A first-person view — from a camera mounted near the operator's head, showing what they see themselves. — the view from the operator's perspective.
On top of that comes a neurological layer. Operators wear an EEG?EEG: Electroencephalography — measuring the brain's electrical activity via electrodes placed on the scalp. headset from Zander Labs, with the work supervised by neuroscientist Lucas Gehrke.
Andrew Ceja, one of the operators who previously worked at Scale, sums up the job briefly.
It's something new every day!
— Andrew Ceja, operator, Encord.
Why brain waves at all
The idea rests on a so-called passive brain–computer interface?Passive brain–computer interface: An interface that reads brain states in the background, without the user consciously controlling it..
Zander Labs, maker of the SAMANAI platform and EEG hardware, designs its tools to run in the background and read cognitive states without conscious effort from the user. The goal is to capture moments of error, intent and surprise — signals that do not show up in the image alone.
That annotation layer is meant to raise the value of the data. A densely labeled recording — with tags like „right hand tightens bolt” paired with language-based understanding — is, per Encord, worth about 100× more than raw egocentric footage.
It also costs about 20× more to produce. Brain waves are meant to help the model recognize when to apply more compute and where to focus attention.
Not just the brain: signals from muscles
Encord adds a second modality — forearm EMG?EMG: Electromyography — measuring the electrical signals muscles produce as they work. sensors that measure electrical signals from muscles. From these, a three-dimensional model of the hand's position is built.
It answers a limitation of video alone: a camera rarely sees the whole hand, especially when the fingers are hidden by the object being grasped. It is worth stressing that this is an experiment, not a finished technology — a trial is under way to determine whether brain-wave-tagged data actually improves model performance.
Why it matters
Lack of data is today's main bottleneck in robotics. Where language models could learn from the ready-made internet, physical AI needs data that someone has to create first — motion by motion, in the physical world. That makes any method for cheaper or denser annotation strategically valuable.
The Encord and Zander Labs approach is interesting because it shifts the burden from quantity to signal quality. Instead of recording yet more thousands of hours of video, it tries to extract from the operator information the camera never captures: intent and the moment of a mistake. If it works, the cost of building useful datasets could fall, and companies without access to giant robot fleets would gain an alternative.
There is also a flip side. Passively reading employees' mental states raises questions about privacy and the limits of monitoring. Zander Labs says it processes data locally, but bringing EEG into the workplace is itself a precedent that reaches beyond the lab.
What's next
- Encord is running a trial to assess whether brain-wave-tagged data improves model performance — only a positive result will trigger scaling.
- The next step is combining modalities: video, EEG and muscle signals into a single training set for physical AI.
- An open question remains around regulation of employees' neurological data, which the pilot does not yet resolve.
Sources
- TechCrunch — Are brain waves the next unlock for physical AI?
- Zander Labs — SAMANAI passive BCI platform
