The data bottleneck in embodied AI
Compute and model architectures are ready. Real-world robot data is not. Inside the data bottleneck holding embodied AI back, and the pyramid behind it.
GPT-class language models trained on something close to the entire public web: trillions of words that billions of people had already written, for their own reasons, and left lying around. The robotics field would love an equivalent. It does not have one. There is no giant corpus of a robot picking up a cup ten million times, and no web crawler can go and make it.
This is the quiet fact behind a lot of embodied-AI hype. The models are good. The compute is available. The recipe is, by 2026 standards, fairly well understood. What is missing is the fuel: real-world data of bodies doing physical tasks, in the quantity and quality that made language models work. Data is the binding constraint, and it is worth being precise about why.
Compute and architecture stopped being the wall
A few years ago you could blame the model. Not really anymore. The dominant design for embodied control, the vision-language-action (VLA) model, is a transformer that takes in pixels and an instruction and emits actions, and it looks a lot like the architectures already proven on text and images. Open X-Embodiment showed that a single policy trained across many robot types beats policies trained on each one alone. NVIDIA Isaac GR00T and Toyota Research Institute's Large Behavior Models are, in architecture terms, not exotic. They are hungry.
Feed one of these models more good data and it improves in the smooth, predictable way we saw with language. That is the encouraging part and the frustrating part at once. The lever works. The lever needs data we do not have.
Why language had it easy
Text is the exhaust of human thought. We produce it constantly, for our own purposes, and it is already digitized. Training a language model meant harvesting a byproduct that existed anyway. Physical actions leave no such trace. When you unload a dishwasher, no file is written. The joint angles of your wrist, the force in your fingers, the order you chose to grab things, all of it evaporates the instant you finish.
Compute scales with capital, and architecture is largely a shared recipe. The one input you cannot buy off a shelf is a million hours of hands doing real tasks.
So embodied AI cannot scrape its way to a foundation model. It has to generate its training set, action by action, and that changes the economics completely.
The data pyramid
A useful way to hold the problem is a pyramid, widest and cheapest at the bottom, narrowest and most valuable at the top. Every tier trades abundance against fidelity to a real robot.
| Tier | Source | Abundance | Fidelity to a robot | Rough cost per useful hour |
|---|---|---|---|---|
| Base | Web text and images | Effectively unlimited | Low: semantics only, no actions | Near zero |
| Base | Internet human video | Very large | Low to medium: shows tasks, no action or force labels | Low |
| Middle | Simulation | Large, on demand | Medium: physics is approximate, the sim-to-real gap bites | Low, plus engineering |
| Top | Teleoperated robot demos | Small | High: real robot, real contact | High |
| Top | Purpose-built human demonstration | Small but growing | High: real hands, multimodal, needs retargeting | Medium to high |
Most serious pipelines use the whole pyramid. You pretrain low for broad visual and semantic priors, you inflate volume in the middle with simulation, and you spend your scarce budget at the top, where the data actually resembles a robot making contact with the world. The fight is over that top tier. It is where policies get reliable, and it is the expensive part.
The ratios matter more than they look. A policy can soak up enormous amounts of cheap base data and still miss the last mile without a few hundred high-fidelity demonstrations of the exact task, on the exact hardware. Cheap data sets the priors. Expensive data sets the behavior. Get the mix wrong and you waste both.
Teleoperation is expensive, human video is cheap and raw
Two ways to fill the top of the pyramid, with opposite problems.
Teleoperation puts a human in the loop driving the actual robot, so every recording is perfectly labeled with the robot's own actions and sensors. It is the gold standard for fidelity. It is also slow and costly: one operator, one robot, roughly real-time, one task at a time. DROID and community collections grew large only through months of coordination across many institutions, and they are still tiny beside a web corpus.
Human video flips the tradeoff. People are fast, cheap, and dexterous, and they can generate demonstrations far quicker than any teleoperated robot. But raw human video carries no action labels, records no force, and shows a human hand that has to be retargeted onto a different end-effector. It is cheaper to collect and more expensive to process. That processing, extracting hand pose, recovering trajectories, adding the sensor channels a video lacks, is the actual work.
Neither path is a shortcut. Teleoperation buys fidelity with time you cannot compress. Human video buys volume against a processing bill you pay later. The teams making real progress treat both as a supply chain to be engineered, not a dataset to be downloaded once.
Which is why capture is becoming a discipline
The realization spreading across the field is that you cannot buy this in bulk and you cannot fully fake it. Google DeepMind and other groups have leaned on large, carefully collected real-world demonstration sets, and they keep hitting the same wall: you get out roughly what you put in, and putting it in is manual. Datasets like RH20T exist because someone decided to capture contact-rich manipulation on purpose, at scale, with the sensors attached.
None of this makes the model work unimportant. It reframes it. The interesting question for embodied AI is shifting from can we design a policy that learns dexterous tasks to can we produce enough of the right data to feed it. The first question is close to answered. The second is a logistics problem wearing a research problem's clothes, and it is the one that will decide how fast robots actually get good.