Sample Efficiency in Robot Learning

Two teams set out to teach a robot the same drawer-opening skill. One collects ten thousand demonstrations and gets a policy that works most of the time. The other collects two hundred and gets one that works better. The robot was not the difference. What each team brought to the problem before the first demonstration was.

Sample efficiency is the amount of skill you extract per demonstration. In language modeling it barely comes up, because text is nearly free and scraped by the terabyte. In robotics it is the whole ballgame. A demonstration is a human operating hardware in real time, and there is no web to scrape for it. Every trajectory costs minutes, wear, and attention.

So the interesting question is not how to collect more. It is how to need less. Three levers do most of the work: the priors you start from, the structure you impose on the problem, and the quality of the data you collect.

Priors: paying once so you pay less later

The fastest way to cut demonstrations is to not start from scratch. A policy that already understands objects, contact, and language before it sees your task can pick it up from a fraction of the data a blank network would need.

This is the whole argument for robot foundation models. A network pretrained across many tasks and bodies, like NVIDIA Isaac GR00T, carries priors that a fresh model lacks, so fine-tuning on a new skill converges faster and on less data. The generalist policies described on the Physical Intelligence blog make the same bet: absorb broad experience once, then adapt cheaply. Pretraining does not make demonstrations free. It changes how many you need to reach a given success rate, which is the number that actually shows up on the bill.

Web video is a prior too. A model that has watched people manipulate objects arrives with a sense of how hands and things behave, even without action labels. That prior is coarse. It still shaves demonstrations off the fine-tuning budget.

The lesson is blunt. Pay the compute to build a strong prior once, and every downstream team that later collects demonstrations pays a smaller bill for years. This is why the economics of robot learning increasingly reward groups that can amortize one large pretraining run across many tasks, instead of starting each new project from a cold, uninformed network.

Structure: the shape of the problem decides its cost

The second lever is representation. How you frame the observation and the action changes how much data the policy needs, sometimes by an order of magnitude.

Consider the action space. Predicting raw joint torques at high rate is expressive but data-hungry, because the policy must learn dynamics the hard way. Predicting end-effector waypoints hands much of that burden to a controller, so the policy learns fewer, more meaningful decisions. Keypoint and object-centric representations go further: a policy that reasons about where the mug handle is, rather than raw pixels, generalizes to new mugs from far fewer examples.

Skill structure matters just as much. Breaking a long task into reusable primitives means each primitive is trained on all the episodes that contain it, not just the whole-task demonstrations. The same two hundred kitchen demos yield far more grasping examples once you segment them, so the grasp primitive learns from a bigger effective sample than the raw episode count suggests.

The cheapest demonstration is the one you never had to collect, because a prior or a better representation already covered it.

Data quality beats data volume

The third lever is the one teams reach for last and should reach for first. Not all demonstrations carry the same information. A thousand near-identical pick-ups of the same block from the same spot teach a policy almost nothing after the first fifty. Fifty pick-ups spread across lighting, clutter, object shape, and starting pose teach it to generalize.

Coverage is the metric that matters, not count. The DROID dataset was built around exactly this idea: many scenes, many objects, many operators, so a policy trained on it sees variety rather than repetition. Work from the Berkeley BAIR blog has repeatedly shown that diversity in the training set predicts generalization better than sheer size.

Failure and recovery data belong here too. A corpus of only clean successes teaches a policy what to do when everything goes right, and nothing about how to recover when it does not. Since deployment is mostly the messy middle, a smaller set that includes near-misses and corrections often outperforms a larger set of flawless runs. Operators feel this in the room: the demonstrations that teach a policy the most are rarely the smoothest ones to record.

Measuring efficiency honestly

Sample efficiency is easy to fool yourself about. If your test set resembles your training set, more data always looks like it helps, because you are rewarding memorization. The honest measure is the demonstration count needed to reach a target success rate on genuinely held-out conditions: new objects, new backgrounds, new starting states. Open tooling like Hugging Face LeRobot makes this kind of controlled comparison easier to run and to reproduce.

The instinct to fix a weak policy by collecting more data is usually the expensive answer to the wrong question. More data helps when the new data is different. When it repeats what the model already saw, it inflates the corpus and the bill without moving the success rate. The teams that win on cost are not the ones with the biggest datasets. They are the ones that start from strong priors, frame the problem so the policy learns less, and spend their capture budget on coverage rather than repetition.