Open X-Embodiment vs DROID vs RH20T: How They Differ
Open X-Embodiment, DROID, and RH20T compared on scale, sensors, and provenance, and why raw trajectory counts mislead more than they reveal.
Start with a number everyone quotes: Open X-Embodiment ships with more than one million real robot trajectories. It sounds decisive. Then you look closer and find the million is stitched together from over 60 separate datasets, recorded on 22 different robots that do not agree on what a single action even means. One robot's "move" is a joint-velocity command; another's is an end-effector pose; a third logs a discretized gripper toggle. The headline scale is real. The comparability is not.
That gap is the whole story of modern manipulation data. Three of the most cited collections, Open X-Embodiment, DROID, and RH20T, are three different answers to the same question: where do you get enough demonstrations to train a policy that generalizes. Open X-Embodiment pools everything the community already had. DROID puts one standardized arm in hundreds of real rooms. RH20T narrows the scope and instruments it with every sensor worth recording. None of them is "the biggest" in a way that settles an argument, because they are not measuring the same thing.
The useful comparison runs along three axes, and only the first is the one people cite. Scale tells you how many episodes exist and, more importantly, what one episode is worth. Sensors tell you what a model can actually perceive. Provenance, meaning who collected the data, on what hardware, under what protocol and license, tells you whether the numbers compare at all and whether you are legally allowed to train on them. Buyers underweight the third axis until it stops them.
What "scale" actually hides
Take the three headline figures at face value first. Open X-Embodiment aggregates more than one million trajectories from over 60 prior datasets and 22 robot embodiments. DROID contributes roughly 76,000 teleoperated trajectories, about 350 hours, across 564 scenes and 86 tasks. RH20T holds more than 110,000 robot episodes paired with human demonstrations across 147 tasks. By raw count, Open X-Embodiment wins by an order of magnitude.
Now discount for what a single episode is worth. Open X-Embodiment's million is heterogeneous by construction: different arms, grippers, control frequencies, camera counts, and action spaces, normalized into a shared RLDS format but not into shared physics. A policy has to learn, implicitly, which robot it is looking at. DROID's episodes are uniform because the hardware is fixed, so variation lives where you want it, in scenes and tasks, not in the platform. RH20T trades aggregate trajectory count for depth: narrower in scope than the pooled aggregate, but each interaction is recorded with contact information most datasets never capture.
| Axis | Open X-Embodiment | DROID | RH20T |
|---|---|---|---|
| Scale | 1M+ trajectories | ~76,000 trajectories, ~350 h | 110,000+ episodes plus human demos |
| Embodiments | 22 robots, 60+ datasets | 1 (Franka Panda) | Multiple arm-gripper configs |
| Cameras and depth | Inherited, mixed | 3 stereo (2 scene, 1 wrist) | Multi-view RGB-D |
| Extra sensing | Varies by source | Stereo depth | Force-torque, audio |
| Provenance | Federated aggregation | Standardized, 13 institutions | Single program, paired human |
| Tasks and scenes | Very broad, uneven | 86 tasks, 564 scenes | 147 tasks |
Three sensor philosophies
Sensors decide what a policy can even attend to, and here the three diverge sharply. Open X-Embodiment inherits whatever each contributing lab happened to record. Many source datasets carry a single RGB camera and no depth; some add a wrist camera; a few include proprioception at high rate. The union is broad but ragged, and a model trained on it must tolerate missing channels.
DROID standardizes the rig. Every episode uses a Franka Panda with two scene-mounted ZED stereo cameras and one wrist ZED, giving calibrated stereo depth on every frame. That consistency is the point: if you want to study whether a policy generalizes across kitchens and offices, you do not want the camera setup changing underneath you. Depth comes from stereo, which is cheaper and more portable than active sensors but noisier on textureless surfaces.
RH20T goes the other direction and instruments for contact. Alongside multiple calibrated RGB-D views it records six-axis force-torque and audio, the channels that matter when a task involves insertion, wiping, or anything where vision alone cannot tell you whether the gripper is pressing too hard. For contact-rich manipulation, that extra sensing is often worth more than another 100,000 vision-only clips.
A trajectory count is a vanity metric until you know the robot, the sensors, and the license behind each frame.
Provenance decides what you can trust
Provenance is the axis that decides whether the other two are even usable. Open X-Embodiment is a federation: its strength is that it unified dozens of independent efforts, and its liability is that it inherits their licenses, consent status, and documentation quality wholesale. Some sub-datasets are permissively licensed and well described; others are thin on metadata. If you train a commercial policy on the pool, you inherit that patchwork, and you have to check it dataset by dataset.
DROID was collected as one coordinated program: 50 collectors across 13 institutions in North America, Asia, and Europe, running the same protocol on the same hardware for about a year. That coordination buys you clean, comparable metadata and a single license to reason about. RH20T is narrower still, a single research program with a fixed sensor stack and paired human demonstrations recorded for the same tasks, which is what makes human-to-robot transfer experiments tractable on it.
For an engineer this is not paperwork. Provenance determines reproducibility, whether you can filter by robot or scene, whether you can legally ship a model trained on the data, and whether a partner's due-diligence team will sign off. The cleaner the provenance, the fewer surprises at audit.
What each dataset trains well
These differences map onto different uses. Open X-Embodiment exists to demonstrate cross-embodiment transfer: the RT-X models trained on it showed that pooling data across robots can lift performance on any single one, and the pooled corpus has since become a default pretraining base for generalist policies. Cross-embodiment foundation efforts such as NVIDIA Isaac GR00T and the open Hugging Face LeRobot ecosystem lean on exactly this premise, that breadth of embodiment is a feature, not noise.
DROID is built for generalization studies on a fixed platform: hold the robot constant, vary the world, and measure what transfers. RH20T is built for skill learning where contact and force matter, and for one-shot imitation from a paired human demonstration. If your target is a Franka-class arm doing varied household tasks, DROID is closer to your distribution. If your target is contact-rich assembly, RH20T's force channel is hard to replace. If your target is a robot none of them used, Open X-Embodiment's breadth is the hedge.
Choosing between them
There is no single winner, and treating trajectory count as the ranking is how teams end up surprised. Pick Open X-Embodiment when you need embodiment breadth and can absorb the license and normalization work. Pick DROID when you want controlled scene diversity on a known arm. Pick RH20T when contact and force decide the task. Then read the sources yourself: Open X-Embodiment, DROID, and RH20T each document their own tradeoffs more honestly than any summary table can. The number on the box was never the point.