Wrist Camera vs Third-Person View
A wrist camera sees the fingertips, a third-person view sees the room. Here is what each teaches a manipulation policy, and why you capture both.
Mount a camera on a robot's wrist and it sees a world most cameras never do: the gripper closing, the exact lip of a mug, the thread of a screw catching. Mount it over the robot's shoulder instead and you see the whole workspace, the arm, the table, the human nearby, but the fingertips shrink to a few blurry pixels. Same task, two cameras, two completely different lessons.
The eye-in-hand versus third-person choice is one of the oldest in robot manipulation, and it keeps coming back because neither view wins outright. Each throws away exactly what the other keeps.
The decision is not academic. It sets what a policy can attend to, and it sets what your capture rig has to record and calibrate. Get it wrong and you collect thousands of hours that quietly cannot teach the skill you wanted.
What the wrist camera teaches
An eye-in-hand camera lives near the action. Its biggest gift is invariance. Because the camera moves with the gripper, a grasp looks similar whether the object sits left, right, near, or far, so a policy needs fewer demonstrations to cover the same space. Large multi-view sets like the DROID dataset include a wrist camera for exactly this reason.
It also sees contact. The instant a finger touches an edge, the wrist view captures it at full resolution, which matters for insertion, alignment, and any task decided in the last centimeter. Contact-rich collections like RH20T lean on close views for this.
The costs are real. The wrist view has no memory of the room, so it is poor at global planning. It swings violently as the arm moves, which brings motion blur. And the gripper itself occludes the very contact you care about at the worst moment. A wrist camera is a specialist: brilliant up close, blind at a distance.
There is a second, quieter benefit. Resolution is not spread evenly. A wrist camera spends most of its pixels on the object and the gripper, exactly where the task is decided. A fixed camera spends most of its pixels on the tabletop, the wall, and empty air. For the same sensor budget, the eye-in-hand view simply puts detail where it matters.
What the third-person view teaches
An over-the-shoulder camera keeps a stable, wide frame. It sees the whole scene: the target, the obstacles, the other hand, the person giving an instruction. That makes it the natural home for long-horizon planning and for grounding language, since the words usually refer to objects across the whole workspace. Most trajectories in Open X-Embodiment come from fixed third-person cameras.
The weakness is precision. The gripper and object occupy a tiny patch of the image, so fine alignment is hard to read. The arm frequently occludes the target. And because the camera is fixed to the world, the same object at a new position looks different, so you need more data to cover the space a wrist camera covers for free.
Teams often run more than one exocentric camera to fight occlusion. Two or three fixed views from different angles mean the arm rarely blocks all of them at once, and a policy can triangulate rough depth from the disparity. The price is calibration. Every added camera is another set of intrinsics and extrinsics to solve and keep stable across a session.
| Property | Wrist (eye-in-hand) | Third-person (exocentric) |
|---|---|---|
| View of contact | Full resolution | Small, often occluded |
| Global scene context | Almost none | Complete |
| Invariance to object position | High | Low |
| Language grounding | Weak | Strong |
| Motion blur | Frequent | Rare |
| Demos to cover a task | Fewer | More |
Why serious stacks record both
The two views are complements, not rivals. A recurring finding across the Berkeley BAIR blog and the systems targeted by NVIDIA Isaac GR00T is that fusing a stable third-person view with a wrist view raises success on precise manipulation more than either view alone. The third-person stream carries the plan. The wrist stream carries the contact. A policy that reads both can decide where to go, then close the loop when it gets there.
Ablations tend to agree. Drop the wrist camera and insertion-style tasks fall off first. Drop the third-person camera and long, multi-step tasks lose the thread.
None of this is free to record. Each stream adds bandwidth, storage, and a calibration burden, and the more cameras you mount, the more ways the rig can fall out of sync. The engineering question is not how many views are theoretically useful. It is how many you can keep aligned, session after session, without the timestamps drifting apart.
The third-person camera tells the policy where to go; the wrist camera tells it what is happening when it arrives.
What this asks of human capture
Human demonstration complicates the picture in a useful way. A head-mounted egocentric camera is not a wrist camera and not a fixed third-person camera; it is a moving first-person view that tracks where attention goes. It sees the hands during manipulation and the scene during reaching, shifting between the two as the person looks. That is close to what a mobile robot actually experiences.
But it is not enough on its own. To transfer cleanly, capture benefits from adding near-hand views and a stable exocentric camera, all time-synced and calibrated to a shared frame. Record only the head view and you lose the fixed anchor. Record only a fixed camera and you lose the intimate contact detail.
Gaze adds a layer a robot rarely captures. A person looks where they are about to act, so the egocentric view is not just first-person; it is foveated by intention, lingering on the hands at the moment of contact and sweeping ahead during a reach. That attention signal is itself a kind of label, telling a downstream model which part of the frame mattered.
So the honest answer to eye-in-hand versus third-person is not a winner. It is a division of labor. The wide view frames the task and grounds the language; the close view resolves the contact that decides success. A dataset that captures both, aligned to the millisecond, keeps every downstream option open. One that captures only one has already chosen, whether it meant to or not.