Grasp Synthesis from Demonstration, Not Solvers
Learning grasp synthesis from human hands captures functional, task-aware grips that analytic force-closure planners miss, and scales to dexterous hands.
For decades, grasping was a physics problem. Find a set of contact points on an object such that the forces they apply resist any external wrench, declare force closure, and command the hand there. It was elegant, provable, and it worked, right up until the object was a crumpled bag, a wine glass, or anything the planner had no clean model for.
The analytic approach did not fail because the math was wrong. It failed because it assumed knowledge the real world withholds: exact geometry, known friction, a static object. Strip those away and force closure has nothing to stand on.
The alternative that is winning is older than any planner. Watch how a human hand does it, and learn the grasp from the demonstration.
Grasping used to be a physics problem
Classical grasp synthesis treats the hand and object as rigid bodies and searches for contacts that satisfy a stability condition. Tools in this lineage, from GraspIt-style planners to large simulated grasp datasets, can generate millions of candidate grasps for known meshes. For a bin of identical machined parts, this is still hard to beat.
The trouble starts with variety, and with contact that is not a point. An analytic planner has no opinion about which grasp is right for the task, only which grasps are stable. It will happily grab a knife by the blade if the blade offers force closure. Stability is not the same as usefulness.
What analytic planners get wrong
Two gaps matter most. The first is task-agnosticism. A stable grasp on a mug is not the grasp you want if the next step is pouring; you need a grip that leaves the handle free and the rim controlled. Function, not just stability, decides the right grasp, and a pure physics planner cannot see function.
The second is the geometry assumption. Force closure wants a known shape and known friction. Deformables, transparent objects, and cluttered piles violate all of it. This is where analytic methods hit the long tail and stop, and where learned methods, trained on how real hands cope, keep going.
A physics planner finds a grasp that will not slip. A human demonstration shows the grasp that will get the job done. Those are not always the same grasp.
Three ways to generate a grasp
| Strategy | Handles novel objects | Task-aware | Data or model cost |
|---|---|---|---|
| Analytic force closure | Only with a known model | No | Needs geometry and friction |
| Simulated or self-supervised | Moderate, sim-to-real gap | Rarely | Millions of sim grasps, then transfer |
| Human demonstration | Well, inherits human priors | Yes | Human-hours plus hand and contact capture |
The simulated route, exemplified by large self-supervised grasp datasets, scales cheaply and covers many shapes, but it inherits the simulator's blind spots and rarely knows what a grasp is for. Human demonstration is costlier per sample and carries something the others lack: a prior about which grasp a competent agent would actually choose.
Why human hands are the right prior
A person grasping an object solves several problems at once, mostly without thinking. They pre-shape the hand in flight, so the fingers arrive already in roughly the right configuration. They pick a grip that suits the next action. They modulate force to the object, gentle on the glass, firm on the wrench. Each of those is a hard sub-problem for an analytic planner, and each is demonstrated, for free, every time a human does the task.
Dexterous, multi-finger hands are where this matters most. The space of possible finger placements on a five-fingered hand is enormous, and searching it analytically is brutal. A human demonstration collapses that search: it hands you one good, functional configuration to imitate and adapt. Work shared on the Physical Intelligence blog and systems like NVIDIA Isaac GR00T treat these human-derived grasps as a starting distribution rather than solving grasp geometry from first principles.
The retargeting gap
Human grasps do not drop onto robot hands unchanged. A human hand has different proportions, more degrees of freedom, and softer contact than most robot hands, so a captured grasp has to be retargeted to the robot's kinematics. Some things transfer well: the approach direction, the choice of which part of the object to grab, the coarse hand shape. Some transfer poorly: exact fingertip forces and fine in-hand reorientation.
This is why the raw capture matters more than a polished video. To retarget a grasp you need the hand pose and the contact, not just an image of a hand near an object. Datasets that record contact-rich manipulation, such as RH20T, and multi-view first-person collections like Ego-Exo4D, exist precisely because a flat RGB clip throws away the signal retargeting depends on.
A grasp is the opening move
A grasp is not a static pose; it is the first move of a manipulation. Part of why human grasps transfer so well is that they are chosen with the whole action in mind, so what looks like a single grip is really a bet about the next several seconds. A demonstration captures that bet. An analytic planner, solving one frame at a time, cannot.
This reframes what a successful grasp means in the data. A grasp that holds firmly but blocks the next step is a failure the moment you zoom out. Labeling grasps by whether the downstream task succeeded, not just by whether the object left the table, is a small change in annotation that sharply improves what a policy learns from each demonstration.
The physics of force closure is not wrong, and it will keep earning its place on rigid, known parts. But general grasping, across the messy variety of real objects and toward genuinely dexterous hands, is not going to come from a better solver. It is going to come from watching competent hands and learning what they already know. The planner asks whether a grasp will hold. The human shows which grasp to pick. For robots to grasp like us, they need to learn from the demonstration, not just the mathematics.