Diffusion policy, explained: generative robot control
Why generative models, diffusion and flow matching, became the default way robot foundation models produce multimodal action, and what it asks of the data.
Picture a gripper that has to reach a mug sitting in the middle of a cluttered counter. Two demonstrations taught it the task: one swings left around the mug, the other swings right. Both are correct. Now train a network to imitate them by minimizing squared error, and watch what it learns, the average of the two paths, a straight line that plows through the mug it was supposed to avoid.
That failure has a name, mode averaging, and for years it quietly capped how well behavioral cloning worked on real manipulation. The fix that stuck did not come from robotics at all. It came from image generation. A diffusion model, the same class of method behind text-to-image systems, turns out to be an unusually good way to represent the set of actions a robot could reasonably take. Between 2023 and 2026 it went from a clever paper to the default action head across much of the field: Toyota Research Institute builds its Large Behavior Models on it, Physical Intelligence ships a flow-matching variant in π0, and NVIDIA's GR00T foundation models generate motion the same way.
This piece explains why a generative model, rather than a plain regressor or a classifier, became the natural fit for multimodal robot action, how a diffusion policy works under the hood, and what it quietly demands from the data that trains it.
The problem plain regression cannot solve
Standard behavioral cloning treats control as regression: given the current camera images and robot state, predict the next action, and train by pushing the prediction toward the demonstrated action with a mean-squared-error loss. That loss has a hidden assumption baked in. It presumes the correct answer is a single point, and that being close to it in Euclidean distance is always good.
Manipulation breaks both assumptions. Human demonstrations are richly multimodal: there are many good ways to grasp a mug, reroute around clutter, or reorient a part, and they are not near each other in action space. Average two valid trajectories and you often get an invalid one. A Gaussian-headed policy, forced to name one action, either collapses onto the mean or jitters between modes. Discretizing the action space and predicting tokens autoregressively, the RT-1 and OpenVLA approach, sidesteps the averaging but pays in sequence length and struggles with fine continuous control.
A generative policy reframes the question. Instead of "what is the single best action," it models "what is the distribution of good actions," and then samples from it. Sample the left path or the right path, each a clean, self-consistent trajectory, and never the ruinous blend of the two.
Regression asks a policy to name the one best action. Generation asks it to describe the space of good ones. Contact-rich manipulation lives in the gap between those two questions.
How a diffusion policy actually works
A diffusion policy borrows the denoising recipe wholesale. Start from a chunk of pure Gaussian noise shaped like a short sequence of future actions. A neural network, conditioned on the recent observations, predicts how to nudge that noise a little closer to a plausible action sequence. Repeat the nudge for a fixed number of steps, and the noise resolves into a coherent trajectory. Training is the reverse: take a real demonstrated action chunk, add noise at a random level, and teach the network to predict the noise it must remove.
Two design choices do most of the heavy lifting. First, the policy predicts an action chunk, a sequence of the next dozen or so timesteps, not a single step. Chunking makes the target distribution smoother, encodes short-term intent, and cuts the compounding error that plagues step-by-step cloning. Second, it runs receding-horizon control: predict a chunk, execute only the first few actions, then re-plan from fresh observations. The policy stays reactive while still committing to locally consistent motion. The original ACT work on the ALOHA setup popularized chunking; diffusion added the generative backbone that made the chunks properly multimodal.
Diffusion, flow matching, and the race for speed
The visible frontier has largely moved from classic diffusion to flow matching, a closely related generative recipe that learns a direct velocity field from noise to data rather than a long stochastic denoising chain. Practically, flow matching can produce a good action chunk in a handful of integration steps instead of dozens of denoising steps, which matters enormously when the model has to run inside a control loop. Physical Intelligence's π0 pairs a vision-language backbone with a flow-matching "action expert" that emits high-frequency continuous commands; NVIDIA's GR00T N1 uses a diffusion-transformer action module on a similar template. The pattern is now standard: a large multimodal model for perception and semantics, a generative head for the fine, fast, multimodal motion.
What it costs, and the tricks that pay it down
Generation is not free. Every sampled action chunk needs several forward passes through the action network, which is why naive diffusion policies ran their control loops slowly and why real deployments lean hard on acceleration. Teams reach for DDIM-style samplers that cut the step count, distillation and consistency-style objectives that push toward one or two steps, and flow matching precisely because it needs fewer function evaluations. Receding-horizon execution helps too: a chunk of a dozen actions amortizes one round of sampling across many control ticks, so a policy can sample at a few hertz while commanding motors at a much higher rate. The table below sketches the landscape of action representations and where diffusion sits.
| Approach | Handles multimodal actions | Inference cost per chunk | Representative systems |
|---|---|---|---|
| MSE regression (single Gaussian) | Poor, averages modes | One forward pass | Early behavioral-cloning baselines |
| Discrete action tokens (autoregressive) | Yes, but coarse | Many token steps | RT-1, RT-2, OpenVLA |
| Diffusion (DDPM / DDIM) | Yes | Roughly 10-100 denoising steps | Diffusion Policy, TRI Large Behavior Models |
| Flow matching | Yes | A few integration steps | π0, GR00T N1 |
Why the bottleneck moved to data
Here is the part that gets undersold. A diffusion policy can only be as multimodal as the demonstrations it is fed. If every operator in your dataset rounds the mug on the same side, the "distribution of good actions" has exactly one mode, and the generative machinery has nothing to generate. The method's strength, faithfully modeling the spread of human behavior, converts directly into an appetite for data that actually contains that spread: many operators, many object arrangements, many strategies, all captured with enough sensing to reconstruct what the hands and the contacts were doing.
This is why the last two years of progress track datasets as much as architectures. Open X-Embodiment pooled demonstrations across dozens of robots and labs; DROID pushed for scale and scene diversity in the wild; egocentric human-video corpora such as Ego4D and Ego-Exo4D extended the same logic to first-person human behavior. Open implementations in Hugging Face's LeRobot then made training a diffusion policy on that data a matter of days, not quarters.
Where this leaves us
Diffusion did not win robot control because it is fashionable. It won because it answers the right question, describing a space of good actions instead of guessing one, and because contact-rich manipulation is multimodal all the way down. The open problems now are unglamorous and real: shave the sampling cost further, prove the reliability such policies need to leave the lab, and above all feed them demonstrations diverse enough to be worth modeling generatively. The architecture is close to settled. The data is where the next few years of the argument will be had.