Onboard vs Cloud Robot Inference

A humanoid closes its fingers around an egg. The grip has to stop the instant the shell pushes back, and that decision lives in a control loop measured in single-digit milliseconds. Send the "stop" signal to a data center and wait for the reply, and the egg is already in pieces.

Where a robot's policy runs is not an infrastructure footnote. It sets a hard ceiling on how fast the machine can react. That ceiling decides which tasks are even possible, and it quietly dictates what your training data must contain.

The choice is usually posed as onboard versus cloud, as if you have to marry one. The sharper question is narrower: which part of the policy runs where, and at what rate? Below is the real tradeoff, and an argument that the split you pick should be visible in your data long before a robot ever ships.

The latency budget nobody escapes

Every closed-loop skill has a clock. A contact-rich task, threading a bolt or seating a connector, wants a control loop somewhere between 100 and 1000 Hz. At 200 Hz you have five milliseconds to sense, decide, and command. Miss the deadline and the loop stutters, the grip overshoots, the part jams.

Cloud inference cannot meet that clock. A round trip to a nearby data center runs on the order of 30 to 100 milliseconds on a good network, before the model even starts computing. That is fine for a planner that fires a few times a second. It is hopeless for a reflex. Large vision-language-action models such as NVIDIA Isaac GR00T are built to reason about a scene and emit an action chunk, not to babysit a servo loop, which is exactly why the fast loop has to live somewhere closer.

So the first rule is blunt. The higher your required control rate, the closer to the motor your inference has to sit.

What onboard inference actually costs

Running the policy on the robot buys you speed and independence. It also hands you a bill. Every watt spent on an accelerator is a watt not spent moving joints, and it turns into heat inside a sealed torso. A battery-powered humanoid cannot carry a rack of GPUs.

That budget forces compression. Teams quantize weights, distill a big teacher into a small student, and prune anything the task does not need. A model with billions of parameters may have to shrink by an order of magnitude to fit a mobile accelerator and still hit its frame time. The research groups pushing on-robot policies, including Physical Intelligence, spend real effort making a capable policy small enough to run at speed. None of that compression is free. Every gram of accuracy you trade away shows up as a missed grasp somewhere.

What the cloud buys, and how it fails

The cloud removes the size limit. You can serve a model too large for any robot, update it for the whole fleet in one deploy, and pool experience across every unit. For a slow deliberative layer, planning a sequence of steps or checking whether a task is done, this is a genuinely good fit.

There is a subtler upside. When every robot's experience flows back to one model, the fleet improves together: a corner case one unit hits on Tuesday can sharpen the policy every other unit runs on Wednesday. That flywheel is real, and it is one reason cloud-connected stacks stay attractive despite the latency tax.

The failure modes are just as real. Connectivity becomes a dependency, and a dropped link can freeze a robot mid-motion, which is unacceptable near people. Latency is not just high, it is variable, and jitter is harder to design around than a constant delay. Streaming a kitchen or a factory floor to a remote server also raises privacy and security questions that a self-contained robot never has to answer.

The split most serious teams land on

Read enough deployment write-ups and a pattern repeats. A fast reactive layer runs onboard at hundreds of Hz, handling balance, contact, and the tight sensorimotor loop. A slower deliberative layer runs less often, on a beefier onboard chip or in the cloud, choosing what to do next. The framing borrows the old System 1 and System 2 metaphor, and you can see it in how companies like Figure describe their stacks, and in Toyota Research Institute's work on Large Behavior Models.

This is not a compromise. It matches the physics. Reflexes and reasoning run at different speeds in animals for the same reason they should in robots.

Where a policy will run is a data decision, not just a deployment one. You cannot train a reflex you never recorded at reflex speed.

Why the split belongs in your data

Here is the part teams discover late. The onboard-versus-cloud choice reaches back into capture. A 200 Hz reactive controller can only be trained on data recorded at 200 Hz or faster, with force and proprioception sampled fast enough to see the contact events that matter. Log the same demonstration at 10 Hz and the reflex is simply not in the file. It cannot be recovered later.

The corollary is that a single demonstration should serve both layers. High-rate force, joint, and tactile channels feed the reflex. Lower-rate vision and language feed the planner. That only works if every channel carries its own accurate timestamp, so a downstream team can resample to whatever rate their target hardware runs, whether that turns out to be onboard, cloud, or a mix.

The choice you are really making

Onboard or cloud is the wrong binary. Almost every capable humanoid will run a fast loop close to the metal and a slow loop wherever the compute is cheapest. The teams that win will not be the ones who guessed the deployment target correctly. They will be the ones whose data was rich enough, and fast enough, to support either answer once the hardware is real.