Hz vs fps: Robot Capture Rate and Control Frequency

A camera bolted to a laundry-folding rig fires 240 frames every second. The policy driving the gripper wakes up 30 times a second. Same robot, same towel, an eightfold gap between how fast the two halves of the system see the world and how fast they act on it.

That gap is not sloppy engineering. It is a design choice repeated across almost every serious manipulation stack shipping today, and the numbers behind it, 240 Hz here, 30 Hz there, 50 Hz somewhere in the middle, are usually reported loosely enough that newcomers assume they are interchangeable. They are not. Hz and fps measure related but distinct things, and conflating them is how a dataset ends up with beautifully sharp video that a policy cannot actually learn from.

This piece pulls the two rates apart: what a high capture rate buys you, why control frequency stays stubbornly low, and what has to happen in between so a model trained on human demonstration data does not choke on its own timestamps.

Two clocks that rarely agree

Start with vocabulary, because the sloppiness starts there. fps, frames per second, is a sampling rate: how often a sensor writes a sample. Hz is just cycles per second, and it attaches to every clock in the system: camera exposure, IMU sampling, force-torque readout, the control loop, the policy inference step. Saying a robot runs at "240" is meaningless until you say 240 of what.

A typical manipulation rig runs at least four clocks at once. RGB cameras at 30 to 60 fps, depth sometimes lower. An inertial measurement unit at 100 to 1000 Hz. A force-torque sensor at 500 Hz to a few kHz. And the control loop, the thing that actually commands joint targets, often at just 15 to 50 Hz for a learned policy, even while the underlying motor controller closes its current loop at 1 kHz or higher. Five clocks, three orders of magnitude apart, and every one of them has to be reconciled before a model sees a single sample.

Why capture wants to be fast

The case for a high capture rate is physical, not aesthetic. Three reasons dominate.

Contact is fast. The instant a fingertip touches a rigid object, force spikes in a millisecond or two. Sample that channel at 30 Hz and you miss the transient entirely: you see "no contact," then "already pressing," with nothing in between. Grasp quality, slip, and impact all live in that gap, which is why force-torque channels are read at hundreds of hertz or more.

Nyquist is unforgiving. To reconstruct a signal you must sample at more than twice its highest frequency. A tendon twang or tool chatter at 80 Hz needs north of 160 Hz just to avoid aliasing into garbage. Undersampling does not blur the signal, it fabricates a false, lower-frequency one that looks perfectly real.

Blur is lost information. A camera at 30 fps with a long exposure smears a fast gripper across the frame. Higher frame rates with shorter exposures keep edges crisp, which matters more to a model reading fine motion than to a human watching playback.

Capture the world at the speed of the fastest event you care about, then hand the policy the slowest rate it can still act well on. Those are two different numbers, and pretending they are one is the classic beginner mistake.

Why control stays slow

If fast is so good, why not run the policy at 240 Hz too? Because the policy is expensive and the world is forgiving.

A modern vision-language-action model is a large neural network. One forward pass can take tens of milliseconds on a datacenter GPU, and longer on anything you can bolt to a robot. Run inference every control tick and you would need a sub-5 ms forward pass, which no model of interesting size delivers. So the field decouples the two rates with action chunking: the policy looks at the scene once, predicts a short sequence of future actions, and a cheaper controller streams them out while the next inference runs. Physical Intelligence's π0 uses flow matching to emit action chunks at up to roughly 50 Hz while the heavy transformer ticks far less often, and NVIDIA's GR00T follows the same predict-a-chunk pattern.

The second reason is that manipulation tolerates it. Folding a towel or seating a connector is quasi-static: the arm moves slowly relative to the sensing, and a 30 Hz command stream is smooth enough. Dynamic tasks, a running gait, a catch, a whip, demand higher control rates and usually a different, non-learned controller underneath.

The alignment tax

Multiple clocks create a bookkeeping problem that is easy to underestimate. If your camera stamps a frame at t and your force sensor stamps a reading at t plus 3 ms, which action does the model associate with which observation? Get it wrong and you teach the policy a causal lie: press, then see, instead of see, then press.

So every serious dataset resamples. The raw streams arrive at their native rates and are aligned to a common timeline, usually the control rate, with interpolation or nearest-neighbor matching per channel. This is where the "240 Hz capture, 30 Hz output" phrasing earns its keep: you record fast to catch the fast events, then downsample deliberately to the rate the policy consumes, keeping the high-rate channels available for anyone who wants them later. Hugging Face's LeRobot dataset format bakes per-feature timestamps in precisely so this alignment is reproducible rather than folklore.

Throw away the timestamps and you can never redo the alignment. That is the quiet reason raw, high-rate, well-stamped capture is worth more than pre-baked 30 Hz clips: the clips have already made choices you can never revisit.

What the public datasets actually clock

The numbers in the open corpora make the spread concrete.

• DROID pairs a Franka arm with three camera views and publishes trajectories at a control rate on the order of 15 Hz, a deliberately modest number chosen so a learned policy can keep up. See the DROID dataset.
• Ego4D, thousands of hours of first-person human video, is distributed at 30 fps: fine for reading human intent and hand motion, coarse for fast contact, which is exactly the tradeoff its designers accepted for scale. See Ego4D.
• Open X-Embodiment stitches together more than sixty datasets across 22 robot embodiments whose control frequencies span roughly 3 to 30 Hz, which is why anyone training across them has to resample first. A model that ignores the mismatch is learning across inconsistent clocks. See Open X-Embodiment.

So the next time a spec sheet brags a single big number, ask 240 of what, and against which clock. The interesting engineering is never in the peak rate. It is in the honest gap between how fast a robot senses the world and how fast it dares to act, and in keeping enough of the fast signal around that the choice stays yours to remake.