What is a robot foundation model? VLAs explained
GR00T, pi0 and Gemini Robotics are robot foundation models, not chatbots. What a VLA is, how it differs from an LLM, and why data is the hard part.
Ask a chatbot how to fold a shirt and it will hand you a tidy numbered list. Ask a robot arm running that same chatbot to actually fold the shirt, and nothing happens. The words are all there. The shirt is exactly where it was. Somewhere between the paragraph and the fabric, the intelligence runs out.
Closing that gap is the entire project behind a robot foundation model. A system like NVIDIA's Isaac GR00T, Physical Intelligence's pi0, or Google DeepMind's Gemini Robotics is not a chatbot with arms bolted on. It is a different kind of model, trained on a different kind of data, and graded by a blunter test: did the object actually move.
The shorthand for this family is a VLA, short for vision-language-action model. The name is the idea. It reads pixels, it reads an instruction in plain language, and instead of emitting a sentence it emits an action. That third letter is where the difficulty lives.
What the word foundation is promising
Borrowed from language, the term foundation model carries a specific promise: train one big model on broad data, then adapt it to many tasks without starting over. In text that promise has largely been kept. One base model now drafts email, writes code, and answers legal questions with a little steering.
Robotics wants the same leverage. A single pretrained policy you can point at a new kitchen, a new part, a new gripper, with a modest amount of task-specific data rather than a fresh multi-year project each time. That is the pitch. Whether the physical world is as generous to this idea as text turned out to be is still an open question, and it is being tested right now.
From words to motor commands
Begin with what does not change. A VLA and a large language model share most of their plumbing. Both are usually transformers. Both are pretrained on a large corpus and then aligned. Many VLAs are built directly on top of a pretrained vision-language model, so the robot arrives already knowing what a mug is and what the word "next to" means, before it is asked to move anything.
The split happens at the output. An LLM predicts the next token of text. A VLA predicts the next action: a set of joint angles, a gripper opening or closing, a small nudge of the wrist, tens of times a second. Google's RT-2 made the move legible by treating robot actions as just another language the model could be taught to speak. The same network that could name the objects on a table could also produce the numbers that drove a hand toward one. Gemini Robotics is the current descendant of that line.
Why a VLA is not an LLM with arms
Sharing architecture hides how different the job is. Line the two up and the divergence is stark.
| Property | Language model | Vision-language-action model |
|---|---|---|
| Output | The next text token | The next motor action: joint angles, gripper state |
| Loop | Open loop, one reply and done | Closed loop, re-deciding many times a second |
| Clock | A slow answer is merely annoying | Late by 200ms and the cup is on the floor |
| Training data | Text humanity already wrote | Demonstrations someone must physically record |
| A wrong answer | An edit, a retry | A dropped glass, a bent part, a safety event |
| Embodiment | None; text is text anywhere | Bound to one body's joints, reach, and hand |
Read down the right column and the trouble compounds. A VLA runs in a closed loop: it acts, the world changes, it looks again, it acts again, and it has to sustain that in real time while friction and contact do things no caption anticipated. It is tied to a body, so a skill learned on one robot does not automatically survive a move to another with different arms. And it gets no undo. That last point is not a detail.
A language model can be wrong and you press retry. A robot foundation model that is wrong has already knocked the glass off the table. Nothing in the physical world ships with an undo button.
The models worth knowing by name
Three reference points cover most conversations. NVIDIA's Isaac GR00T targets humanoids and leans hard on simulation and synthetic motion to manufacture training data at volume. Physical Intelligence's pi0 is a cross-embodiment policy designed to absorb demonstrations from many different robots and generalize across hardware. Gemini Robotics, from Google DeepMind, puts a strong vision-language backbone behind the action head, so broad web knowledge carries into physical tasks. They differ in emphasis, not in kind. All three are VLAs. All three bet that a general policy beats a thousand hand-tuned scripts.
Why data, not architecture, is the hard part
Here is the twist that catches people used to language models. The architectures are largely shared and largely public. You can read the papers. The blocker is not a secret network design. It is the data.
Text scaled because the internet is a byproduct of people communicating; the training set wrote itself. There is no equivalent exhaust for manipulation. Nobody was wearing a sensor rig, logging force and hand pose, while they loaded ten thousand dishwashers. That record has to be produced on purpose. Efforts like Open X-Embodiment pooled robot demonstrations across dozens of labs, and open stacks like Hugging Face's LeRobot lowered the barrier to collecting and sharing more, but the totals are tiny next to what a text model consumes without noticing. The model can only imitate what it was shown. Show it too little of the physical world and it stays a very articulate system that cannot fold the shirt.
So a robot foundation model is not a language model that learned to move. It is a wager that intelligence about the physical world can be trained the way intelligence about text was: one large model, broad data, then adapt. The architecture question is mostly answered. The open one is whether we can gather enough of the right demonstrations to make the wager pay, and that is a problem you solve with sensors and patience, not with a cleverer network.