Goal-Conditioned Policies: One Model, Many Goals

Train a robot to stack a red block on a blue one. Now ask it to stack the blue on the red. If it was trained the old way, one policy for one task, you are out of luck: that is a different task, and you retrain. A goal-conditioned policy treats the goal as an input, not a constant baked into the weights, so the same model can pursue both without retraining.

That single design choice, feeding the goal into the policy, is one of the quiet reasons a single humanoid model can now attempt hundreds of tasks. It also reshapes what your dataset has to contain. A policy that accepts any goal must have seen many goals, each paired with the behavior that reaches it, and getting that pairing right is harder than it sounds.

Goal as input, not assumption

A plain policy maps observation to action. A goal-conditioned policy maps observation and goal to action. The goal is an extra argument the network reads at every step, and the same weights produce different behavior depending on what you ask for.

The payoff is reuse. One set of weights covers a family of tasks that share dynamics but differ in intent. This is the backbone of modern multi-task systems, and it is why foundation models like NVIDIA Isaac GR00T and the generalist policies described on the Physical Intelligence blog are framed as one model serving many instructions, rather than a shelf of single-task controllers.

Contrast that with the alternative. Without goal-conditioning you train one network per task, and nothing learned about stacking blocks helps with pouring water. Shared structure goes to waste. A goal-conditioned model instead amortizes what is common across tasks, the reaching, the grasping, the sense of contact, and spends its capacity only on what differs. That is why a single set of weights can absorb hundreds of instructions without ballooning in size.

How you tell a robot what you want

A goal is abstract until you choose a representation, and the choice carries real consequences.

• Language: "put the mug in the sink." Flexible and human-friendly, but ambiguous, and it needs paired language-action data to ground the words in motion. This is the VLA route taken by systems like Gemini Robotics.
• Goal image: a picture of the desired end state. Precise and unambiguous, but you need that image for every example, and specifying one at test time can be awkward.
• Coordinates or poses: exact target positions. Great for well-defined motion, useless for "tidy the table."

Each representation demands a different label on every trajectory. That labeling burden is the hidden cost of going multi-task, and it is why open corpora like Open X-Embodiment spend real effort standardizing how goals and instructions are recorded across dozens of contributing datasets.

In practice, teams rarely pick just one. A modern system often accepts a language instruction from the human, then grounds it internally into an image or coordinate goal the low-level controller can chase. That layering means the same trajectory may need several goal labels at once, which raises the bar on how richly each demonstration is recorded.

A goal-conditioned policy is only as general as the set of goals in its data. Narrow goals, narrow robot, no matter how large the model.

The data problem hiding in "many goals"

Here is the trap. To train a policy that reaches any goal, you seem to need demonstrations for every goal, which is impossible. The escape is a relabeling trick called hindsight.

Hindsight goal relabeling is simple and powerful. A robot tries to reach goal A and instead ends up at state B. That trajectory is a failure for A, but it is a perfect demonstration of how to reach B. So you relabel it: same actions, new goal, now a success. One failed attempt becomes training data for a goal nobody asked for. This is what lets goal-conditioned learning squeeze many goals out of finite data, and it changes how a dataset should be built.

The idea came out of reinforcement learning, where it is known as hindsight experience replay, and it turns the sparse-reward problem inside out. Instead of a robot failing a thousand times and learning nothing until it stumbles onto the goal, every attempt teaches something, because every attempt reaches a state worth labeling. The consequence for data collection is direct. A trajectory has value even when the demonstrator missed, as long as you recorded where it actually ended up.

Why relabeling changes what you capture

If failed attempts and unplanned end-states are training data, then a capture pipeline that only keeps clean successes is throwing away half its yield. Hindsight rewards messy, exploratory, imperfect data, because every trajectory reaches some state, and every reached state is a valid goal for that trajectory.

This inverts the usual instinct to film only the perfect take. Under hindsight, the clumsy attempt that drifted to the wrong shelf is not a discard, it is a labeled example of reaching that wrong shelf on purpose. The demonstrator's error becomes someone else's target. A pipeline built around that idea captures more usable data per hour than one chasing spotless demonstrations.

Open frameworks make this concrete. The Hugging Face LeRobot stack ships goal-conditioned and multi-task recipes that expect data in a standard episodic format, with goals and outcomes recorded per episode. The lesson is consistent: the format you record in decides whether you can relabel later, or whether that signal is lost.

General in name, or in data

One model, many goals is not a modeling trick alone. It is a bet about data. The policy can only chase goals it has effectively seen, and the cheapest way to see more goals is to stop discarding the states your demonstrators reached by accident. Decide the goal representation early, record outcomes as carefully as intentions, and a finite pile of demonstrations stretches across a much wider set of tasks. Skip that, and a powerful multi-goal architecture ends up trained on a narrow slice of goals, general in name only.