Generalist Robot Policies

ACT and diffusion policies can produce very impressive dexterous behavior.

They are strong when the setup is fixed: one robot, one task family, one action space, one data distribution.

The limitation is scale. If the robot changes, the task changes, or the environment changes, you often need to retrain or heavily adapt the policy.

The goal of generalist robot policies is to control many embodiments with one model.

In the ideal version, the same policy can work across robot arms, bimanual setups, quadrupeds, navigation robots, drones, and humanoids.

The dream is almost like downloading a robot brain: install it on your robot, give it a task, and get reasonable behavior out of the box or after light adaptation.

NLP and computer vision moved from many task-specific models to large models trained on broad, diverse data.

A language model can learn code completion, translation, question answering, and summarization from one large training mixture.

Robot learning wants the same effect: train on diverse physical experience so one policy can outperform narrow policies and transfer to new tasks.

Robotics has extra problems that language models do not face in the same way.

Robot data is scarce and expensive. Collecting data needs human supervision. Every robot can have different cameras, proprioception, grippers, control frequencies, and action spaces. (Multi modal and Heterogeneous)

The policy must also run in real time, and real-world evaluation becomes expensive as the number of tasks and robots grows.

To train a generalist policy, we need much more than a few demonstrations for one robot.

But teleoperating robots is slow, costly, and physically annoying. Labs often collect small datasets for their own papers, and those datasets are usually not easy to combine.

The first question is therefore not only "what model should we train?" It is "where does the robot experience come from?"

Open X-Embodiment pooled robot datasets from many labs and converted them into a common format.

The lecture frames this as a key enabler: instead of each lab training only on its own small dataset, the community can co-train on a much larger mixture.

This created the first serious sparks of cross-robot transfer: one model trained on many datasets could outperform models tailored to individual setups.

Robot datasets are not automatically compatible.

One dataset may have one camera; another may have wrist cameras. One robot may output Cartesian end-effector actions; another may output joint commands. One task may use language, another goal images.

A shared format makes it possible to train one model across these differences instead of treating every dataset as an isolated island.

A modern robot policy can be viewed like a vision-language model with a different output type.

A VLM does:

A VLA-style robot policy does:

image tokens + text tokens -> text answer

image tokens + language tokens + robot state -> action sequence

The answer is not a sentence. The answer is movement.

Text and images already have mature tokenization and transformer recipes.

Robot actions are harder because they are often continuous vectors: joint deltas, end-effector poses, velocities, and gripper commands.

To reuse transformer machinery, many generalist policy approaches turn actions into tokens or train action heads that translate model representations into continuous control commands.

Octo is presented as an early open-source generalist robot policy.

It is a transformer-based policy trained on a large subset of Open X-Embodiment robot trajectories.

The model can take flexible task specifications, such as language instructions or goal images, and predict action chunks for manipulation tasks.

Octo uses a pretrained language encoder for task text and an observation tokenizer for robot images.

Those tokens are concatenated and passed through a transformer. Special readout tokens summarize the information that later feeds the action heads.

The action heads are diffusion-style decoders, chosen because they handle multimodal action distributions better than a single deterministic prediction.

Octo is trained with goal-conditioned behavior cloning.

Instead of only asking for the next action, the model gets the current observation plus a goal specification and predicts the action sequence that should move toward that goal.

The goal can be expressed as language or as a goal image, which makes the same policy interface useful across different robot-learning setups.

Octo invested most of its capacity in the transformer backbone rather than in heavy separate image encoders.

The idea was that a shared transformer could become the place where cross-dataset and cross-task knowledge accumulates.

This also made fine-tuning attractive: when adding a new sensor or action space, most of the model's learned knowledge stays inside the shared backbone.

The Octo team did not fully align every coordinate frame across datasets, because that would hurt scalability.

But they did align the gripper dimension because many failures were gripper-related.

For example, the model should not see "1" mean open in one dataset and closed in another. If basic action semantics conflict, transfer becomes much harder.

Octo could control some robot setups out of the box, including evaluations on robots whose setting had shifted from the original training data.

But the deeper value of a generalist policy is as an initialization for new robots and tasks.

If a lab adds a force-torque sensor, changes cameras, or switches from Cartesian to joint control, the model should be fine-tunable rather than trained from scratch.

The lecture emphasizes a key result: fine-tuning from a generalist robot policy works better than starting from generic visual representation backbones.

That makes sense because the robot foundation model has already learned something about actions, control, and physical interaction.

It is not merely a good image model. It is a representation shaped by robot behavior.

Octo mainly focused on manipulation-style data.

CrossFormer expands the idea to more embodiments: single-arm manipulation, bimanual manipulation, navigation, locomotion, and aviation-style control.

The important claim is that the same neural network weights can control very different robots, rather than training a separate checkpoint per robot.

Different robots have different sensors, actuators, action dimensions, and control frequencies.

A quadruped, a wheeled robot, a drone, and a bimanual arm do not naturally share the same output shape.

The modeling challenge is to share as much useful knowledge as possible while still respecting embodiment-specific control requirements.

CrossFormer-style models can use a shared transformer backbone with separate action heads for each embodiment family.

The shared backbone learns common perception and behavior representations. The embodiment head maps those representations into the right action format.

The tradeoff is explicit routing: at test time, the user or configuration must choose which head to use.

Another style, used in pi0-like recipes, uses one shared action expert with a unified padded action space.

Every embodiment's action vector is padded to a common maximum size, and the same head predicts actions for all robots.

This is architecturally simpler, but the model must infer the embodiment from cues like camera view, proprioception, and language instruction.

A slow navigation robot and a high-frequency dexterous manipulation robot should not predict the same kind of action chunk.

One robot may operate around 5 Hz; another may need 50 Hz control.

That changes how long the action sequence should be and how much detail the policy must output per inference step.

The lecture highlights a TRI study that rigorously tested whether multitask pretraining improves later fine-tuning.

The answer was yes, especially when the downstream task has little data.

Pretraining made post-training several times more sample-efficient, which is exactly the behavior we want from a robot foundation model.

One practical lesson from the lecture: robot-learning performance often depends on unglamorous details.

Normalization choices can affect downstream success more than architectural changes.

That is part of why robotics is hard. The model, dataset, control interface, action scaling, and evaluation environment all interact.

Robot-only data is tiny compared with web-scale image and language data.

So the next question is whether pretrained vision-language models can transfer useful semantic knowledge into robot control.

The VLA recipe starts from a VLM-like backbone and teaches it to output robot actions instead of only text.

One early VLA approach (llava) is to discretize robot actions into tokens and add those tokens to the LLM vocabulary.

Then the model can train with the same next-token prediction objective:

Models like RT-2 and OpenVLA used simple per-dimension, per-timestep action binning as an early version of this idea.

image + language + history -> next action token

For autoregressive VLAs, the lecture gives a practical rule of thumb: wait until action token accuracy is around 95 percent before attempting serious real-world rollouts.

This metric is computed under teacher forcing, so it does not equal real-world success.

But if action-token prediction is still poor offline, the real robot is unlikely to behave well online.

Action token accuracy asks whether the exact discretized bin was predicted correctly.

But two wrong bins may be very different: one could be next to the correct bin, another could be far away.

So after detokenizing and unnormalizing actions, it is also useful to track continuous L1 or L2 action error against the ground truth trajectory.

If one robot has one camera and another has three, their input token counts differ.

The simple batching strategy is to pad every sample to the longest input shape and mask attention appropriately.

This is easy and reliable, but with highly heterogeneous data, much of the batch can become zeros. That wastes GPU compute.

Sequence packing puts variable-length samples into one long token sequence instead of padding them separately.

This improves throughput because every token in the batch carries real data.

The tradeoff is implementation complexity: you need block-diagonal attention masks so samples do not attend to each other, and you must reset positional encodings at sample boundaries.

Large robot datasets are often streamed from shards into a shuffle buffer.

If the shuffle buffer is too small, batches remain correlated, and the model may generalize poorly.

The lecture notes that increasing randomness in the data pipeline had a large performance impact for Octo, even when the model and data mixture stayed the same.

Another loading strategy is to maintain a flat index and randomly seek into the dataset to build each batch.

This gives more unbiased randomness than a small streaming buffer.

But it requires fast storage and efficient IO, because robot datasets are too large to simply keep in memory.

The lecture summarizes a newer recipe inspired by pi0.5-style systems.

First, train the VLA backbone with next-token prediction on a mixture of FAST-tokenized robot actions and web data.

The web data helps preserve the vision-language backbone's semantic abilities. FAST tokens give a compact action representation for learning robot behavior.

Then train an action expert with a flow-matching head to output continuous actions.

CrossFormer Style - User specifies which head to use as part of the prompt

Pi style - Model infers embodiment from obs, proprio and task.

In the newer VLA recipe, the action expert reads from the VLA backbone but does not freely update it.

A stop-gradient-style separation insulates the backbone so robot-action training does not destroy the pretrained semantic knowledge.

This helps avoid catastrophic forgetting while still letting the model learn to act.

Once policies are general, evaluating them becomes exhausting.

A small camera shift, hardware repair, motor replacement, changed proprioception reading, or moved object can invalidate a rollout setup.

Real robot evaluation is also slow. A single rollout can take minutes, and meaningful comparisons require many trials.

SIMPLER was created to evaluate policies trained on real-world data inside simulated environments.

The hope is not that simulation gives exact real-world success rates. The hope is that it gives correlated comparisons.

If policy A ranks above policy B in SIMPLER, it should tend to rank above policy B on the real robot too.

For simulation evaluation to be meaningful, actions must produce similar effects in simulation and reality.

That requires system identification: fitting the simulator/controller so the same action creates similar robot motion.

The visuals must also look close enough to the real setup. SIMPLER used techniques like compositing simulated assets into real backgrounds and projecting real object textures onto simulated objects.

A simulation success rate of 55 percent does not mean the real robot will get exactly 55 percent.

The useful claim is pairwise or ranking correlation.

If simulation says policy A is better than policy B, and that tends to match the real world, then simulation becomes a useful tool for cheaper iteration.

Generalist robot policies need three things working together.

First: aggregate offline robot data into a common format so co-training is possible.

Second: train models that can handle heterogeneous sensors, tasks, embodiments, and action spaces.

Third: build scalable evaluation methods so progress is not limited by slow, fragile real-world rollouts.

The lecture does not say "just train a big transformer and robotics is solved."

It says the foundation-model idea is useful only after adapting it to robot data scarcity, embodiment mismatch, real-time action, control frequencies, action tokenization, data loading, and evaluation.

Generalist robot policies are the foundation-model dream translated into the messy, physical world.