World Models

A useful starting point is the baseball example. A fast ball reaches the plate so quickly that a batter cannot wait for vision, think from scratch, and then swing.

The batter succeeds because the brain predicts where the ball will go and starts acting from that prediction.

That is the intuition for world models in robotics: the robot should not only react to the current image. It should simulate likely consequences and act ahead of raw sensory feedback.

A policy directly maps situation and goal to action:

current state + goal → action

A world model predicts dynamics:

current state + action + memory → next state

So the policy is about choosing behavior. The world model is about understanding consequences.

A vision-language-action policy can learn that certain images and instructions correspond to certain actions.

But it may not explicitly know why an action causes a physical outcome. It can imitate without representing contact, friction, object motion, or delayed effects.

A world model is useful because it tries to learn the transition itself: what changes in the world when the robot acts.

A traditional simulator is built by experts. It has explicit equations for rigid bodies, contacts, collisions, friction, and geometry.

That can be fast and precise when the simulator matches the real world. But it struggles when the relevant physics is hard to model, like deformable cloth, biological tissue, messy contacts, or real-world visual variation.

A world model flips the recipe. Instead of hand-coding the physics, it learns a transition distribution from data.

An important distinction: a video model is not automatically a world model for robotics. P(video | text)

A useful control world model must answer counterfactual questions:

That action conditioning is what makes the model useful for planning.

P(sₜ₊₁ | sₜ, aₜ, hₜ)

One early robotics example was Chelsea Finn's action-conditioned video prediction work.

Instead of generating the next image from scratch, the model predicted how pixels should move. It learned a flow field that warps the current image into the next image.

This makes the prediction problem more structured: moving existing pixels is easier than hallucinating an entire future frame from nothing.

Visual foresight uses the learned video model inside Model Predictive Control.

The user can specify a goal, for example by clicking where an object should move. The planner samples possible action sequences, rolls each one forward through the video model, scores the imagined outcomes, and refines the action sequence.

Then the robot executes the first action, observes again, and replans.

Pixel-space world models have a nice property: the robot can collect random interaction data and train from it without human labels. It also generalizes to unseen objects due to the motion objective.

But the approach has real costs. Pixel losses can make uncertain predictions blurry. Long rollouts accumulate errors. Evaluating full future images for many candidate action sequences is also computationally heavy.

That motivates moving the world model into a smaller latent space.

A raw image contains many pixels that may not matter for action: background texture, lighting details, or irrelevant objects.

A latent world model first compresses the observation into a compact code. The dynamics model predicts forward in that latent space, and the controller acts from the latent state plus memory.

The dream is cheaper planning: simulate in a small useful representation instead of full images.

The classic World Models architecture has three parts.

The vision model is a VAE that compresses images into latent variables. The memory model is an RNN that predicts how those latents evolve over time. The controller is a small network that maps latent state and memory to actions.

The design says: make the world model do the representation work, so the controller can stay simple.

The latent reconstructions in early world models were not pixel-perfect. They were blurry and abstract.

That is acceptable if the latent keeps the information needed for action. The controller does not need a beautiful picture; it needs a state representation that predicts what matters for reward and control.

This is a recurring theme: world models are judged by usefulness for decision-making, not only by visual realism.

A single latent frame can say where things are now, but not necessarily where they are going.

Without memory, the agent can lose information about velocity, curvature, momentum, and recent contact. In a car-racing example, removing memory makes the agent quickly fail because it cannot anticipate the road from history.

World models need memory because physical state is often partially observable.

A key failure mode appears during imagined rollouts.

During training, the encoder sees real observations and produces real latent codes. During imagination, the model has no real observations, so it feeds its own predicted latents back into itself.

That creates a distribution mismatch: the model is trained on real latents but tested on its own hallucinated ones. Errors compound and the rollout drifts.

The Recurrent State Space Model separates state into two parts.

hₜ is deterministic memory carried by the recurrent model. It tracks history and is not sampled.

zₜ is a stochastic latent state that captures uncertainty. During training, it is inferred from the real observation. During imagination, it is sampled from a learned prior conditioned on memory.

A KL loss keeps the imagined prior close to the observed posterior, so imagined states stay in regions the model understands.

PlaNet used the world model with CEM planning at every step. That means sampling many action sequences at test time, rolling them out, and repeatedly refining the best ones.

Dreamer asks: why do all that planning at deployment time?

It trains an actor-critic policy inside imagined latent rollouts. The policy learns to internalize the planning process, so at test time it can act directly without an expensive search loop.

DayDreamer applies the Dreamer idea to physical robots.

A striking example is learning quadruped walking from about one hour of real interaction. The world model turns that limited real data into many imagined training rollouts.

This is the appeal: collect a little real experience, learn a dynamics model, and then train much more cheaply in imagination.

The path from early World Models to PlaNet and the Dreamer model family is useful to trace.

PlaNet introduced the RSSM machinery. Dreamer V1 replaced test-time CEM with an actor-critic trained in imagination. Later Dreamer versions improved the latent variables, KL balancing, training stability, and scale.

Dreamer V3 reaching diamond in Minecraft is important because it shows long-horizon behavior can emerge from imagined latent training.

Dreamer V4 adds two important shifts.

First, the dynamics model can be pretrained on unlabeled video before being finetuned with actions. That means the model can use video sources where robot actions are not available.

Second, transformer memory and KV cache replace the small fixed RNN hidden state, giving longer and more expressive temporal context.

Classic latent world models work well when trained for a specific domain: a game, a robot, or a narrow environment.

But their latent spaces are usually tied to that domain. A Minecraft latent, a car-racing latent, and a robot-pushing latent are not automatically shared representations of the general visual world.

If we want broader world models, we need to scale beyond one environment.

The next move is to train on large-scale video.

Instead of learning a compact world model for one robot task, a video model learns broad visual dynamics: how objects move, how people interact, and how scenes change over time.

The hope is that this gives robotics a prior over physical motion before seeing much robot-specific data.

A naive video transformer would tokenize every frame like an image.

That explodes quickly. A short clip at ordinary frame rates can become tens of thousands of patch tokens, even at modest resolution.

So video world models need tokenizers that are compact and temporally aware before the transformer can scale usefully.

Video compression can be described along three axes.

Spatial compression uses encoders, strides, or VQ-style tokenizers to reduce each frame.

Temporal compression merges information across frames, either non-causally with 3D patches or causally for streaming and control settings.

Adaptive compression spends more tokens on complex dynamic scenes and fewer tokens on simple static ones.

In classic action-conditioned world models, actions are inputs:

state + action → future state

But internet video usually has no robot action labels. So newer robotics approaches ask where the action should enter the system.

Some models generate video and action together. Others freeze a video model and learn a smaller module that maps video features to actions.

World Action Models, or WAMs, jointly predict future video and the actions that produce it.

Instead of treating the world model and policy as separate systems, the model learns both at once:

past video + language + proprioception → future video + actions

DreamZero is one example: a single causal diffusion transformer denoises video latents and action tokens together.

Video Action Models, or VAMs, take a different route.

They use a pretrained video backbone to represent visual dynamics, then train a lightweight inverse dynamics model on robot data.

The video model does the heavy lifting: it encodes how the scene is likely to evolve. The action decoder only has to translate that visual plan into low-level robot actions.

A standard VLA often starts from a vision-language model trained on static image-text tasks.

That gives useful semantics: the model knows objects, captions, and instructions. But it does not automatically know contact dynamics, grasp consequences, or how a scene changes after an action.

A video backbone is trained on motion. It brings both semantics and temporal dynamics, so robot finetuning can focus more on control and less on teaching physics from scratch.

In the mimic-video framing, the pretrained video model creates a latent visual plan.

The robot-specific decoder then translates that plan into an action trajectory. Because the plan already contains motion information, the decoder can be much simpler than a full VLA policy learning dynamics from scratch. The video model activations condition a lightweight flow-matching action decoder.

That led to better sample efficiency than Pi0.5-style VLA models.

An oracle experiment makes this clear: if the action decoder is given latent features from the true future frames, action prediction becomes nearly perfect.

That means the decoder is not the main bottleneck. The hard part is predicting the right future visual representation.

So in this paradigm, better video models should directly improve downstream robot performance.

DreamZero generates chunks of future frames and corresponding actions autoregressively.

After the robot executes an action, the real observation can replace the generated frame in the model's KV cache.

That matters because pure autoregressive video generation accumulates visual errors. In closed-loop robotics, the real world corrects the model after each step.

WAMs tightly couple video prediction and action prediction in one model. That can give strong control behavior, but the model has to learn the joint structure of video and action.

VAMs reuse a robot-agnostic video backbone and keep the robot-specific part smaller. That helps cross-embodiment transfer because only the action decoder has to adapt to the robot.

Action-conditioned latent world models still have a different advantage: they can support reinforcement learning in imagination, beyond pure imitation.

The final family asks a sharper question: do we need to reconstruct future pixels?

A JEPA-style model predicts future embeddings instead. It encodes the current frame, predicts the next latent given action, encodes the real next frame, and makes the predicted latent match the real latent.

No decoder. No pixel loss. The model spends its capacity on representation-space prediction rather than visual reconstruction.

If a model only predicts embeddings, there is a dangerous trivial solution: map every frame to the same vector.

Then prediction loss becomes easy, but the representation contains no useful world state.

Different approaches avoid this in different ways: a moving target encoder, a frozen pretrained encoder like DINO, or regularizers that force the latent distribution to stay spread out.

All the approaches answer the same core question:

Given state, action, and memory, what happens next?

They differ in three choices.

First: how is state represented? Pixels, latents, video tokens, or embeddings.

Second: where do actions live? As inputs, outputs, decoder targets, or joint tokens.

Third: what data can the model use? Robot trajectories, random interaction, internet video, offline datasets, or imagined rollouts.

That is the main story: a world model is not just prediction for prediction's sake. It is prediction shaped so the robot can plan, learn, and act more efficiently.