DreamZero: World Action Models are Zero-shot Policies
9 min read
Learn how the world moves from video, then jointly predict the visual future and the robot actions that produce it.
Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, et al. · arXiv:2602.15922v1 · February 17, 2026. Note added October 22, 2026.
The idea to remember
A video model already contains knowledge about motion. DreamZero turns that knowledge into a policy by learning to generate future video and robot actions together. Its video prediction acts as an implicit visual plan, while the action prediction connects that plan to the robot's motors.
DreamZero is a foundation policy built on a video world model. Predicting future observations is part of the policy itself. Unlike Dreamer's reward-driven learning inside imagined rollouts, DreamZero learns from demonstrations with a joint video-action prediction objective.
From observation to motor commands
Step 1
Observe
Encode camera views, the instruction, and proprioception: the robot's measured joint state. Retain recent visual history as cached attention keys and values, so the next prediction has temporal context.
Step 2
Predict video and actions together
Start from noise and jointly generate the next video chunk and matching motor commands. The predicted visual future guides the actions through their shared transformer representation.
Step 3
Execute and observe again
Run the action chunk while inference works on the next observation. Replace predicted visual history with real camera frames in the cache. This corrects the next prediction's context without updating model weights.
One learned policy, with two coupled predictions
For an instruction such as “untie the shoelace,” the input is the current scene, recent visual context, and measured robot state. DreamZero jointly predicts how the grippers and shoelace should move on camera and a chunk of relative joint-position commands. These outputs come from one model. There is no separate language-model planner or library of hand-coded skills.
The backbone is Wan2.1-I2V-14B-480P, a pretrained 14-billion-parameter image-to-video model. A variational autoencoder (VAE) compresses video into latent representations. Added state and action encoders feed robot information into the diffusion transformer, and an action decoder maps its output to motor commands. Multiple camera views are concatenated into one image.
The model generates video autoregressively: each chunk depends on earlier visual chunks. Cached attention keys and values reuse the computation for that history. After execution, real observations replace predicted history, reducing the accumulation of imagined-world errors. A wrong prediction can still produce a wrong action before the next observation arrives.
What learns, and what stays fixed
Training uses flow matching: mix real future video and actions with noise, then learn the direction from noise back toward the demonstrated data. Teacher forcing supplies clean preceding video chunks while the model learns to denoise the current chunk. This is supervised prediction, rather than reinforcement learning from a task reward.
Robot training updates all diffusion-transformer blocks plus the state encoder, action encoder, and action decoder. The text encoder, image encoder, and VAE stay frozen. AgiBot and DROID-Franka models are pretrained separately. The AgiBot model uses roughly 500 hours of teleoperated data from 22 environments, emphasizing varied behavior over repeated demonstrations of a few tasks.
At deployment, observations update the cache and the controller executes new commands. The weights stay fixed. Learning from additional human video or adapting to a new robot requires another training phase.
Architecture and objective, Section 3.1 · Training setup, Section 4.1
Generalization measured on real robots
62.2% vs. 27.4%
Seen tasks, new environments
Average AgiBot task progress versus the best pretrained VLA baseline, pi0.5. Ten tasks, 80 rollouts per checkpoint. This measures partial completion.
39.5% vs. 16.3%
Unseen AgiBot tasks
Average task progress versus pretrained pi0.5 on ten tasks absent from the AgiBot training data. Eight rollouts per task across four robots.
49% progress
Unseen DROID verbs
Twenty tasks, two rollouts per task. Full success is 22.5%; GR00T N1.6 reaches 31% progress and 12.5% success, while pi0.5 reaches 33% and 7.5%.
Both benchmarks use unseen environments and objects. For unseen tasks, zero-shot describes evaluation after robot-policy pretraining, without demonstrations of the evaluated task for that model. The pretrained VLA baselines may have encountered some evaluated tasks in their earlier cross-embodiment datasets. Free-form gallery videos illustrate additional capabilities, but do not establish an aggregate success rate.
Two different kinds of embodiment transfer
Video-only transfer improves the existing AgiBot policy. The authors collect 72 demonstrations of nine previously unseen tasks from humans or a YAM robot, then co-train with AgiBot data for 10,000 steps. These extra videos use only the video objective; AgiBot data retains both video and action supervision.
Table 2 reports 38.3% task progress before transfer, 54.3% with 12 minutes of human video, and 55.4% with 20 minutes of YAM video. The gains are 16.0 and 17.1 percentage points, respectively. These are demonstrations of the evaluated tasks, so the improved results measure transfer after extra training.
New-robot adaptation changes the policy to control YAM itself. The AgiBot checkpoint is post-trained on 55 action-labeled trajectories spanning 11 tasks, totaling 30 minutes. Figure 12 shows qualitative transfer to novel objects and language instructions. The 30 minutes is adaptation data on top of video and AgiBot pretraining; this experiment does not provide a comparable aggregate success table.
Fast inference has a hardware and accuracy cost
DreamZero-Flash trains actions to become clean while the accompanying video remains noisy, allowing a single denoising step. Parallel computation, caching, compilation, quantization, and asynchronous execution also reduce latency. The reported 150 ms per chunk, approximately 7 Hz, uses two GB200 GPUs. That is the rate of new model predictions; the low-level bimanual controller runs at 30 Hz.
Table 3 shows the tradeoff on table bussing: four-step DreamZero reaches 83% task progress at 350 ms, ordinary one-step DreamZero reaches 52% at 150 ms, and one-step Flash recovers 74% at 150 ms. Flash remains nine percentage points below the four-step model. The paper reports standard errors of 6.1, 10.2, and 10.1 percentage points, respectively.
Human work and remaining limits
People supply teleoperated robot trajectories, language annotations, task-specific demonstrations for post-training, and the human or YAM videos used for transfer. Benchmark task definitions, initial scenes, and partial-completion scoring also require human design. The model's closed-loop feedback is camera and robot-state input, not an autonomous evaluator that retrains the policy after each trial.
Novel-task completion remains uneven, with modest full-success rates. Incorrect generated futures can become incorrect motor behavior. The context is roughly six seconds, memory-essential tasks are not explicitly evaluated, and long-horizon planning and high-precision manipulation remain open problems. The authors leave formal scaling laws and large-scale human-video transfer for future work.