Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models
10 min read
A video-diffusion policy that learns physical dynamics from human experience, then transfers that representation to robot action with very little robot data.
architecture
Two diffusion streams, joined where temporal reasoning is strongest
Each modality gets its own tokens and DiT stack. The shallow action transformer taps the early video layers, then emits continuous action chunks without rolling out a future video.
Language instruction
Encoded as text and cross-attended by video tokens.
Observed frames
The visual context the policy reacts to.
Noised future video zₜ
Present when the video velocity field is trained or sampled.
Proprioception + noised actions aₜ
Robot state is embedded directly into the action stream.
mixture of transformers
Video transformer
Deeper stack · causal video masking
action attends to video in early layers
Action transformer
Shallow stack · bidirectional attention
video velocity field
uᵛⁱᵈ(zₜ; t, c) → z − εᶻ
action velocity field
uᵃᶜᵗ(aₜ; t, c) → a − εᵃ
Future video
A world-modeling target that forces the trunk to learn contact, motion, geometry, and object response.
Future action chunk
Continuous end-effector motion and grasp commands for reactive control.
Fast policy path
At policy inference, actions do not consume or require a generated future video. The shallower action branch stays reactive.
Video self-attention
Causal masking preserves the direction of time.
Action self-attention
Bidirectional across the whole noised action chunk.
Language route
Text conditions video tokens, not action tokens directly.
training data
Turn human experience into two kinds of supervision
Action labels are expensive. Future-video targets are abundant. Dyna-2 trains on both instead of discarding clips whose hand tracks do not clear the action-quality bar.
Egocentric human video
Head-mounted recordings of cooking, tidying, folding, assembling, and other everyday manipulation.
Extract and filter 3D hands
A cleaning, pose-estimation, validation, and filtering pipeline keeps only reliable tracks for action supervision.
Derive pseudo-actions
Wrist poses become end-effector trajectories; thumb-index aperture becomes a continuous grasp signal.
Keep unlabeled video useful
Clips that lack action labels can still contribute to the future-video objective at scale.
labeled human clips
video + pseudo-action
Train both marginal velocity fields.
video-only clips
future video target
Expand physical coverage without action labels.
important control
No human-to-robot alignment or embodiment-specific visual cleanup is used in the scaling study.
flow-matching objective
Corrupt both futures, then learn the straight path back
Video and action share a trunk, but the model fits two separate marginal velocity fields. The video loss can improve the representation without forcing action inference to generate pixels.
sample and corrupt
future video latent
zₜ = t z + (1 − t) εᶻ
future action chunk
aₜ = t a + (1 − t) εᵃ
εᶻ and εᵃ are independent Gaussian noise. The same interpolation time identifies each point on the path.
predict velocities
co-training loss
Lco = ||uvid − (z − εᶻ)||²
+ λ ||uact − (a − εᵃ)||²
Shared trunk
World prediction shapes features used by action prediction.
Separate fields
The action field never takes noised future video as an input.
What the two squared L2 terms supervise
These are errors in velocity space. The video term is not a direct RGB reconstruction loss, and the action term is not L2 weight decay on the model parameters.
MSE versus squared L2
MSE(r) = (1 / N) ||r||²2
They measure the same squared residual. MSE averages over its N elements, while a squared L2 norm sums them. With a fixed output size, that normalization changes the loss scale, not the optimum.
The report does not name a separate MIS-E objective. Its displayed training equation uses squared L2 norms; MSE and L1 appear later as action-prediction evaluation metrics.
what actually scales
World prediction is the transfer mechanism
The controlled ablation holds architecture and action-data scale fixed, then changes only the objective and the availability of extra video.
Action-only
Lact
Action-labeled human clips
Overfits unpredictably and does not transfer reliably.
Joint
Lvid + λ Lact
The same clips supervise video and action
Beats action-only on all 39 robot tasks at every action scale.
Video co-training
Joint + extra Lvid
Additional human clips need no action labels
The only recipe that improves cross-embodiment transfer with scale.
Fixed action data
One sweep holds 50K action hours fixed and scales extra video from 0 to 1K, 10K, and 50K hours. A second holds 250K action hours fixed and adds 0, 250K, or 750K video-only hours.
Monotonic robot generalization
In both sweeps, adding video-only data improves zero-shot prediction on held-out robot data. The improvement is specifically cross-embodiment: the same video scaling does not improve held-out human action metrics.
pre-train, then post-train
Human-video scale survives the trip to a physical robot
Every scale rung uses a nested human-only subset. The robot adaptation recipe is held fixed, so the changing variable is the amount of human experience in pre-training.
human pre-training
1K
10K
100K
1M
Exact-hour, nested subsets preserve source proportions. A fixed, disjoint 100-hour human set measures held-out scaling.
robot post-training
Same recipe at every rung
Fourteen tasks, each with at most ten hours of task-specific robot demonstrations. No human-robot co-training or alignment stage is introduced.
Bottle-cap untwisting uses roughly 10 minutes of demonstrations.
physical evaluation
20%
1K
28%
10K
45%
100K
53%
1M
Mean normalized score across 14 tasks rises from 20% to 53%. The 1M checkpoint is best on 9 of 14 tasks.
39 tasks
zero-shot offline robot suite
39 / 39
joint beats action-only at every scale
3 embodiments
used in physical post-training tests
the core idea
Predicting how the world changes is not an auxiliary flourish. In Dyna-2, it is the training signal that turns abundant human video into a robot representation that keeps improving across the embodiment gap.
The mental model
Dyna-2 is one generative model with two coupled jobs: denoise the future video and denoise the future action chunk. Video and action get separate DiT stacks, but the action stack reads the video representation in the early layers where the backbone concentrates temporal reasoning. This keeps the action path shallow enough for real-time control while letting world prediction shape the shared physical representation.
Pre-training turns first-person human video into a scalable robot learning source. Clean 3D hand tracks supply pseudo-actions for wrist motion and grasp, while clips without reliable hand labels can still teach future-video prediction. Both outputs use flow matching: corrupt a real future toward Gaussian noise, then predict the velocity back to data with an MSE-style squared L2 loss.
The decisive result is not simply that more data helps. Action-only training fails to scale reliably across embodiments. Joint future prediction improves every task in the controlled comparison, and adding more video-only data is the recipe that makes zero-shot robot prediction improve monotonically. After this human-only pre-training, a few hours of task-specific robot demonstrations are enough to adapt the policy to several new robot bodies.
Source