MolmoAct2: Action Reasoning Models for Real-world Deployment
9 min read
A fully open VLA that turns embodied visual reasoning into fast continuous robot control through an action-aware VLM and a flow-matching expert.
architecture
A spatial VLM thinks; a dedicated expert moves
MolmoAct2 keeps visual-language reasoning in the backbone and hands smooth trajectory generation to a continuous action model.
Context tokens
Images / video
One or more robot camera views
Instruction
Natural-language task
Robot setup
Embodiment + control mode
Current state
256-bin discretized tokens
Molmo2-ER
Molmo2 specialized for pointing, spatial QA, motion, multi-view correspondence, and egocentric reasoning.
Action-aware KV states
Next-token supervision keeps the backbone predicting discrete FAST actions during post-training.
projected + detached
DiT-style action expert
Cross-attends to matching VLM layers and learns a flow field from noisy actions to an executable trajectory.
Gaussian noise → velocity field → action chunk
Up to 30 timesteps × 32 action dimensions during post-training.
No target-token shortcut
The expert sees the task, observations, setup, and state, but its conditioning mask hides the target FAST action span.
Knowledge insulation
Flow loss trains the expert and KV adapters without flowing back into the VLM. Language-model loss keeps training the backbone.
training
Build the perception brain before asking it to move
The staged recipe avoids learning spatial reasoning, a robot token interface, and continuous control all at once.
Embodied reasoning
Molmo2 → Molmo2-ER
Specialize on 3.3M spatial and embodied examples, then rehearse with general VLM and instruction data.
Robot pre-training
Molmo2-ER → MolmoAct2-Pretrain
Add setup, control, state, and FAST action tokens so the VLM becomes an autoregressive robot policy.
Action post-training
Attach the action expert
Connect the DiT expert through per-layer KV conditioning and learn continuous trajectories on mixed robot data.
Embodiment fine-tuning
Adapt to one robot
Tune the same architecture for a target setup such as Franka, SO-100/101, bimanual YAM, or LIBERO.
token bridge
FAST makes continuous motion legible to the VLM
The action tokenizer is a training scaffold. The deployed continuous controller is still the flow-matching expert.
Input
1 second of robot motion
Mixed embodiments, rates, and control spaces
Normalize
1st to 99th percentiles
Pad
Shared 32-D action space
Transform
Frequency-domain coefficients
Quantize + BPE
2,048 action-token vocabulary
State tokenizer
Current robot values use simple normalization and 256 uniform bins. These tokens go in the prompt.
different jobs
FAST action tokenizer
Future motion is compressed into action tokens used as prediction targets. The expert never reads the target span.
MolmoAct2-FAST was trained on one million action sequences spanning absolute joint and delta end-effector control. For the mechanics behind frequency-space tokenization, see the separate FAST note.
molmoact2-think
Spend depth reasoning only where the scene changed
A cached depth buffer preserves static geometry while dynamic regions receive fresh depth-token predictions.
Current frame
Adaptive depth router
Updated depth context
[cached depth] + [new depth]
Geometry-aware KV states
Continuous action expert
The paper reports a 17% latency reduction versus predicting dense depth tokens at every frame.
evidence
Designed for the deployment constraints that usually get skipped
The evaluation spans embodied reasoning, simulation, real-world zero-shot control, and adaptation.
63.8
Molmo2-ER average
Across 13 embodied-reasoning benchmarks
180 ms
Base action call
LIBERO on one NVIDIA H100
87.1%
Zero-shot Franka
Average success across real-world tasks
98.1%
LIBERO with Think
Average success after post-training
The remaining constraint is important: MolmoAct2 executes chunks of 10 to 30 actions before observing again. It cannot react inside a chunk, and out-of-the-box support is limited to heavily trained embodiments.
one-line summary
MolmoAct2 teaches a spatial VLM to speak robot motion with discrete tokens, then lets a flow-matching expert read the VLM's layer-by-layer attention state and turn noise into continuous control.
The mental model
MolmoAct2 starts by making the perception backbone more useful for robotics. Molmo2-ER is trained to reason about pointing, metric space, motion, camera viewpoints, and the agent's own frame of reference. The specialize-then-rehearse recipe adds those embodied skills without discarding the general visual and language ability inherited from Molmo2.
Robot pre-training then turns that spatial VLM into a discrete action policy. Current state values become simple state tokens, while one second of future motion is compressed by the open MolmoAct2-FAST tokenizer. Predicting those action tokens is not the final control interface. It is the scaffold that makes the VLM's internal attention state carry information about what the robot should do next.
Post-training attaches a DiT-style expert for continuous control. Every expert layer cross-attends to the keys and values from its matching VLM layer, so it can use grounded scene and task context while learning to transform noise into a smooth trajectory. The target action tokens are hidden from this path, and the flow loss is insulated from the VLM during post-training. MolmoAct2-Think adds explicit depth reasoning, but only regenerates depth codes for regions that changed since the previous frame.
Sources