RPG: Reconstruct, Practice, Go Real

7 min read

Use offline videos to build a practice curriculum, repair shared robot skills in simulation, and freeze the improved system before physical deployment.

UC Berkeley / Amazon · October 2026

The idea to remember

A robot can improve persistently by changing the code it calls and the instructions it follows. RPG uses privileged simulation runs to diagnose failures, then tests every proposed repair across the task suite before keeping it.

What improves: Reusable symbolic skills and the Runtime Agent’s system prompt. The underlying multimodal model weights stay fixed.

The architecture

How the improvement loop works

RPG pipeline connecting source videos, runtime and privileged rollouts, failure analysis, code revisions, and cross-task validation. — Paper Figure 2. The six roles improve a shared execution system while the runtime uses only deployable observations. Original figure from the authors; select it to view at full resolution.

Step 1
Reconstruct useful practice
The Constructor reads offline videos and text, identifies manipulation capabilities, and builds related MuJoCo tasks with success checks. The scene need not copy the demonstration exactly.
Step 2
Practice and diagnose
Runtime and privileged agents attempt matching initial states. A video analyst compares their traces and source videos; an implementor proposes changes to skills and the system prompt.
Step 3
Validate, merge, go real
Test candidates across all practice tasks, merge eligible edits, and test again. After common hardware calibration, freeze the system and deploy it on the physical YAM.

What the agents do

Constructor: Builds simulation environments, task evaluators, and the initial higher-level skills from offline video trajectories and descriptions.
Runtime Agent: Selects and composes skills using deployable observations and execution history. This is the agent that controls the physical robot.
Privileged Agent: Runs separate episodes with the same model and skills, plus true simulator state such as object poses and velocities. Its successful executions provide diagnostic references.
Video Analyzer: Finds the first observed failure by comparing runtime traces, privileged executions, and available source videos. Recommends a concrete repair without editing the system.
Implementor: Writes new skills, repairs existing ones, and revises the system prompt. Each candidate branches independently from the current shared system.
Merger: Combines eligible candidate revisions, resolves overlapping edits, and submits the merged system to the same regression gate.

The regression gate is the learning rule

Each candidate is compared with the current system on five fixed development seeds per task, across 22 tasks. Mean task success must increase, and no individual task may lose more than one success out of five. The tolerance is therefore a 20 percentage-point drop on any one task, rather than a guarantee of zero regressions.

The merged revision must pass the same gate again. Two individually useful patches can interact badly when combined, so checking only the separate candidates would miss integration failures. Held-out evaluation uses ten different seeds per task after the systems are frozen.

Why the privileged execution helps

If the privileged agent succeeds with the same skills while the runtime agent fails, the traces can expose problems in perception or skill selection. When both fail, the common skill implementation becomes a likely place to investigate. This comparison guides diagnosis; it is not a trained privileged critic.

In the bottle-placement example, the repair validates the full placement path, delays release until the bottle is inside the bin, and checks the final placement. The agent stores a reusable control change instead of merely retrying the original action.

What the experiments show

28.6% → 95.0%
Held-out simulation success: Round 1 to round 15 on 22 tasks × 10 held-out seeds. This is one improvement run, evaluated retrospectively.; 30 / 30
Physical trials: Ten trials each for storing a ball in a drawer, folding a towel, and transferring a bowl. The bowl uses table-supported staging.; 78.2%
Five-round diagnostic ablation: Full RPG versus 51.8% without the Video Analyzer and 50.9% without the Privileged Agent, after five rounds.

Original RPG graph of task success over 15 practice rounds and accepted skill additions, modifications, and prompt revisions. — Paper Figure 4. Panel A evaluates frozen round-end systems on held-out seeds. Panel B counts accepted skill edits; purple squares indicate prompt revisions, not edit counts. Rounds are not matched compute budgets. Original figure from the authors; select it to view at full resolution.

Where humans still contribute

Offline dataset videos and text provide the source material. Humans inspect the generated success checks before practice and handle setup and the decision to stop practice.
Physical deployment includes coordinate calibration and shared hardware adaptation. During the quantitative trials, no human assistance, manual reset, or system revision is allowed.

How to read the result

Thirty successful physical trials cover three tasks, not the full 22-task simulation suite. Additional physical examples are qualitative.
The 28.6% starting point is after the first practice round. The curve is not a compute-matched comparison between methods or independent repeated training runs.
The Privileged Agent and Video Analyzer support simulation practice; they are absent from physical deployment.

Read the paper

Figures and numerical results are attributed to the linked paper. The step cards are explanatory summaries.