RATs: Playful Agentic Robot Learning
7 min read
Let robot agent teams invent useful practice, learn executable skills through play, and reuse the library when downstream tasks finally arrive.
UC Berkeley-led collaboration · June 2026
The idea to remember
Practice can happen before someone supplies the final task. RATs proposes novel but learnable goals, diagnoses attempts, and turns the code that worked into a reusable library without finetuning its language model.
What improves: A callable code skill library and a compact failure memory. The underlying model weights remain fixed.
The architecture
How the improvement loop works
Step 1
Propose a useful challenge
Generate candidate goals from the current scene and memory. Favor novelty and tasks near the competence frontier rather than only easy wins.
Step 2
Plan, execute, diagnose
Write a Code-as-Policy program, verify intermediate steps, localize failures, and retry while preserving the parts that already work.
Step 3
Distill and reuse
Extract reusable skills from success and lessons from failure. Curate the memory, then freeze the library for evaluation on unseen downstream tasks.
What the agents do
- Task Proposer team
- Generates candidate practice tasks and ranks them using novelty and estimated learnability based on skill reliability.
- Execution team
- Planner and policy writer generate programs; verifiers check intermediate progress; a failure diagnoser supports targeted retries.
- Memory-Management team
- Extracts reusable code from successful runs, keeps concise failure lessons, tracks skill reliability, and prunes or merges duplicates.
- Downstream executor
- Retrieves the learned library to solve held-out tasks. Skills can also be plugged into another Code-as-Policy agent as context.
Why practice near a 50% success estimate?
The competence-frontier term is 4r(1 − r), where r summarizes empirical reliability of the skills needed for a candidate task. It peaks at r = 0.5. This is a selection preference for challenges that seem learnable, not a claim that every practiced task empirically succeeds exactly half the time.
Novelty also matters. The proposer uses scene observations, existing skills, and failure memory to seek interactions that expand capabilities rather than repeatedly rehearsing what the library already covers.
Memory stores both capability and caution
A successful episode can yield a callable helper function. A failed episode can yield a missing precondition or correction useful for the next attempt. Keeping both stores lets task proposal and execution use what the robot has learned, even when the latest trial failed.
Periodic curation merges near-duplicate skills and removes redundant lessons. Helpers proposed to fill missing capabilities enter as experimental skills and gain reliability through later use. During evaluation, the learned library is frozen so success reflects transfer from prior play.
What the experiments show
- +20.6 pp
- LIBERO-PRO
- Held-out downstream improvement over CaP-Agent0 after autonomous play.
- +17.0 pp
- MolmoSpaces
- Improvement over CaP-Agent0 on the downstream evaluation in the second play environment.
- +8.9 pp
- Cross-environment transfer
- LIBERO-derived skills improve RoboSuite when plugged into another Code-as-Policy executor. Real-world transfer improves by 8.8 points.
Where humans still contribute
- Play does not require human-specified practice tasks or task demonstration trajectories. The system still starts with environment interfaces and a library of basic robot primitives.
- Researchers supply the environments, hardware setup, and downstream evaluation protocol. The no-demonstration play setting should not be confused with constructing an entire robot stack without human engineering.
How to read the result
- The strongest evidence concerns skills acquired in simulated play environments and reused for held-out tasks. Real-world transfer does not establish unlimited autonomous physical play.
- The 50% target is an estimated competence-frontier preference. Its quality depends on skill reliability estimates and the proposal model.
- The graph below measures library and memory growth, not downstream success. A larger library alone does not establish better performance; the reported benchmark comparisons supply that evidence.
Figures and numerical results are attributed to the linked paper. The step cards are explanatory summaries.