ASPIRE: Agentic /Skills Discovery for Robotics
7 min read
Debug robot programs from detailed execution traces, validate the repair, and preserve the lesson as a skill that future coding agents can retrieve.
NVIDIA · June 2026
The idea to remember
The lasting output of debugging is a reusable explanation of what failed, when the repair applies, and how to implement it. ASPIRE compounds those validated lessons across tasks and can transfer them from simulation to real robot programming.
What improves: Executable control programs and a retrieved skill library. The coding model stays frozen; improvement happens through program search and accumulated context.
The architecture
How the improvement loop works
Step 1
Execute with observability
A robot execution engine exposes per-primitive camera frames, perception outputs, geometry, action calls, and structured failure traces.
Step 2
Debug and explore programs
Task-specific coding agents diagnose failures and test patches. Evolutionary search explores multiple candidate programs using top survivors and remaining failure traces.
Step 3
Validate and remember
Check repairs on environment variations before the coordinator admits them to the shared library. Later agents retrieve relevant skills as in-context guidance.
What the agents do
- Coordinator
- Launches one coding actor per task, enabling parallel work, and admits validated repair knowledge into the shared skill library.
- Coding actors
- Claude Code agents write, run, inspect, and debug robot programs. They use execution evidence to attribute failure to perception, planning, or control.
- Evolutionary program search
- Generates a population of candidate programs from strong previous programs and residual failures. Execution scores select what survives to the next round.
- Real-robot coding agent
- In the transfer study, Codex uses selected simulation-discovered skills as guidance while adapting programs to a bimanual YAM with different APIs.
A trace turns a failed rollout into a repair target
A task-level failure says very little about why the robot failed. ASPIRE records intermediate tool calls and multimodal evidence, allowing the agent to distinguish a bad object localization from an unreachable approach pose or a failed grasp.
For the navigate-and-pick-up-radio task, repeated planning errors reveal that approach goals fall inside the table’s collision-avoidance buffer. A multi-angle approach routine makes the object reachable. Once validated, this becomes reusable navigation knowledge rather than a task-specific transcript.
The skill is a guarded lesson, not a memorized trajectory
Library entries preserve a failure signature, applicability conditions, and a repair sketch. Future actors retrieve these entries into context and synthesize programs for their own scene, robot, and API. Generalization checks keep a locally successful patch from automatically becoming shared advice.
Evolutionary search changes the control program, not the model weights. It broadens the search beyond a single chain of edits, while the library lets later tasks start with knowledge from earlier debugging.
What the experiments show
- Up to +77 pp
- LIBERO-Pro
- Largest reported gain over the strongest baseline in the perturbation suites. This is a percentage-point improvement, not a relative percentage.
- 31% vs 4%
- Unseen long-horizon tasks
- Zero-shot library transfer to LIBERO-Pro Long, with no additional debugging at test time.
- 13/20 → 19/20
- Real soda-can lifting
- Simulation skill guidance raises held-out success and reduces debugging tokens by nearly an order of magnitude in this task.
Where humans still contribute
- The system starts from robot APIs, environment interfaces, task instructions, and success checks rather than task demonstration trajectories for the skill-discovery loop.
- The physical transfer study uses a prepared YAM station and selected simulation skills. The coding agent then runs and debugs programs without task-specific human guidance during that loop.
How to read the result
- The nearly 10× token reduction is a task-specific real-robot result. The effect on final success varies: bowl placement succeeds in both settings, while drawer manipulation still reaches only 11/20 with skills.
- Simulation experiments use a strong frozen Claude Opus 4.6 coding agent with a large context window. The paper does not establish that smaller models can sustain the same loop.
- Success depends on the execution engine and the reliability of validation. The graph below reports held-out Robosuite results; the +77-point headline comes from LIBERO-Pro.
Figures and numerical results are attributed to the linked paper. The step cards are explanatory summaries.