FIND: Find Something You Can’t Do

7 min read

Use the scene left by the last attempt to choose the next useful task, judge its outcome, and train a small correction to a frozen VLA.

TU Darmstadt · September 2026, revised October 2026

The idea to remember

A persistent workspace can become its own curriculum. FIND chooses weak but currently feasible tasks and uses before/after images for supervision, so practice can continue without restoring a fixed initial scene after every rollout.

What improves: A bounded residual actor and critic learn from demonstrations and online experience. The fine-tuned π₀.₅ base policy stays frozen during online RL; the VLM supervises task choice and success.

The architecture

How the improvement loop works

FIND architecture showing scene-conditioned task sampling, paired-image evaluation, frozen VLA actions, critic-gated residual execution, and asynchronous actor-critic training. — Paper Figure 1. The supervisory loop chooses and evaluates tasks; the learned residual handles control adaptation. Original figure from the authors; select it to view at full resolution.

Step 1
Select feasible practice
A VLM reads the current workspace. Mask infeasible tasks from a predefined library and prioritize feasible tasks with lower recent success rates.
Step 2
Execute a gated correction
The frozen VLA proposes an action. A learned residual suggests a bounded adjustment, and the critic applies it only when the predicted value improvement clears a threshold.
Step 3
Judge and continue
Compare pre- and post-execution images for a binary outcome. Use it as a reward and update curriculum statistics. The resulting scene becomes the next starting state.

What the agents do

Scene and outcome supervisor: The VLM identifies feasible tasks and evaluates success from paired observations. It does not write control code or directly produce low-level robot actions.
Curriculum sampler: Tracks a rolling history per task, gives weak tasks larger sampling weights, and normalizes over the feasible subset. A weight floor keeps reliable tasks eligible.
Frozen VLA controller: π₀.₅ supplies language-conditioned base actions from camera views and proprioception. Offline demonstrations fine-tune this model before online learning.
Residual actor and critic: A small actor proposes bounded action corrections. An ensemble critic trains off-policy and gates execution, falling back to the VLA when the residual lacks predicted benefit.

Reset-free practice means following the scene

Opening a drawer makes closing it feasible; putting a cube into a bowl makes taking it out possible. FIND uses a predefined library of eight manipulation tasks and feasibility relations, so the agent chooses among supported behaviors rather than inventing arbitrary new tasks.

Selection combines two questions: what can the current scene support, and where is the policy still weak? After each rollout, both the scene and the recent success history change, producing a new distribution over practice tasks.

The VLM supplies a reward, not the motor policy

Paired observations help the evaluator identify a change caused by the latest attempt. The predicted outcome updates both replay rewards and the task’s recent performance history. Evaluation errors can therefore affect learning and future data allocation simultaneously.

The control backend mixes offline demonstrations with online trajectories. Its residual actor learns bounded corrections to the frozen VLA, while critic gating retains the base action unless the correction is predicted to help. This is policy learning through real interaction rather than code-library improvement.

What the experiments show

55% → 71.9%
Independent human evaluation: Mean across eight tasks and five independent runs. Evaluation uses 20 human-assessed rollouts per task at each checkpoint.; 456 episodes
Representative 6-hour run: Online experience in a persistent workspace with 30 human scene-recovery interventions.; 400 demos
Offline initialization: 50 Meta Quest teleoperation trajectories per task, used for base-policy fine-tuning and the offline replay buffer.

Original FIND graph with online and human-assessed success curves, changing task-sampling probabilities, and per-task practice statistics. — Paper Figure 3. Solid curves show mean online rolling success with ±1 standard deviation across five runs. Dashed curves are independent human evaluations every 20k steps. The sampling panels show one representative run. Original figure from the authors; select it to view at full resolution.

Where humans still contribute

People collect the 400 offline teleoperation demonstrations and define the task library and workspace. The frozen online base policy has already been fine-tuned on this data.
A representative six-hour run needs 30 manual scene recoveries. Online learning receives no human reward labels, while separate human-assessed evaluation measures the actual success rate.

How to read the result

The workspace is not manually restored after every rollout, but the experiment is not intervention-free. Exceptional scene recovery remains necessary.
The reported 71.9% comes from independent human evaluation. Online rolling success uses VLM judgments and is a different measurement; evaluation rollouts are excluded from replay and curriculum statistics.
The eight tasks and their feasibility relations are predefined. This demonstrates autonomous scheduling and residual learning within that space, not open-ended discovery of new robot skills.

Read the paper

Figures and numerical results are attributed to the linked paper. The step cards are explanatory summaries.