Code as Policies: Language Model Programs for Embodied Control
7 min read
Turn language into executable Python that reads perception, computes geometry, and calls robot controls. Recursively generate helper functions when the program needs them.
Robotics at Google · September 2022, revised May 2023
The idea to remember
The policy can be a program that the language model writes. Python provides the loops, numerical computation, and reusable functions needed to connect an instruction to perception and robot actions.
What improves: Generated policy code and helper functions, with no additional language-model training. This is a foundation for agentic robot programming; it does not add the autonomous practice, failure-driven repair, and cross-task acceptance loops of the later papers.
The architecture
How the improvement loop works
Step 1
Prompt with robot interfaces
Provide available perception and action APIs, naming and type conventions, and a few instruction-to-code examples. The new user instruction is formatted as a comment.
Step 2
Generate and expand the program
The model writes Python that calls existing tools and can reference undefined helper functions. A function-generation LMP recursively implements those helpers and their dependencies.
Step 3
Execute against the robot
Run the generated code in a scope containing the robot APIs and generated functions. Its conditionals and loops can read fresh perception and issue commands during execution.
What the agents do
- Instruction and session LMP
- Interprets user commands, tracks conversational context, and generates policy logic that calls parsing and action APIs.
- Object-parsing LMP
- Maps descriptions such as the right-most object to names or objects using available detection results and geometric calculations.
- Waypoint or position LMP
- Converts spatial descriptions into coordinates or trajectories. NumPy and Shapely make precise geometric computation available to the policy.
- Function-generation LMP
- Defines missing functions recursively so a high-level program can be decomposed into smaller pieces. These are prompted program roles, not independently trained robot policies.
A program can express more than a list of skills
A high-level planner might say “move the object a bit right” and assume that exact skill already exists. Code as Policies can instead read the object position, compute an offset, and pass the new coordinates to a supplied pick-and-place primitive. The model decides which computation to write; Python carries it out.
Generated code can also contain feedback loops. A loop can keep reading an object detector while issuing velocity commands, or implement a simple reactive controller through available control APIs. This gives the model influence over how observations affect actions, beyond choosing a fixed sequence of skill names.
Hierarchical generation fills gaps in the program
The top-level program is allowed to call a helper that has not been defined yet. The function generator synthesizes that helper, finds further missing functions, and repeats until the dependencies are available. Existing and generated functions can then be reused in later program construction.
The four roles above describe the prompt decomposition used in the drawing and tabletop systems. They are language-model programs, or LMPs, rather than a team of agents conducting separate experiments. A generated while-loop can react to sensor feedback without asking the language model to re-plan on every control step.
What the experiments actually measure
RoboCodeGen contains 37 robotics-themed function-generation problems, scored by human-written unit tests. With the larger Codex model, hierarchical generation raises the pass rate from 81% to 95%. This measures generated functions, not successful physical robot trials.
HumanEval is a general coding benchmark. The reported 39.8% is pass@1 with sampled decoding; the hierarchical greedy result is 53.0%, which uses a different decoding protocol. Neither number is a robot task success rate.
Quantitative tabletop manipulation is evaluated in simulation with 50 trials per task. Under jointly unseen attributes and instructions, CaP achieves 80% on long-horizon tasks and 62% on spatial-geometric tasks. The paper also demonstrates real drawing, tabletop manipulation, and mobile manipulation, but does not provide a comparable quantitative physical success evaluation for those open-ended demonstrations.
The connection to the later agentic papers
Code as Policies supplies the representation: executable programs can compose perception, geometric reasoning, and control without retraining a language model. ASPIRE, RATs, and RPG later study how to improve and retain this kind of robot-programming knowledge through experience.
CaP itself assumes the requested command is feasible and has no general mechanism to certify a generated response before execution. Its appendix describes human prompt engineering to repair recurring errors, rather than an automated, regression-gated skill-improvement process.
What the experiments show
- 81% → 95%
- RoboCodeGen pass rate
- Flat versus hierarchical generation with the larger Codex model on 37 robotics-themed function problems, evaluated with unit tests.
- 39.8%
- HumanEval pass@1
- General code generation with hierarchical prompting and sampled decoding. The separate greedy decoding result is 53.0%.
- 80% / 62%
- Jointly unseen simulation tasks
- Long-horizon / spatial-geometric success with unseen attributes and instructions. Each task has 50 trials.
Where humans still contribute
- Humans provide the instruction-to-code few-shot examples, API definitions, and prompt conventions. These are programming examples, not a requirement to collect new robot demonstration trajectories for every command.
- Perception models, motion primitives, robot setup, and controllers already exist. Some supplied components are pretrained, so no additional training of the code-writing model does not mean the full robot stack is learned without data.
- The paper describes manual prompt engineering to correct API-use errors, ambiguous types, and inconsistent examples. Human-written unit tests evaluate the code benchmarks; physical examples are researcher-prepared demonstrations.
How to read the result
- The available perception outputs and control primitives bound what the generated policy can do. Code generation does not supply missing sensing or a motor skill that the interfaces cannot express.
- Commands much longer, more complex, or at a different abstraction level from the examples can fail. The system assumes instructions are feasible and cannot establish correctness before execution.
- Recursive decomposition is not uniformly beneficial for every model or generalization type. Figure 4 includes negative changes for the smaller code-cushman model.
- The paper’s checks on imports, special variables, and exec/eval calls constrain generated code, but they do not establish task correctness or physical safety. The quantitative benchmark scores and qualitative real-robot demonstrations should be read separately.
Figures and numerical results are attributed to the linked paper. The step cards are explanatory summaries.