Introduction to Robot Learning

Robot learning is a missing piece in the robotics curriculum.

Robotics already has decades of control, planning, perception, and hardware progress. Machine learning has brought foundation models, large-scale data, and generalization.

Robot learning sits at the intersection: it asks how we can use data-driven methods to make physical machines more adaptive, general, and useful.

Robot learning is unusual because it touches both high-level intelligence and low-level hardware.

At one end, we talk about foundation models, multimodal reasoning, sequence modeling, and large datasets.

At the other end, the robot still needs PID control, motor commands, calibration, sensors, and physical interaction.

The field is exciting because the abstract model has to survive contact with motors, friction, gravity, and objects.

Computer vision and NLP moved from many specialized models to broad foundation models.

Robotics is now trying to make a similar transition: from hand-engineered robot systems and task-specific policies toward more general robot foundation models.

The hope is not just better demos. The hope is a robot policy that can generalize across tasks, environments, and hardware.

Robots through fiction and culture: early mechanical characters, industrial automation, service robots, social robots, and companion-like agents.

This history matters because it shows how expectations changed over time.

At first, robots were imagined as machines that automate labor. Later, people started expecting them to collaborate, help, reason, and adapt.

Robots are not only built because they are impressive.

They are useful in dangerous, distant, dirty, or demanding environments: space exploration, deep oceans, mining, search and rescue, disaster response, medicine, and elder care.

The big motivation is simple: robots can extend human capability while keeping people out of harm's way.

Shakey, developed at SRI in the 1960s, is an early landmark.

It sensed the world with cameras and range sensors, planned over a known environment, and acted by moving or pushing objects.

This is the classic loop:

sense → plan → act

A historical note: A-star search was developed in this era for Shakey-style planning.

Industrial robot arms extended the same basic idea into factories.

They can sense or assume a structured world, execute a geometric plan, and repeat the same motions with high precision.

This works extremely well when the environment is controlled. But if a part shifts, an object changes, or the assumptions break, the robot may not adapt.

That brittleness is one reason robot learning is needed.

Traditional automation often behaves like a carefully programmed state machine.

That is powerful when every case is known in advance. But homes, hospitals, kitchens, streets, and disaster zones are full of edge cases.

Robot learning tries to move beyond closed-world assumptions by letting the robot learn from data and experience instead of relying only on hand-coded logic.

Humanoid examples, including DARPA Robotics Challenge-style failures and Boston Dynamics-style demos, make an important point.

Great hardware is not enough. Many impressive motions rely on near-perfect maps, markers, QR codes, or state estimation.

If perception is off by a few centimeters, a robot can fall, miss a contact, or fail the task. Physical intelligence has very little tolerance for wrong assumptions.

Moravec's paradox says that things humans find intellectually hard can be easier for computers, while things toddlers do effortlessly can be brutally hard for AI.

Chess, math, and strategic reasoning can be solved in clean symbolic or digital spaces.

Grasping a mug, using a tool, walking on rough terrain, or opening a door requires perception, contact, balance, timing, and physical adaptation.

That is why robotics remains hard even after huge progress in AI.

Robot learning can be defined as solving robotics with machine learning.

More concretely, it studies algorithms that let robots acquire skills or adapt to unstructured environments from data and experience.

Instead of writing every behavior by hand, we collect demonstrations, interactions, rewards, videos, or robot trajectories and train policies or models from them.

Robotics brings the physical side: perception, control, geometry, dynamics, sensors, and actuators.

Machine learning brings the data-driven side: imitation learning, reinforcement learning, representation learning, sequence modeling, and scalable models.

The bridge is the key: use learning to handle the parts of physical behavior that are too complex to engineer manually.

Consider whether a dishwasher, a surgical robot, and ChatGPT on a phone should count as robots.

The useful distinction is that a robot is not only intelligent and not only mechanical. It must have some physical embodiment and affect the world.

Autonomy also matters. A teleoperated surgical arm changes the world, but the human is still making the decisions. ChatGPT can reason, but without embodiment it does not directly act physically.

A 2007 Stanford PR1 demo shows the robot doing useful home tasks like tidying.

The striking point is that the hardware could already perform many motions, but the robot was teleoperated.

So the missing piece was not only arms, grippers, or motors. The missing piece was the autonomous brain that decides what to do from perception.

Since early home-robot demos, hardware has improved dramatically.

Humanoids and manipulators from companies like Figure, Tesla, and Boston Dynamics are much more capable than earlier research prototypes.

At the same time, GPUs and large-scale AI training have made it possible to train models that were unrealistic in 2007.

This convergence is why the field feels newly alive.

Language models learn from enormous text corpora. Vision models learn from huge image and video datasets.

Robotics does not have that luxury. Robot data is expensive because someone often has to operate, supervise, reset, or maintain a physical robot.

That data bottleneck is one of the biggest differences between training LLMs and training general robot policies.

The long-term goal is a general robot policy.

Instead of one policy for one task on one robot in one environment, we want models that can use large heterogeneous robot datasets and output useful actions across many robots and many tasks.

In the ideal future, you could download a robot brain and get reasonable behavior on a new robot with minimal adaptation.

The Open X-Embodiment dataset is a major step toward shared robot data.

The idea is to collect data from many institutions, robots, tasks, and embodiments into a common format.

This gives the community a way to train and compare more general robot policies, similar in spirit to how ImageNet accelerated computer vision.

A modern robot policy often takes inputs like:

camera image + language instruction → action sequence

For example:

image of a table + "pick up the spoon" → end-effector motions

The policy does not output a caption. It outputs motor-relevant actions that move the robot toward the goal.

If a vision-language model maps an image and instruction to text, a robot policy can be viewed as a similar multimodal model whose answer is an action.

Text VLM:

image + prompt → text answer

Robot VLA:

image + task instruction → robot actions

This analogy is powerful because it lets robotics borrow ideas from vision, language, and sequence modeling.

Modern robot learning can be framed as a sequence problem with different token types.

A model may consume language tokens, image tokens, proprioception, and past actions, then produce future action tokens or action vectors.

This is why transformers and foundation-model ideas are becoming central to robot learning.

However, the simple idea that we can just train a huge transformer on robot data and be done does not hold up.

Robotics is harder for several reasons. Data is scarce. Robots have different sensors and actuators. Policies must run in real time. Hardware evaluation is slow, expensive, and sometimes risky.

So the foundation-model recipe must be adapted to the physical world.

Imitation learning starts from expert data.

The dataset contains observations and expert actions, and the model learns:

observation → expert action

This is similar to supervised learning, but the output is a physical action rather than a label or sentence.

The difficulty is that small action errors can push the robot into states that were not common in the demonstrations.

Reinforcement learning does not only imitate expert actions.

The robot tries actions in an environment and learns a policy that maximizes reward over time.

This is powerful because it can discover strategies, but robotics makes it hard: every action changes the future data distribution, and collecting real-world trials can be slow or expensive.

Classic supervised learning often assumes examples are independent and identically distributed (IID).

Robotics breaks this assumption. If the robot makes a bad action, the next observation changes because of that action.

So the policy is not just predicting on a static dataset. It is creating the future situations it will have to handle.

Comparing older teleoperated or heavily engineered demos with newer autonomous systems from companies like Physical Intelligence, Tesla, and Figure reveals a broader shift.

The underlying shift is the same across manipulation, autonomous driving, and humanoids:

hand-coded rules → learned behavior from data

The big idea is learned physical intelligence: systems that acquire useful behavior from experience rather than only from explicit programming.

Robot learning is interdisciplinary. It forces you to think across mechanics, control, perception, machine learning, software, hardware, data, and evaluation.

It is also broadly useful. The same frameworks can apply to manipulators, self-driving cars, drones, humanoids, assistive robots, and space robots.

This is one of the most exciting times to enter the field because robotics may be approaching its own foundation-model breakthrough.

Hardware is no longer the only bottleneck.

We have better robots, more compute, stronger AI models, and the first serious attempts at large multi-robot datasets.

The missing question is how to turn all of that into policies that act reliably in the real world.

That is what robot learning is trying to solve.