Models & learning
Behaviour cloning
Behaviour cloning is a form of imitation learning that fits a policy to expert observation–action pairs as a supervised prediction problem. For humanoid robots, the training examples typically align camera or proprioceptive observations with commands recorded during demonstrations, so the learned policy can reproduce similar behaviour without an explicit reward model.
Also known as: behavior cloning, BC
Updated
How behaviour cloning learns
A demonstration supplies observations and the expert actions paired with them. Behaviour cloning trains a classifier, regressor or generative model to predict those actions from the observations. This is the direct state-to-action approach described in the Carnegie Mellon survey of robot learning from demonstration. It does not require the designer to specify a reward function.
For a humanoid, one training step might pair head-camera images and joint positions with target joint angles, end-effector motion or another control command. Timing and coordinate frames matter: a command detached from the observation that caused it is a poor label. The ALOHA study, for example, records camera views and leader–follower robot motion for bimanual imitation learning.
Why deployment is harder than training
Ordinary supervised training sees states visited by the demonstrator. Once deployed, the learned policy influences its next observation. A small error can therefore move the robot into states absent from the dataset, allowing errors to compound. The DAgger paper formalised this distribution-shift problem and proposed collecting expert labels on states visited by the learner.
Demonstrations may also contain several valid responses to a similar observation. Averaging incompatible actions can produce an invalid motion, especially around contact or whole-body balance. Sequence models and generative methods such as Diffusion Policy can represent richer action distributions, but their performance still depends on demonstration coverage, action quality and accurate synchronisation.
Sources
- Argall et al., A Survey of Robot Learning from Demonstration
- Ross, Gordon and Bagnell, A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
- Zhao et al., Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
- Chi et al., Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Related terms
Models & learning
Imitation learning
Imitation learning is a family of methods that learns a policy from examples of expert behaviour rather than specifying every control rule by hand. In robotics, demonstrations pair observations or states with actions, trajectories or inferred objectives. Behaviour cloning is one imitation-learning method; interactive and inverse approaches address different supervision and distribution-shift problems.
Data & collection
Demonstration
A demonstration is a recorded example of how an intended task or behaviour is performed, usually represented as a time-aligned sequence of observations, states and actions. For humanoid robot learning, demonstrations may come from teleoperation, kinaesthetic guidance, motion capture or autonomous experts and provide targets for imitation.
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Data & collection
Trajectory
A trajectory is a time-ordered sequence of states or observations, actions and, where applicable, rewards generated as an agent or robot evolves. A complete episode or policy rollout often yields a trajectory, but the terms are not universally identical: trajectories may be partial, while episodes have dataset- or environment-defined boundaries.