Models & learning
Diffusion policy
A diffusion policy is a learned robot policy that generates an action or action sequence through iterative denoising conditioned on observations and, where applicable, a task goal. This formulation can represent several plausible behaviours for the same scene and is commonly trained by imitation on time-aligned trajectories of observations and expert actions.
Also known as: action diffusion policy, diffusion-based policy
Updated
How action denoising works
During training, a diffusion policy learns to reverse noise added to expert action sequences while conditioning on observations such as images and robot state. At inference, it begins with a noisy action sample and refines it through several denoising steps. The result is an action chunk that the controller can execute before observing the world again and replanning.
The original Diffusion Policy paper formulated visuomotor control as conditional denoising in robot action space and combined it with receding-horizon control.
Why use a distribution over actions
Demonstrations can contain more than one valid response to the same scene. A humanoid might pass an obstacle on either side or grasp an object with either hand. Averaging those alternatives can produce an action that matches neither. A diffusion model can represent a multimodal action distribution and sample a coherent sequence from it.
The method also handles high-dimensional action chunks, but those properties do not guarantee that sampled motion is safe or dynamically feasible. The evidence in the original work came from manipulation tasks, so whole-body humanoid control requires its own evaluation.
Training and deployment requirements
Training trajectories need synchronised observations and actions, consistent units and a declared control frequency. The chunk length and prediction horizon determine what temporal structure the model learns. Demonstrations should include corrections and varied initial states if the policy is expected to recover from drift.
Iterative denoising also introduces inference work before an action is available. Implementations therefore choose the number of denoising steps, action horizon and replanning rate together. The authors’ official implementation publishes the training and evaluation configuration used for the reported tasks.
Sources
Related terms
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Models & learning
Imitation learning
Imitation learning is a family of methods that learns a policy from examples of expert behaviour rather than specifying every control rule by hand. In robotics, demonstrations pair observations or states with actions, trajectories or inferred objectives. Behaviour cloning is one imitation-learning method; interactive and inverse approaches address different supervision and distribution-shift problems.
Models & learning
Behaviour cloning
Behaviour cloning is a form of imitation learning that fits a policy to expert observation–action pairs as a supervised prediction problem. For humanoid robots, the training examples typically align camera or proprioceptive observations with commands recorded during demonstrations, so the learned policy can reproduce similar behaviour without an explicit reward model.
Data & collection
Trajectory
A trajectory is a time-ordered sequence of states or observations, actions and, where applicable, rewards generated as an agent or robot evolves. A complete episode or policy rollout often yields a trajectory, but the terms are not universally identical: trajectories may be partial, while episodes have dataset- or environment-defined boundaries.