Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Also known as: robot policy, control policy
Updated
What a policy controls
Sutton and Barto define a policy as the agent’s way of behaving: a mapping from perceived states to actions, which may be deterministic or stochastic. A robot policy uses the same idea but often receives observations rather than a complete state. These can include camera frames, joint positions, velocities, contact measurements and a task instruction.
The output depends on the control stack. A policy may command joint positions, torques, end-effector changes or a named skill. Lower-level controllers can then turn those outputs into motor commands. The policy therefore need not replace balance, impedance or safety control.
How a policy is learned
A policy can be programmed, optimised from reward, fitted to demonstrations or trained with a mixture of these methods. Imitation learning uses recorded expert behaviour as supervision. Reinforcement learning changes the policy according to outcomes obtained through interaction. In both cases, the deployed policy acts repeatedly in a closed loop as new observations arrive.
RT-1 provides a concrete learned example: it takes a short image history and a language instruction, then predicts discretised arm, base and gripper actions at each step.
Why the training record matters
Policy data must preserve the relationship between observation, instruction, robot state and action. Timestamp errors can make a correct action appear to follow the wrong scene. Action units, reference frames, control frequency and embodiment also need to be known. For a humanoid, useful coverage includes ordinary execution, disturbances and recoveries; otherwise small errors may take the robot into states absent from its demonstrations.
Sources
Related terms
Models & learning
Imitation learning
Imitation learning is a family of methods that learns a policy from examples of expert behaviour rather than specifying every control rule by hand. In robotics, demonstrations pair observations or states with actions, trajectories or inferred objectives. Behaviour cloning is one imitation-learning method; interactive and inverse approaches address different supervision and distribution-shift problems.
Models & learning
Reinforcement learning
Reinforcement learning is a method in which an agent learns a policy by interacting with an environment and optimising cumulative reward. In humanoid robotics, actions change the robot and world, while observations, rewards and episode endings provide experience for improving balance, locomotion or manipulation behaviour.
Data & collection
Trajectory
A trajectory is a time-ordered sequence of states or observations, actions and, where applicable, rewards generated as an agent or robot evolves. A complete episode or policy rollout often yields a trajectory, but the terms are not universally identical: trajectories may be partial, while episodes have dataset- or environment-defined boundaries.
Hardware & control
Proprioception
Proprioception is sensing of a robot’s own internal configuration and motion rather than the external scene. For a humanoid it commonly includes joint positions and velocities, actuator effort or torque, and inertial measurements of body rotation and acceleration. These signals support state estimation and feedback control but do not, by themselves, directly describe nearby objects or terrain.