Models & learning
World model
A world model is a learned predictive model of how an environment changes, often conditioned on a robot’s current state and candidate actions. It may forecast pixels, latent states, rewards or other task-relevant quantities. A policy or planner can use those predictions to compare possible futures before commanding the physical robot.
Also known as: learned world model, environment dynamics model
Updated
What the model predicts
A world model learns regularities in how observations and states evolve over time. It may generate future images, but photorealistic video is not required. The prediction can instead live in a compressed latent space and include quantities such as reward or task progress. In World Models, a recurrent generative model learned compressed spatial and temporal representations that were supplied to a compact controller.
For robotics, the useful question is usually conditional: what is likely to happen if the robot takes a particular action? This separates a controllable dynamics model from a video predictor that forecasts motion without knowing the robot’s command.
How a world model supports control
A planner can roll out several candidate action sequences inside the model, score their predicted outcomes and execute the first action from the best sequence. It then observes the real result and plans again. PlaNet demonstrated this pattern by learning image-based dynamics and planning online in latent space.
A policy can also be trained against imagined experience. In either case, model errors matter: a plausible-looking future may still contain the wrong contact, balance or object motion.
Data needs for humanoid robotics
Training examples should align each observation and robot state with the action taken and the resulting next observation. Humanoids may need proprioception, contact and base motion alongside video because visually similar poses can have different balance states. Multi-step evaluation should compare predicted rollouts with real trajectories, especially around impacts and recoveries, where small transition errors can accumulate.
Sources
Related terms
Models & learning
World-action model
A world-action model is an emerging, non-standardised name for a robot model that couples prediction of future world states with generation of executable actions, often by adapting a video or world-model backbone as a policy. It differs from an action-conditioned world model that only predicts consequences of supplied actions, and from Barrett’s WAM, the Whole Arm Manipulator.
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Models & learning
Reinforcement learning
Reinforcement learning is a method in which an agent learns a policy by interacting with an environment and optimising cumulative reward. In humanoid robotics, actions change the robot and world, while observations, rewards and episode endings provide experience for improving balance, locomotion or manipulation behaviour.
Data & collection
Trajectory
A trajectory is a time-ordered sequence of states or observations, actions and, where applicable, rewards generated as an agent or robot evolves. A complete episode or policy rollout often yields a trajectory, but the terms are not universally identical: trajectories may be partial, while episodes have dataset- or environment-defined boundaries.