Models & learning
Observation horizon
An observation horizon is the span or number of recent sensor and state observations that a robot policy uses to predict its next action or action sequence. It is distinct from the action horizon, which describes how far into the future the policy's output extends.
Also known as: observation history, history window, observation window
Updated
History supplied to the policy
The observation horizon specifies how much recent evidence conditions one policy decision. It may be expressed as a frame count, a number of control steps, or physical time. The last form is easiest to compare across systems only when sampling rates are also stated.
Diffusion Policy separates its observation horizon from its prediction and action horizons. A policy can read several observations, predict a longer action sequence, and execute only the first part before replanning. These lengths describe different interfaces and should not be merged into one “sequence length”.
A longer horizon can reveal hidden state
Recent observations can expose motion direction, velocity, contact changes, recovery attempts, and which substeps have already happened. A single frame may show an object beside a gripper without showing whether it is approaching, slipping, or already released.
Long-WAM varies real observation history while holding its forecast and action horizons fixed. Its results show that longer history can improve control, but also that access alone is insufficient: the pretrained representation and deployment path determine whether the policy uses that evidence.
Duration, padding, and latency belong together
An observation horizon is not automatically filled with unique information. Early in an episode, missing history may be padded or the initial frame repeated. Overlapping windows may also reuse most of the same source footage. Dataset documentation should report real duration, sampling, padding, and episode boundaries.
Longer windows increase encoding, attention, and cache costs. A horizon that improves offline prediction can still harm a robot if it delays the next action. Evaluate task success, end-to-end latency, and the fraction of real observations together rather than choosing the largest window the model accepts.
Sources
Related terms
Models & learning
Action chunking
Action chunking is a policy design in which one observation is used to predict a sequence of several future robot actions rather than a single next action. The chunk can capture short-term motion structure and reduce the effective number of sequential predictions, while its execution and replanning schedule determine how quickly the robot can react to new observations.
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Hardware & control
Control latency
Control latency is the elapsed time between a relevant event or observation and the physical effect of the resulting robot command. It can include sensor exposure and transport, preprocessing, policy inference, network transfer, controller scheduling, actuator response, and communication back to a human operator. The measured boundary must be stated because each component has a different latency.
Data & collection
Data synchronisation
Data synchronisation is the process of placing sensor, state, action, annotation, and outcome records on a common timeline so samples that describe the same physical instant or transition can be matched. It requires trustworthy timestamps or trigger relationships and an explicit rule for handling streams with different rates, delays, dropped samples, and clock offsets.
Models & learning
World-action model
A world-action model is an emerging, non-standardised name for a robot model that couples prediction of future world states with generation of executable actions, often by adapting a video or world-model backbone as a policy. It differs from an action-conditioned world model that only predicts consequences of supplied actions, and from Barrett’s WAM, the Whole Arm Manipulator.