Models & learning
Action space
An action space is the set and representation of commands that an agent or robot policy is allowed to choose. In robotics, actions may be discrete choices or continuous values such as joint targets, motor torques, end-effector pose changes, base velocities, or gripper commands. The action space defines what the policy outputs, not necessarily the motion the hardware ultimately executes.
Also known as: robot action space, action spaces
Updated
The policy can choose only represented actions
In a Markov decision process, the action space specifies the choices available to the agent. Sutton and Barto use actions as the signal through which an agent affects its environment. The set can be discrete, continuous, or structured from several components.
A robot action might be a desired joint position, velocity or torque; a Cartesian change in end-effector pose; a mobile-base command; a binary gripper state; or a higher-level skill such as “grasp”. These choices create different learning problems even when the visible task is identical.
A policy output is not the executed motion
The action normally passes through controllers, safety limits and hardware before the robot moves. A joint-position target may be clipped at a limit, tracked imperfectly by a feedback controller or interrupted after contact. The recorded command, controller output and measured state are therefore different quantities.
The distinction becomes important across embodiments. Open X-Embodiment aligned many datasets to a seven-dimensional end-effector representation, but retained differences such as absolute versus relative commands, coordinate frames and control conventions. The same numerical vector can produce different motion on two robots.
What action data must document
A usable dataset should identify each action field, its units, reference frame, limits, control mode and update rate. It should state whether values are absolute targets, deltas, velocities, forces or torques; whether they were normalised; and how gripper or hand commands are encoded.
The schema should also separate requested actions from measured execution where both are available. Without that information, a model can learn a mapping whose output has the right shape but the wrong physical meaning.
Sources
Related terms
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Models & learning
Markov decision process
A Markov decision process is a mathematical model of sequential decision-making defined by states, actions, transition probabilities and rewards. After an agent chooses an action, the current state and action determine the distribution of the next state and reward. Reinforcement-learning methods use this structure to compare policies by expected cumulative reward.
Hardware & control
Robot embodiment
A robot embodiment is the particular body and sensorimotor interface through which a robot perceives and acts. It includes morphology and kinematics, actuators, end effectors, sensors, physical limits, and the observation and action conventions exposed to a controller or learned policy. Two robots can perform the same task while having different embodiments.
Data & collection
Cross-embodiment data
Cross-embodiment data is robot training data drawn from multiple physical embodiments, such as arms, mobile manipulators, quadrupeds or humanoids with different kinematics, sensors and action spaces. The datasets are aligned or packaged so models can learn jointly from experience produced by different robots, although shared formatting does not make their observations or controls physically equivalent.
Models & learning
Vision-language-action model
A vision-language-action model is a robot policy that conditions on visual observations and natural-language instructions to predict executable actions. It adapts a pretrained vision-language model, or a similar multimodal backbone, using robot trajectories so that semantic knowledge is connected to an embodiment’s action space. The action representation may use discrete tokens or continuous values.