Models & learning
Vision-language-action model
A vision-language-action model is a robot policy that conditions on visual observations and natural-language instructions to predict executable actions. It adapts a pretrained vision-language model, or a similar multimodal backbone, using robot trajectories so that semantic knowledge is connected to an embodiment’s action space. The action representation may use discrete tokens or continuous values.
Also known as: VLA, VLA model, vision language action model, vision-language-action policy
Updated
How vision, language and action connect
A VLA receives visual context and a language goal, then produces robot actions rather than only a caption or plan. The RT-2 paper introduced the term for vision-language models co-fine-tuned on web vision-language tasks and robot trajectories. RT-2 represented robot actions as text tokens, allowing one model to process images and instructions and emit actions for closed-loop control.
Tokenisation is one design, not part of the definition. OpenVLA also predicts tokenised actions, while π0 uses flow matching to produce continuous action chunks. What makes the model a VLA is the learned connection from visual and linguistic input to an executable action space.
Why robot trajectories are still required
Web data can teach visual categories and language relationships, but it does not normally specify the control command that a particular robot should issue. Robot trajectories provide aligned images, instructions, states and actions. The same hand movement may require different joint commands on two humanoids, so action units, reference frames and per-embodiment normalisation must remain attached to the data.
Cross-embodiment training can broaden the experience available to a VLA, but a shared dataset does not erase hardware differences. Cameras, joint limits, hands and control rates still affect what the output means.
What a VLA does not replace
A VLA is a policy class, not a complete robot control and safety system. Semantic task understanding does not by itself enforce balance, collision limits or safe contact forces. Humanoid deployments can place fast whole-body or joint controllers beneath the VLA and validate its action outputs before execution. Evaluation should therefore separate instruction understanding from physical task completion and recovery.
Sources
Related terms
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Models & learning
Robot foundation model
A robot foundation model is a broadly pretrained model intended to provide a reusable starting point for multiple robot tasks, environments or embodiments. It learns from diverse robotics and sometimes web or human data, then acts directly or is adapted with target-domain data. The term describes a training and reuse strategy, not one fixed architecture.
Data & collection
Cross-embodiment data
Cross-embodiment data is robot training data drawn from multiple physical embodiments, such as arms, mobile manipulators, quadrupeds or humanoids with different kinematics, sensors and action spaces. The datasets are aligned or packaged so models can learn jointly from experience produced by different robots, although shared formatting does not make their observations or controls physically equivalent.
Models & learning
Imitation learning
Imitation learning is a family of methods that learns a policy from examples of expert behaviour rather than specifying every control rule by hand. In robotics, demonstrations pair observations or states with actions, trajectories or inferred objectives. Behaviour cloning is one imitation-learning method; interactive and inverse approaches address different supervision and distribution-shift problems.