Models & learning
Asynchronous inference
Asynchronous inference is a deployment pattern that overlaps robot action execution with prediction of a later action chunk. A robot client consumes queued actions while a policy process or server evaluates a newer observation, then merges or replaces the remaining queue when the new chunk arrives. The pattern reduces idle inference gaps but does not remove latency, stale observations or safety constraints.
Also known as: async inference, asynchronous policy inference
Updated
Prediction and execution run concurrently
Sequential inference can leave a robot idle after its action queue empties and before the policy returns another chunk. LeRobot’s asynchronous inference design separates a RobotClient from a PolicyServer. The client sends observations, receives action chunks and executes one action per control tick while the server calculates a later chunk.
The server may run on the same machine or on remote accelerated hardware. Network transport, preprocessing and inference all contribute to the time between observation capture and chunk arrival.
Tuning the action queue
actions_per_chunk controls how many commands arrive together. A queue threshold controls how early the client requests another prediction. Requesting too late causes queue starvation; requesting at every step increases compute and bandwidth and can repeatedly merge nearly identical chunks.
The threshold should account for control frequency and a high latency percentile, not only average inference time. If the model consumes more control ticks than a complete chunk contains, queue tuning alone cannot hide the delay.
Safety and freshness
Asynchronous inference keeps the robot moving, which makes local validation more important. Actions should carry timestamps or sequence positions, and the client should reject stale results, enforce joint and velocity limits, handle server failure and define a safe stop.
The latest observation may describe a world that has already changed by the time its action chunk arrives. Asynchronous execution improves throughput and responsiveness relative to waiting, but it does not make an old observation current or a predicted action safe.
Sources
Related terms
Models & learning
Action chunking
Action chunking is a policy design in which one observation is used to predict a sequence of several future robot actions rather than a single next action. The chunk can capture short-term motion structure and reduce the effective number of sequential predictions, while its execution and replanning schedule determine how quickly the robot can react to new observations.
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Models & learning
Vision-language-action model
A vision-language-action model is a robot policy that conditions on visual observations and natural-language instructions to predict executable actions. It adapts a pretrained vision-language model, or a similar multimodal backbone, using robot trajectories so that semantic knowledge is connected to an embodiment’s action space. The action representation may use discrete tokens or continuous values.
Models & learning
Robot foundation model
A robot foundation model is a broadly pretrained model intended to provide a reusable starting point for multiple robot tasks, environments or embodiments. It learns from diverse robotics and sometimes web or human data, then acts directly or is adapted with target-domain data. The term describes a training and reuse strategy, not one fixed architecture.