Models & learning
Cross-entropy method
The cross-entropy method is a sampling-based optimisation algorithm that draws candidate solutions from a proposal distribution, keeps an elite set with the best objective values, and updates the proposal towards those samples. Robotics systems use it for trajectory, action, or command search when gradients are unavailable or inconvenient.
Also known as: CEM, cross-entropy optimisation, cross-entropy optimization
Updated
Sample, select, and refit
The cross-entropy method maintains a probability distribution over candidate solutions. One iteration samples a population, evaluates every candidate with an objective or cost, selects the best-performing elite subset, and fits the next proposal distribution to those elites. Repeating the loop concentrates future samples around regions that have scored well.
The method does not require a derivative of the objective. That makes it useful when a candidate must be evaluated through a simulator, a non-differentiable controller, collision checks, or a cost containing discontinuous failures.
Receding-horizon robot planning
Pinneri and colleagues study CEM for real-time model-based planning. A robot can optimise a finite sequence, execute only its first part, observe the next state, and then plan again. This receding-horizon use resembles model-predictive control, although CEM is the numerical search procedure rather than the control problem itself.
The search variables need not be actuator commands. They can be waypoints, pose residuals, latent actions, grasp parameters, or reference corrections. The objective and decoder determine what a sampled vector means physically.
KODA searches for trackable commands
KODA uses CEM twice. Whole-body motion generation searches over sparse pelvis and joint residuals so a tracker follows human-derived head and hand references while remaining feasible in simulation. Controller-specific command generation then searches for SONIC or HEFT inputs that track the fixed canonical robot motion.
The second search illustrates why a shared trajectory is not a shared action. Two controllers can receive different optimised commands while being evaluated against the same intended body motion.
What the result does not guarantee
CEM is stochastic and finite-budget. Its result depends on the proposal family, initial variance, population size, elite count, number of iterations, parameterisation, simulator, and objective. A diagonal Gaussian cannot represent every multimodal or correlated search distribution. Selecting the best sampled candidate also does not prove a global optimum or physical safety.
For reproducible robot data, retain the random seed, search bounds, proposal updates, sample and elite counts, decoder, cost terms, horizon, execution prefix, failure handling, simulator version, and final tracking residuals. A stored “CEM action” without that search context conceals how much optimisation was performed and what the candidates were rewarded for doing.
Sources
Related terms
Hardware & control
Model predictive control
Model predictive control is a feedback-control method that uses a dynamics model to optimise a sequence of future actions, applies the first action or short part of that sequence, and replans from an updated state estimate. In receding-horizon MPC, the planning window moves forward at each update.
Hardware & control
Motion planning
Motion planning is the computation of a feasible path or trajectory that moves a robot from an initial condition toward a goal while satisfying constraints such as collision avoidance, joint limits, contact, balance, or dynamics. The planner searches over possible robot motions; a controller is still needed to execute the result.
Data & collection
Trajectory
A trajectory is a time-ordered sequence of states or observations, actions and, where applicable, rewards generated as an agent or robot evolves. A complete episode or policy rollout often yields a trajectory, but the terms are not universally identical: trajectories may be partial, while episodes have dataset- or environment-defined boundaries.
Models & learning
Action space
An action space is the set and representation of commands that an agent or robot policy is allowed to choose. In robotics, actions may be discrete choices or continuous values such as joint targets, motor torques, end-effector pose changes, base velocities, or gripper commands. The action space defines what the policy outputs, not necessarily the motion the hardware ultimately executes.
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.