Models & learning
Residual policy
A residual policy is a learned controller that predicts a corrective action added to the output of an existing base controller or policy. The base supplies nominal behaviour such as balance, tracking, or a hand-designed skill, while the residual adapts errors or task-specific effects; its safety and meaning depend on a shared action space, explicit scaling, and limits on the combined command.
Also known as: residual controller, residual control policy, additive policy correction
Updated
A correction reuses what already works
Residual policy learning writes the final action as a base action plus a learned correction. The original residual policy paper uses the decomposition to improve a conventional controller with reinforcement learning instead of asking the learned policy to reproduce the entire behaviour from scratch.
The base may be a hand-designed controller, planner, motion tracker, or previously trained policy. It can supply stable nominal behaviour, while the residual focuses on effects the base misses: contact variation, model error, a new payload, interaction with another agent, or a task-specific adjustment.
This reuse is not automatic transfer. Base and residual outputs need the same joint ordering, coordinate convention, units, timing, and action semantics. A correction trained for joint-position targets cannot simply be added to torque commands.
Scale and saturation are part of the policy
An additive correction can overwhelm the controller it is meant to preserve. Implementations commonly bound, scale, filter, or otherwise constrain the residual before composing the final command. The logged record should contain the raw residual, applied scale, clipped residual, base action, combined action, and any command rejected by a safety layer.
OmniInteract freezes a 29-dimensional SONIC whole-body action and adds a 29-dimensional partner-dependent residual. Each task expert sees the ego humanoid, its partner, their recent histories, and future paired references. The paper bounds the expert correction with a scale of 0.3; its reported sensitivity study finds that both 0.1 and 0.5 perform worse under the evaluated simulation metrics.
That result is specific to the model, normalisation, robot, and benchmark. It is not a universal residual scale.
A residual does not inherit every guarantee of the base
A stable base controller can become unstable after an unrestricted correction. The combined system must be evaluated under the disturbances, delays, contacts, and actuator limits expected at deployment. A residual also cannot recover information absent from its observation: a partner-aware correction needs a credible partner state, and a contact correction may need force or geometry that the base tracker never observes.
For provenance, store which base revision produced each nominal action and which residual revision changed it. Reporting only the final action hides whether success or failure came from the reused controller, the learned correction, or their composition.
Sources
Related terms
Models & learning
Flow matching
Flow matching is a generative-model training method that learns a time-dependent vector field transporting samples from a simple prior distribution to a data distribution. Training regresses the vector field along chosen probability paths without solving the flow during each update; generation starts from the prior and numerically integrates the learned ordinary differential equation.
Models & learning
Motion tracking
Motion tracking is a control task in which a robot policy follows a supplied reference pose or trajectory over time while maintaining physical stability. Success measures how closely and reliably the robot reproduces that reference; it does not by itself show that the robot can perceive a goal, choose a task, or plan a motion.
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Models & learning
Policy distillation
Policy distillation transfers behaviour from one or more teacher policies into a student policy by training the student to match teacher outputs on sampled states. In robotics it can consolidate specialist controllers into one deployable policy, but the student remains limited by the states it visits and the quality and coverage of its teachers.
Models & learning
Reinforcement learning
Reinforcement learning is a method in which an agent learns a policy by interacting with an environment and optimising cumulative reward. In humanoid robotics, actions change the robot and world, while observations, rewards and episode endings provide experience for improving balance, locomotion or manipulation behaviour.
Hardware & control
Whole-body control
Whole-body control coordinates a humanoid’s full multibody state, actuated joints and environmental contacts to pursue several motion or force objectives while respecting constraints such as balance, joint limits and friction. It is a family of hierarchical or optimisation-based methods, not one algorithm; implementations may output joint positions, accelerations or torques.