Models & learning
Dataset aggregation
Dataset aggregation is an interactive imitation-learning method that repeatedly runs the current learner, asks an expert to label the states that learner actually visits, and adds those state-action examples to a cumulative training dataset. The canonical algorithm is DAgger; it reduces train-deployment distribution shift but still depends on an expert, a safe collection environment, and faithful labels.
Also known as: DAgger, data aggregation algorithm
Updated
The learner collects the states that need labels
Ordinary behavioural cloning trains on a fixed set of expert demonstrations. At deployment, a small prediction error can move the learner into a state the expert dataset rarely contains. Later predictions then have to operate outside the distribution on which they were trained.
DAgger changes the collection loop. It runs the current policy, records the states produced by that policy, queries an expert for the action that should be taken in each state, aggregates those examples with earlier data, and retrains. Repeating this process makes training data reflect the learner's own state distribution rather than only clean expert trajectories.
The expert can be a person, controller, planner, simulator policy, or other system able to label the visited state. Dataset aggregation describes how examples enter the training set; it does not specify the model architecture or require reinforcement learning.
DAgger is interactive even when the learner is not updated online
The deployed policy may remain fixed during one collection rollout. The process is interactive because later training data depends on states visited by the learner, and the newly aggregated labels affect a later policy revision.
This differs from online robot exploration, where environment outcomes may teach the system without an expert action label. It also differs from ordinary policy distillation on a fixed teacher dataset. DAgger queries the teacher on student-induced states.
In OmniInteract, a shared flow policy controls two simulated humanoids above a frozen motion tracker. At every paired state the student visits, the relevant residual expert labels one corrective action for each humanoid before the simulator advances. Those two examples update the same student. The task identity routes the teacher but is withheld from the student input.
Aggregation moves rather than removes collection risk
DAgger needs a way to visit imperfect-policy states without unacceptable harm. A physical robot may require a simulator, supervisor, shield, recovery controller, emergency stop, or a mixture schedule that limits learner control. Expert labels can also be delayed, inconsistent, or impossible for unsafe states.
A reusable aggregation record should retain the policy revision, rollout seed, observation, learner action, expert label, expert identity and version, intervention, timing, safety event, and dataset round. Mixing labels from different experts or policy versions without provenance makes later failures difficult to diagnose.
Sources
Related terms
Data & collection
Demonstration
A demonstration is a recorded example of how an intended task or behaviour is performed, usually represented as a time-aligned sequence of observations, states and actions. For humanoid robot learning, demonstrations may come from teleoperation, kinaesthetic guidance, motion capture or autonomous experts and provide targets for imitation.
Models & learning
Imitation learning
Imitation learning is a family of methods that learns a policy from examples of expert behaviour rather than specifying every control rule by hand. In robotics, demonstrations pair observations or states with actions, trajectories or inferred objectives. Behaviour cloning is one imitation-learning method; interactive and inverse approaches address different supervision and distribution-shift problems.
Data & collection
Online robot exploration
Online robot exploration is a learning and data-collection process in which an agent acts in a robot environment, observes the resulting state changes and failures, and uses that newly gathered experience to change later decisions or executable artefacts. It differs from offline learning on a fixed dataset and does not by itself imply reinforcement learning, random action selection, or safe autonomous deployment.
Models & learning
Policy
A policy is the decision rule that maps a robot’s current observations or estimated state, and sometimes a task instruction, to an action or probability distribution over actions. It can be hand-designed or learned from demonstrations, rewards or both. In humanoid robotics, its outputs may be joint targets, torques, end-effector changes or higher-level skills.
Models & learning
Policy distillation
Policy distillation transfers behaviour from one or more teacher policies into a student policy by training the student to match teacher outputs on sampled states. In robotics it can consolidate specialist controllers into one deployable policy, but the student remains limited by the states it visits and the quality and coverage of its teachers.
Data & collection
Trajectory
A trajectory is a time-ordered sequence of states or observations, actions and, where applicable, rewards generated as an agent or robot evolves. A complete episode or policy rollout often yields a trajectory, but the terms are not universally identical: trajectories may be partial, while episodes have dataset- or environment-defined boundaries.