Data & collection
Online robot exploration
Online robot exploration is a learning and data-collection process in which an agent acts in a robot environment, observes the resulting state changes and failures, and uses that newly gathered experience to change later decisions or executable artefacts. It differs from offline learning on a fixed dataset and does not by itself imply reinforcement learning, random action selection, or safe autonomous deployment.
Also known as: online exploration, robot agent exploration, autonomous robot exploration
Updated
Online describes when experience enters the loop
An online process gathers experience while the learner is interacting with an environment. The next action can therefore depend on an observation, error, or success that did not exist in a fixed dataset at the start of the run.
In RAX, a frozen coding agent explores a robot task, records measurements and failure lessons, changes code and notes in a persistent workspace, and rehearses the resulting program. The model parameters do not update. The process is still online because the robot's new interaction record changes the submitted artefact.
This timing distinguishes online exploration from offline reinforcement learning, which learns from a fixed logged dataset without collecting new environment interactions during training. It also distinguishes the source of experience from the optimisation method: an online system may update a policy, a world model, a search tree, a memory, or executable code.
Exploration is not a synonym for random action
Exploration means gathering information through action. The actions can be random, uncertainty-directed, goal-directed, planned, human-guided, or chosen by an agent debugging a task. The term alone says nothing about the quality of the objective or the safety of trying an action.
It also differs from robot play data. Play can be task-agnostic continuous interaction without a named success condition. Online exploration may pursue a specific instruction under a fixed episode budget and use failures to revise a task solution. One collection can satisfy both descriptions, but neither implies the other.
Interactive imitation learning is another neighbouring case. The original DAgger paper repeatedly runs the current policy, asks an expert to label states that policy visits, and adds those labels to the training set. It is online with respect to state collection even though the supervision comes from an expert. RAX instead gathers robot outcomes and changes programs without requesting expert action labels for the target task.
Useful exploration data includes failed decisions
An exploration dataset should preserve the decision context, not only successful trajectories. That includes observations, robot state, commands, tool responses, timestamps, task and success definitions, resets, code or policy versions, human interventions, rejected commands, safety stops, calibration changes, and the boundary between exploration and held-out evaluation.
The operating envelope is part of the record. Exploration performed behind a motion planner, workspace limit, force threshold, simulator, or human-controlled emergency stop should not be described as evidence that the learner can act safely without those controls.
Sources
Related terms
Hardware & control
Autonomy
Robot autonomy is the ability to perform an intended task from the robot’s current state and sensor information without human intervention during that task. Autonomy is contextual and can differ by function, environment and operating phase; it is not a single permanent capability level for the whole robot.
Data & collection
Demonstration
A demonstration is a recorded example of how an intended task or behaviour is performed, usually represented as a time-aligned sequence of observations, states and actions. For humanoid robot learning, demonstrations may come from teleoperation, kinaesthetic guidance, motion capture or autonomous experts and provide targets for imitation.
Models & learning
Imitation learning
Imitation learning is a family of methods that learns a policy from examples of expert behaviour rather than specifying every control rule by hand. In robotics, demonstrations pair observations or states with actions, trajectories or inferred objectives. Behaviour cloning is one imitation-learning method; interactive and inverse approaches address different supervision and distribution-shift problems.
Models & learning
Offline reinforcement learning
Offline reinforcement learning is reinforcement learning in which a policy is learned from a fixed dataset of previously collected interactions without collecting additional environment interactions during training. The central difficulty is evaluating and improving actions that may be poorly represented or absent in that dataset.
Models & learning
Reinforcement learning
Reinforcement learning is a method in which an agent learns a policy by interacting with an environment and optimising cumulative reward. In humanoid robotics, actions change the robot and world, while observations, rewards and episode endings provide experience for improving balance, locomotion or manipulation behaviour.
Data & collection
Robot play data
Robot play data is interaction experience collected through exploratory control rather than a fixed sequence of labelled task demonstrations. It can preserve observations, actions, state changes, contact, and outcomes without requiring every segment to begin from a standard reset or complete a named task; its usefulness depends on how the learning objective interprets that off-task behaviour.
Data & collection
Trajectory
A trajectory is a time-ordered sequence of states or observations, actions and, where applicable, rewards generated as an agent or robot evolves. A complete episode or policy rollout often yields a trajectory, but the terms are not universally identical: trajectories may be partial, while episodes have dataset- or environment-defined boundaries.