How Many Robot Demonstrations Do You Need to Train a Policy?

There is no universal number of demonstrations needed to train a robot policy. The useful unit is not episodes by themselves. It is the coverage each episode adds across tasks, objects, environments, operators, viewpoints, embodiments, and failure modes.

A narrow policy can learn from a surprisingly small, carefully collected set. A policy expected to handle new objects and rooms needs variation across those dimensions, not merely more repetitions of the same scene. A recent study of more than 40,000 demonstrations and 15,000 real-world rollouts found that environment and object diversity mattered more than the absolute number of demonstrations once each setting had enough examples. Its recommendation of 50 demonstrations per environment-object pair applied to tasks of similar difficulty, not to robot learning in general. Read the study's full protocol and qualifications.

Contents

• The short answer
• Why episode count is a weak planning unit
• What published robot-learning studies actually collected
• How to plan a first collection
• How to tell whether more data is helping
• What a training-ready count should include
• The practical answer

The short answer

Use the first collection to answer a narrower question than “how many episodes do we need?”

• Can the task be learned at all? Collect enough consistent examples to test the recording, conversion, loader, and training loop on the target embodiment.
• Can the policy repeat the task in the collection setting? Add variation in poses, trajectories, speeds, and successful approaches while keeping the evaluation separate.
• Can it generalise to new objects or environments? Spend the next collection effort on new object-environment combinations, not only more repetitions of the original pair.
• Can it recover from mistakes? Preserve failed attempts, interventions, retries, and recoveries, or collect them deliberately if the policy will need them.
• Can it transfer across robots? Count embodiments and action conventions as well as episodes. A million trajectories from another body does not equal a million target-robot demonstrations.

The first pilot does not need a universal episode target. It needs enough valid episodes to expose whether the complete capture-to-loader path works. Set that pilot size from the task's duration, reset cost, and evaluation design, then increase it in coverage-aware batches.

A good stopping rule is therefore empirical: keep a held-out evaluation set, add a defined batch of coverage, and stop when the improvement in the intended evaluation is smaller than the collection and integration cost. The batch size is an experimental choice, not a law of robot learning.

Why episode count is a weak planning unit

An episode is a container for experience. It does not say how much independent information the experience contains.

One hundred demonstrations can mean:

• one operator repeating one pick-and-place motion in one room;
• ten operators working across ten object placements;
• a mixture of successes, failed grasps, recoveries, and interrupted episodes; or
• a generated dataset adapted from a much smaller set of human demonstrations.

Those collections have the same headline count and different coverage. They will not answer the same generalisation question.

The RLDS dataset model makes the structure explicit: an episode is an ordered sequence of steps, and its final step can be marked as last without being terminal. That distinction matters when counting valid training examples. A truncated or interrupted episode may be useful for failure analysis, but it should not silently become another successful demonstration. The robot training data entry explains why episode boundaries, action meaning, timing, and provenance belong in the record.

For each proposed batch, count at least these dimensions:

This is why a dataset card that reports only “10,000 episodes” is incomplete. The number becomes useful once the reader knows what varied and what remained fixed.

What published robot-learning studies actually collected

Published work gives useful reference points, but its counts answer different questions. None is a universal minimum.

The studies also differ in policy architecture, action representation, hardware, evaluation protocol, and task difficulty. Comparing their counts without those qualifications turns examples into misleading benchmarks.

The ALOHA result is especially easy to misread. “Ten minutes” does not mean that ten minutes is enough for every bimanual or dexterous task. It describes a particular low-cost hardware system, a particular collection procedure, six tasks, and the authors' evaluation. The same paper explains why fine manipulation is sensitive to compounding errors and contact precision.

DROID is the opposite kind of reference. Its 76,000 trajectories are valuable partly because the collection moved through many scenes and tasks while keeping the platform consistent. The count is evidence of a broad data operation, not a prescription to gather 76,000 examples for a new pick-and-place task.

How to plan a first collection

A useful collection plan starts from the generalisation claim you want to make.

1. Define the held-out question before recording

Write down what will remain unseen during training. It might be a new object, a new kitchen, a different operator, a new camera placement, or a target robot configuration. If every evaluation example resembles a training episode, more demonstrations can improve the score without proving useful generalisation.

Keep the evaluation set physically and administratively separate from the collection loop. Record its task, object, environment, and success criteria before deciding how much training data to buy or collect.

2. Collect a pipeline pilot, not a fake final dataset

The first batch should exercise the complete path from sensor capture to the training loader. Check that:

• episodes start and end where the operator expects;
• camera, state, action, force, and language streams are time-aligned;
• action labels have defined units, frames, and controller semantics;
• resets, failed attempts, and interruptions have explicit outcomes;
• the exported trajectory can be replayed and traced to its source; and
• the policy can train without silently dropping fields or padding invalid actions.

A small pilot that exposes a timestamp or action-space error is more valuable than a larger batch of corrupted repetitions. The camera and action timing checklist covers the physical meaning of timestamps and the difference between a command time and an executed response.

3. Allocate collection across the variation axes

Once the pipeline works, choose which axes need breadth. If the target is one object in one fixture, repeated successful trajectories may be appropriate. If the target is a household task, alternate environments, object instances, placements, lighting, and operators deliberately.

The Data Scaling Laws study provides a concrete example: for tasks of similar difficulty, its authors recommend 50 demonstrations per environment-object pair after studying how the number of environments, objects, and demonstrations affected generalisation. The phrase “per pair” is the important part. It is not a recommendation for 50 total demonstrations, and it should not be transferred to harder dexterous or whole-body tasks without new evidence.

A collection spreadsheet or manifest should make the intended coverage visible. For every episode, record the task, object instance, environment, operator, embodiment, sensor profile, outcome, and split. That makes it possible to discover that a large dataset is actually concentrated in one room or one operator's motion style.

4. Decide what to do with synthetic expansion

Synthetic or transformed demonstrations can increase coverage without requiring a person to repeat every scenario. MimicGen demonstrates this with more than 50,000 generated demonstrations from 200 human demonstrations, while RoboCasa documents much larger synthetic counts than its human target sets.

Treat generated episodes as derived data. Keep the source episode, transformation parameters, simulator or generator version, and validity checks. A generated count should not be added to a human-demonstration count as though the two had identical provenance or physical fidelity.

5. Add the failures your deployment will actually encounter

A clean success-only set may teach the nominal path while leaving the policy unprepared for a shifted object, a slipped grasp, a blocked view, a balance correction, or a late command. Failure data is not automatically good training data: it needs an outcome label, a clear intervention record, and a decision about whether the policy should imitate, avoid, or recover from it.

For a humanoid, include whole-body context when it affects feasibility. A hand trajectory without torso, foot, balance, or contact state can look complete while omitting the information that kept the robot stable.

How to tell whether more data is helping

Do not decide from training loss or the number of files alone. Compare successive data batches on the same held-out evaluation.

Track at least:

1. Generalisation by the intended axis. Report separate results for seen and unseen objects, environments, operators, and embodiments where relevant.
2. Success and failure composition. A higher average success rate can hide a regression on a safety-critical subtask or a collapse on recovery episodes.
3. Marginal gain per collection cost. Compare the improvement from the next batch with recording time, annotation, storage, conversion, and review cost.
4. Data validity. Count episodes rejected for missing frames, timing gaps, corrupted video, invalid actions, or ambiguous outcomes. A nominal episode count is not a valid-episode count.
5. Duplicate and near-duplicate coverage. Similar trajectories from the same scene may increase optimiser steps without widening the deployment distribution.
6. Transfer evidence. If the goal is a different robot, evaluate on that embodiment rather than treating source-robot validation as proof of transfer.

A learning curve is most useful when the x-axis is meaningful. Plotting performance against total episodes can hide the fact that one batch added new environments while another only added repeats. Keep a second view grouped by the variation dimension that the policy is meant to master.

The Open X-Embodiment project illustrates the cross-embodiment qualification. Its release combines more than one million real-robot trajectories across 22 embodiments and uses a common RLDS-oriented ecosystem, but the robots still have different action spaces and input modalities. The aggregate count supports a research mixture; it does not erase embodiment-specific evaluation.

What a training-ready count should include

Before calling a dataset “10,000 demonstrations”, define the denominator.

Include or report separately:

• recorded episodes and valid episodes;
• successful, failed, truncated, and interrupted episodes;
• human demonstrations and generated or retargeted episodes;
• source episodes and transformed copies;
• episodes used for training, validation, and held-out evaluation;
• episodes with complete video, state, action, force, tactile, and language streams; and
• episodes captured on each robot, hand, camera, controller, and software version.

This is not bureaucracy. A training pipeline can discard failures, skip corrupted videos, filter by task success, or sample only one robot profile. If the published count does not distinguish those subsets, a reader cannot reproduce the training mixture or compare it with another dataset.

The format comparison for LeRobot v3, RLDS, HDF5, and Zarr explains how episode boundaries, metadata, timestamps, and action conventions survive storage and conversion. Format choice cannot create missing coverage, but poor packaging can hide the coverage that was collected.

The practical answer

For a first policy experiment, collect enough consistent demonstrations to validate the full data and training path on the target task and embodiment. Then stop asking whether you have “enough episodes” in the abstract. Ask which object, environment, operator, failure mode, or embodiment is still absent from the claim you want to make.

Use published numbers as reference designs:

• a short, tightly controlled collection can support a narrow proof of concept;
• a robust single-task policy needs variation in the conditions that will change at deployment;
• cross-environment or cross-object generalisation needs coverage across those combinations;
• generated data can multiply a source set but does not remove the need to document its transformations; and
• cross-embodiment pretraining needs explicit action and sensor semantics plus target-robot evaluation.

The right stopping point is where a new, well-defined batch no longer improves the held-out behaviour that matters. If that point cannot be measured, the next investment should be in evaluation and provenance rather than another round of indistinguishable demonstrations.