Why Two Humanoid Trackers Still Miss Each Other

Two capable humanoid motion trackers can reproduce two human poses and still miss the handshake, slide past the shoulder, or pull an embrace apart. The missing information is the relationship between the bodies. A close interaction depends on where their surfaces meet, how the nearby limbs are arranged, and how each robot corrects for the partner once contact changes both motions.

The new OmniInteract study addresses that gap in two stages. It jointly retargets a human-human recording into a paired robot reference, then keeps a frozen single-humanoid tracker and adds a partner-aware residual policy for the interaction. In the authors' simulation benchmark, the resulting shared controller reaches 84.32% success across 60 selected interactions, compared with 63.32% for two independently executed copies of the underlying SONIC tracker. Physical tests on two Unitree G1s are promising but narrower: three contact tasks, one robot model, a controlled mapped space, and 54 listed attempts per method.

That makes OmniInteract useful evidence for a precise claim. Physical coordination is not obtained by synchronising two solo trajectories. The data and controller must represent the pair.

Two solo trajectories do not define an interaction

A joint recording gives both actors a common clock, but timing alone does not say which geometric relationships must survive a change of robot embodiment. Human arms, hands, torsos, and joint limits do not match a G1. Independently retargeting each person can produce two individually plausible robot motions whose palms no longer meet.

The site's analysis of MATE collaboration data explains the first part of this problem: a collaborative episode should retain both agents and the shared scene on one timeline. OmniInteract asks the next question. Once a paired human motion has been recorded, how can it become a paired robot reference that preserves the interaction itself?

The paper evaluates motion retargeting on the full InterHuman and Inter-X motion sets. Its control benchmark then selects 30 motions from each dataset, spanning handshakes, dancing, pushing, embraces, sparring, fencing, linked walking, and other paired behaviours. These are human-human motion records, not robot commands or measured robot contact forces. OmniInteract derives robot references from them before learning execution.

SCAR retargets a relationship, not two bodies separately

Surface Contact-Aware Retargeting, or SCAR, binds sampled points on a human body to corresponding points on a robot body in a canonical pose. Forward kinematics updates the robot points as its joints move. SCAR then solves for both humanoids together rather than fitting one and repairing the other later.

Its objective preserves four things at once:

• each actor's overall pose and surface orientation;
• relative vectors between sampled surface points that are in contact;
• the arrangement of nearby, non-contact surfaces around that interface; and
• joint, speed, floor, collision, temporal, and stance-foot constraints.

The distinction between contact and nearby context is important. Matching a palm to a point on a partner's back does not determine the elbow, shoulder, and torso configuration around it. Conversely, preserving only the broad neighbourhood can leave the palm visibly separated. The paper's ablation reports that removing its contact-interface objective raises contact interaction error on both datasets, while removing the near-field objective raises the corresponding neighbourhood error.

On the authors' retargeting evaluation, SCAR reports the lowest penetration, contact, and near-field errors of the compared methods. Its downstream success rate reaches 75.4% on InterHuman and 72.1% on Inter-X, versus 58.7% and 55.1% for the strongest listed alternatives on that metric. Those are simulation results under the paper's thresholds, not proof that every generated reference is safe or executable.

The appendix makes the boundary unusually explicit. SCAR uses numerical collision and floor tolerances, and its validity flag is an implementation diagnostic rather than a physical-executability certificate. It produces coupled kinematic references. The controller still has to sustain them under gravity, impacts, estimation error, and the partner's response.

Partner-dependent corrections sit above a frozen tracker

OmniInteract does not ask one new policy to rediscover walking, balance, reference tracking, and interaction at once. It freezes SONIC, a general whole-body motion tracker, and trains an additive correction for each interaction. The base tracker provides the nominal 29-joint action. A task-specific residual expert sees recent self and partner states, their relative geometry, previous actions, and future reference windows, then predicts a bounded 29-dimensional correction.

This decomposition turns a difficult control problem into a narrower one: what should change because another body is here? The correction can move a hand toward a drifting shoulder, change stance under a push, or adjust two relative poses while the frozen tracker continues to supply the underlying motor skill.

The paper's simulation comparison separates that effect from model capacity. A peer-aware controller trained from scratch reports 68.49% success on the 60-motion benchmark. Distilling residual specialists into a deterministic MLP reaches 80.68%. OmniInteract's flow-based residual controller reports 84.32%, with the lowest tracking and interaction-geometry errors in the table.

These clips are a matched qualitative comparison, not an independent benchmark. They clarify the mechanism: the base motion can look correct while the pairwise contact is wrong.

DAgger labels the states the shared policy creates

One specialist per interaction does not scale to a general controller. OmniInteract consolidates the experts into one shared flow policy using dataset aggregation, commonly called DAgger.

The distinction from ordinary offline distillation is material. The current student runs in the paired simulator, so its own mistakes determine which states appear. At every visited state, the corresponding fixed expert labels the correction that each humanoid should have taken before the simulation advances. Both labels enter the growing dataset. Task identity chooses the teacher, but it is not an input to the student.

This collection loop reduces the mismatch between clean expert trajectories and the states created by an imperfect shared policy. It does not make the system expert-free: the residual expert bank remains the label source during training. It also does not retrain SONIC. Every target lives in the same residual action space above that frozen motor prior.

The shared policy uses flow matching to model those expert corrections. At deployment, it starts from noise and integrates a learned vector field into one residual for each humanoid. Both robots share the same weights, but each evaluates from its own ego perspective with the partner's state included.

The three evaluation sets answer different questions

The paper's results combine three scopes that should remain separate in a dataset or model card.

On the unseen set, OmniInteract reports 72.89% success, compared with 68.20% for the residual MLP and 62.65% for independent SONIC. “Unseen” therefore means held out from residual-expert training and distillation. It does not mean a new embodiment, sensor stack, human partner, environment, or interaction source.

Success is also a compound threshold. In simulation, a rollout must keep root-relative mean joint-position error below 10 cm, interaction-geometry error below 20%, and avoid a fall or early termination. A single percentage hides which condition failed. Releasing the component errors and termination reasons per rollout would make the result easier to diagnose and reuse.

Two physical G1s establish contact in a mapped space

The hardware evaluation compares independent SONIC tracking with OmniInteract on high-five, hug, and shoulder-hold tasks. The table lists 14, 22, and 18 attempts per method. OmniInteract reports 85.7%, 72.7%, and 77.8% success; SONIC reports 28.6%, 22.7%, and 11.1%. Here success is based on establishing contact at specified keyframes, not completing a productive collaborative task.

The deployment stack matters as much as the headline. Each G1 runs the frozen tracker and residual policy onboard. Point-LIO combines LiDAR and IMU data at 10 Hz; a pre-built point-cloud map supplies a common frame; GICP and a Kalman filter refine the poses; and the robots exchange estimated pose and peer state through LCM. Their paired references advance with a common motion phase. The learned policy runs at 50 Hz, while the low-level interface publishes joint targets and gains at 500 Hz.

This is qualitative evidence of closed-loop correction under one disturbance, not a measured disturbance-rejection envelope. The paper does not report force magnitude, direction distribution, recovery-time statistics, or safety trials with people in the robots' contact space.

A reusable interaction record has to preserve the pair

For a training-data team, OmniInteract changes the minimum useful record. Two motion files with matching start times are insufficient. A paired interaction example should preserve:

• both source actors in one coordinate system, with body model, scale, surface mesh, timestamps, and contact candidates;
• canonical human-to-robot surface bindings, robot models, joint ordering, collision geometry, and every retargeting objective and tolerance;
• separate source motion, generated robot reference, simulator execution, and physical execution rather than calling all four a demonstration;
• self and partner observations, relative poses, history windows, future reference windows, base actions, residual actions, and final commanded actions;
• expert identity, DAgger round, student revision, flow noise and solver settings, rollout seed, reward components, and termination reason;
• localization, map, synchronization, communication delay, calibration, interventions, contacts, falls, and safety stops on hardware.

That provenance distinguishes a contact captured from human motion from a contact achieved by a robot. It also makes the residual legible: a buyer can see whether success came from the frozen motor prior, the interaction correction, the state estimator, or a favourable reset.

The strongest result is also the narrowest conclusion

OmniInteract shows that paired geometry and partner-conditioned correction can close a measurable gap left by independent humanoid tracking. Its evidence is unusually useful because retargeting, dynamic control, held-out motions, and physical contact are evaluated separately.

The release boundary is less complete. As inspected on 20 October 2026, the project page provides the paper, method figures, result tables, captions, and extensive official videos, but its code field is empty. The hardware evidence covers two 29-DoF Unitree G1 robots in a mapped, controlled setting. The policies exchange partner state and follow synchronised reference phases; they do not infer an unconstrained person's intent, negotiate a task, or establish that close physical interaction is safe around humans.

The practical lesson is not that one controller has solved humanoid collaboration. It is that interaction must be represented before it can be controlled. Preserve the contact interface and its neighbourhood when motion crosses embodiments, keep the partner inside the feedback loop, and retain the labels and failures created by the shared policy itself. Without those links, two excellent solo trajectories remain two solo trajectories.