Hardware & control
Sensor fusion
Sensor fusion is the process of combining measurements from multiple sensors or information sources to estimate a robot's state or its environment. A fusion system must account for what each measurement observes, its coordinate frame, timing, uncertainty, and failure modes; merely placing sensor channels in the same record is not sensor fusion.
Also known as: multisensor fusion, multi-sensor fusion
Updated
Fusion estimates a common quantity
A humanoid may combine encoders, an IMU, cameras, force sensors and external tracking. Those devices do not automatically measure the same thing. Encoders describe joint configuration, an IMU measures inertial motion, and a camera constrains motion through observed scene structure. Fusion uses a model to relate such measurements to a common state or environmental quantity.
OpenVINS derives a measurement update by combining a prior state distribution with a noisy measurement model. The update depends on measurement uncertainty and on how the measurement is expected to change with the state. This is more specific than concatenating several inputs for a neural network, which may be multimodal learning without an explicit fused estimate.
Frames and timestamps are part of the measurement
Measurements must refer to compatible times and coordinate frames. A camera pose delayed by one frame can disagree with an IMU measurement even when both sensors are accurate. An incorrect sensor-to-body transform can appear as persistent motion or bias.
The ROS robot_localization documentation makes these assumptions visible through timestamped inputs, coordinate-frame parameters, covariances and selectable state variables. A dataset intended for fusion should retain timestamps, clocks, frame identifiers, calibration transforms, units and uncertainty information rather than only the final fused trajectory.
More sensors do not guarantee a better estimate
Two sensors can share a failure mode, provide correlated evidence or observe the same limited directions. Treating correlated measurements as independent can make an estimator overconfident. A sensor can also degrade under motion blur, magnetic interference, impacts, occlusion or contact.
Evaluation should therefore include ablations and failure cases, not only aggregate accuracy with every sensor enabled. Report which inputs were available, how they were synchronised, whether covariance was measured or tuned, and how missing or rejected measurements were handled. The fused output is an estimate; preserve the raw observations needed to audit it.
Sources
Related terms
Hardware & control
State estimation
State estimation is the process of inferring variables that are not known exactly—such as a robot’s base pose, velocity, contact state or sensor bias—from noisy measurements, control inputs and a model of how the system evolves. An estimator should provide both an estimate and, where possible, a representation of its uncertainty.
Hardware & control
Observability
Observability is the property that a system's internal state can be determined from its measured outputs over time, given the known inputs and the assumed model. If different states can generate indistinguishable observations, those differences are unobservable under that sensing setup.
Data & collection
Sensor calibration
Sensor calibration is the estimation and documentation of parameters that map raw sensor readings into physically meaningful values and known spatial relationships. In robotics it can include scale, bias, distortion, intrinsic camera parameters, sensor-to-sensor or sensor-to-robot transforms, and timing offsets. Calibration does not remove all noise or drift.
Data & collection
Data synchronisation
Data synchronisation is the process of placing sensor, state, action, annotation, and outcome records on a common timeline so samples that describe the same physical instant or transition can be matched. It requires trustworthy timestamps or trigger relationships and an explicit rule for handling streams with different rates, delays, dropped samples, and clock offsets.
Hardware & control
Coordinate frame
A coordinate frame is a defined origin and set of oriented axes used to express positions, orientations, motions, forces, or other spatial quantities. A value has no complete geometric meaning until its frame and convention are known. Transformations relate measurements expressed in frames such as world, robot base, camera, end effector, object, or sensor.
Hardware & control
Proprioception
Proprioception is sensing of a robot’s own internal configuration and motion rather than the external scene. For a humanoid it commonly includes joint positions and velocities, actuator effort or torque, and inertial measurements of body rotation and acceleration. These signals support state estimation and feedback control but do not, by themselves, directly describe nearby objects or terrain.