Models & learning
Tactile pretraining
Tactile pretraining is an initial learning stage that uses touch-related data or objectives to initialise a model before downstream robot-task fine-tuning. Touch may be supplied as a model input, a prediction or representation target, or training-only supervision; the term does not imply that tactile sensors are required at deployment.
Also known as: tactile pre-training, touch pretraining, touch pre-training
Updated
Touch can be an input, a target, or both
Pretraining establishes parameters or representations before a model is adapted to a downstream task. Tactile pretraining makes contact-related data part of that initial stage. The model might encode pressure or deformation directly, predict future touch, align visual features with contact locations, or learn a shared representation across people, robots, and sensor types.
Those choices are not interchangeable. FTP-1 projects heterogeneous tactile sensors into shared morphology-aware tokens and transfers the tactile policy across hardware. Human-Centric Transferable Tactile Pre-Training uses human tactile-action data and unified human-to-robot tactile and action spaces. Both retain touch as an important model signal, but their sensor and transfer assumptions differ.
Training-only touch does not create a tactile sensor
Spark-0 demonstrates a second use. Contact maps derived from tactile gloves supervise visual tokens during pretraining, then the contact head is removed. A downstream policy can therefore run without touch while retaining parameters shaped by tactile-derived objectives. A separate tactile encoder supports deployments that do have online touch.
Calling that vision-only deployment does not mean pressure can be reconstructed perfectly from pixels. It means the deployed observation omits touch. Any gain must be evaluated against a matched model that saw the same visual data and compute without the tactile objectives.
A tactile-pretraining record needs provenance
The term alone says nothing about sensor layout, units, calibration, human-to-robot mapping, or dataset rights. A reproducible record should name the tactile hardware, taxels or image representation, sample rate, synchronization, filtering, missing-sensor treatment, pretraining mixture, objectives, checkpoint, and downstream fine-tuning data.
Evaluation should also separate three effects: generic pretraining versus training from scratch, tactile objectives versus matched vision-only pretraining, and online tactile feedback versus no-touch deployment. Without those comparisons, a reported tactile gain can mix several sources of improvement.
Sources
Related terms
Hardware & control
Tactile sensing
Tactile sensing is the detection and measurement of physical contact properties at a robot's surface or contact interface. Depending on the sensor, it can report pressure or force distribution, contact location, shear, vibration, slip, texture, temperature, or deformation. Tactile data complements vision by measuring interactions that may be hidden at the point of contact.
Models & learning
Transfer learning
Transfer learning uses knowledge learned from one or more source tasks or domains to improve learning or performance on a different target task or domain. In robotics, transfer may reuse representations, model weights, skills, data, or policies across robots, tasks, environments, sensing configurations, or simulation and reality.
Models & learning
Vision-language-action model
A vision-language-action model is a robot policy that conditions on visual observations and natural-language instructions to predict executable actions. It adapts a pretrained vision-language model, or a similar multimodal backbone, using robot trajectories so that semantic knowledge is connected to an embodiment’s action space. The action representation may use discrete tokens or continuous values.
Data & collection
Robot training data
Robot training data is recorded experience used to train, fine-tune, or adapt models for robot perception, prediction, planning, or control. It can include sensor observations, robot state, actions, task instructions, rewards or outcomes, demonstrations, failures, and embodiment metadata. Not every dataset contains every field, but their timing and physical meaning must be clear.
Data & collection
Cross-embodiment data
Cross-embodiment data is robot training data drawn from multiple physical embodiments, such as arms, mobile manipulators, quadrupeds or humanoids with different kinematics, sensors and action spaces. The datasets are aligned or packaged so models can learn jointly from experience produced by different robots, although shared formatting does not make their observations or controls physically equivalent.