[Paper Notes] FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation
Published:
FTP-1 is a foundation-policy attempt for tactile manipulation. The paper asks whether tactile feedback can be pretrained at scale across many sensor types and then reused for downstream contact-rich manipulation, including sensor setups that were never seen during pretraining.
My read: the important idea is not simply “add touch to a VLA.” FTP-1 is about making heterogeneous tactile sensors look like a shared robot-policy interface. It defines a morphology-aware tactile token space, uses sensor-specific encoders to map raw tactile signals into that space, and trains a separate tactile Transformer expert so tactile knowledge can be reused across sensors without disturbing the vision-language backbone.
Paper Info
The paper is “FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation” by Chengbo Yuan, Zicheng Zhang, Mingjie Zhou, Wendi Chen, Yi Wang, Zhuoyang Liu, Dantong Niu, Shuo Wang, Hui Zhang, Wenkang Zhang, Yingdong Hu, Yuanqing Gong, Wanli Xing, Chuan Wen, Cewu Lu, Kaifeng Zhang, and Yang Gao. It is available as arXiv:2606.13102, with project page ftp1-policy.github.io.
The Problem
Vision-based generalist robot policies can aggregate large heterogeneous datasets because camera observations are relatively easy to standardize. Tactile sensing is different. Across hardware, tactile data may be images, arrays, force-torque states, glove readings, fingertip signals, wrist forces, or hand-specific contact fields. Sensors vary in resolution, placement, morphology, physical response, and data format.
This makes tactile policy learning fragmented. A tactile policy trained for one sensor or hand often has little direct path to another sensor or embodiment. FTP-1 tries to build the tactile analogue of a foundation policy: one pretrained model that absorbs many tactile experiences and becomes a useful initialization for new contact-rich tasks.
Policy Interface
FTP-1 predicts action chunks from language, vision, proprioception, and tactile observations:
[ \hat{A}{t:t+H-1}=\pi\theta(\ell, I_t, s_t, X_t). ]
Here (\ell) is the language instruction, (I_t) is multi-view RGB, (s_t) is proprioception, and (X_t) is tactile input. The action lives in a predefined Unified Action Space (UAS). Following UniDex-style alignment, UAS represents different robots with fixed sparse action slots for left arm, right arm, head pose, and supplementary controls. Dexterous hand joints are mapped through function-aligned slots, so different hands can share action semantics where their functions match.
That action-space design matters because tactile pretraining is only useful if both input and output heterogeneity are handled. FTP-1 pairs a unified tactile input interface with a unified robot-action interface.
Morphology-Aware Tactile Token Space
The core design is Morphology-Aware Tactile Token Space (MTTS). MTTS defines 24 functional-area slots. Slots 0-14 represent in-hand functional regions, slots 15-20 represent wrist and finger force/torque signals, and slots 21-23 are reserved. For a parallel gripper, the two side sensors are mapped to the thumb-tip and index-fingertip slots because those are their functional equivalents.
Each tactile input is grouped by functional area and converted into one token per area. FTP-1 adds a learnable functional-area embedding to each token, shared across sensors, with separate left/right hand embeddings where needed. This is the paper’s key abstraction: a token carries both the sensor reading and the functional contact region of the hand or robot.
This is also what makes cross-sensor transfer plausible. A GelSight fingertip image, a tactile array, or a force state can all contribute to a similar functional-area slot if they correspond to a similar contact role.
Heterogeneous Tactile Encoders
MTTS gives the common target space, but raw tactile observations still have different modalities. FTP-1 uses different encoders depending on the input type.
For image-type tactile inputs, such as GelSight-style signals, the model uses a lightweight sensor-specific ViT followed by a shared pretrained T3 Transformer tactile encoder. The final CLS token becomes the tactile token.
For array-type inputs, the model applies Fourier encoding on the signal dimension, then uses a CNN plus MLP to compress the functional area into one token.
For state-type inputs, such as force-torque vectors, the model applies Fourier encoding and uses an MLP.
All tactile tokens are LayerNorm-normalized, augmented with functional-area embeddings, and projected into the tactile expert dimension. The result is a set of unified tactile tokens with sensor-specific front ends and shared downstream semantics.
Separate Tactile Expert
FTP-1 is built on the (\pi0.5) VLA codebase. It keeps a pretrained vision-language expert and a flow-matching action expert, then adds a separate 300M-parameter tactile Transformer expert. The action expert attends to tactile-expert outputs, while the tactile expert does not attend back into the action expert.
This modularity is important. Prior tactile-augmented VLA methods often inject tactile tokens into the vision-language expert through adapters. FTP-1 argues that this can interfere with pretrained vision-language knowledge and may not provide a reusable tactile module. A separate tactile expert lets the model reuse tactile knowledge across sensors while preserving the vision-language backbone.
The paper also injects proprioception through adaptive RMSNorm instead of a plain proprioceptive token, which improved generalization in their preliminary experiments.
Pretraining Dataset
The FTP-1 dataset aggregates 26 data sources covering 21 tactile sensors and roughly 3,000 hours of tactile manipulation data. The sensors include 7 image-type, 5 array-type, and 9 state-type tactile or force sensors.
The data mix spans human-hand demonstrations, dexterous-hand robot data, gripper robot data, and UMI-style data. After resampling, the mixture is approximately 20% human hand, 30% dexterous hand, and 50% gripper data. The authors also collect Sharpa North-FTP-1, with 4,000 long-horizon dexterous demonstrations.
During pretraining, the vision encoder, tokenizer, vision-language expert, and action expert are initialized from (\pi0.5). The tactile encoder, tactile expert, adaptive RMSNorm proprioception injector, and action projector are trained from scratch. Pretraining uses 48 NVIDIA H20 GPUs for 50k steps with global batch size 768.
Evaluation
FTP-1 is evaluated by distributing checkpoints to independent institutions for downstream finetuning across five hardware setups. The seen-sensor setups include UniVTAC in simulation with GelSight-Mini, Sharpa North with Sharpa DTC, and Sharpa&Dexmate with Sharpa DTC. The unseen-sensor setups are FlexivXense with Xense image tactile sensors and TactileUMI with Contactile array sensors.
The baselines separate three questions.
- (\pi0.5): strong vision-language-action policy without tactile input.
- Tactile-VLA: tactile tokens injected into the VLM expert without a separate tactile expert.
- FTP-(\pi0.5): FTP-1 architecture initialized from (\pi0.5), without large-scale FTP-1 tactile pretraining.
On the UniVTAC simulation benchmark, FTP-1 reaches 66.66% average success, and 59.5% when the two lift tasks are excluded. This is about +17.5 points over the next strongest result under both averages.
On seen real tactile sensors, FTP-1 averages 62.5% across Sharpa North and Sharpa&Dexmate tasks, compared with 45.3% for (\pi0.5), 35.8% for Tactile-VLA, and 41.6% for FTP-(\pi0.5). This is a useful caution: adding tactile input through a weak fusion design can hurt compared with a strong vision-only VLA.
On unseen tactile sensors, FTP-1 is strongest again. It reaches 46.6% average success across Insert Hanoi, Insert USB, and Wipe Board, compared with 15.0% for (\pi0.5), 8.3% for Tactile-VLA, and 15.0% for FTP-(\pi0.5). The paper reports this as a +31.6 point gain over the architecture without FTP-1 tactile pretraining.
Does the Gain Come from Tactile Pretraining?
The paper includes a useful ablation called NTP-1. It pretrains on the same data distribution but removes tactile inputs and tactile-specific architecture during pretraining. During finetuning, the tactile architecture is added back. This tests whether the gains come from just seeing similar robot-task data, or from tactile-branch pretraining itself.
On UniVTAC, NTP-1 improves over FTP-(\pi0.5), suggesting that data distribution does help. But FTP-1 remains clearly better. On FlexivXense, FTP-1 outperforms NTP-1 by +37.5 points, supporting the stronger claim that pretrained tactile knowledge transfers to unseen tactile-sensor setups.
Strengths and Limitations
The strength of FTP-1 is its interface design. MTTS turns sensor-specific touch into morphology-aware functional tokens. Heterogeneous encoders respect sensor modality differences. The tactile expert gives touch its own modeling capacity while allowing the action expert to consume tactile information when needed. This combination is much more concrete than a generic “add tactile input” recipe.
The limitation is also explicit in the paper: FTP-1 mainly focuses on general tactile perception and policy finetuning, and it does not yet solve tactile- or force-based servoing and low-level control. It is still an action-prediction foundation policy, not a force-control system. The authors also note that the tactile pretraining dataset is still limited in scale and diversity, especially compared with vision-language robot data.
Takeaway
FTP-1 is best read as an infrastructure paper for tactile foundation policies. Its main contribution is a shared representation and pretraining recipe that makes heterogeneous tactile data usable across sensors and embodiments.
For contact-rich manipulation, the interesting result is the unseen-sensor transfer. If MTTS plus a shared tactile expert can make tactile pretraining useful even when the downstream sensor is new, then tactile policy learning can move from isolated sensor-specific pipelines toward reusable model-level starting points.