[Paper Notes] ContactBench 1.0: A Benchmark for Everyday Contact-Rich Robotic Manipulation
Published:
TL;DR
A robot can open a familiar door almost perfectly and fail when its knob becomes just 5 mm smaller. ContactBench 1.0 makes this kind of failure measurable through 36 tasks, more than 40 printable assets, and matching simulation models. Its most useful design choice is to vary the mechanics of an interaction, including resistance and required rotation, while controlling what the robot can see.
The real-world study covers six tasks, with tactile-policy comparisons on four of them. Adding tactile input produces no consistent improvement in those original task settings. Yet Reactive Diffusion Policy (RDP) adapts when a visually unchanged door requires more rotation to unlock. These results put the emphasis on whether contact feedback changes the action at the right moment.
Paper and reading version
ContactBench 1.0: A Benchmark for Everyday Contact-Rich Robotic Manipulation, by Jingyi Xiang, Yixuan Wang, Jeannette Bohg, and Wenzhen Yuan, from the University of Illinois Urbana-Champaign and Stanford University. These notes use the 23-page project-hosted PDF, checked on October 26, 2026. The supplied version does not identify a publication venue or version number.
The project website provides asset and task pages; its simulation-code link is still marked “coming soon” at the time of reading. The discussion below separates the simulation system described in the paper from code availability.
Eight skills, with explicit stopping conditions
The benchmark contains 31 short-horizon tasks and five longer tasks that combine several skills. Its eight categories cover grasping thin or small objects, grasping multiple objects, extracting objects from clutter, placing under environmental constraints, in-hand repositioning, extrinsic contact control, articulated-object manipulation, and insertion/alignment. The categories overlap: tightening a screw with an Allen key involves both alignment and maintaining contact between the tool and screw.
That overlap matters for diagnosis. A controller may reach the right pose but lose the contact needed to transmit torque. It may also tighten the screw and then keep repeating the operation. ContactBench specifies when to stop and release: several task definitions require disengaging from the objects and terminating motion after the physical goal is achieved. For door opening, success requires opening the door by at least 5°. These definitions make completion and termination part of the evaluation. §III-A/B and Table II
Reproducing a force response across laboratories
A tight fit between printed parts can create insertion resistance, but small differences in printer tolerances also change that resistance. ContactBench instead uses compliant PLA spring structures to set forces through geometry and material properties. A door handle contains a torsional spring; an outlet uses a compression spring. Assets use standard PLA and M3 screws and nuts.
The authors test three handle variants and two charger–outlet variants printed on a Bambu Lab P1S and a Prusa XL, with the same filament and print settings. Each physical asset is measured ten times. Printed handles require 5.01–9.54 N, compared with 5.45–9.53 N for the tested real handles. Printed outlets require 8.04–29.26 N, within the 5.24–37.18 N range of the tested commercial outlets. Differences between the two printers reach at most 0.5 N for handles and 1.19 N for outlets. This is evidence for the tested mechanisms and fabrication settings; it does not establish reproducibility for every asset or printer. §III-D and Figure 5
The design also gives behavioral generalization a concrete test. Changing a hidden spring or the rotation required to unlock a door can alter the necessary action without providing a clear visual cue. Changing knob diameter alters both geometry and the required grasp. Keeping these variations separate helps identify which feedback the policy uses.
Simulation preserves the mechanism’s resistance
The simulation suite uses Newton with the MuJoCo Warp solver. CAD assemblies become rigid parts connected by joints, limits, and supports. Printed springs become prismatic or revolute spring–damper drives with adjustable stiffness and preload. For the door handle, tuning stiffness $K$ and preload torque $\tau_0$ reproduces the shape and ordering of the measured force profiles. The simulator approximates the spring’s mechanical response without resolving its full material deformation.
Contact-critical geometry, including threads and insertion passages, uses mesh signed-distance fields; simpler regions use collision primitives. Thin geometry requires particular care: the 1.2 mm card uses a maximum SDF resolution of 512 cells. The optional GelSight Mini model converts hydroelastic contact into pressure, indentation, and shear fields, then renders tactile images using Taxim-based optical shading. Keyboard, gamepad, and SpaceMouse teleoperation and task-specific scripted policies support demonstration collection. These scripted executions demonstrate the modeled mechanisms; the learned-policy comparisons below are real-world experiments. §IV and Figures 6–9
What was trained and measured
The robot is a UR5e with a WSG50 parallel-jaw gripper and one RealSense D435i wrist camera. Four tasks use GelSight Mini fingers. The utensil and card tasks use compliant Fin-Ray fingers because the GelSight hardware’s shape and sensing coverage are unsuitable; tactile policies are therefore not evaluated on those two tasks.
The authors collect 100 UMI demonstrations per task, with HTC VIVE motion tracking replacing SLAM; one utensil demonstration is discarded during quality checks. DP and RDP train from scratch, while $\pi_{0.5}$ and FTP-1 are fully fine-tuned per task. They are evaluated as task specialists. RDP receives tactile marker displacements and runs a 12 Hz slow policy with a 24 Hz fast policy; FTP-1 receives tactile images at 24 Hz. These differences matter when interpreting the comparison as evidence about tactile sensing. §V-A and Appendix B, Table B1
Each evaluated method–task pair has 30 rollouts with a one-minute limit and matched initial conditions. Object positions are sampled within a 0.15 × 0.15 m area; Velcro fixes task fixtures to the workspace. The four reported outcomes are success, checkpoint progress, the fraction of episodes exceeding 40 N peak wrist force, and completion time conditioned on success.
| Task | DP | RDP | $\pi_{0.5}$ | FTP-1 |
|---|---|---|---|---|
| Thread nut onto screw | 8/30 | 0/30 | 14/30 | 14/30 |
| Tighten screw with Allen key | 0/30 | 0/30 | 17/30 | 3/30 |
| Plug in charger | 2/30 | 4/30 | 6/30 | 4/30 |
| Turn doorknob to open door | 30/30 | 29/30 | 30/30 | 29/30 |
| Retrieve one utensil from bin | 0/30 | Not evaluated | 16/30 | Not evaluated |
| Pick up card from platform | 14/30 | Not evaluated | 2/30 | Not evaluated |
These are successful episodes, transcribed from Table III. The often-quoted 56.7% is the best method–task result outside door opening, $17/30$ for $\pi_{0.5}$ using the Allen key. It is not a suite-wide average.
Success alone hides substantial differences. For threading the nut, $\pi_{0.5}$ and FTP-1 both succeed 14 times, but 27% and 83% of their episodes, respectively, exceed 40 N. On door opening, DP succeeds 30 times and RDP 29 times, while their force-exceedance rates are 10% and 63%. The threshold is a reported diagnostic, not a universal boundary for safe interaction. Completion times also need their denominator: FTP-1’s 0.84× human time on Allen-key tightening comes from only three successful episodes.
Why touch does not consistently improve the base tasks
The rollout analysis identifies failures at several points between sensing and action. Some collisions occur on the back of a gripper finger, outside the inner fingertip sensing area. A policy cannot use tactile evidence that the sensor never captures. Even when contact information is available, the policies often fail to learn the small search and alignment motions that demonstrators use when the charger occludes its own pins.
There is also a more specific mistake: both tactile policies sometimes stop tightening after the Allen key slips in the grasp. Slip can occur during normal progress or after misalignment, so it does not reliably mean that the screw is tight. Figure 13 contrasts this incidental cue with the large in-plane torque associated with actual tightening. The failure concerns how the policy interprets contact history, as well as what its sensors observe. §V-B/C and Figure 13
The human demonstrations expose another gap. Operators see the same wrist-camera stream, but their hands feel contact across the UMI gripper body. GelSight Mini captures a smaller contact region. Matching the visual viewpoint therefore leaves a difference in the tactile evidence available to teacher and learner.
A hidden rotation change reveals adaptation
All four policies nearly solve the training door, which unlocks after roughly 60° of rotation. On visually indistinguishable doors requiring 72° and 80°, RDP retains 80% and 53.3% success, respectively, across 30 trials per condition. The other methods fall to roughly half success at 72° and near zero at 80°.
The wrist-angle distributions are more informative than success alone: RDP shifts toward larger rotations as the required angle increases. DP, $\pi_{0.5}$, and FTP-1 keep approximately the same distributions. Their occasional successes can come from the existing spread of actions reaching the new threshold. This is the paper’s clearest evidence that a tactile policy can adapt its behavior despite showing no improvement on the base task. It does not isolate which component of RDP causes the advantage. §V-D and Figure 14
Changing knob diameter from 60 to 55 mm defeats all four policies: each records 0/10 successes. Their gripper widths remain around 59 mm, so they fail to establish the smaller grasp. The authors stop at ten trials because failure is immediate and consistent; this experiment does not use the full 30-trial protocol. Contact feedback helps only after the controller takes actions that make useful contact possible.
Reading ContactBench alongside GALATEA
GALATEA reconstructs hand–object motion from generated videos and learns a simulator-grounded tracking controller. ContactBench provides a way to ask what happens when the motion reference is plausible but the interaction mechanics change. A useful follow-up would hold the intended task fixed, vary resistance or the unlocking threshold, and measure whether execution changes while preserving completion and acceptable forces. This is a proposed experiment; the two papers do not report a joint evaluation.
I would use ContactBench first to test that feedback-to-action connection. Its current results cover six tasks on one arm–gripper setup, while the full suite includes skills requiring other embodiments. The assets focus on rigid and articulated objects, and printed layer texture can introduce tactile cues unlike those of manufactured objects. Surface friction is not systematically characterized or varied. For a transfer study, I would measure the force response of the actual printed fixtures and keep success, force, and termination behavior visible as separate outcomes. Discussion, §VI