[Paper Notes] ForceBand: Learning Forceful Manipulation with sEMG
Published:
ForceBand adds a missing physical variable to learning from human demonstrations: contact force. A low-cost wristband records surface electromyography (sEMG) and inertial signals, and an EMG2Force model converts them into synchronized per-finger force traces. Human videos can then supervise a robot policy with both motion and force, while the demonstrator keeps bare fingertips during target-task collection.
The paper reports a 10-hour multimodal pretraining dataset, more than 50% lower force-prediction error than vision baselines, and 87% success on real-robot pick–squeeze–place tasks. Its most reusable idea is a sensing decomposition: direct fingertip sensors provide temporary calibration labels; wrist muscle activity supplies force labels at scale; robot-side force sensing closes the execution loop.
Paper Info
“ForceBand: Learning Forceful Manipulation with sEMG” is by Botao He, Zhi Wang, Linna Kuang, Ishaan Ghosh, Jitendra Malik, Cornelia Fermuller, Tingfan Wu, Jiayuan Mao, Ruoshi Liu, Haozhi Qi, and Yiannis Aloimonos, from Amazon FAR, the University of Maryland, and Johns Hopkins University. The paper is an arXiv preprint, arXiv:2606.26093, released in June 2026. The project page provides videos and links to the open-source code and bill of materials; the dataset is marked as forthcoming at the time of writing.
The Missing Channel in Human Demonstrations
Human videos and motion-capture trajectories preserve appearance and kinematics. Force remains ambiguous: identical-looking grasps can hold a rigid bottle lightly, prevent a heavy container from slipping, or squeeze a deformable tube. Occlusion makes visual force estimation especially unreliable for the ring and little fingers. Instrumented tactile gloves provide direct measurements, although fingertip sensors and wiring can obscure the hand and interfere with natural interaction.
ForceBand moves the measurement site from fingertips to the wrist. Forearm muscle activation precedes and accompanies force production, so sEMG offers an unobtrusive force proxy. The resulting pipeline has three linked learning problems:
flowchart TD
A["Paired calibration data<br/>sEMG + IMU + fingertip force"] --> B["EMG2Force pretraining<br/>and user calibration"]
B --> C["Natural human demonstrations<br/>ForceBand + video"]
C --> D["Per-finger force labels"]
C --> E["Retargeted motion and<br/>embodiment-normalized video"]
D --> F["Force-augmented<br/>flow-matching policy"]
E --> F
F --> G["Robot motion + target force"]
G --> H["Robot force feedback<br/>and PD tracking"]
The system separates force-label acquisition from task demonstration collection. Fingertip sensors are present while building the pretraining dataset and during a short user calibration. Target-task demonstrations use the wristband and video alone.
ForceBand Hardware
The evaluated wristband uses eight bipolar sEMG channels and a 10-D IMU. Seven electrode pairs cover forearm muscles associated with finger flexion and extension; one covers wrist flexion. The anatomically guided layout concentrates sensing around thumb, index, middle, and multi-finger activation; a uniform wrist ring serves as the placement baseline.
An OpenBCI Cyton board with an ADS1299 front end acquires the low-amplitude biopotential signals. Bipolar differential measurements suppress common-mode noise and motion artifacts. The mechanical parts are 3D printable, the components are commercially available, and the stated hardware cost can be as low as $300. Daisy chaining supports a 16-channel extension without changing the downstream pipeline.
Thin-film force-sensitive resistors inside transparent gel finger cots provide the ground-truth fingertip forces used for pretraining and calibration. The transparent, palm-routed design preserves much of the hand appearance for video tracking. These sensors are removed after calibration.
A 10-Hour Multimodal Dataset
The authors collect synchronized egocentric video, eight-channel sEMG, IMU, and five-finger force measurements. The 10-hour dataset covers four action groups:
- pinch/grasp: 46%;
- pick-and-place: 27%;
- open/close: 16%;
- pouring: 11%.
It also spans two-, three-, and five-finger gestures plus free-form daily interactions, with objects varying in geometry, size, and weight. This dataset teaches the general relation between forearm activity and fingertip loading. A new user subsequently contributes about 15 minutes of paired calibration data to adapt that relation to personal anatomy, electrode placement, and muscle activation patterns.
EMG2Force: From Muscles to Five Force Traces
For each five-second window, the model receives
[ X_{\mathrm{EMG}}\in\mathbb{R}^{8\times 1250}, \qquad X_{\mathrm{IMU}}\in\mathbb{R}^{10\times 1250}, ]
sampled at 250 Hz. The IMU contains acceleration, angular velocity, and orientation. EMG2Force reads the signals through two complementary representations:
- a 1-D convolutional encoder processes the raw time series;
- an STFT converts the signals to time–frequency spectrograms, which a pretrained DINOv3 encoder processes.
The features are fused in a transformer decoder that predicts
[ F_{\mathrm{ftp}}\in\mathbb{R}^{5\times 1250}, ]
one synchronized force trace for each finger. The frequency branch captures characteristic muscle-activation spectra, while the IMU helps separate force-related activation from wrist-motion artifacts.
The ablation supports both inputs. The full sEMG + spectrogram + IMU model reaches 0.92 N MAE; removing the IMU raises it to 1.02 N, and removing the spectrogram raises it to 1.14 N.
Three-Step Deployment
1. Calibrate the user
The user wears ForceBand and fingertip force sensors for roughly 15 minutes of varied interaction. EMG2Force adapts to the user’s muscle patterns and the current electrode placement.
2. Collect force-enriched human demonstrations
The fingertip sensors are removed. The user performs the target task with ForceBand and video, and calibrated EMG2Force supplies the missing per-finger force labels. The robot experiment uses 15 human demonstrations per object.
3. Retarget, learn, and execute
Human hand and object motion is converted into a parallel-jaw robot representation. Aria MPS hand keypoints define the end-effector pose: the thumb–index midpoint gives position, selected MCP joints define orientation, and normalized thumb–index distance gives the gripper aperture. Grounding DINO, SAM2, CoTracker, and Orient Anything recover object masks, tracks, and 6-DoF poses. The human arm is removed with SAM2 and LaMa inpainting, then a virtual gripper and object keypoints are rendered into the scene.
This preprocessing preserves task geometry while reducing the visual embodiment gap between a human hand and a parallel-jaw gripper.
Force-Augmented Flow Matching
The robot action is
[ a_t=[p_t; r_t^{6D}; g_t; f_t]\in\mathbb{R}^{11}, ]
where (p_t) is 3-D end-effector position, (r_t^{6D}) is a continuous rotation representation, (g_t) is gripper aperture, and (f_t) is desired grip force. A transformer predicts a (K=50)-step action chunk using conditional flow matching.
Force appears on both sides of the policy interface. The observation tokens contain current grip force, while the action chunk contains future target forces. Spatial-relation tokens encode each hand/object entity, its 6-DoF pose, and its relation to the manipulator. This gives the policy a closed-loop mapping from current contact to future motion and loading.
EMG2Force produces five finger forces, but the parallel-jaw policy uses a single scalar:
[ f_t=\frac{F_{\mathrm{thumb},t}+F_{\mathrm{index},t}}{2}. ]
The rich human-side signal is therefore compressed to the force degree of freedom available on the evaluated gripper. A multi-finger robot hand could potentially retain more of the per-finger structure.
What “Robot-Data-Free” Means
The policy is trained from human demonstrations without teleoperated or autonomously collected robot trajectories. This is the paper’s robot-data-free claim. Robot execution still uses a sensor-feedback loop.
The real system uses a UR-5 arm, a Robotiq parallel-jaw gripper, a ZED 2i RGB-D camera, and four Paxini fingertip force sensors mounted on the gripper. When the policy requests closure, execution pauses briefly while the gripper establishes a stable 5 N pre-grasp. A PD controller then tracks the predicted force trajectory during squeezing and placement.
This distinction matters: ForceBand transfers a human-derived force prior without robot demonstrations; robot-side force sensing and classical feedback control make that prior executable.
Experiments and Main Results
Hardware placement
More channels improve force estimation: MAE falls from 1.89 N with one channel to 0.85 N with eight channels on the channel-count comparison. Under a separate matched 30-minute protocol, anatomical placement reaches 0.77 N MAE, compared with 0.94 N for an evenly spaced eight-channel ring—an 18% reduction.
Force estimation
ForceBand reduces force-regression error by more than 50% relative to the vision baselines. Its advantage is largest on occluded fingers. Ring-finger contact PR AUC improves from 0.398 to 0.763, and little-finger PR AUC from 0.314 to 0.590, relative to the FEEL vision baseline.
Real-robot policy learning
The real-robot benchmark uses nine everyday objects spanning 43–650 g and 1–72 mm grasp widths. The task requires picking, applying an object-specific squeeze, and placing. The paper reports 87% success for ForceBand across these tasks.
A binary gripper often completes pick-and-place yet produces zero successful squeeze behaviors. Continuous aperture control sometimes squeezes deformable objects, but aperture is an unreliable force proxy on rigid objects and under noisy human-hand tracking. ForceBand generates object-dependent peak forces from approximately 3.2 N to 19.3 N and transfers meaningful force profiles to held-out objects.
The policy also preserves the three-stage task structure under novel backgrounds, objects, extreme lighting, and distractors. Background and texture changes can shift the precise squeeze magnitude, revealing continued reliance on visual appearance for force selection.
Strengths
ForceBand addresses a real supervision bottleneck with a practical sensor placement. Its wrist-only target-task collection keeps the hand visually accessible, and the three-step workflow cleanly separates expensive ground truth from scalable demonstrations. The paper evaluates the full chain from electrode design through force inference to closed-loop robot behavior. It also makes force a predicted action variable, giving the representation a direct control consequence.
The combination of learning and control is well chosen. EMG supplies the prior for when and how hard to squeeze; robot fingertip sensing and a PD loop handle execution errors and gripper dynamics.
Limitations
Force prediction remains less accurate than direct fingertip sensing. The study covers four users and requires per-user calibration, so broad cross-user and day-to-day robustness remain open. Calibration still needs fingertip force sensors, and electrode displacement or skin-condition changes may alter the sEMG mapping.
The downstream robot policy compresses five predicted finger forces into one parallel-jaw grip-force scalar. The evaluated behavior is a focused pick–squeeze–place benchmark with nine objects and 15 demonstrations per object. Transfer to articulated hands, shear forces, torque, tool use, and long-horizon contact-rich tasks remains open. Robot deployment relies on added Paxini sensors, a 5 N pre-grasp routine, and PD force control.
Visual domain shifts affect exact force magnitude, and the force estimator uses five-second windows that may smooth or delay small peaks. The public dataset is still forthcoming, which currently limits independent reproduction of the complete training pipeline.
Takeaways
ForceBand shows that scalable human data can carry dynamics as well as kinematics when the sensing interface targets the body’s motor signals. Its core recipe is broadly reusable:
- collect a modest paired dataset with accurate but intrusive sensors;
- train a wearable proxy model for the hidden physical quantity;
- remove the intrusive sensors during large-scale task collection;
- retain robot-side feedback for stable execution.
The deeper contribution is the conversion of a transient calibration instrument into persistent supervision. Applied beyond grip force, the same pattern could connect human physiological sensing to robot compliance, fatigue, effort, or contact-state labels.