[Paper Notes] ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning
Published:
This post supports English / 中文 switching via the site language toggle in the top navigation.
TL;DR
ADEPT treats dexterous reinforcement learning as a pre-training and post-training problem. A generic object-reposing policy first learns reusable reach, grasp, lift, in-hand reorientation, and transport skills in simulation. Each downstream contact-rich task then starts from this behavioral prior. The central technical finding is that ordinary PPO fine-tuning quickly destroys the pretrained behavior because the reward and observation spaces change while the old critic gives unreliable advantages. ADEPT stabilizes transfer with three operations: behavior-cloning distillation into the downstream actor, critic warm-up with the actor frozen, and conservative PPO updates with a much smaller actor learning rate.
The resulting state-based task teacher is distilled into a deployable student that consumes stereo RGB, proprioception, and geometric-fabric state. The Flexiv-Sharpa version also consumes five fingertip tactile maps. Real deployment therefore runs without an online object-pose estimator: object and receptacle poses are inferred implicitly inside the visual or visuo-tactile policy. On real hardware, the Kuka-Allegro vision student achieves 5/10 on a star-peg insertion, 3/10 on an asymmetric square/round peg, and 6/10 on dish-rack placement. On the square/round peg, adding fingertip touch raises the Flexiv-Sharpa result from 3/10 to 8/10.
My main takeaway is that ADEPT contributes more than a dexterous policy. It provides a practical recipe for preserving motor competence while changing the task, a perception curriculum for converting privileged simulation policies into raw-sensor controllers, and a safety-oriented action interface shared by simulation and hardware.
Paper Info
“ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning” is by Jayjun Lee, Jessica Yin, Asif Rana, Nicholas Blauch, Sam Mady, Mohak Bhardwaj, Nima Fazeli, Nathan Ratliff, Karl Van Wyk, and Ankur Handa, from NVIDIA and the University of Michigan. This note covers the 31-page arXiv:2608.19182v1, submitted on August 19, 2026. The project page contains real-time rollout videos.
1. Why Dexterous RL Needs a Reusable Starting Point
A high-DoF arm-hand policy trained from scratch must discover a long sequence before receiving useful contact-rich task reward: reach the object, form a stable grasp, lift it, reorient it, transport it, align it, and finally insert or place it. The early skills recur across tasks, yet task-specific RL repeatedly relearns them. Sparse rewards and high-dimensional contact make that rediscovery slow and seed-sensitive.
ADEPT separates the shared motor foundation from the downstream interaction. Pre-training covers the recurring object-manipulation repertoire. Post-training adds the task-specific contact behavior. The policy initialization is the useful artifact: it places downstream optimization near meaningful grasps and object motions, where exploration can reach the final interaction.
This view also explains a result that initially looks surprising. The dish used in the rack-placement experiment is much flatter and larger than every pre-training primitive, and the pretrained policy cannot grasp it successfully. It still reaches the plate and produces plausible contact attempts. Post-training turns that useful starting distribution into a new plate grasp and even discovers flip-and-regrasp behavior. A prior can help without already containing the complete downstream solution.
2. The End-to-End ADEPT Pipeline
flowchart TD
A["Generic object reposing<br/>16 randomized primitives"] --> B["Pretrained state teacher<br/>reach, grasp, lift, reorient, transport"]
B --> C["BC distillation into<br/>downstream observation space"]
C --> D["Freeze actor and<br/>warm up a new critic"]
D --> E["Conservative PPO<br/>on contact-rich task"]
E --> F["Post-trained<br/>task-specialist teacher"]
F --> G["Two-stage DAgger distillation<br/>with 8-keypoint auxiliary loss"]
G --> H["Stereo-RGB student"]
G --> I["Stereo-RGB + fingertip-touch student"]
H --> J["Zero-shot real deployment<br/>Kuka-Allegro"]
I --> K["Zero-shot real deployment<br/>Flexiv-Sharpa"]
The pipeline has three learning stages and one shared control layer:
- Reposing pre-training learns foundational dexterity with privileged simulation state.
- Structured post-training preserves that behavior while adapting to a new contact-rich reward and observation space.
- Teacher-student distillation converts the state policy into a raw visual or visuo-tactile policy.
- Full-Cspace Geometric Fabric mediates every policy action in simulation and on hardware.
The first two stages solve motor learning; the third solves deployable perception; the fabric handles smooth action generation and hardware constraints.
3. Stage 1: Reposing as Dexterity Pre-Training
At the beginning of every episode, ADEPT samples one of 16 primitive shapes—cylinders, cuboids, spheres, and cones—with randomized scale. The robot must reach, grasp, lift, reorient, transport, and repose the object at a sampled target pose. Objects are represented with point clouds, allowing the state teacher to share one policy across geometry and scale.
Training uses PPO with an asymmetric actor-critic, Automatic Domain Randomization (ADR), and Population-Based Training (PBT). ADR gradually increases goal difficulty and environmental variation as success improves. Gravity is annealed from zero to (-9.81\,\mathrm{m/s^2}), so early learning begins with an easier contact problem and eventually reaches normal gravity.
The reposing checkpoint costs about 8 billion environment steps. That cost is paid once per robot embodiment. On unseen objects, the pretrained Kuka-Allegro teacher reaches 0.76 episodic success on the two FMB pegs and 0.77 on 152 VisDex objects, close to or slightly above its 0.73 success on the training primitives. The Flexiv-Sharpa teacher similarly remains near its in-distribution result.
Pre-training does not solve insertion. Along the FMB curriculum, zero-shot success stays above 50% through ADR level 35, covering much of lift, transport, and alignment, then falls to approximately zero at the final contact-rich insertion goal. This boundary motivates downstream learning: reuse the free-space dexterity and spend new exploration on the interaction that pre-training did not cover.
4. Why Direct PPO Fine-Tuning Collapses
Let (\mathcal M_{\text{pre}}) denote the reposing MDP and (\mathcal M_{\text{post}}) the downstream MDP. They share the joint action space,
[ \mathcal A=[-1,1]^{n_q}, ]
but their observations and rewards differ. The downstream observation adds task-specific signals such as receptacle pose and object-receptacle contact forces. Its reward values insertion or placement, while the old critic (V_{\text{pre}}) was calibrated for reposing.
If PPO begins updating immediately, the critic supplies inaccurate value and advantage estimates under the new reward. Large early actor updates move the policy away from its pretrained behavior before the critic can recover. The next rollouts come from a worse actor and train an already-miscalibrated critic, forming a destructive feedback loop. In the reported experiment, direct fine-tuning drives success at the transfer point to zero within a few updates.
ADEPT repairs each mismatch explicitly.
Step A: behavior-cloning actor distillation
The pretrained actor is distilled for 40,000 supervised iterations into a new actor (\pi_{\text{post}}) that accepts the downstream observation space. This transfers behavior while introducing new input dimensions cleanly.
Step B: critic warm-up
ADEPT initializes a fresh downstream critic (V_{\text{post}}), freezes the actor, and trains the critic for 20 PPO iterations under the new reward—about one million environment steps per GPU with 4,096 parallel environments. Policy behavior stays fixed while the value function becomes useful for the new task.
Step C: conservative PPO
The actor is unfrozen and optimized with a much smaller learning rate. The paper reduces the actor learning rate from (10^{-3}) to (10^{-5}), reduces PPO clipping from 0.20 to 0.05, and holds the critic learning rate at (5\times10^{-5}).
The ablation gives a precise interpretation. The low actor learning rate is the component that prevents collapse: every (10^{-3}) variant fails at the transfer point. With the low learning rate already in place, critic warm-up adds 17.6 success points, and BC reduces time to the final curriculum level from 35.2 to 19.9 hours. Tightening the PPO clip is not essential; a 0.20 clip slightly outperforms the deployed 0.05 setting in the ablation. A KL penalty to the pretrained policy also fails to rescue direct fine-tuning.
ADEPT needs roughly 3 billion additional environment steps for the downstream teacher. The full first task therefore costs 11 billion steps, while later tasks reuse the 8-billion-step foundation and pay only the marginal post-training cost.
5. Stage 3: From State Teacher to Raw Perception
The post-trained teacher still depends on simulator state and cannot be deployed directly. ADEPT trains a student with DAgger: the student collects trajectories, and the teacher provides target action distributions on those visited states. The deployable observation contains proprioception, geometric-fabric state, and two RGB images. Flexiv-Sharpa also includes one tactile stream from each of its five fingertips.
Action cloning alone does not reliably recover peg orientation under occlusion. ADEPT adds an auxiliary head that predicts eight object keypoints from the shared stereo visual features:
[ \mathcal L=\mathcal L_{\mathrm{BC}}+\mathcal L_{\mathrm{aux}}. ]
The visual curriculum also has two stages. A student first imitates the generic reposing teacher while learning to detect, track, and reorient the peg. It then initializes distillation from the post-trained insertion teacher. This schedule lets the image encoder acquire object geometry before contact-rich action imitation dominates optimization. A single-stage student scores 0/10 on both real pegs; the two-stage curriculum is therefore a necessary part of the reported sim-to-real system.
For touch, each fingertip produces a geometry-consistent penetration-depth map and a thresholded binary contact map. A shared per-finger CNN encodes them, and fingertip position conditions the features so the policy knows which contact occurred where. The same TacMap-style representation exists in simulation and on the real sensor.
Training randomizes physics, object disturbances, lighting, backgrounds, camera intrinsics and poses, proprioceptive noise, and contact signals. This prepares the student for zero-shot deployment without real-world fine-tuning.
6. How ADEPT Locates Real Objects
ADEPT does not run FoundationPose or another explicit 6D tracker during real execution. Two calibrated RealSense cameras observe the workspace from left and center views. The student network maps these RGB images, robot state, and optional touch directly to joint actions. Object and receptacle pose remain latent inside the learned visual representation.
The eight-keypoint auxiliary loss supplies explicit geometric supervision during simulation training, but the predicted keypoints are not passed through a separate real-world pose-estimation-and-control pipeline. This distinction separates ADEPT from state-based deployment systems such as SimToolReal and Play2Perfect, whose policies consume externally estimated object poses.
End-to-end perception removes a brittle tracker interface and introduces a harder representation-learning problem. The paper’s failure analysis confirms this trade-off: orientation mistakes under hand-object occlusion are the main bottleneck. Touch resolves contact ambiguity, but robust object-centric perception under occlusion remains open.
7. Full-Cspace Geometric Fabric
ADEPT places a joint-configuration-space Geometric Fabric between the learned policy and the robot:
[ \mathbf M_f(\mathbf q_f,\dot{\mathbf q}f)\ddot{\mathbf q}_f +\mathbf f_f(\mathbf q_f,\dot{\mathbf q}_f) +\mathbf f\pi(\mathbf a)=0. ]
Here (\mathbf f_f) contains autonomous geometric and dissipative terms for collision avoidance, joint-limit repulsion, damping, and speed control; (\mathbf f_\pi) converts policy output into a forcing term. The policy emits one relative target per arm-hand joint, retaining the full 23 DoF on Kuka-Allegro and 29 DoF on Flexiv-Sharpa.
Earlier fabric-guided hand policies often restricted finger motion to a low-dimensional PCA grasp space. ADEPT exposes the full configuration space so finger gaiting, in-hand reorientation, and contact-rich corrections can emerge. The identical fabric runs in simulation and hardware, reducing the low-level controller gap while enforcing safety constraints.
8. Results on Two Arm-Hand Platforms
The hardware consists of a 7-DoF Kuka iiwa7 with a 16-DoF Allegro hand and a 7-DoF Flexiv Rizon with a 22-DoF Sharpa hand. Every task and embodiment receives its own post-trained specialist and perceptive student; the reusable component is the embodiment-specific reposing checkpoint.
| Modality and robot | Real-world task | Success |
|---|---|---|
| Vision, Kuka-Allegro | FMB star peg | 5/10 |
| Vision, Kuka-Allegro | FMB square/round peg | 3/10 |
| Vision, Kuka-Allegro | Dish-rack placement | 6/10 |
| Vision, Flexiv-Sharpa | FMB square/round peg | 3/10 |
| Visuo-tactile, Flexiv-Sharpa | FMB square/round peg | 8/10 |
The tactile comparison is the clearest real-world result. The vision-only Sharpa policy frequently forms a valid grasp and then reopens because it cannot determine whether contact succeeded. Those errors propagate into lifting and reorientation. The visuo-tactile policy grasps and lifts in all 10 trials, then reaches 9/10 after reorientation and 8/10 after alignment and insertion.
ADEPT executes the complete reach-to-insert behavior as one continuous policy in roughly 5–10 seconds. The referenced parallel-jaw FMB pipeline uses external fixtures and decomposes the behavior into multiple regrasp stages, taking 20–70 seconds. This produces the reported 2–14× execution-time difference, although the systems use different end effectors and manipulation strategies.
9. Relationship to SimToolReal and Play2Perfect
All three methods use generic 6D object manipulation to acquire reusable dexterity, but they optimize different deployment goals.
| Method | Use of generic reposing/play | Downstream adaptation | Real-world perception | Main target |
|---|---|---|---|---|
| SimToolReal | Final generalist policy | None | Explicit object pose and grasp box from SAM/FoundationPose | Unseen tools and goal trajectories |
| Play2Perfect | Initialization for assembly | Sparse-reward RL per CAD task | Explicit part and fixture poses from FoundationPose | Precise insertion, assembly, and screwing |
| ADEPT | Embodiment-specific motor prior | Structured post-training per task | Raw stereo RGB; optional fingertip touch | Long-horizon contact-rich control without a pose tracker |
SimToolReal has the strongest zero-shot generalization across unseen tools and trajectories. Play2Perfect makes task-specific precision and sample efficiency its center. ADEPT invests most heavily in stable policy transfer, raw perception, tactile feedback, full-DoF safety control, and multiple embodiments. Their reported success rates are not directly comparable because their tasks, tolerances, observations, and metrics differ.
10. Strengths, Limitations, and Open Questions
The strongest part of ADEPT is the diagnosis of transfer collapse. The ablation separates the roles of actor learning rate, critic warm-up, BC initialization, PPO clipping, and KL regularization. This turns “pre-train and fine-tune” into a reproducible procedure with a clear optimization explanation.
The system integration is equally important. The same work connects scalable RL, point-cloud-conditioned state teachers, raw stereo vision, tactile simulation, DAgger, and a safety controller shared across simulation and hardware. The tactile result demonstrates that contact information can close a failure mode that more RGB alone does not resolve reliably.
The evidence also has boundaries:
- Pre-training uses only 16 primitives and two embodiments; wider interaction coverage remains untested.
- Every downstream task and robot embodiment requires its own specialist training and distillation.
- The real-world evaluation uses 10 trials per condition, so success estimates have high uncertainty.
- Perception under occlusion remains the main deployment bottleneck.
- The current downstream set covers peg insertion and dish placement. Tool use, clutter, bimanual manipulation, and broader task families remain future work.
- Pre-training is computationally expensive at 8 billion environment steps, even though that cost is amortized across tasks.
Takeaway
ADEPT offers a useful decomposition for dexterous robot learning. Generic reposing creates a parameter-space region containing competent grasps and object motions. Structured post-training keeps the actor near that region while a new critic and task reward add contact behavior. Teacher-student distillation converts privileged competence into a raw-sensor policy, and the Geometric Fabric turns learned actions into safe full-joint commands.
The paper’s broader message is that reusable motor skill is a behavioral prior. Effective transfer requires attention to observation changes, value calibration, policy drift, perception, and the low-level control interface.