[Paper Notes] Extreme-RGMT: Continual Learning of Highly Dynamic Skills for Robust Generalist Humanoid Control
Published:
This post supports English / 中文 switching via the site language toggle in the top navigation.
TL;DR
Extreme-RGMT extends a robust generalist humanoid tracker toward backflips, aerial cartwheels, kip-ups, and twisting aerial motions. Its central challenge is the generalist-specialist trade-off. Rare highly dynamic motions need concentrated optimization on short failure-critical segments, while direct specialist fine-tuning shifts the policy away from the state-action mappings that support walking, crawling, calisthenics, and other mastered behaviors.
The paper organizes learning into two stages. Stage I trains an improved RGMT-style base policy over 3.096 hours of multi-source motion data. It separately encodes proprioceptive and action histories, uses recent closed-loop dynamics to query a local reference window, and applies an FSQ bottleneck to stabilize the aggregated command representation. The base policy’s completion statistics then divide the corpus into 2.82 hours of mastered motions and 0.28 hours of challenging motions.
Stage II combines two mechanisms. PACE assigns challenging clips to acquisition environments trained with PPO and mastered clips to consolidation environments constrained toward a frozen copy of the base policy. STAR finds high-failure temporal bins and resamples the top high-advantage trajectory fragments from them, increasing the learning value of scarce successful experience around aerial adjustment, contact switching, and landing recovery.
This changes XtremeMotion success from 21.42% to 100% and AMASS challenging-motion success from 18.18% to 90.91%. Generalist success remains 99.76% on in-source motions and reaches 96.68% on unseen motions. On a 29-DoF Unitree G1, the complete policy reports 90% success for fixed highly dynamic replay, 85% for online highly dynamic Xsens teleoperation, and 100% for generalist Xsens teleoperation.
Paper Info
“Extreme-RGMT: Continual Learning of Highly Dynamic Skills for Robust Generalist Humanoid Control” is by Yubiao Ma, Han Yu, Kai Guo, Changtai Lv, Zhengquan Mao, Boyang Xing, Xuemei Ren, and Dongdong Zheng, with affiliations at Beijing Institute of Technology, Humanoid Robotics (Shanghai) Co., Ltd., and Shandong University. This note covers arXiv:2607.20110v1, posted on July 22, 2026. The project page contains simulation and hardware videos. The current version is a 16-page preprint.
This is a direct follow-up to RGMT from substantially the same author team. RGMT focuses on robust command interpretation and integrated recovery. Extreme-RGMT keeps that dynamics-guided foundation and develops a continual-learning recipe for rare highly dynamic skills.
1. Why Generalists Struggle with Extreme Motion
A large motion corpus is dominated by smooth, regular behaviors. Highly dynamic clips occupy a small fraction, and their control difficulty is concentrated in even shorter intervals: takeoff, aerial posture adjustment, rapid contact transition, and landing recovery. When these clips are mixed uniformly into general training, their gradients are diluted by ordinary motion. When they fail early, a rollout also contains few useful transitions after the failure point.
Specialist training creates the opposite problem. Continuing PPO only on flips and aerial skills supplies strong learning pressure, but it modifies shared state-action mappings and gradually weakens the original repertoire. The learning problem therefore requires both plasticity for extreme skills and stability for mastered skills.
flowchart TD
A["Stage I: multi-source generalist training"] --> B["Base policy"]
B --> C["Evaluate every motion clip"]
C --> D["Mastered set: completion at least 80%"]
C --> E["Challenging set: completion below 80%"]
D --> F["PACE consolidation environments"]
E --> G["PACE acquisition environments"]
G --> H["STAR selects critical high-advantage fragments"]
F --> I["Single augmented policy"]
H --> I
I --> J["Generalist capability plus highly dynamic skills"]
The framework is continual learning within one goal-conditioned tracking problem. There is no sequence of separately named tasks; the boundary comes from performance-based stratification of an imbalanced motion distribution.
2. Stage I: A More Structured RGMT Base Policy
At each step, the actor receives ten frames of proprioceptive history, ten previous actions, and a 21-token reference window. Proprioception contains projected gravity, base angular velocity, joint-position offset, and joint velocity. Each reference token contains base linear velocity, base angular velocity, gravity direction, and 29 joint positions. The policy outputs a residual joint-position target:
[ q_t^{\mathrm{tar}}=q_t^{\mathrm{ref}}+a_t, ]
followed by low-level PD control.
Extreme-RGMT modifies the original RGMT encoder in three useful ways.
Separate state and action histories
Proprioceptive observations and past actions use separate MLP encoders and LayerNorm interfaces:
[ z_\tau^o=\mathrm{LN}o(f_o(o\tau^{\mathrm{prop}})), \qquad z_{\tau-1}^a=\mathrm{LN}a(f_a(a{\tau-1})). ]
The resulting tokens are interleaved in time and passed through a causal history encoder:
[ \mathcal H_t=[z_{t-H-1}^a,z_{t-H}^o,\ldots,z_{t-1}^a,z_t^o], \qquad h_t=\mathrm{Enc}_{\mathrm{hist}}(\mathcal H_t). ]
State history describes what the robot experienced; action history describes what the controller recently commanded. Separate normalization reduces interference between their different scales, while interleaving preserves the closed-loop response sequence.
Dynamics-guided reference aggregation
The history representation again acts as the query over a local command window:
[ u_t=\mathrm{CrossAttn}(Q=W_qh_t,\;K=Z_t^g,\;V=Z_t^g). ]
This preserves RGMT’s core idea: the relevant point in a reference window depends on the robot’s current physical phase. The dependence is particularly valuable during aerial motion, where a small timing deviation changes the command that should receive attention.
FSQ command bottleneck
After cross-attention, the command representation passes through finite scalar quantization:
[ \hat u_t=\mathcal Q_{\mathrm{FSQ}}(u_t). ]
The implementation factorizes (u_t) into two 32-dimensional tokens and quantizes them before actor fusion. This bounded, discrete latent interface regularizes local variation in the aggregated command. Hardware ablations suggest that it improves robustness to rapid state changes and imperfect inertial-motion references.
3. Motion Data and Performance-Based Stratification
Stage I uses retargeted motions from LAFAN1, AMASS, and an in-house Xsens inertial-motion-capture set. Every sequence is resampled to the 50 Hz policy frequency.
| Source | Duration | Share |
|---|---|---|
| LAFAN1 | 2.444 h | 78.94% |
| AMASS | 0.511 h | 16.51% |
| In-house Xsens | 0.141 h | 4.55% |
| Total | 3.096 h | 100% |
After Stage I, sequences longer than ten seconds are divided into ten-second clips. Each clip receives five randomized rollouts. Completion of at least 80% assigns it to the mastered set (\mathcal D_m); the remaining clips enter the challenging set (\mathcal D_c).
| Set | Duration | Stage-II role |
|---|---|---|
| Mastered (\mathcal D_m) | 2.82 h | Capability consolidation and broad coverage |
| Challenging (\mathcal D_c) | 0.28 h | Highly dynamic skill acquisition |
The challenging set has higher root and joint dynamics and more airborne motion. Its small size makes the imbalance concrete: roughly nine percent of the reference duration carries most of the pressure for extreme capability expansion.
4. PACE: Acquisition and Consolidation in Parallel
Stage II initializes the trainable policy from the Stage-I base policy and divides parallel simulation environments by role. The acquisition fraction is (\xi=0.8): most environments train on difficult clips, while the remaining environments revisit mastered clips.
Acquisition environments optimize the normal clipped PPO objective on (\mathcal D_c). Consolidation environments run (\mathcal D_m) and compare the trainable policy with a frozen base-policy reference:
[ \mathcal L_{\mathrm{con}}^{\mathcal D_m} =\mathbb E_{s\sim d_{\mathcal D_m}} \left[|a_\theta(s)-a_{\mathrm{ref}}(s)|_2^2\right]. ]
The combined objective is
[ \min_\theta \left{ \mathcal L_{\mathrm{acq}}^{\mathcal D_c} +\lambda_{\mathrm{con}}^t \mathcal L_{\mathrm{con}}^{\mathcal D_m} \right}. ]
The consolidation weight changes with the realized ratio of valid acquisition samples. Early in training, extreme rollouts fail quickly and useful acquisition data are scarce, so a strong constraint would prevent the policy from discovering new control responses. As more acquisition samples survive, (\lambda_{\mathrm{con}}^t) grows and more strongly limits drift on mastered motions.
This is the key difference from ordinary replay. PACE gives the two data subsets distinct objectives: difficult motions drive PPO updates; mastered motions define an action-space retention constraint against the frozen base controller.
5. STAR: Learning from Failure-Critical Segments
Adaptive motion sampling already initializes more rollouts from temporal bins with high failure or tracking error. STAR adds a second level of prioritization inside the acquisition rollout buffer.
Its procedure is:
- Use the adaptive-sampling difficulty prior to identify high-difficulty temporal bins.
- Normalize advantages within difficulty groups, avoiding a global scale that can suppress useful samples from difficult regions.
- Group transitions into contiguous trajectory fragments associated with each bin.
- Rank valid bin-fragment pairs by their average raw advantage and retain the top 5% for every difficult bin.
- Construct each acquisition mini-batch with 25% resampled transitions from the selected pool and 75% standard rollout samples.
The selected samples combine two properties: they occur where failures are common, and their advantages indicate useful improvement directions. Bin-wise selection also preserves coverage across different critical moments instead of letting one abundant segment dominate the pool.
STAR has its largest effect on noisy Xsens motions. Specialist success on in-house Xsens clips rises from 45.5% without STAR to 86.3% with STAR, a 40.8-point gain. On AMASS challenging motions, the improvement is 82.2% to 90.9%. The gap supports the paper’s claim that low-quality inertial references create a stronger need to reuse scarce informative fragments.
6. Generalist and Specialist Results
Evaluation uses MuJoCo and averages tracking metrics over five random seeds. Success requires completing the reference without root-height deviation exceeding 0.2 m.
Generalist capability
| Method | In-source success | Unseen-motion success |
|---|---|---|
| ExBody2 | 85.63% | 66.78% |
| BeyondMimic | 94.72% | 71.34% |
| SONIC | 99.33% | 93.67% |
| RGMT | 99.12% | 94.58% |
| Extreme-RGMT Stage I | 99.54% | 95.13% |
| Extreme-RGMT Full | 99.76% | 96.68% |
Stage II preserves and slightly improves completion. Tracking error reveals a small trade-off: unseen-motion MPJPE changes from 45.80 mm at Stage I to 46.91 mm in the full model, while unseen completion increases from 95.13% to 96.68%. Capability retention is therefore strong, though not every fidelity metric improves simultaneously.
Highly dynamic capability
| Method | XtremeMotion success | AMASS challenging success |
|---|---|---|
| OmniXtreme | 100.00% | 36.16% |
| Direct fine-tuning | 71.43% | 54.55% |
| Extreme-RGMT Stage I | 21.42% | 18.18% |
| Extreme-RGMT Full | 100.00% | 90.91% |
OmniXtreme is the strongest narrow specialist on its matched XtremeMotion set and achieves the lowest pose error there. Its success drops sharply on AMASS challenging motions. Extreme-RGMT trades a small amount of matched-set fidelity for much broader specialist coverage while maintaining its generalist repertoire.
The direct-fine-tuning baseline improves extreme motion but progressively reduces generalist performance. Mixed training over mastered and challenging clips preserves general capability yet supplies too little pressure for specialist learning. PACE addresses both failure modes through role-specific environments and the retention loss; STAR drives continued improvement after ordinary difficulty sampling reaches a plateau.
7. Real-World Extreme Motion and Teleoperation
The 29-DoF Unitree G1 runs the learned policy at 50 Hz and its PD loop at 500 Hz. Fixed-reference tests include a Webster-style flip, butterfly kick, twisting back-handspring, and aerial cartwheel. Online Xsens tests include a standing tucked flip, air-twist kick, running side flip, and aerial body twist. Each category contains four motions with five trials per motion.
| Hardware evaluation | Complete model success |
|---|---|
| Fixed AMASS highly dynamic replay | 90.0% |
| Online Xsens highly dynamic teleoperation | 85.0% |
| Online Xsens generalist teleoperation | 100.0% |
Removing STAR causes the largest teleoperation drop: online highly dynamic success falls to 45%. Removing FSQ yields 65%, and using a unified state-action encoder yields 75%. These component ablations connect representation design and trajectory resampling to hardware behavior.
Online tracking is more demanding than fixed replay because the policy receives a continuously arriving inertial stream without access to a complete future trajectory. Timing variation, root drift, and pose inconsistency remain present. The demonstrations include extreme motions absent from the training corpus, providing evidence of zero-shot transfer within related coordination patterns.
8. What Extreme-RGMT Adds to RGMT
| RGMT | Extreme-RGMT |
|---|---|
| Filters imperfect reference commands using current dynamics | Retains this mechanism and strengthens its representation |
| Jointly learns tracking and fall recovery | Progressively expands toward rare aerial and high-contact skills |
| Single-stage generalist training | Two-stage generalist training plus continual skill expansion |
| Main mechanism: causal history plus cross-attention | Main mechanisms: separate histories, FSQ, PACE, and STAR |
| Robustness to noise, disturbances, and ground interaction | Balance between specialist acquisition and generalist retention |
Extreme-RGMT is therefore a training-framework extension as much as an architecture extension. Its main idea is to organize optimization pressure: reserve most Stage-II environments for the rare skills, keep a protected channel for old capabilities, and spend more gradient updates on the short trajectory fragments that contain useful recovery signals.
9. Strengths, Limitations, and Takeaways
The paper defines a compelling capability frontier. General motion and expert-level dynamic motion share one policy, yet they occupy strongly imbalanced data and control distributions. PACE turns retention into an explicit objective, and STAR connects motion-level difficulty with fragment-level learning value. The ablations trace both mechanisms to the intended outcomes.
The evidence also has limits. Hardware results use four representative motions and five trials per motion in each setting, so the 85–100% rates come from small, curated evaluation suites. Some real motions are unseen during training, but they remain related to the coordination patterns represented in the extreme-motion corpus. The paper reports that substantially different timing, contact patterns, or coordination may still require dedicated practice.
Like RGMT, the controller tracks root-relative references and lacks global position or heading input. Long-duration execution can accumulate global drift. Online adaptation and global localization remain future directions. Highly dynamic hardware experiments also demand careful safety infrastructure; simulation success alone is insufficient evidence for deployment on a new platform.
The research takeaway is broader than the specific controller. Rare capabilities are often defined by short critical intervals. A useful continual-learning system needs three levels of organization: separate old and new capability roles, constrain policy drift where old behavior already works, and concentrate new learning on the small fragments where success or failure is decided.