[Paper Notes] ENPIRE: Agentic Robot Policy Self-Improvement in the Real World
Published:
ENPIRE is a harness for letting coding agents improve real robot policies through a closed physical feedback loop. The loop is deliberately simple: reset the scene, run the policy, verify the outcome, inspect logs, change the code or training recipe, then try again. The paper’s main claim is that this loop can be made concrete enough for frontier coding agents to operate on real manipulation hardware, reaching high success on tasks such as Push-T, pin insertion, GPU insertion, and zip-tie cutting.
My read: the important move is the separation between building the environment interface and running autonomous policy improvement. The authors are not asking a language model to directly “do robotics” from raw pixels and motors. They first construct a reusable physical API around safety, reset, rollout, and reward. Once that interface is stable, the coding agent can work like an experimental scientist with a robot budget: propose a hypothesis, edit code, launch rollouts, read results, and keep the variants that improve success.
Paper Info
The paper is “ENPIRE: Agentic Robot Policy Self-Improvement in the Real World” by Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian “Max” Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, Jimmy Wu, Guanzhi Wang, S. Shankar Sastry, Ken Goldberg, Linxi “Jim” Fan, Yuke Zhu, and Guanya Shi. The affiliations are NVIDIA, CMU, and UC Berkeley.
The paper is available as arXiv:2606.19980. The project page is research.nvidia.com/labs/gear/enpire.
The Problem: Physical Autoresearch
Coding agents have become useful in digital environments because each trial is cheap: run code, see the error, patch, repeat. Real-world robot learning has a different bottleneck. Every trial consumes robot time, hardware can be damaged, the scene must be reset, and success must be measured reliably. Human supervision often remains hidden inside the “learning” pipeline through data collection, resetting, evaluation, reward engineering, and algorithm tuning.
ENPIRE frames the target as physical autoresearch: an agent should be able to run the research loop on hardware under an explicit robot-time budget. The missing abstraction extends beyond a better policy class to a repeatable experimental interface that turns robot interaction into structured feedback.
The paper decomposes that interface into four modules:
| Module | Role |
|---|---|
| EN: Environment | hard safety constraints, automatic reset, automatic verification |
| PI: Policy Improvement | agent-driven code edits, training recipes, algorithm variants |
| R: Rollout | execution on one or more physical robots, logs, videos, rewards |
| E: Evolution | multi-agent hypothesis selection, Git-based sharing, branch merging |
This naming is useful because it makes the dependency clear. Policy improvement only becomes autonomous after the Environment and Rollout modules provide reliable feedback.
Stage 1: Build the Environment From Human Feedback
The first stage is human-guided. A person provides the task objective and feedback while the coding agent constructs the environment APIs. This stage is a one-time setup cost for a task; after it is validated, the API becomes an immutable interface reused during autonomous improvement.
There are three pieces.
Hard safety constraints restrict the robot’s configuration space and kinematic behavior. Violating the safe region immediately terminates the episode and triggers reset. This is both a physical safety layer and a clean episode boundary for learning.
Automated verification converts sensor streams into reward or success signals. Given a few minutes of success and failure demonstrations, the agent synthesizes procedural reward code using videos and proprioception, then optimizes for prediction accuracy and low latency. In pin insertion, the reward combines visual alignment, end-effector height, and force estimates. In zip-tie insertion, the reward uses cropped segmentation from two camera views to detect whether the strap passes through the head, with latency reduced below 150 ms.
Automated reset restores the scene after success or failure. For contact-rich tasks, ENPIRE uses modular manipulation tools inspired by CaP-X to reset directly to the critical phase, such as hovering above a pin hole, seating a GPU near a slot, or positioning scissors before cutting. This matters because policy learning should spend robot time on the hard part of the task, not on repeatedly reproducing easy setup motions.
The appendix gives the more concrete tool stack: SAM3 for open-vocabulary segmentation, BundleSDF for 6-DoF pose tracking, cuRobo for collision-free planning, RGB-D and proprioceptive state, and gripper torque as a tactile-like signal for slip and force control.
Stage 2: Let the Agent Improve the Policy
After the environment is stable, the second stage becomes autonomous. The agent receives the task objective, a writable training codebase, access to rollout logs and videos, and the goal of maximizing real-world success rate. It can review literature, formulate hypotheses, edit behavior cloning or RL code, tune infrastructure, and launch real rollouts through the environment API.
The pin insertion task is the paper’s clearest example. The robot must insert a pin into a hole with a tight 4 mm clearance, and agents are asked to reach 50 consecutive successes. During the run, agents try behavior cloning, iterative BC with online data aggregation, online RL, offline RL, offline-to-online RL, and BC-regularized RL. They also tune practical variables such as batch size, actor-critic update rate, and the BC regularization weight.
The mechanism is closer to an automated lab notebook than to a single learned controller. ENPIRE gives the coding agent enough observability to ask: did the last idea improve success, what failures remain, which branch found a better recipe, and which code changes should be merged?
Fleet Scaling: Evolution Through Git
ENPIRE scales by assigning one agent to one robot station. The paper uses a fleet of eight bimanual YAM robot stations. Each station has its own arms, cameras, compute, and coding agent. Hardware-control requests go through a local FastAPI server, while coordination across stations happens through Git: agents push branches, pull peer branches, cherry-pick useful commits, and merge promising training recipes.
This is the Evolution part of ENPIRE. The fleet does not require a central optimizer that collects all state. It uses a decentralized version-control protocol where ideas compete through measured success rates. In the pin-insertion idea tree, a few changes account for most of the improvement, including BC regularization at +10.8 percentage points, online RL mixed with demonstrations at +3.8 pp, batch-size tuning at +0.9 pp, and controller compensation at +1.3 pp.
This framing is nice because it treats code changes as experimental hypotheses. A branch is not just software state; it is a record of a policy-improvement idea tested on hardware.
Experiments and Results
The real-world tasks cover several forms of dexterity:
| Task | Core challenge |
|---|---|
| Push-T | non-prehensile pushing to align a T-shaped block |
| Pin insertion | plug pins into 4 mm holes |
| GPU insertion | seat GPU chips into thin motherboard sockets |
| Zip-tie cutting | grasp scissors and cut the zip-tie tail |
Success is measured as completing the task in one rollout with up to eight retries, where later retries can react to earlier failures. This is stricter in a useful way: it measures precision together with recovery under real-world uncertainty.
Several results stand out.
First, heuristic learning works in simulation but is much harder on real hardware. In Gym-PushT, Claude Code and Codex reach 95% success in about two hours, while Kimi Code takes roughly twice as long. In the real Push-T setup, two of the three agents fail, showing that real contact, friction, dynamics, and scene variation are the real test.
Second, ENPIRE can support gradient-based real-world policy improvement. For pin insertion, agents use BC, online data aggregation, and RL variants to hill-climb toward the 50-success objective. The paper reports policy convergence to 100% in pin insertion, faster than a frontier human-in-the-loop method.
Third, fleet size reduces wall-clock time. In Push-T, scaling from one to eight agent-robot pairs reduces the time to reach a 1.0 normalized score from roughly five hours to two hours. In pin insertion, scaling from one to eight agents reduces time to near-perfect success from more than 1.5 hours to about 40 minutes.
Fourth, ENPIRE can discover hybrid strategies that combine code-based policies and VLAs. In RoboCasa365, the agent improves over GR00T and zero-shot CaP-X by using perception and motion-planning tools to hover above an object before grasping. The same style of strategy transfers to the real scissors and zip-tie task.
Resource Metrics: MRU and MTU
The paper’s resource metrics are a useful contribution because robot learning is constrained by scarce hardware as well as GPU time.
Mean Robot Utilization (MRU) is the fraction of research wall-clock time during which the robot is actively executing experiments. GPU utilization measures the analogous active GPU fraction. Mean Token Utilization (MTU) measures average token consumption across the agent fleet, and the paper also tracks tokens-to-success and time-to-success.
The observed tradeoff is intuitive and important. Larger fleets reach success faster, while token consumption grows faster than ideal linear scaling. MRU drops as the fleet grows because agents spend more time reading logs, debugging, summarizing peer branches, and waiting on language-model calls. GPU utilization rises, although agents still do not saturate the hardware. The result is a real speed-cost frontier: more robots and agents buy wall-clock acceleration, while the token budget can grow disproportionately.
System Details
Each station uses two 6-DoF YAM arms with 1-DoF parallel-jaw grippers, giving 14 actuated joints across the bimanual pair. The arms use PD control with gravity compensation; the grippers use torque-limited compliant grasping so failed contact stalls safely instead of forcing rigid closure. This is a small hardware detail with large autonomy consequences: unattended robot learning needs bounded force behavior.
Perception uses Intel RealSense D405 cameras: one top-down camera and two wrist cameras for most tasks. GPU insertion adds a RealSense D435i side camera. Policies run at 30 Hz, while low-level joint controllers run at 100 Hz. Each station has a local workstation with one NVIDIA RTX 5090 32 GB GPU, an Intel Core Ultra 9 285K CPU, 128 GB RAM, Ubuntu 22.04, and CUDA 13.2.
The real-world RL integration follows a three-tier design: deployment records episodes and runs hardware, the learner trains an actor-critic with visual features, and the actor exposes a Portal/ZMQ endpoint for policy inference. Rollouts are written to disk with synchronized videos and action-source labels; a daemon ingests finalized episodes and mixes RL-generated transitions with human/manual demonstrations in an RLPD-style buffer.
Limitations
ENPIRE still leaves resource utilization on the table. Robots wait while agents inspect logs, write code, debug, summarize branches, or wait for model responses. Larger fleets reduce wall-clock time but increase coordination overhead, and token cost grows super-linearly at eight agents.
The environment construction stage also remains human-guided. A person still supplies the task objective, validates reward/reset behavior, and provides representative success and failure demonstrations. ENPIRE amortizes this work across later autonomous improvement; it does not remove it.
Finally, the success of the loop depends heavily on the quality of exposed tools. In RoboCasa, the paper identifies SAM3 perception failures on small or ambiguous objects. Prompt search and higher image resolution help, but perception remains a bottleneck. This generalizes to real hardware: if reset, verification, or perception APIs are brittle, the agent will optimize against brittle feedback.
Takeaways
ENPIRE is interesting because it gives coding agents a robotics-shaped experimental substrate. The core contribution is not a new policy architecture. It is the construction of a repeatable physical feedback loop where reward, reset, rollout, code editing, and branch-level hypothesis selection are all visible to the agent.
For robot learning, the practical lesson is that autonomy starts before policy training. A robot fleet becomes useful to coding agents only after the environment exposes safe resets, reliable rewards, structured logs, and reproducible execution. Once that substrate exists, agentic code search can improve policies, discover training recipes, and trade tokens for robot-time acceleration.