[Paper Notes] Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots
Published:
This post supports English / 中文 switching via the site language toggle in the top navigation.
TL;DR
Evolution Gym (EvoGym) is a benchmark for jointly optimizing a soft robot’s body and controller. A robot is a connected grid of fewer than 100 voxels selected from five types: empty, rigid, soft, horizontal actuator, and vertical actuator. This small material alphabet creates a large combinatorial morphology space while remaining cheap enough for repeated controller training.
The benchmark formalizes co-design as a bilevel loop. An outer optimizer proposes a body; an inner reinforcement-learning procedure trains a controller for that body; the achieved task reward becomes the body’s fitness. EvoGym supplies a fast 2D mass-spring simulator, a Gym-style Python interface, and more than 30 locomotion and manipulation environments. The paper evaluates three outer-loop methods—genetic algorithm, Bayesian optimization, and CPPN-NEAT—with PPO in the inner loop.
The central result is deliberately mixed. Automatically evolved robots outperform hand-designed morphologies on many tasks and develop recognizable structures such as grippers, feet, and friction-maximizing backs. A simple genetic algorithm is the strongest overall baseline. Every tested method fails on the hardest environments, exposing how little the field had solved about morphology-control co-optimization. EvoGym’s main contribution is therefore a standardized research problem and failure surface, not a single winning evolution algorithm.
Paper Info
The paper is “Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots” by Jagdeep Singh Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, and Wojciech Matusik from MIT CSAIL. It appeared at NeurIPS 2021.
- Paper: arXiv:2201.09863
- Project, environments, documentation, and tutorials: evogym.csail.mit.edu
1. Why Co-Design Is a Different Problem
Most robot-learning benchmarks hold the body fixed and optimize a controller:
[ \pi^\star = \arg\max_\pi J(D_{\text{fixed}},\pi;T), ]
where (D) is robot design, (\pi) is the controller, and (T) is the task. EvoGym expands the decision space:
[ (D^\star,\pi^\star) = \arg\max_{D,\pi} J(D,\pi;T). ]
This change is more difficult than adding another policy parameter. Altering the body changes observation dimension, action dimension, dynamics, contact geometry, and which behaviors are physically reachable. A body can look poor because its controller was undertrained; a controller can look poor because the body makes the task impossible. Evaluation must separate controller-training noise from morphology quality.
The paper adopts a practical bilevel approximation:
[ \pi_D^\star \approx \operatorname{OptimizeControl}(T,D), \qquad D^\star \approx \arg\max_D J(D,\pi_D^\star;T). ]
Every body evaluation contains a full control-learning problem. This nested cost is the main computational bottleneck in robot evolution and one reason earlier work used only a handful of simple environments.
2. Multi-Material Voxel Robots
EvoGym represents a robot as a material matrix (M) plus a connectivity list (C). Each cell in (M) has one of five labels:
- empty,
- rigid,
- soft,
- horizontal actuator,
- vertical actuator.
The connectivity list records links between adjacent occupied voxels. Valid designs must form a connected body and contain at least one actuator. Changing an occupied cell to empty changes topology; changing its material changes compliance or actuation.
This direct encoding is important for benchmarking. All search methods operate on the same explicit morphology, so performance differences come from optimization instead of incompatible simulators or body representations. The encoding is also expressive: a small grid and a few material types can form legs, cavities, grippers, compliant contact regions, and asymmetric appendages.
There is a trade-off. A voxel grid makes topology editing easy and simulation fast, but it limits geometry to a 2D lattice and discrete material categories. EvoGym studies algorithmic co-design under this abstraction; it does not claim that an evolved voxel body can be fabricated directly as a complete physical robot.
3. Observation and Action Change with the Body
Let (N) be the number of unique voxel corner points. Robot state includes every corner’s 2D position relative to the robot center of mass, plus center-of-mass velocity and orientation:
[ o_{\text{robot}}\in\mathbb R^{2N+3}. ]
Tasks may append a local terrain-height window and goal-specific state. Manipulation environments, for example, include the object’s orientation, velocity, and position relative to the robot.
Each action component controls one actuator voxel. The command (u\in[0.6,1.6]) specifies a target deformation as a fraction of that voxel’s rest length. Horizontal and vertical actuators change their preferred shape along different axes.
Consequently, observation and action dimensions depend on morphology. The benchmark trains a separate controller for each proposed body, avoiding the need for a universal policy architecture across arbitrary voxel counts. This choice simplifies baseline evaluation while making the inner loop expensive.
4. The 2D Soft-Body Simulator
The simulator models objects and terrain as a mass-spring system. Each voxel begins as a cross-braced square; its edges behave as ideal springs with stiffness determined by material type. The system advances with symplectic RK-4 integration.
Collision detection uses a bounding-box tree. Normal and friction forces are penalty-based and scale with penetration depth. The backend is written in C++, with Python bindings designed around the OpenAI Gym API.
The simulator’s simplicity is a feature. Co-design may require evaluating thousands of bodies, and every evaluation trains a controller. High-fidelity 3D simulation would make a benchmark-scale comparison prohibitively expensive. EvoGym chooses fast, consistent dynamics so researchers can spend computation on the co-design algorithm itself.
The same decision defines the benchmark’s boundary. Results demonstrate optimization inside the 2D mass-spring world. They provide limited evidence about 3D mechanics, actuator bandwidth, manufacturing tolerances, material hysteresis, or sim-to-real transfer.
5. More Than 30 Tasks
The environment suite spans locomotion, manipulation, and combinations of the two. Tasks are labeled easy, medium, or hard based on baseline performance.
Representative locomotion tasks include:
- Walker: maximize speed on flat ground.
- Bridge Walker: cross soft rope bridges separated by rigid supports.
- Up Stepper: climb stairs with varying step lengths.
- Climber: move upward between two walls using contact and friction.
- Traverser: cross a pit filled with rigid blocks without sinking.
Representative manipulation tasks include:
- Carrier: catch a falling object and carry it forward.
- Thrower: throw an object far while keeping the body near its start.
- Beam Slider: reach a beam resting on separated platforms and slide it forward.
- Catcher: catch a spinning object dropped from a random high position.
- Lifter: grasp an object and lift it out of a hole.
These environments force morphology to serve different functions. Flat walking rewards efficient periodic motion. Climbing rewards contact area and traction. Catching needs impact absorption and containment. Lifting requires an appendage that behaves like a gripper. A morphology optimizer that performs well across the suite must discover task-conditioned structure, not a single fast walker.
6. The Bilevel Evolution Loop
For a population of (p) designs over (n) generations, the generic loop is:
- sample or propose robot designs;
- train a controller for every design;
- evaluate the optimized design-controller pair;
- update the design optimizer from the accumulated fitness data;
- repeat and return the best pair.
In compact form,
[ \mathcal S \leftarrow \mathcal S\cup {(D_j,\pi_j,r_j)}{j=1}^{p}, \qquad D{1:p}^{\text{next}} = \operatorname{OptimizeDesigns}(\mathcal S,p). ]
The dataset (\mathcal S) preserves each body, its trained controller, and achieved reward. The outer loop sees only the expensive, noisy result of inner-loop learning.
7. Three Design Optimizers
7.1 Genetic Algorithm
The GA retains an elite fraction of the current population and constructs offspring by mutating survivors. Each voxel has a 10% probability of changing type. Mutation to or from empty edits body topology. The survivor fraction decreases from 60% toward zero over the run, shifting population turnover across generations. The implementation does not use crossover.
This baseline is intentionally simple. Its strong results later show that direct local mutation is well matched to EvoGym’s discrete grid.
7.2 Bayesian Optimization
BO treats morphology evaluation as an expensive black-box problem. It fits a Gaussian-process surrogate over categorical voxel inputs, uses batch Thompson sampling, and optimizes the acquisition function with L-BFGS.
The setting is hostile to standard BO: the categorical morphology vector is high-dimensional, topology validity is structured, and PPO introduces noisy fitness estimates. The results confirm that an inaccurate surrogate cannot guide this design space effectively.
7.3 CPPN-NEAT
A Compositional Pattern Producing Network receives voxel coordinates and outputs a material type. NEAT evolves the CPPN topology and weights. This indirect encoding favors spatial regularity and can generate repeated, smooth material patterns with few parameters.
That inductive bias often helps locomotion. It can become restrictive for manipulation, where an irregular hook, gripper, or cavity may be essential. EvoGym makes this encoding-task interaction visible.
8. PPO as the Inner-Loop Controller
Previous soft-robot evolution often optimized open-loop periodic actuation or a CPPN that generated actuator phases and frequencies. Those controllers fit regular locomotion but struggle when terrain changes or an object arrives unpredictably.
EvoGym trains a feedback policy with Proximal Policy Optimization (PPO) for every proposed morphology. The controller can react to voxel state, local terrain, and task variables. This allows non-periodic behaviors such as catching, lifting, climbing uneven geometry, and manipulating an object.
PPO greatly expands behavioral expressiveness, while multiplying evaluation cost and adding variance. The benchmark therefore tests two optimization capabilities at once: exploration of a combinatorial body space and reliable comparison of bodies under imperfect controller training.
9. What the Baselines Reveal
There is no universally best outer-loop method, but the simple GA performs best overall. CPPN-NEAT is competitive on locomotion and weak on complex manipulation. BO performs poorly on most tasks.
The likely reasons align with each method’s inductive bias:
- GA mutations make local, discrete changes directly in the evaluated representation.
- CPPN-NEAT favors regular morphology, which suits repeated locomotion patterns.
- BO must learn a smooth surrogate over a noisy, high-dimensional categorical space with strong topology constraints.
The result is valuable because it resists an easy “more sophisticated optimizer wins” story. Representation and task structure determine which search bias is useful.
10. Morphology Actually Changes During Evolution
The qualitative sequences show increasing functional specialization.
In Carrier, early survivors already contain legs and a pocket-like region that catches the falling object. Later generations improve locomotion while retaining containment. Fitness couples two body requirements: object security and forward speed.
In Lifter, evolution creates a parallel-gripper-like structure beneath the body. This structure is absent from the initial random population and emerges because lifting rewards a specific manipulation affordance.
In Bridge Walker, successful bodies grow a large front foot. The increased contact area and friction help the robot move across compliant rope bridges.
These examples are the paper’s strongest illustration of embodied intelligence. The controller learns how to use a body, and the outer loop changes the body so that useful control strategies become easier or possible.
11. Comparison with Hand-Designed Robots
The authors build several bio-inspired robots manually and optimize their controllers with the same PPO procedure. On every evaluated task, at least one co-design method finds a body that outperforms the hand-designed set.
The Climber result is especially revealing. Human designs struggle to balance upward propulsion and wall traction. Evolution finds legs that generate progress, a long flat back that increases frictional contact, and a central hole that supports the resulting gait.
Some tasks are closer. A natural hand-designed Carrier nearly matches the best evolved designs because the required morphology—a container plus locomotor appendages—is intuitive. On Beam Slider, neither human nor evolved designs are satisfactory. One GA robot moves the beam across several supports, yet remains far from an effective solution.
The conclusion is narrower than “evolution beats engineers.” Automated search helps when body-task interactions are unintuitive, and the hardest tasks defeat both search and intuition.
12. Strengths and Limitations
Strengths. EvoGym turns scattered soft-robot evolution experiments into a comparable benchmark. It standardizes morphology, simulation, task interfaces, and evaluation while allowing researchers to replace the co-design algorithm. The task suite goes beyond flat-ground locomotion and includes contact-rich manipulation. Open-source code and a familiar Gym interface lower the barrier to entry.
Limitations. The world is 2D, voxelized, and simulated with simplified mass-spring physics. Morphologies use a small discrete material set, and the paper does not demonstrate fabrication or real-world transfer. A separate PPO controller is trained from scratch for each body, making body evaluation costly and noisy. Direct encoding also scales poorly as grid resolution increases.
Difficulty labels depend on the chosen baselines, so “hard” means unsolved by these algorithms under their budgets. The benchmark can expose failure without identifying whether it comes from design search, controller optimization, reward shaping, or insufficient computation.
13. My Takeaway
Evolution Gym’s enduring idea is to treat morphology as part of the learning problem and give that problem a reproducible interface. Its voxel world is deliberately modest, yet the bilevel optimization structure captures a fundamental challenge: every body defines a new control problem, and every controller determines how that body is judged.
The strongest research opportunities follow directly from this bottleneck: reuse controllers across related morphologies, learn morphology-conditioned value estimates, differentiate through physics, jointly update body and policy, discover compact generative encodings, and optimize bodies for multiple tasks.
EvoGym should therefore be read as an invitation to improve co-design methodology. Its hardest environments are useful precisely because the included baselines fail. A benchmark becomes valuable when it measures progress that has not happened yet.