# evorl | EMI-Group | Knowledge Base | Instagit

EvoRL is a fully GPU-accelerated framework for Evolutionary Reinforcement Learning, implemented with JAX. It supports Reinforcement Learning (RL), Evolutionary Computation (EC), Evolution-guided Reinforcement Learning (ERL), AutoRL, and seamless integration with GPU-optimized simulation environments.

GitHub Stars: 264

Repository: https://github.com/emi-group/evorl

---

## Articles

### [How to Debug JIT Compilation Errors and Profile JAX Training Pipelines in EvoRL](/emi-group/evorl/how-to-debug-jit-compilation-errors-and-profile-jax-pipelines)

Debug JIT errors and profile JAX training in EvoRL easily. Disable JIT, inspect compiled code with jax.debug.print, and analyze performance using jax.profiler and TensorBoard.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Create Custom Neural Network Architectures for Policies in EvoRL](/emi-group/evorl/how-to-create-custom-neural-network-architectures-for-policies)

Learn to build custom neural network architectures for policies in EvoRL. Use the flexible make_policy_network factory to easily assemble MLP, CNN, and normalization components for your AI agents.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Save and Load Model Checkpoints Using Orbax in EvoRL](/emi-group/evorl/how-to-save-and-load-model-checkpoints-with-orbax)

Learn to efficiently save and load model checkpoints using Orbax in EvoRL. Explore helper functions and the Checkpoint Manager for seamless training.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement On-Policy Algorithms (PPO, A2C, IMPALA) in EvoRL](/emi-group/evorl/how-to-implement-on-policy-algorithms-ppo-a2c-impala-evorl)

Implement PPO A2C and IMPALA on policy algorithms in EvoRL easily. Discover how EvoRL's modular workflow simplifies training pipelines with custom agent factories and loss functions.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement Off-Policy Algorithms (SAC, TD3, DDPG) in EvoRL: A Complete Guide](/emi-group/evorl/how-to-implement-off-policy-algorithms-sac-td3-ddpg-evorl)

Easily implement SAC, TD3, and DDPG off-policy algorithms in EvoRL. Follow our guide to create agent classes and leverage OffPolicyWorkflowTemplate for efficient training.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Optimize Training Performance Using `jax.jit` and `jax.vmap` in EvoRL](/emi-group/evorl/how-to-optimize-training-performance-with-jax-jit-and-jax-vmap)

Optimize EvoRL training with jax.jit and jax.vmap. Compile loops and vectorize operations for faster execution and better GPU/TPU use. Eliminate Python overhead.

- Tags: performance
- Published: 2026-03-01

### [How to Integrate Custom Evolutionary Computation (EC) Components with RL Workflows in EvoRL](/emi-group/evorl/how-to-integrate-custom-ec-components-with-rl-workflows)

Integrate custom EC components into EvoRL by implementing the EvoOptimizer abstract base class with init ask and tell methods. Learn how to inject your optimizer into hybrid ERL or pure EC workflows in this guide.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Leverage EvoRL's Object-Oriented Functional Programming Model with JAX jit](/emi-group/evorl/how-to-use-oo-functional-programming-model-with-jax-jit)

Learn how EvoRL uses JAX jit with its object-oriented functional programming model. Discover how class instances act as static containers for pure functional kernels and immutable pytrees.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Create Custom Environment Wrappers in EvoRL: A Complete Guide](/emi-group/evorl/how-to-create-custom-environment-wrappers-actionsquashwrapper)

Learn to create custom environment wrappers in EvoRL by subclassing Wrapper. Override step and expose new spaces to transform actions, observations, and rewards effectively.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Set Up Logging with LogRecorder and WandbRecorder in EvoRL](/emi-group/evorl/how-to-set-up-logging-with-logrecorder-and-wandbrecorder)

Learn to set up logging in EvoRL with LogRecorder and WandbRecorder. Effortlessly track local metrics and visualize progress on Weights & Biases dashboards. Get started now.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement Cross-Entropy Method for RL (CEM-RL) with Covariance Adaptation in EvoRL](/emi-group/evorl/how-to-implement-cem-rl-with-covariance-adaptation-evorl)

Implement Cross-Entropy Method for RL with covariance adaptation in EvoRL. Configure SepCEM optimizer with ExponentialScheduleSpec for adaptive diagonal covariance decay. Get started today.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Use PyTreeDict, PyTreeData, and PyTreeNode for JAX-Compatible Data Structures in EvoRL](/emi-group/evorl/how-to-use-pytredict-pytreedata-pytreenode-jax-data-structures)

Leverage PyTreeDict, PyTreeData, and PyTreeNode in EvoRL for JAX-compatible data structures. Effectively manage configurations, immutable data, and mutable state with JAX's JIT compilation and vectorization.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement Evolution-guided Reinforcement Learning (ERL) Workflows in EvoRL](/emi-group/evorl/how-to-implement-evolution-guided-reinforcement-learning-erl-workflows)

Learn to implement Evolution-guided Reinforcement Learning ERL workflows in EvoRL. This guide explains the dual-workflow architecture combining gradient-based RL and evolutionary optimization.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Use Distributed Training with Multiple GPUs and Data Sharding in EvoRL](/emi-group/evorl/how-to-use-distributed-training-with-multiple-gpus-and-sharding)

Learn distributed training with multiple GPUs and data sharding in EvoRL. Optimize large-scale population evolution using JAX pmap and shard_map primitives. Get started today!

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement Custom Evaluators for Agent Evaluation in EvoRL](/emi-group/evorl/how-to-implement-custom-evaluators-for-agent-evaluation)

Learn to implement custom evaluators for agent evaluation in EvoRL. Subclass the Evaluator class, define metrics, and override the evaluate method for custom metrics.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Use the ReplayBuffer for Off-Policy Learning Algorithms in EvoRL](/emi-group/evorl/how-to-use-replaybuffer-for-off-policy-learning-algorithms)

Learn to use the EvoRL ReplayBuffer for off-policy algorithms. Discover JAX-compatible methods add sample and can_sample for efficient trajectory storage and retrieval for TD3 and SAC.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement Multi-Agent RL Algorithms Using JaxMARL Environments in EvoRL](/emi-group/evorl/how-to-implement-multi-agent-rl-algorithms-using-jaxmarl-environments)

Learn to implement multi-agent RL algorithms in EvoRL using JaxMARL environments. Master batching and centralized rollouts for efficient training of shared policy networks.

- Tags: tutorial
- Published: 2026-03-01

### [How to Configure Population-Based Training (PBT) with Hydra Configuration Files in EvoRL](/emi-group/evorl/how-to-configure-population-based-training-pbt-with-hydra-configs)

Learn to configure Population-Based Training PBT in EvoRL using Hydra. Combine PBT and RL workflows, define population size, hyperparameter search spaces, and exploit-explore ratios.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement Evolutionary Algorithms Using the EC Module in EvoRL (OpenES, CMA-ES)](/emi-group/evorl/how-to-implement-evolutionary-algorithms-using-ec-module-evorl)

Learn to implement evolutionary algorithms like OpenES and CMA-ES using the EvoRL EC module. Integrate advanced EC into your RL workflows easily with this plug-and-play solution.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Use SampleBatch for Trajectory Data Storage and Processing in EvoRL Rollouts](/emi-group/evorl/how-to-use-samplebatch-for-trajectory-data-storage-and-processing)

Learn how to use SampleBatch in EvoRL for efficient trajectory data storage and processing. Optimize rollouts with JAX-compatible PyTree containers for batching and advantage computation.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Add Custom Loss Functions to an Agent in EvoRL for Gradient-Based Updates](/emi-group/evorl/how-to-add-custom-loss-functions-to-agent-in-evorl)

Learn to add custom loss functions in EvoRL for gradient-based updates. Implement new loss methods in your Agent subclass and register weights in your config for seamless integration.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Integrate Different Environment Backends (Brax, Gymnax, and MuJoCo Playground) in EvoRL](/emi-group/evorl/how-to-integrate-different-environment-backends-brax-gymnax-mujocho-playground)

Integrate Brax Gymnax and MuJoCo Playground in EvoRL using adapters training wrappers and a factory function Learn how EvoRL unifies diverse environment backends for streamlined reinforcement learning.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Implement a Custom Workflow Class for Training Loop Logic in EvoRL](/emi-group/evorl/how-to-implement-custom-workflow-class-for-training-loop-logic)

Learn to implement a custom Workflow class for training loop logic in EvoRL. Subclass base workflows, implement factory methods, and register your class. Optimize your RL training.

- Tags: how-to-guide
- Published: 2026-03-01

### [How to Create a Custom Agent in EvoRL with compute_actions and evaluate_actions Methods](/emi-group/evorl/how-to-create-custom-agent-in-evorl-with-compute_actions-and-evaluate_actions-methods)

Learn to create a custom Agent in EvoRL by implementing compute_actions and evaluate_actions methods. Subclass the Agent class for flexible agent development.

- Tags: how-to-guide
- Published: 2026-03-01