OpenEnv Custom Environment Creation: A Complete Developer's Guide

Yes, OpenEnv is explicitly designed for custom environment creation, providing a full-stack framework that lets you define, package, deploy, and consume isolated execution environments for agentic reinforcement learning through typed APIs and containerized deployment.

OpenEnv is a Hugging Face framework that streamlines the development of isolated reinforcement learning environments. Whether you're building game simulations, robotics controllers, or data-driven tasks, OpenEnv custom environment creation follows a standardized pattern that separates environment logic from client access while ensuring type safety across the distributed stack.

Core Architecture for Custom Environments

The Environment Base Class

The foundation of OpenEnv custom environment creation lies in src/openenv/core/env_server/interfaces.py. This module defines the abstract Environment class that implements the Gym-style API requiring reset(), step(), and state methods. You subclass this to encode your custom dynamics, using the generic type signature Environment[Act, Obs, State] to ensure type safety across the client-server boundary.

Typed Data Models

Action, Observation, and State containers are defined in your environment's models.py file (scaffolded at envs/<my_env>/models.py). These Pydantic-like typed data structures ensure that both the client and server share consistent data contracts. The Echo environment in envs/echo_env/models.py demonstrates this pattern with strictly typed fields for messages and responses.

FastAPI Server Wrapper

Located in src/openenv/core/env_server/http_server.py, the server wrapper transforms your Environment subclass into a WebSocket service exposing standardized endpoints at /reset, /step, and /state. The generated app.py in your environment's server/ directory utilizes this wrapper to create the FastAPI entry point.

Containerized Execution Runtime

OpenEnv achieves isolation through Docker-based container providers in src/openenv/core/containers/runtime/. These providers support local Docker, Docker Swarm, and Kubernetes orchestration, ensuring that custom dependencies—such as Unity binaries, Atari emulators, or external simulators—do not pollute the host system.

Client Interface

The EnvClient class in src/openenv/core/env_client.py provides an async client that communicates with the server over WebSocket. It includes a synchronous .sync() wrapper for compatibility with standard RL training loops, allowing both async and sync consumption patterns without code changes.

CLI Scaffolding and Project Structure

The openenv init <name> command generates a complete project scaffold from templates located in src/openenv/cli/templates/openenv_env/. This creates a standardized directory structure:

• models.py – Type definitions for Action, Observation, and State
• server/ – Directory containing your Environment subclass and app.py FastAPI entry point
• Dockerfile – Container definition for isolated deployment
• openenv.yaml – Manifest describing the environment to the CLI and deployment pipeline
• client.py – Thin wrapper extending EnvClient for your specific types

Step-by-Step OpenEnv Custom Environment Creation Workflow

1. Scaffold – Execute openenv init my_custom_env to generate the complete directory tree and boilerplate files.

2. Define Models – Edit models.py to specify the fields your actions, observations, and internal state require, inheriting from Action, Observation, and State base classes.

3. Implement Logic – Subclass Environment[YourAction, YourObservation, YourState] in server/my_custom_environment.py, implementing the reset(), step(), and state methods.

4. Configure Transforms – Optionally attach Transform[Obs] objects to modify observations or compute rewards without altering core environment logic.

5. Add Rubrics – For complex, trajectory-based reward calculation, implement a Rubric object using the framework in src/openenv/core/rubrics/ instead of modifying your step() method.

6. Serve Locally – Run openenv serve my_custom_env to start the FastAPI server at http://localhost:8000 without Docker overhead for rapid iteration.

7. Deploy – Execute openenv build my_custom_env to create a Docker image, then openenv push my_custom_env to upload to a Hugging Face Space, making the environment reachable via EnvClient from anywhere.

Complete Implementation Example: The Echo Environment

Below is a self-contained implementation that mirrors the tutorial in docs/source/guides/first-environment.md. Copy this into my_custom_env/server/echo_environment.py:

from openenv.core.env_server.interfaces import Environment, Action, Observation, State

# ---- 1. Define the typed models -------------------------------------------------

class EchoAction(Action):
    """Message to echo."""
    message: str

class EchoObservation(Observation):
    """Result returned to the client."""
    echo: str
    reward: float = 0.0
    done: bool = False

class EchoState(State):
    """Internal episode state."""
    last_message: str = ""

# ---- 2. Implement the environment logic -----------------------------------------

class EchoEnvironment(Environment[EchoAction, EchoObservation, EchoState]):
    """A trivial environment that echoes back the submitted string."""

    def reset(self, seed=None, episode_id=None, **kwargs) -> EchoObservation:
        self._state = EchoState()
        return EchoObservation(echo="Ready!")

    def step(self, action: EchoAction, timeout_s=None, **kwargs) -> EchoObservation:
        self._state.last_message = action.message
        return EchoObservation(echo=action.message)

    @property
    def state(self) -> EchoState:
        return self._state

Running the environment locally:

# From the repository root

openenv init my_echo_env          # scaffolds the folder structure

# Replace the generated server file with the snippet above

openenv serve my_echo_env         # starts FastAPI at http://localhost:8000

Client usage (async pattern):

import asyncio
from my_echo_env.client import EchoEnv, EchoAction

async def demo():
    async with EchoEnv(base_url="http://localhost:8000") as env:
        await env.reset()
        result = await env.step(EchoAction(message="Hello, OpenEnv!"))
        print(result.observation.echo)   # → Hello, OpenEnv!

asyncio.run(demo())

The same client code functions identically after you deploy the environment to a Hugging Face Space using openenv push.

Advanced Features for Complex Simulations

Plugin Transforms

You can attach Transform[Obs] objects that modify observations before they reach the client. This pattern supports reward shaping, information masking, or observation preprocessing without cluttering the core environment logic in your step() method.

Rubric Integration

For environments requiring complex, trajectory-based reward calculation, OpenEnv provides a Rubric abstraction in src/openenv/core/rubrics/. This allows you to define evaluation criteria that span multiple timesteps while keeping your primary Environment implementation focused on state transition dynamics.

Reference Implementations

The repository includes two canonical examples demonstrating OpenEnv custom environment creation patterns:

• envs/echo_env/ – A minimal reference showing the basic scaffold structure and typed model definitions.
• envs/coding_env/server/python_codeact_env.py – A complex implementation featuring sandboxed Python code execution with comprehensive state management and security isolation.

Summary

• Typed API Architecture – The generic Environment[Act, Obs, State] abstraction enables type-safe custom environment creation for any simulation domain.
• CLI Automation – The openenv init command generates complete project scaffolds including models.py, Dockerfile, and openenv.yaml manifest.
• Containerized Isolation – Docker-based runtime in src/openenv/core/containers/runtime/ guarantees that custom dependencies remain isolated from host systems.
• Dual Client APIs – Both async and synchronous client interfaces are auto-generated, ensuring compatibility with downstream RL libraries like TRL or SkyRL.
• Extensible Reward Systems – Transform and Rubric plugins in src/openenv/core/rubrics/ support sophisticated reward shaping without modifying core environment logic.

Frequently Asked Questions

What programming languages does OpenEnv support for custom environment creation?

OpenEnv primarily supports Python for environment logic, as the Environment base class and type system rely on Python's type hints and Pydantic-like models defined in src/openenv/core/env_server/interfaces.py. However, the containerized architecture allows you to wrap external binaries or services written in other languages within the generated Dockerfile, making them accessible through the Python server wrapper.

How does OpenEnv handle environment state persistence across episodes?

The State model defined in your environment's models.py encapsulates internal episode state, which your Environment subclass manages through the reset() and step() methods. The FastAPI server implementation in src/openenv/core/env_server/http_server.py maintains state instances across WebSocket connections, while the EnvClient in src/openenv/core/env_client.py automatically handles session lifecycle management and timeout configurations via the timeout_s parameter.

Can I deploy OpenEnv environments to cloud providers other than Hugging Face?

Yes. While the openenv push command targets Hugging Face Spaces by default, the containerized architecture using providers in src/openenv/core/containers/runtime/ supports Docker Swarm, Kubernetes, and custom container registries. The generated Dockerfile and openenv.yaml manifest provide portable deployment artifacts suitable for any container orchestration platform, including AWS ECS, Google Cloud Run, or Azure Container Instances.

Does OpenEnv require Docker for local development and testing?

No. You can develop and test environments locally using openenv serve, which starts the FastAPI server directly at http://localhost:8000 without containerization overhead. Docker is only required when you execute openenv build to create production images or openenv push to deploy remotely, ensuring isolated execution of custom dependencies in production environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}