# OpenEnv Custom Environment Creation: A Complete Developer's Guide

> Create custom AI environments with OpenEnv. This guide details how to define, package, deploy, and consume isolated execution environments for agentic reinforcement learning.

- Repository: [Hugging Face/OpenEnv](https://github.com/huggingface/OpenEnv)
- Tags: how-to-guide
- Published: 2026-06-16

---

**Yes, OpenEnv is explicitly designed for custom environment creation, providing a full-stack framework that lets you define, package, deploy, and consume isolated execution environments for agentic reinforcement learning through typed APIs and containerized deployment.**

OpenEnv is a Hugging Face framework that streamlines the development of isolated reinforcement learning environments. Whether you're building game simulations, robotics controllers, or data-driven tasks, OpenEnv custom environment creation follows a standardized pattern that separates environment logic from client access while ensuring type safety across the distributed stack.

## Core Architecture for Custom Environments

### The Environment Base Class

The foundation of OpenEnv custom environment creation lies in [`src/openenv/core/env_server/interfaces.py`](https://github.com/huggingface/OpenEnv/blob/main/src/openenv/core/env_server/interfaces.py). This module defines the abstract **Environment** class that implements the Gym-style API requiring `reset()`, `step()`, and `state` methods. You subclass this to encode your custom dynamics, using the generic type signature `Environment[Act, Obs, State]` to ensure type safety across the client-server boundary.

### Typed Data Models

Action, Observation, and State containers are defined in your environment's [`models.py`](https://github.com/huggingface/OpenEnv/blob/main/models.py) file (scaffolded at `envs/<my_env>/models.py`). These Pydantic-like typed data structures ensure that both the client and server share consistent data contracts. The **Echo** environment in [`envs/echo_env/models.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/echo_env/models.py) demonstrates this pattern with strictly typed fields for messages and responses.

### FastAPI Server Wrapper

Located in [`src/openenv/core/env_server/http_server.py`](https://github.com/huggingface/OpenEnv/blob/main/src/openenv/core/env_server/http_server.py), the server wrapper transforms your `Environment` subclass into a WebSocket service exposing standardized endpoints at `/reset`, `/step`, and `/state`. The generated [`app.py`](https://github.com/huggingface/OpenEnv/blob/main/app.py) in your environment's `server/` directory utilizes this wrapper to create the FastAPI entry point.

### Containerized Execution Runtime

OpenEnv achieves isolation through Docker-based container providers in `src/openenv/core/containers/runtime/`. These providers support local Docker, Docker Swarm, and Kubernetes orchestration, ensuring that custom dependencies—such as Unity binaries, Atari emulators, or external simulators—do not pollute the host system.

### Client Interface

The `EnvClient` class in [`src/openenv/core/env_client.py`](https://github.com/huggingface/OpenEnv/blob/main/src/openenv/core/env_client.py) provides an async client that communicates with the server over WebSocket. It includes a synchronous `.sync()` wrapper for compatibility with standard RL training loops, allowing both async and sync consumption patterns without code changes.

## CLI Scaffolding and Project Structure

The `openenv init <name>` command generates a complete project scaffold from templates located in `src/openenv/cli/templates/openenv_env/`. This creates a standardized directory structure:

- [`models.py`](https://github.com/huggingface/OpenEnv/blob/main/models.py) – Type definitions for Action, Observation, and State
- `server/` – Directory containing your `Environment` subclass and [`app.py`](https://github.com/huggingface/OpenEnv/blob/main/app.py) FastAPI entry point
- `Dockerfile` – Container definition for isolated deployment
- [`openenv.yaml`](https://github.com/huggingface/OpenEnv/blob/main/openenv.yaml) – Manifest describing the environment to the CLI and deployment pipeline
- [`client.py`](https://github.com/huggingface/OpenEnv/blob/main/client.py) – Thin wrapper extending `EnvClient` for your specific types

## Step-by-Step OpenEnv Custom Environment Creation Workflow

1. **Scaffold** – Execute `openenv init my_custom_env` to generate the complete directory tree and boilerplate files.

2. **Define Models** – Edit [`models.py`](https://github.com/huggingface/OpenEnv/blob/main/models.py) to specify the fields your actions, observations, and internal state require, inheriting from `Action`, `Observation`, and `State` base classes.

3. **Implement Logic** – Subclass `Environment[YourAction, YourObservation, YourState]` in [`server/my_custom_environment.py`](https://github.com/huggingface/OpenEnv/blob/main/server/my_custom_environment.py), implementing the `reset()`, `step()`, and `state` methods.

4. **Configure Transforms** – Optionally attach `Transform[Obs]` objects to modify observations or compute rewards without altering core environment logic.

5. **Add Rubrics** – For complex, trajectory-based reward calculation, implement a `Rubric` object using the framework in `src/openenv/core/rubrics/` instead of modifying your `step()` method.

6. **Serve Locally** – Run `openenv serve my_custom_env` to start the FastAPI server at `http://localhost:8000` without Docker overhead for rapid iteration.

7. **Deploy** – Execute `openenv build my_custom_env` to create a Docker image, then `openenv push my_custom_env` to upload to a Hugging Face Space, making the environment reachable via `EnvClient` from anywhere.

## Complete Implementation Example: The Echo Environment

Below is a self-contained implementation that mirrors the tutorial in [`docs/source/guides/first-environment.md`](https://github.com/huggingface/OpenEnv/blob/main/docs/source/guides/first-environment.md). Copy this into [`my_custom_env/server/echo_environment.py`](https://github.com/huggingface/OpenEnv/blob/main/my_custom_env/server/echo_environment.py):

```python
from openenv.core.env_server.interfaces import Environment, Action, Observation, State

# ---- 1. Define the typed models -------------------------------------------------

class EchoAction(Action):
    """Message to echo."""
    message: str

class EchoObservation(Observation):
    """Result returned to the client."""
    echo: str
    reward: float = 0.0
    done: bool = False

class EchoState(State):
    """Internal episode state."""
    last_message: str = ""

# ---- 2. Implement the environment logic -----------------------------------------

class EchoEnvironment(Environment[EchoAction, EchoObservation, EchoState]):
    """A trivial environment that echoes back the submitted string."""

    def reset(self, seed=None, episode_id=None, **kwargs) -> EchoObservation:
        self._state = EchoState()
        return EchoObservation(echo="Ready!")

    def step(self, action: EchoAction, timeout_s=None, **kwargs) -> EchoObservation:
        self._state.last_message = action.message
        return EchoObservation(echo=action.message)

    @property
    def state(self) -> EchoState:
        return self._state

```

**Running the environment locally:**

```bash

# From the repository root

openenv init my_echo_env          # scaffolds the folder structure

# Replace the generated server file with the snippet above

openenv serve my_echo_env         # starts FastAPI at http://localhost:8000

```

**Client usage (async pattern):**

```python
import asyncio
from my_echo_env.client import EchoEnv, EchoAction

async def demo():
    async with EchoEnv(base_url="http://localhost:8000") as env:
        await env.reset()
        result = await env.step(EchoAction(message="Hello, OpenEnv!"))
        print(result.observation.echo)   # → Hello, OpenEnv!

asyncio.run(demo())

```

The same client code functions identically after you deploy the environment to a Hugging Face Space using `openenv push`.

## Advanced Features for Complex Simulations

### Plugin Transforms

You can attach `Transform[Obs]` objects that modify observations before they reach the client. This pattern supports reward shaping, information masking, or observation preprocessing without cluttering the core environment logic in your `step()` method.

### Rubric Integration

For environments requiring complex, trajectory-based reward calculation, OpenEnv provides a `Rubric` abstraction in `src/openenv/core/rubrics/`. This allows you to define evaluation criteria that span multiple timesteps while keeping your primary `Environment` implementation focused on state transition dynamics.

## Reference Implementations

The repository includes two canonical examples demonstrating OpenEnv custom environment creation patterns:

- **`envs/echo_env/`** – A minimal reference showing the basic scaffold structure and typed model definitions.
- **[`envs/coding_env/server/python_codeact_env.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/coding_env/server/python_codeact_env.py)** – A complex implementation featuring sandboxed Python code execution with comprehensive state management and security isolation.

## Summary

- **Typed API Architecture** – The generic `Environment[Act, Obs, State]` abstraction enables type-safe custom environment creation for any simulation domain.
- **CLI Automation** – The `openenv init` command generates complete project scaffolds including [`models.py`](https://github.com/huggingface/OpenEnv/blob/main/models.py), `Dockerfile`, and [`openenv.yaml`](https://github.com/huggingface/OpenEnv/blob/main/openenv.yaml) manifest.
- **Containerized Isolation** – Docker-based runtime in `src/openenv/core/containers/runtime/` guarantees that custom dependencies remain isolated from host systems.
- **Dual Client APIs** – Both async and synchronous client interfaces are auto-generated, ensuring compatibility with downstream RL libraries like TRL or SkyRL.
- **Extensible Reward Systems** – Transform and Rubric plugins in `src/openenv/core/rubrics/` support sophisticated reward shaping without modifying core environment logic.

## Frequently Asked Questions

### What programming languages does OpenEnv support for custom environment creation?

OpenEnv primarily supports **Python** for environment logic, as the `Environment` base class and type system rely on Python's type hints and Pydantic-like models defined in [`src/openenv/core/env_server/interfaces.py`](https://github.com/huggingface/OpenEnv/blob/main/src/openenv/core/env_server/interfaces.py). However, the containerized architecture allows you to wrap external binaries or services written in other languages within the generated `Dockerfile`, making them accessible through the Python server wrapper.

### How does OpenEnv handle environment state persistence across episodes?

The `State` model defined in your environment's [`models.py`](https://github.com/huggingface/OpenEnv/blob/main/models.py) encapsulates internal episode state, which your `Environment` subclass manages through the `reset()` and `step()` methods. The FastAPI server implementation in [`src/openenv/core/env_server/http_server.py`](https://github.com/huggingface/OpenEnv/blob/main/src/openenv/core/env_server/http_server.py) maintains state instances across WebSocket connections, while the `EnvClient` in [`src/openenv/core/env_client.py`](https://github.com/huggingface/OpenEnv/blob/main/src/openenv/core/env_client.py) automatically handles session lifecycle management and timeout configurations via the `timeout_s` parameter.

### Can I deploy OpenEnv environments to cloud providers other than Hugging Face?

Yes. While the `openenv push` command targets Hugging Face Spaces by default, the containerized architecture using providers in `src/openenv/core/containers/runtime/` supports Docker Swarm, Kubernetes, and custom container registries. The generated `Dockerfile` and [`openenv.yaml`](https://github.com/huggingface/OpenEnv/blob/main/openenv.yaml) manifest provide portable deployment artifacts suitable for any container orchestration platform, including AWS ECS, Google Cloud Run, or Azure Container Instances.

### Does OpenEnv require Docker for local development and testing?

No. You can develop and test environments locally using `openenv serve`, which starts the FastAPI server directly at `http://localhost:8000` without containerization overhead. Docker is only required when you execute `openenv build` to create production images or `openenv push` to deploy remotely, ensuring isolated execution of custom dependencies in production environments.