# OpenEnv Tutorial: Complete Guide to Building, Deploying, and Training Agentic Environments

> Master OpenEnv with our complete tutorial. Learn to build, deploy, and train agentic environments using the OpenEnv repository. Start your RL journey today.

- Repository: [Hugging Face/OpenEnv](https://github.com/huggingface/OpenEnv)
- Tags: tutorial
- Published: 2026-06-16

---

**OpenEnv ships with a comprehensive, step-by-step tutorial series that guides users from environment creation to production deployment and RL training, housed in the `tutorial/` directory of the huggingface/OpenEnv repository.**

The huggingface/OpenEnv repository provides a modular OpenEnv tutorial designed to onboard developers through the entire lifecycle of agentic environment development. This learning resource progresses from core architectural concepts to scalable production deployments, featuring hands-on code examples, command-line workflows, and an interactive Jupyter notebook that demonstrates end-to-end GRPO training.

## OpenEnv Tutorial Structure and Learning Path

The tutorial is organized as a modular curriculum anchored by [`tutorial/README.md`](https://github.com/huggingface/OpenEnv/blob/main/tutorial/README.md), which serves as the central entry point. According to the source code, the learning path follows four distinct phases:

- **Fundamentals** – Understanding the Gymnasium-style API (`reset`, `step`, `state`) and WebSocket-based client architecture
- **Local Development** – Scaffolding environments with the CLI and testing them locally
- **Deployment** – Containerizing with Docker and pushing to Hugging Face Spaces
- **Scaling & Training** – Production load balancing and integrating with TRL for reinforcement learning

Each phase corresponds to a dedicated markdown file in the `tutorial/` directory, supplemented by runnable code in `tutorial/examples/`.

## Core Concepts in [`01-environments.md`](https://github.com/huggingface/OpenEnv/blob/main/01-environments.md)

The first module, located at [`tutorial/01-environments.md`](https://github.com/huggingface/OpenEnv/blob/main/tutorial/01-environments.md), establishes the architectural foundation of OpenEnv. This section explains how environments expose a standard **Gymnasium-style API** through methods like `reset()` and `step()`, while maintaining state via WebSocket connections.

Key implementations covered include:
- The **`EnvClient`** class, which manages asynchronous communication with environment servers
- **OpenSpiel integration**, demonstrating how to wrap existing game implementations
- The **`CallToolAction`** pattern for structuring agent interactions

The tutorial emphasizes that every environment must implement three core primitives: state initialization, action processing, and observation rendering.

## Local Development and Deployment ([`02-deployment.md`](https://github.com/huggingface/OpenEnv/blob/main/02-deployment.md))

The second phase, documented in [`tutorial/02-deployment.md`](https://github.com/huggingface/OpenEnv/blob/main/tutorial/02-deployment.md), transitions from theory to implementation using the OpenEnv CLI. Developers scaffold new projects using:

```bash
openenv init my_game
cd my_game
pip install -e .

```

Local testing uses the auto-reload server:

```bash
openenv serve --reload

```

For production, the tutorial details containerization workflows:
1. Building Docker images with the generated `Dockerfile`
2. Testing containers locally before deployment
3. Pushing to Hugging Face Spaces via `openenv push --repo-id your-username/my_game_env`

This module also covers CI/CD pipeline configuration for automated testing and deployment.

## Production Scaling Strategies ([`03-scaling.md`](https://github.com/huggingface/OpenEnv/blob/main/03-scaling.md))

The third tutorial module addresses high-throughput scenarios in [`tutorial/03-scaling.md`](https://github.com/huggingface/OpenEnv/blob/main/tutorial/03-scaling.md). This section moves beyond single-container deployments to explore **WebSocket load balancing** and horizontal scaling patterns.

Implementation details include:
- Configuring the **`LocalDockerProvider`** for multi-container orchestration on a single host
- Kubernetes provider setup for cloud-native deployments
- Latency benchmarking results and optimization strategies
- Connection pooling and session management for concurrent agent interactions

The tutorial provides configuration examples for running environments behind reverse proxies while maintaining real-time WebSocket connectivity.

## End-to-End Training with [`04-training.md`](https://github.com/huggingface/OpenEnv/blob/main/04-training.md)

The final instructional component, [`tutorial/04-training.md`](https://github.com/huggingface/OpenEnv/blob/main/tutorial/04-training.md), bridges environment development with modern RL pipelines. This module features a complete **GRPO (Generalized Reward-Penalty Optimization) training implementation** using the Wordle environment.

Key technical elements:
- Integration with the **TRL (Transformer Reinforcement Learning)** library
- Reward shaping strategies for linguistic environments
- Distributed training configuration across multiple environment instances

The accompanying **`tutorial/examples/OpenEnv_Tutorial.ipynb`** notebook provides a ready-to-run Colab implementation, allowing users to train LLM agents without local GPU infrastructure.

## Hands-On Code Examples

### Connecting to a Remote Environment

The tutorial demonstrates client-side interaction using the Echo environment example from `envs/echo_env/`:

```python
import asyncio
from echo_env import CallToolAction, EchoEnv

async def main():
    # Connect to a running Echo Space

    async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as client:
        # Start a fresh episode

        result = await client.reset()
        print(result.observation.echoed_message)   # → "Echo environment ready!"

        # Send a message to the environment

        result = await client.step(
            CallToolAction(
                tool_name="echo_message",
                arguments={"message": "Hello, OpenEnv!"},
            )
        )
        print(result.observation.result)          # → "Hello, OpenEnv!"

        print(result.reward)                      # Reward for the action

asyncio.run(main())

```

### Running the Tutorial Notebook Locally

For users preferring local execution over Colab:

```bash

# Clone the repository

git clone https://github.com/huggingface/OpenEnv.git
cd OpenEnv

# Install dependencies

pip install -e .
pip install jupyterlab

# Launch the interactive tutorial

jupyter lab tutorial/examples/OpenEnv_Tutorial.ipynb

```

## Summary

- The **OpenEnv tutorial** is a four-part modular curriculum covering fundamentals, deployment, scaling, and training
- Entry point is [`tutorial/README.md`](https://github.com/huggingface/OpenEnv/blob/main/tutorial/README.md), with deep dives in [`01-environments.md`](https://github.com/huggingface/OpenEnv/blob/main/01-environments.md) through [`04-training.md`](https://github.com/huggingface/OpenEnv/blob/main/04-training.md)
- Environments implement a **Gymnasium-style API** (`reset`, `step`) over WebSocket connections via `EnvClient`
- The CLI provides `openenv init`, `openenv serve`, and `openenv push` for the full development lifecycle
- Production scaling uses **LocalDockerProvider** or Kubernetes with WebSocket load balancing
- **GRPO training** examples use TRL integration with the Wordle environment, available as a runnable Colab notebook

## Frequently Asked Questions

### Where is the OpenEnv tutorial located?

The tutorial resides in the `tutorial/` directory of the huggingface/OpenEnv repository. Start with [`tutorial/README.md`](https://github.com/huggingface/OpenEnv/blob/main/tutorial/README.md) for the overview, then progress through the numbered markdown files ([`01-environments.md`](https://github.com/huggingface/OpenEnv/blob/main/01-environments.md) through [`04-training.md`](https://github.com/huggingface/OpenEnv/blob/main/04-training.md)). The interactive version is available at `tutorial/examples/OpenEnv_Tutorial.ipynb`.

### Do I need Docker to complete the OpenEnv tutorial?

Docker is not required for the initial development phases. You can scaffold and test environments locally using `openenv serve --reload`. However, Docker is necessary for the deployment module ([`02-deployment.md`](https://github.com/huggingface/OpenEnv/blob/main/02-deployment.md)) and scaling sections that cover container orchestration and Hugging Face Spaces integration.

### What reinforcement learning algorithms does the tutorial cover?

The tutorial focuses on **GRPO (Generalized Reward-Penalty Optimization)** implementation using the TRL library. The [`04-training.md`](https://github.com/huggingface/OpenEnv/blob/main/04-training.md) module and accompanying notebook demonstrate end-to-end training on a Wordle environment, including reward shaping and distributed training across multiple environment instances.

### Can I run the tutorial without installing OpenEnv locally?

Yes. While local installation via `pip install -e .` enables development, the tutorial provides a ready-to-run Colab notebook (`OpenEnv_Tutorial.ipynb`) that executes entirely in the cloud. This allows you to experiment with GRPO training and environment interaction without configuring local dependencies or GPU drivers.