# Agentic Environments in Miles: Harbor, HUD, NeMo Gym & OpenEnv Integration Guide

> Discover agentic environments supported by Miles including Harbor, HUD, NeMo Gym, and OpenEnv. Learn how lightweight HTTP adaptors integrate these environments seamlessly.

- Repository: [RadixArk/miles](https://github.com/radixark/miles)
- Tags: how-to-guide
- Published: 2026-09-06

---

**Miles supports four agentic environments—Harbor, HUD, NeMo Gym, and OpenEnv—each integrated through lightweight HTTP adaptors that translate Miles' sampling API into environment-specific protocols while preserving reward computation and metadata flow.**

The Miles reinforcement learning framework treats diverse agentic evaluation suites as interchangeable backends. By wrapping Harbor, HUD, NeMo Gym, and OpenEnv with thin adaptor modules, Miles enables seamless **agentic environment integration** for GRPO training without modifying core rollout logic. Each adaptor follows a consistent request-response pattern: resolve the Miles session URL, build a parameterized request dict, POST via async HTTP, and normalize the JSON response into Miles' `Sample` schema.

## Harbor Integration: Agent Server HTTP Interface

Miles connects to Harbor-based agent servers through direct HTTP POST requests. The [`swe_agent_function.py`](https://github.com/radixark/miles/blob/main/swe_agent_function.py) adaptor constructs requests containing the Miles session URL, model identifier, and sampling parameters, then transmits them to `<AGENT_SERVER_URL>/run`.

**Key implementation details from the Miles source code:**

- **Request construction**: The `run` function in [`examples/swe-agent-harbor-docker/swe_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/swe-agent-harbor-docker/swe_agent_function.py) assembles a dict with `base_url`, model name, `request_kwargs`, and metadata including `max_seq_len` and `session_server_id`
- **Response parsing**: Returns JSON with `reward`, `exit_status`, and optional `agent_metrics`
- **Abort handling**: An `abort` hook calls the server's `/flush` endpoint to terminate in-flight trials

```python
import asyncio
from examples.swe-agent-harbor-docker.swe_agent_function import run

metadata = {"max_seq_len": 2048}
sampling_kwargs = {"temperature": 0.7, "top_p": 0.9}

result = asyncio.run(
    run(
        base_url="http://localhost:12345",   # Miles router URL

        prompt="Explain the impact of quantum computing on cryptography.",
        request_kwargs=sampling_kwargs,
        metadata=metadata,
    )
)
print(result)   # {'reward': …, 'exit_status': …, …}

```

## HUD Integration: OpenAI-Compatible Token ID Contract

HUD provides an OpenAI-compatible chat endpoint returning token IDs. Miles integrates HUD through [`sglang_compat.py`](https://github.com/radixark/miles/blob/main/sglang_compat.py), an async thin client that forwards requests without altering HUD's server code.

The adaptor preserves HUD's **token-ID contract** by extracting `choice.token_ids` alongside generated text. It injects HUD task metadata—environment, template ID, and arguments—into sample metadata for downstream reward model consumption.

```python
from examples.experimental.hud.sglang_compat import AsyncOpenAIChatAgent

agent = AsyncOpenAIChatAgent(model="gpt-4")
response = await agent.chat(
    messages=[{"role": "user", "content": "Summarize the paper 'Attention Is All You Need'."}]
)
print(response.choices[0].token_ids)   # token-ID contract preserved

```

## NeMo Gym Integration: Responses-API Mapping

NeMo Gym integration routes through the `mini_swe_agent_2` sandbox agent. The [`nemogym_agent_function.py`](https://github.com/radixark/miles/blob/main/nemogym_agent_function.py) adaptor maps Miles sampling arguments onto NeMo Gym's **Responses-API parameters**, handles timeouts and retries, and embeds the NeMo Gym URL in metadata.

The implementation in [`examples/experimental/nemo-gym/nemogym_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/experimental/nemo-gym/nemogym_agent_function.py) targets the `/run` endpoint and translates JSON responses into Miles-standard reward and metrics.

```python
import asyncio
from examples.experimental.nemo_gym.nemogym_agent_function import run

result = asyncio.run(
    run(
        base_url="http://localhost:8000",
        prompt="Implement a binary search tree in Python.",
        request_kwargs={"temperature": 0.0},
    )
)
print(result["reward"])

```

## OpenEnv Integration: Multi-Backend Sandbox Dispatch

OpenEnv supplies sandbox backends through Daytona, E2B, and Modal. Miles implements a **two-layer architecture**:

1. **Registry layer**: [`openenv_sandbox_common.py`](https://github.com/radixark/miles/blob/main/openenv_sandbox_common.py) maps backend names to provider-specific modules
2. **Dispatch layer**: [`openenv_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_agent_function.py) resolves `OPENENV_SANDBOX_BACKEND` from environment variables and constructs OpenEnv-compatible requests

Backend-specific modules handle authentication and endpoint URLs:
- [`openenv_daytona_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_daytona_agent_function.py)
- [`openenv_e2b_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_e2b_agent_function.py)
- [`openenv_modal_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_modal_agent_function.py)

```python
import asyncio
from examples.experimental.openenv.openenv_agent_function import run

result = asyncio.run(
    run(
        base_url="http://localhost:9000",
        prompt="Write a bash script that backs up /var/log.",
        request_kwargs={"max_new_tokens": 256},
        metadata={"openenv_sandbox_backend": "daytona"},
    )
)
print(result)

```

## Common Integration Pattern Across All Four Environments

Every Miles **agentic environment adaptor** follows five standardized steps as implemented in the source:

1. **Resolve session URL** — [`miles/rollout/agentic/session.py`](https://github.com/radixark/miles/blob/main/miles/rollout/agentic/session.py) generates the trainer's base URL for environment callbacks
2. **Build request dict** — Encapsulates model ID, sampling parameters, and metadata
3. **Async HTTP POST** — `httpx.AsyncClient` with keep-alive sockets for long-running trials
4. **Parse and normalize** — Extract `reward`, `exit_status`, `eval_report`, and `agent_metrics` into Miles `Sample` schema
5. **Optional abort/flush** — Clean up pending trials for environments like Harbor

This uniformity lets researchers swap environments through configuration changes alone.

## Source File Reference

| Component | Path | Purpose |
|-----------|------|---------|
| Harbor adaptor | [`examples/swe-agent-harbor-docker/swe_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/swe-agent-harbor-docker/swe_agent_function.py) | HTTP interface to Harbor agent server with abort/flush |
| HUD adaptor | [`examples/experimental/hud/sglang_compat.py`](https://github.com/radixark/miles/blob/main/examples/experimental/hud/sglang_compat.py) | Token-ID-preserving async client |
| NeMo Gym adaptor | [`examples/experimental/nemo-gym/nemogym_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/experimental/nemo-gym/nemogym_agent_function.py) | Responses-API mapping with retry logic |
| OpenEnv core | [`examples/experimental/openenv/openenv_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/experimental/openenv/openenv_agent_function.py) | Backend-agnostic dispatch |
| OpenEnv registry | [`examples/experimental/openenv/openenv_sandbox_common.py`](https://github.com/radixark/miles/blob/main/examples/experimental/openenv/openenv_sandbox_common.py) | Backend name-to-module mapping |
| Daytona backend | [`examples/experimental/openenv/openenv_daytona_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/experimental/openenv/openenv_daytona_agent_function.py) | Daytona-specific auth and endpoints |
| E2B backend | [`examples/experimental/openenv/openenv_e2b_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/experimental/openenv/openenv_e2b_agent_function.py) | E2B sandbox integration |
| Modal backend | [`examples/experimental/openenv/openenv_modal_agent_function.py`](https://github.com/radixark/miles/blob/main/examples/experimental/openenv/openenv_modal_agent_function.py) | Modal serverless integration |
| Session resolver | [`miles/rollout/agentic/session.py`](https://github.com/radixark/miles/blob/main/miles/rollout/agentic/session.py) | URL resolution for environment callbacks |

## Summary

- **Harbor**: Direct HTTP POST to agent server with abort/flush lifecycle management
- **HUD**: OpenAI-compatible wrapper preserving token-ID contracts and task metadata
- **NeMo Gym**: Responses-API adaptor with timeout handling and URL embedding
- **OpenEnv**: Pluggable backend system supporting Daytona, E2B, and Modal sandboxes
- **Unified interface**: All four environments implement the same five-step integration pattern, enabling drop-in replacement during GRPO training

## Frequently Asked Questions

### How does Miles handle authentication for different OpenEnv backends?

Backend-specific modules in `examples/experimental/openenv/` encapsulate provider authentication. [`openenv_daytona_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_daytona_agent_function.py), [`openenv_e2b_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_e2b_agent_function.py), and [`openenv_modal_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_modal_agent_function.py) each manage their respective API keys and endpoint configurations, while the generic [`openenv_agent_function.py`](https://github.com/radixark/miles/blob/main/openenv_agent_function.py) dispatches based on the `OPENENV_SANDBOX_BACKEND` environment variable or metadata override.

### Can Miles abort long-running trials in Harbor environments?

Yes. The Harbor adaptor exposes an `abort` hook that POSTs to the agent server's `/flush` endpoint. This terminates in-flight trials without waiting for completion, implemented in [`swe_agent_function.py`](https://github.com/radixark/miles/blob/main/swe_agent_function.py) alongside the main `run` function.

### What metadata does Miles inject for downstream reward computation?

Each adaptor injects environment-specific context: Harbor includes session and model identifiers; HUD adds task row data (environment, template ID, arguments); NeMo Gym embeds the server URL; OpenEnv records the selected backend. This metadata flows through Miles' `Sample` schema to reward models.

### Is the HUD integration compatible with standard OpenAI SDK usage?

The [`sglang_compat.py`](https://github.com/radixark/miles/blob/main/sglang_compat.py) adaptor maintains compatibility while extending functionality. It returns `token_ids` alongside standard text responses, satisfying HUD's contract without requiring modifications to either HUD's server code or typical OpenAI client patterns.