Agentic Environments in Miles: Harbor, HUD, NeMo Gym & OpenEnv Integration Guide

Miles supports four agentic environments—Harbor, HUD, NeMo Gym, and OpenEnv—each integrated through lightweight HTTP adaptors that translate Miles' sampling API into environment-specific protocols while preserving reward computation and metadata flow.

The Miles reinforcement learning framework treats diverse agentic evaluation suites as interchangeable backends. By wrapping Harbor, HUD, NeMo Gym, and OpenEnv with thin adaptor modules, Miles enables seamless agentic environment integration for GRPO training without modifying core rollout logic. Each adaptor follows a consistent request-response pattern: resolve the Miles session URL, build a parameterized request dict, POST via async HTTP, and normalize the JSON response into Miles' Sample schema.

Harbor Integration: Agent Server HTTP Interface

Miles connects to Harbor-based agent servers through direct HTTP POST requests. The swe_agent_function.py adaptor constructs requests containing the Miles session URL, model identifier, and sampling parameters, then transmits them to <AGENT_SERVER_URL>/run.

Key implementation details from the Miles source code:

• Request construction: The run function in examples/swe-agent-harbor-docker/swe_agent_function.py assembles a dict with base_url, model name, request_kwargs, and metadata including max_seq_len and session_server_id
• Response parsing: Returns JSON with reward, exit_status, and optional agent_metrics
• Abort handling: An abort hook calls the server's /flush endpoint to terminate in-flight trials

import asyncio
from examples.swe-agent-harbor-docker.swe_agent_function import run

metadata = {"max_seq_len": 2048}
sampling_kwargs = {"temperature": 0.7, "top_p": 0.9}

result = asyncio.run(
    run(
        base_url="http://localhost:12345",   # Miles router URL

        prompt="Explain the impact of quantum computing on cryptography.",
        request_kwargs=sampling_kwargs,
        metadata=metadata,
    )
)
print(result)   # {'reward': …, 'exit_status': …, …}

HUD Integration: OpenAI-Compatible Token ID Contract

HUD provides an OpenAI-compatible chat endpoint returning token IDs. Miles integrates HUD through sglang_compat.py, an async thin client that forwards requests without altering HUD's server code.

The adaptor preserves HUD's token-ID contract by extracting choice.token_ids alongside generated text. It injects HUD task metadata—environment, template ID, and arguments—into sample metadata for downstream reward model consumption.

from examples.experimental.hud.sglang_compat import AsyncOpenAIChatAgent

agent = AsyncOpenAIChatAgent(model="gpt-4")
response = await agent.chat(
    messages=[{"role": "user", "content": "Summarize the paper 'Attention Is All You Need'."}]
)
print(response.choices[0].token_ids)   # token-ID contract preserved

NeMo Gym Integration: Responses-API Mapping

NeMo Gym integration routes through the mini_swe_agent_2 sandbox agent. The nemogym_agent_function.py adaptor maps Miles sampling arguments onto NeMo Gym's Responses-API parameters, handles timeouts and retries, and embeds the NeMo Gym URL in metadata.

The implementation in examples/experimental/nemo-gym/nemogym_agent_function.py targets the /run endpoint and translates JSON responses into Miles-standard reward and metrics.

import asyncio
from examples.experimental.nemo_gym.nemogym_agent_function import run

result = asyncio.run(
    run(
        base_url="http://localhost:8000",
        prompt="Implement a binary search tree in Python.",
        request_kwargs={"temperature": 0.0},
    )
)
print(result["reward"])

OpenEnv Integration: Multi-Backend Sandbox Dispatch

OpenEnv supplies sandbox backends through Daytona, E2B, and Modal. Miles implements a two-layer architecture:

1. Registry layer: openenv_sandbox_common.py maps backend names to provider-specific modules
2. Dispatch layer: openenv_agent_function.py resolves OPENENV_SANDBOX_BACKEND from environment variables and constructs OpenEnv-compatible requests

Backend-specific modules handle authentication and endpoint URLs:

• openenv_daytona_agent_function.py
• openenv_e2b_agent_function.py
• openenv_modal_agent_function.py

import asyncio
from examples.experimental.openenv.openenv_agent_function import run

result = asyncio.run(
    run(
        base_url="http://localhost:9000",
        prompt="Write a bash script that backs up /var/log.",
        request_kwargs={"max_new_tokens": 256},
        metadata={"openenv_sandbox_backend": "daytona"},
    )
)
print(result)

Common Integration Pattern Across All Four Environments

Every Miles agentic environment adaptor follows five standardized steps as implemented in the source:

1. Resolve session URL — miles/rollout/agentic/session.py generates the trainer's base URL for environment callbacks
2. Build request dict — Encapsulates model ID, sampling parameters, and metadata
3. Async HTTP POST — httpx.AsyncClient with keep-alive sockets for long-running trials
4. Parse and normalize — Extract reward, exit_status, eval_report, and agent_metrics into Miles Sample schema
5. Optional abort/flush — Clean up pending trials for environments like Harbor

This uniformity lets researchers swap environments through configuration changes alone.

Source File Reference

Component	Path	Purpose
Harbor adaptor	examples/swe-agent-harbor-docker/swe_agent_function.py	HTTP interface to Harbor agent server with abort/flush
HUD adaptor	examples/experimental/hud/sglang_compat.py	Token-ID-preserving async client
NeMo Gym adaptor	examples/experimental/nemo-gym/nemogym_agent_function.py	Responses-API mapping with retry logic
OpenEnv core	examples/experimental/openenv/openenv_agent_function.py	Backend-agnostic dispatch
OpenEnv registry	examples/experimental/openenv/openenv_sandbox_common.py	Backend name-to-module mapping
Daytona backend	examples/experimental/openenv/openenv_daytona_agent_function.py	Daytona-specific auth and endpoints
E2B backend	examples/experimental/openenv/openenv_e2b_agent_function.py	E2B sandbox integration
Modal backend	examples/experimental/openenv/openenv_modal_agent_function.py	Modal serverless integration
Session resolver	miles/rollout/agentic/session.py	URL resolution for environment callbacks

Summary

• Harbor: Direct HTTP POST to agent server with abort/flush lifecycle management
• HUD: OpenAI-compatible wrapper preserving token-ID contracts and task metadata
• NeMo Gym: Responses-API adaptor with timeout handling and URL embedding
• OpenEnv: Pluggable backend system supporting Daytona, E2B, and Modal sandboxes
• Unified interface: All four environments implement the same five-step integration pattern, enabling drop-in replacement during GRPO training

Frequently Asked Questions

How does Miles handle authentication for different OpenEnv backends?

Backend-specific modules in examples/experimental/openenv/ encapsulate provider authentication. openenv_daytona_agent_function.py, openenv_e2b_agent_function.py, and openenv_modal_agent_function.py each manage their respective API keys and endpoint configurations, while the generic openenv_agent_function.py dispatches based on the OPENENV_SANDBOX_BACKEND environment variable or metadata override.

Can Miles abort long-running trials in Harbor environments?

Yes. The Harbor adaptor exposes an abort hook that POSTs to the agent server's /flush endpoint. This terminates in-flight trials without waiting for completion, implemented in swe_agent_function.py alongside the main run function.

What metadata does Miles inject for downstream reward computation?

Each adaptor injects environment-specific context: Harbor includes session and model identifiers; HUD adds task row data (environment, template ID, arguments); NeMo Gym embeds the server URL; OpenEnv records the selected backend. This metadata flows through Miles' Sample schema to reward models.

Is the HUD integration compatible with standard OpenAI SDK usage?

The sglang_compat.py adaptor maintains compatibility while extending functionality. It returns token_ids alongside standard text responses, satisfying HUD's contract without requiring modifications to either HUD's server code or typical OpenAI client patterns.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}