# How the Rollout Executor (Worker Manager) Connects to SGLang Engines in Miles

> Discover how the RolloutExecutor connects to SGLang engines in Miles. Learn how this Ray actor manages generation and evaluation requests via HTTP to the SGLang router.

- Repository: [RadixArk/miles](https://github.com/radixark/miles)
- Tags: internals
- Published: 2026-09-06

---

**The RolloutExecutor is a Ray remote actor that stores the SGLang router address and sends all generation and evaluation requests via HTTP to the router, which forwards them to the underlying SGLang engine processes.**

In the Miles reinforcement learning framework, the **rollout executor**—also referred to as the **worker manager**—serves as the critical bridge between the training loop and distributed SGLang inference engines. This component handles data generation, evaluation, and checkpointing by routing requests through a centralized SGLang router. Understanding this connection is essential for debugging, scaling, or modifying Miles inference pipelines.

## Creating the Rollout Manager

The rollout infrastructure is initialized through a single factory function that orchestrates multiple Ray actors.

In [`miles/ray/placement_group.py`](https://github.com/radixark/miles/blob/main/miles/ray/placement_group.py), the `create_rollout_components` function spins up both the `RolloutExecutor` actor and the SGLang engine pool:

```python

# train_async.py, line 33

inference_controller, rollout_executor, num_rollout_per_epoch = await create_rollout_components(args)

```

This call appears in the main training scripts:
- [`train_async.py`](https://github.com/radixark/miles/blob/main/train_async.py)
- [`train.py`](https://github.com/radixark/miles/blob/main/train.py)
- [`train_multi_lora_async.py`](https://github.com/radixark/miles/blob/main/train_multi_lora_async.py)

The `create_rollout_components` function returns three values: an inference controller, the rollout executor handle, and the number of rollouts to generate per epoch.

## RolloutExecutor Initialization and SGLang Router Connection

The `RolloutExecutor` class is decorated as a Ray remote actor (`@ray.remote`). Its constructor captures the full argument namespace and establishes the HTTP connection to the SGLang router.

From [`miles/ray/rollout/rollout_executor.py`](https://github.com/radixark/miles/blob/main/miles/ray/rollout/rollout_executor.py), lines 46-64:

```python
class RolloutExecutor:
    def __init__(self, args: "Args"):
        self.args = args
        # Line 59-60: derive router address from CLI arguments

        router_addr = f"http://{args.sglang_router_ip}:{args.sglang_router_port}"
        
        # Line 62-64: initialize HTTP client unless in debug mode

        if not self.args.debug_train_only:
            init_http_client(args)  # from miles/utils/http_utils.py

            init_tracking(args, primary=False, router_addr=router_addr)

```

The router address construction pulls directly from `args.sglang_router_ip` and `args.sglang_router_port`. These values originate from command-line configuration and point to the SGLang router process that sits in front of the actual engine replicas.

## Sending Rollout Requests to SGLang Engines

When the training loop needs fresh rollout data, it invokes the executor's `get` method:

```python

# Inside training loop

rollout_data = await rollout_executor.get.remote(rollout_id)

```

The `get` method internally calls `call_rollout_function` (or legacy `call_rollout_fn`), which performs an HTTP POST to the router address established during initialization. The SGLang router then:
1. Receives the request
2. Selects an available SGLang engine process
3. Forwards the inference call
4. Returns generated tokens, KV-cache metadata, and logprobs

The executor post-processes this response into training-compatible tensors before returning to the trainer.

## Evaluation Path Through the Same Router

Evaluation requests follow an identical pattern. The `RolloutExecutor.eval` method (also in [`miles/ray/rollout/rollout_executor.py`](https://github.com/radixark/miles/blob/main/miles/ray/rollout/rollout_executor.py)) uses the same HTTP client to reach the router:

```python

# Optional checkpoint evaluation

await rollout_executor.eval.remote(rollout_id, hf_dir=hf_path)

```

For checkpoint evaluation, Miles can optionally spin up a separate "checkpoint-eval fleet" of SGLang engines, but the communication mechanism remains unchanged—HTTP requests routed through the same address.

## Lifecycle Management and Cleanup

The `create_rollout_components` factory registers the executor with the router so the router maintains awareness of active workloads. When training completes, proper shutdown is critical to release GPU resources:

```python

# train_async.py, line 137

await rollout_executor.dispose.remote()

```

This `dispose` call terminates the SGLang engine processes and cleans up the Ray actor.

## Complete Usage Pattern

```python

# 1. Initialize rollout infrastructure

inference_ctl, rollout_executor, n_per_epoch = await create_rollout_components(args)

# 2. Generate rollouts during training

for rollout_id in range(n_per_epoch):
    data = await rollout_executor.get.remote(rollout_id)
    # ... training step ...

# 3. Optional evaluation

await rollout_executor.eval.remote(checkpoint_id, hf_dir="/path/to/checkpoint")

# 4. Shutdown

await rollout_executor.dispose.remote()

```

## Key Source Files

| File | Purpose |
|------|---------|
| [`miles/ray/placement_group.py`](https://github.com/radixark/miles/blob/main/miles/ray/placement_group.py) | `create_rollout_components()` factory; orchestrates Ray actors and SGLang engines |
| [`miles/ray/rollout/rollout_executor.py`](https://github.com/radixark/miles/blob/main/miles/ray/rollout/rollout_executor.py) | `RolloutExecutor` class; Ray remote actor that manages HTTP communication with SGLang router |
| [`train_async.py`](https://github.com/radixark/miles/blob/main/train_async.py), [`train.py`](https://github.com/radixark/miles/blob/main/train.py), [`train_multi_lora_async.py`](https://github.com/radixark/miles/blob/main/train_multi_lora_async.py) | Entry points demonstrating executor usage |
| [`miles/utils/http_utils.py`](https://github.com/radixark/miles/blob/main/miles/utils/http_utils.py) | `init_http_client()` and request helpers |
| [`miles/backends/sglang_utils/sglang_engine.py`](https://github.com/radixark/miles/blob/main/miles/backends/sglang_utils/sglang_engine.py) | SGLang server process management (implicit in router setup) |

## Summary

- The **RolloutExecutor** is a Ray remote actor that acts as the worker manager for Miles inference operations.
- It stores the **SGLang router address** (`sglang_router_ip`:`sglang_router_port`) derived from CLI arguments.
- All **generation and evaluation requests** are sent as HTTP RPCs to this router, which load-balances across SGLang engine processes.
- The executor lifecycle is managed through `create_rollout_components()` initialization and `dispose.remote()` cleanup.
- This architecture decouples the training loop from engine scaling, allowing dynamic adjustment of inference capacity.

## Frequently Asked Questions

### How does the RolloutExecutor discover the SGLang router address?

The router address is passed explicitly through the `args` namespace. At initialization, the executor constructs the full URL as `f"http://{args.sglang_router_ip}:{args.sglang_router_port}"` and passes it to `init_tracking()` in [`miles/ray/rollout/rollout_executor.py`](https://github.com/radixark/miles/blob/main/miles/ray/rollout/rollout_executor.py), lines 59-60.

### What happens if `debug_train_only` is set to True?

When `args.debug_train_only` is enabled, the executor skips `init_http_client()` entirely (line 62-64). This mode allows training logic verification without launching actual SGLang engines or making inference requests.

### Can multiple RolloutExecutors connect to the same SGLang router?

Yes. The SGLang router is designed as a shared entry point that can accept connections from multiple clients. However, Miles typically creates one RolloutExecutor per training job through `create_rollout_components()`, which coordinates the full fleet.

### Where is the HTTP client implementation located?

The HTTP client initialization lives in [`miles/utils/http_utils.py`](https://github.com/radixark/miles/blob/main/miles/utils/http_utils.py) via `init_http_client()`. The actual request logic for rollout calls is implemented in the `call_rollout_function` helper, which performs POST requests to the stored router address.