# Configuring SGLang vs vLLM Inference Backends for Production Workloads in AReaL

> Compare SGLang and vLLM inference backends for AReaL production workloads. Learn which high-throughput option suits your needs best for efficient AI deployment.

- Repository: [inclusionAI/areal](https://github.com/inclusionai/areal)
- Tags: performance
- Published: 2026-03-04

---

**AReaL supports both SGLang and vLLM as swappable high-throughput inference backends, selected via the `--allocation-mode.gen-backend` CLI flag and configured through distinct dataclasses in [`areal/api/cli_args.py`](https://github.com/inclusionai/areal/blob/main/areal/api/cli_args.py).**

When training large language models with reinforcement learning pipelines like PPO or GRPO, the choice of inference backend directly impacts throughput, latency, and scalability. The inclusionai/areal repository abstracts both **SGLang** and **vLLM** behind a unified interface, allowing operators to switch between GPU-centric low-latency generation and scalable tensor-parallel serving without modifying training logic.

## Backend Architecture Overview

AReaL implements backend-specific wrappers that handle process management, while exposing a common `InferenceEngine` interface for rollout collection:

| Feature | **SGLang** | **vLLM** |
|---------|------------|----------|
| **Primary target** | GPU-centric, low-latency generation with fine-grained scheduling (`schedule_policy`, `decode_log_interval`) | Scalable tensor-parallel serving (`tp_size`, `pp_size`) with strong batching and KV-cache sharing |
| **LoRA support** | Built-in, optional (`enable_lora`, `max_lora_rank`) | Built-in, optional (`enable_lora`, `max_lora_rank`) |
| **Distributed launch** | Uses `SGLangServerWrapper` spawning multiple processes via `AREAL_SGLANG_MULTI_NODE_*` env vars | Uses `vLLMServerWrapper` relying on vLLM's native MP launcher; cross-node not yet supported |
| **Configuration** | `SGLangConfig` in [`areal/api/cli_args.py`](https://github.com/inclusionai/areal/blob/main/areal/api/cli_args.py) (lines 1384-1452) | `vLLMConfig` in [`areal/api/cli_args.py`](https://github.com/inclusionai/areal/blob/main/areal/api/cli_args.py) (lines 1422-1475) |
| **Server entry** | `areal.infra.launcher.sglang_server.main` | `areal.infra.launcher.vllm_server.main` |
| **Remote engine** | `RemoteSGLangEngine` ([`areal/engine/sglang_remote.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/sglang_remote.py)) | `RemoteVLLMEngine` ([`areal/engine/vllm_remote.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/vllm_remote.py)) |

## Selecting the Backend

The CLI argument `--allocation-mode.gen-backend` determines which server wrapper instantiates. Both backends share identical high-level workflows for rollout, PPO/GRPO, and weight updates.

### Command-Line Selection

```bash

# SGLang backend

python -m areal.infra.launcher.sglang_server \
    --allocation-mode.gen-backend sglang \
    --allocation-mode.gen-instance-size 8 \
    --sglang.model_path /model/llama-2-70b \
    --sglang.enable_lora True \
    --sglang.max_lora_rank 32

# vLLM backend

python -m areal.infra.launcher.vllm_server \
    --allocation-mode.gen-backend vllm \
    --allocation-mode.gen-instance-size 4 \
    --vllm.model /model/llama-2-70b \
    --vllm.enable_lora True \
    --vllm.gpu_memory_utilization 0.85

```

### Programmatic Switching

```python
from areal.api.cli_args import AllocationMode, parse_cli_args, to_structured_cfg
from areal.infra.launcher.sglang_server import launch_sglang_server
from areal.infra.launcher.vllm_server import launch_vllm_server

def launch_backend(argv: list[str], backend: str):
    cfg, _ = parse_cli_args(argv)
    cfg.allocation_mode = AllocationMode.from_str(cfg.allocation_mode)
    assert cfg.allocation_mode.gen_backend == backend, "Mismatched backend"

    if backend == "sglang":
        cfg.sglang = to_structured_cfg(cfg.sglang, SGLangConfig)
        launch_sglang_server(argv)
    elif backend == "vllm":
        cfg.vllm = to_structured_cfg(cfg.vllm, vLLMConfig)
        launch_vllm_server(argv)
    else:
        raise ValueError(f"Unsupported backend: {backend}")

```

## Core Configuration Differences

While both backends support LoRA, chunked prefill, and CPU offloading, their configuration schemas differ in naming and available optimizations.

| Setting | SGLang (`SGLangConfig`) | vLLM (`vLLMConfig`) |
|---------|--------------------------|---------------------|
| **Model path** | `model_path` | `model` |
| **Random seed** | `random_seed` | `seed` |
| **CUDA graphs** | `disable_cuda_graph`, `cuda_graph_max_bs` | N/A (handled internally) |
| **Chunked prefill** | `chunked_prefill_size` (default `-1` disables) | `no_enable_chunked_prefill` (set `True` to pass `--no-enable-chunked-prefill`) |
| **Prefix caching** | `no_enable_prefix_caching` (default `True`) | Exposed via same field; vLLM enables by default |
| **LoRA** | `enable_lora`, `max_lora_rank`, `max_loaded_loras` | `enable_lora`, `max_lora_rank`, `max_loras` |
| **CPU offload** | `cpu_offload_gb` | `cpu_offload_gb` |
| **GPU memory** | N/A (internal management) | `gpu_memory_utilization` (default `0.9`) |
| **Worker extension** | N/A | `worker_extension_cls` (default `areal.engine.vllm_ext.vllm_worker_extension.VLLMWorkerExtension`) |

**Note on boolean flags:** Both configs use a "no-enable" prefix pattern (`no_enable_*`) because the internal `get_py_cmd` helper skips flags with falsy values. Setting these booleans to `True` forces the `--no-enable-…` flag to be passed to the underlying engine.

## Launch Workflow

Both `SGLangServerWrapper` and `vLLMServerWrapper` follow an identical six-step orchestration pattern defined in [`areal/infra/launcher/sglang_server.py`](https://github.com/inclusionai/areal/blob/main/areal/infra/launcher/sglang_server.py) and [`areal/infra/launcher/vllm_server.py`](https://github.com/inclusionai/areal/blob/main/areal/infra/launcher/vllm_server.py):

1. **Calculate per-process resources** – Derives `gpus_per_server` from `allocation_mode.gen_instance_size`.
2. **Determine port ranges** – `find_free_ports(2, port_range)` in [`areal/utils/network.py`](https://github.com/inclusionai/areal/blob/main/areal/utils/network.py) allocates unique HTTP and NCCL init ports.
3. **Construct command** – `SGLangConfig.build_cmd` or `vLLMConfig.build_cmd` expands dataclasses into CLI arguments.
4. **Spawn subprocesses** – `launch_server_cmd` isolates Triton/vLLM cache directories and starts servers via `subprocess.Popen`.
5. **Health check** – `wait_for_server` polls `/v1/models` until HTTP 200.
6. **Name resolution** – Registers successful servers with `name_resolve.add_subentry` for controller discovery during rollout.

Both wrappers install a **monitor thread** (`_monitor_server_processes`) that terminates the entire launch if any server process dies, ensuring clean failure modes for production clusters.

## Production-Ready Considerations

| Concern | SGLang | vLLM |
|---------|--------|------|
| **Cross-node scaling** | Supported via `AREAL_SGLANG_MULTI_NODE_*` environment variables; adjusts `node_rank`, `master_addr`, and `master_port` when `gpus_per_server > n_gpus_per_node`. | Not supported; launching across multiple nodes raises `NotImplementedError` in `vLLMServerWrapper.run`. |
| **Metrics** | Native Prometheus-style metrics via `enable_metrics=True` and `decode_log_interval`; optional request logging via `log_requests`. | Custom metrics via `VLLMWorkerExtension` in [`areal/engine/vllm_ext/vllm_worker_extension.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/vllm_ext/vllm_worker_extension.py). |
| **Graceful shutdown** | Signal handlers (`SIGTERM`, `SIGINT`) forward shutdown to child processes via `kill_process_tree`. | Same pattern with additional cleanup of vLLM's compile cache (`VLLM_CACHE_ROOT`). |
| **GPU memory** | Use `cpu_offload_gb` and `disable_cuda_graph` to reduce fragmentation for large batches. | Control via `gpu_memory_utilization` (default `0.9`) and `swap_space` for dynamic paging. |
| **LoRA hot-swap** | Disk-based updates only (`weight_update_mode='disk'`) required; distributed updater does not support LoRA. | Disk-based or runtime loading via `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True`. |

## Code Examples

### Switching Backends Programmatically

```python
from areal.api.cli_args import AllocationMode, parse_cli_args, to_structured_cfg
from areal.infra.launcher.sglang_server import launch_sglang_server
from areal.infra.launcher.vllm_server import launch_vllm_server

def launch_backend(argv: list[str], backend: str):
    # Parse base CLI args

    cfg, _ = parse_cli_args(argv)
    cfg.allocation_mode = AllocationMode.from_str(cfg.allocation_mode)
    assert cfg.allocation_mode.gen_backend == backend, "Mismatched backend"

    # Convert the backend-specific config to a structured dataclass

    if backend == "sglang":
        cfg.sglang = to_structured_cfg(cfg.sglang, SGLangConfig)
        launch_sglang_server(argv)
    elif backend == "vllm":
        cfg.vllm = to_structured_cfg(cfg.vllm, vLLMConfig)
        launch_vllm_server(argv)
    else:
        raise ValueError(f"Unsupported backend: {backend}")

# Example usage

launch_backend(sys.argv[1:], backend="sglang")

```

*Key files referenced*: [`sglang_server.py`](https://github.com/inclusionai/areal/blob/main/sglang_server.py), [`vllm_server.py`](https://github.com/inclusionai/areal/blob/main/vllm_server.py), and [`cli_args.py`](https://github.com/inclusionai/areal/blob/main/cli_args.py) for config structs.

### Adding Custom Metrics to the vLLM Worker

```python

# areal/engine/vllm_ext/vllm_worker_extension.py

from areal.utils import stats_tracker

class VLLMWorkerExtension:
    async def on_generation_end(self, request_id: str, metrics: dict):
        # Increment a custom Prometheus counter

        stats_tracker.inc("vllm_generated_tokens", metrics["output_len"])
        # Forward the original metrics upstream

        return metrics

```

The launcher automatically imports this class via `vllm_config.worker_extension_cls`, which defaults to `areal.engine.vllm_ext.vllm_worker_extension.VLLMWorkerExtension`.

### Enabling LoRA Hot-Swap with SGLang

```python

# 1. Create a new LoRA checkpoint on disk

lora_path = "/lora/checkpoint_step_20000"

# 2. Issue a weight-update request via the RemoteSGLangEngine

engine.update_weights_from_disk(
    WeightUpdateMeta(
        path=Path(lora_path),
        use_lora=True,
        lora_name="my_lora",
        version=20000,
        alloc_mode=engine.allocation_mode,
    )
)

```

The backend constructs an HTTP POST to `/load_lora_adapter` via `SGLangBackend.build_disk_weight_update_requests` in [`areal/engine/sglang_remote.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/sglang_remote.py).

## Key Files Reference

| Path | Role |
|------|------|
| [`areal/api/cli_args.py`](https://github.com/inclusionai/areal/blob/main/areal/api/cli_args.py) | Dataclasses for `SGLangConfig`, `vLLMConfig`, and CLI parsing |
| [`areal/infra/launcher/sglang_server.py`](https://github.com/inclusionai/areal/blob/main/areal/infra/launcher/sglang_server.py) | Process manager that spawns one or many SGLang server processes |
| [`areal/infra/launcher/vllm_server.py`](https://github.com/inclusionai/areal/blob/main/areal/infra/launcher/vllm_server.py) | Process manager for vLLM, mirrors the SGLang launch flow |
| [`areal/engine/sglang_remote.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/sglang_remote.py) | Remote inference engine implementation (`RemoteSGLangEngine`) and `SGLangBackend` |
| [`areal/engine/vllm_remote.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/vllm_remote.py) | Remote inference engine implementation (`RemoteVLLMEngine`) |
| [`areal/engine/vllm_ext/vllm_worker_extension.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/vllm_ext/vllm_worker_extension.py) | Custom vLLM worker extension that records AReaL-specific stats |
| [`areal/engine/vllm_ext/areal_vllm_server.py`](https://github.com/inclusionai/areal/blob/main/areal/engine/vllm_ext/areal_vllm_server.py) | Entrypoint used by `vLLMConfig.build_cmd` to start the server with the extension |
| [`areal/utils/network.py`](https://github.com/inclusionai/areal/blob/main/areal/utils/network.py) | Helper `find_free_ports` used by both launchers |
| [`areal/utils/name_resolve.py`](https://github.com/inclusionai/areal/blob/main/areal/utils/name_resolve.py) | Registers server addresses for the rollout controller |

## Summary

- **AReaL abstracts SGLang and vLLM** behind a unified `InferenceEngine` interface, enabling backend swaps via `--allocation-mode.gen-backend`.
- **SGLang excels at single-node, low-latency generation** with fine-grained scheduling and cross-node scaling via `AREAL_SGLANG_MULTI_NODE_*` environment variables.
- **vLLM targets tensor-parallel scalability** with built-in GPU memory utilization controls and custom worker extensions for metrics.
- **Configuration differs in naming conventions** (e.g., `model_path` vs `model`, `random_seed` vs `seed`) and available optimizations (CUDA graphs in SGLang, chunked prefill toggles in both).
- **Production workflows** require monitoring the `_monitor_server_processes` thread for clean failure modes and using `name_resolve.add_subentry` for controller discovery.

## Frequently Asked Questions

### How do I switch between SGLang and vLLM without changing my training script?

Set `--allocation-mode.gen-backend` to either `sglang` or `vllm` when launching the server. The `AllocationMode` dataclass in [`areal/api/cli_args.py`](https://github.com/inclusionai/areal/blob/main/areal/api/cli_args.py) routes to the appropriate wrapper (`SGLangServerWrapper` or `vLLMServerWrapper`), while your training controller continues to use the standard `InferenceEngine` interface for rollouts.

### Can I scale vLLM across multiple nodes like SGLang?

No. As of the current implementation in [`areal/infra/launcher/vllm_server.py`](https://github.com/inclusionai/areal/blob/main/areal/infra/launcher/vllm_server.py), `vLLMServerWrapper.run` raises `NotImplementedError` when attempting multi-node launches. SGLang supports cross-node scaling via the `AREAL_SGLANG_MULTI_NODE_*` environment variables, which configure `node_rank`, `master_addr`, and `master_port` for distributed initialization.

### What is the difference between `model_path` and `model` in the configuration?

These fields refer to the same concept but follow each backend's native naming convention. `SGLangConfig` uses `model_path` to specify the HuggingFace model directory or name, while `vLLMConfig` uses `model`. Similarly, SGLang uses `random_seed` where vLLM uses `seed`. Always consult [`areal/api/cli_args.py`](https://github.com/inclusionai/areal/blob/main/areal/api/cli_args.py) for the exact field names when migrating configurations between backends.

### How does AReaL handle LoRA updates in production?

Both backends support LoRA via `enable_lora` and `max_lora_rank` flags, but hot-swapping implementations differ. SGLang requires disk-based updates through `RemoteSGLangEngine.update_weights_from_disk` with `weight_update_mode='disk'`, posting to `/load_lora_adapter`. vLLM supports both disk-based updates and runtime loading when `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True`, offering more flexibility for dynamic adapter switching during training.