Configuring SGLang vs vLLM Inference Backends for Production Workloads in AReaL

AReaL supports both SGLang and vLLM as swappable high-throughput inference backends, selected via the --allocation-mode.gen-backend CLI flag and configured through distinct dataclasses in areal/api/cli_args.py.

When training large language models with reinforcement learning pipelines like PPO or GRPO, the choice of inference backend directly impacts throughput, latency, and scalability. The inclusionai/areal repository abstracts both SGLang and vLLM behind a unified interface, allowing operators to switch between GPU-centric low-latency generation and scalable tensor-parallel serving without modifying training logic.

Backend Architecture Overview

AReaL implements backend-specific wrappers that handle process management, while exposing a common InferenceEngine interface for rollout collection:

Feature	SGLang	vLLM
Primary target	GPU-centric, low-latency generation with fine-grained scheduling (schedule_policy, decode_log_interval)	Scalable tensor-parallel serving (tp_size, pp_size) with strong batching and KV-cache sharing
LoRA support	Built-in, optional (enable_lora, max_lora_rank)	Built-in, optional (enable_lora, max_lora_rank)
Distributed launch	Uses SGLangServerWrapper spawning multiple processes via AREAL_SGLANG_MULTI_NODE_* env vars	Uses vLLMServerWrapper relying on vLLM's native MP launcher; cross-node not yet supported
Configuration	SGLangConfig in areal/api/cli_args.py (lines 1384-1452)	vLLMConfig in areal/api/cli_args.py (lines 1422-1475)
Server entry	areal.infra.launcher.sglang_server.main	areal.infra.launcher.vllm_server.main
Remote engine	RemoteSGLangEngine (areal/engine/sglang_remote.py)	RemoteVLLMEngine (areal/engine/vllm_remote.py)

Selecting the Backend

The CLI argument --allocation-mode.gen-backend determines which server wrapper instantiates. Both backends share identical high-level workflows for rollout, PPO/GRPO, and weight updates.

Command-Line Selection

# SGLang backend

python -m areal.infra.launcher.sglang_server \
    --allocation-mode.gen-backend sglang \
    --allocation-mode.gen-instance-size 8 \
    --sglang.model_path /model/llama-2-70b \
    --sglang.enable_lora True \
    --sglang.max_lora_rank 32

# vLLM backend

python -m areal.infra.launcher.vllm_server \
    --allocation-mode.gen-backend vllm \
    --allocation-mode.gen-instance-size 4 \
    --vllm.model /model/llama-2-70b \
    --vllm.enable_lora True \
    --vllm.gpu_memory_utilization 0.85

Programmatic Switching

from areal.api.cli_args import AllocationMode, parse_cli_args, to_structured_cfg
from areal.infra.launcher.sglang_server import launch_sglang_server
from areal.infra.launcher.vllm_server import launch_vllm_server

def launch_backend(argv: list[str], backend: str):
    cfg, _ = parse_cli_args(argv)
    cfg.allocation_mode = AllocationMode.from_str(cfg.allocation_mode)
    assert cfg.allocation_mode.gen_backend == backend, "Mismatched backend"

    if backend == "sglang":
        cfg.sglang = to_structured_cfg(cfg.sglang, SGLangConfig)
        launch_sglang_server(argv)
    elif backend == "vllm":
        cfg.vllm = to_structured_cfg(cfg.vllm, vLLMConfig)
        launch_vllm_server(argv)
    else:
        raise ValueError(f"Unsupported backend: {backend}")

Core Configuration Differences

While both backends support LoRA, chunked prefill, and CPU offloading, their configuration schemas differ in naming and available optimizations.

Setting	SGLang (SGLangConfig)	vLLM (vLLMConfig)
Model path	model_path	model
Random seed	random_seed	seed
CUDA graphs	disable_cuda_graph, cuda_graph_max_bs	N/A (handled internally)
Chunked prefill	chunked_prefill_size (default -1 disables)	no_enable_chunked_prefill (set True to pass --no-enable-chunked-prefill)
Prefix caching	no_enable_prefix_caching (default True)	Exposed via same field; vLLM enables by default
LoRA	enable_lora, max_lora_rank, max_loaded_loras	enable_lora, max_lora_rank, max_loras
CPU offload	cpu_offload_gb	cpu_offload_gb
GPU memory	N/A (internal management)	gpu_memory_utilization (default 0.9)
Worker extension	N/A	worker_extension_cls (default areal.engine.vllm_ext.vllm_worker_extension.VLLMWorkerExtension)

Note on boolean flags: Both configs use a "no-enable" prefix pattern (no_enable_*) because the internal get_py_cmd helper skips flags with falsy values. Setting these booleans to True forces the --no-enable-… flag to be passed to the underlying engine.

Launch Workflow

Both SGLangServerWrapper and vLLMServerWrapper follow an identical six-step orchestration pattern defined in areal/infra/launcher/sglang_server.py and areal/infra/launcher/vllm_server.py:

1. Calculate per-process resources – Derives gpus_per_server from allocation_mode.gen_instance_size.
2. Determine port ranges – find_free_ports(2, port_range) in areal/utils/network.py allocates unique HTTP and NCCL init ports.
3. Construct command – SGLangConfig.build_cmd or vLLMConfig.build_cmd expands dataclasses into CLI arguments.
4. Spawn subprocesses – launch_server_cmd isolates Triton/vLLM cache directories and starts servers via subprocess.Popen.
5. Health check – wait_for_server polls /v1/models until HTTP 200.
6. Name resolution – Registers successful servers with name_resolve.add_subentry for controller discovery during rollout.

Both wrappers install a monitor thread (_monitor_server_processes) that terminates the entire launch if any server process dies, ensuring clean failure modes for production clusters.

Production-Ready Considerations

Concern	SGLang	vLLM
Cross-node scaling	Supported via AREAL_SGLANG_MULTI_NODE_* environment variables; adjusts node_rank, master_addr, and master_port when gpus_per_server > n_gpus_per_node.	Not supported; launching across multiple nodes raises NotImplementedError in vLLMServerWrapper.run.
Metrics	Native Prometheus-style metrics via enable_metrics=True and decode_log_interval; optional request logging via log_requests.	Custom metrics via VLLMWorkerExtension in areal/engine/vllm_ext/vllm_worker_extension.py.
Graceful shutdown	Signal handlers (SIGTERM, SIGINT) forward shutdown to child processes via kill_process_tree.	Same pattern with additional cleanup of vLLM's compile cache (VLLM_CACHE_ROOT).
GPU memory	Use cpu_offload_gb and disable_cuda_graph to reduce fragmentation for large batches.	Control via gpu_memory_utilization (default 0.9) and swap_space for dynamic paging.
LoRA hot-swap	Disk-based updates only (weight_update_mode='disk') required; distributed updater does not support LoRA.	Disk-based or runtime loading via VLLM_ALLOW_RUNTIME_LORA_UPDATING=True.

Code Examples

Switching Backends Programmatically

from areal.api.cli_args import AllocationMode, parse_cli_args, to_structured_cfg
from areal.infra.launcher.sglang_server import launch_sglang_server
from areal.infra.launcher.vllm_server import launch_vllm_server

def launch_backend(argv: list[str], backend: str):
    # Parse base CLI args

    cfg, _ = parse_cli_args(argv)
    cfg.allocation_mode = AllocationMode.from_str(cfg.allocation_mode)
    assert cfg.allocation_mode.gen_backend == backend, "Mismatched backend"

    # Convert the backend-specific config to a structured dataclass

    if backend == "sglang":
        cfg.sglang = to_structured_cfg(cfg.sglang, SGLangConfig)
        launch_sglang_server(argv)
    elif backend == "vllm":
        cfg.vllm = to_structured_cfg(cfg.vllm, vLLMConfig)
        launch_vllm_server(argv)
    else:
        raise ValueError(f"Unsupported backend: {backend}")

# Example usage

launch_backend(sys.argv[1:], backend="sglang")

Key files referenced: sglang_server.py, vllm_server.py, and cli_args.py for config structs.

Adding Custom Metrics to the vLLM Worker

# areal/engine/vllm_ext/vllm_worker_extension.py

from areal.utils import stats_tracker

class VLLMWorkerExtension:
    async def on_generation_end(self, request_id: str, metrics: dict):
        # Increment a custom Prometheus counter

        stats_tracker.inc("vllm_generated_tokens", metrics["output_len"])
        # Forward the original metrics upstream

        return metrics

The launcher automatically imports this class via vllm_config.worker_extension_cls, which defaults to areal.engine.vllm_ext.vllm_worker_extension.VLLMWorkerExtension.

Enabling LoRA Hot-Swap with SGLang

# 1. Create a new LoRA checkpoint on disk

lora_path = "/lora/checkpoint_step_20000"

# 2. Issue a weight-update request via the RemoteSGLangEngine

engine.update_weights_from_disk(
    WeightUpdateMeta(
        path=Path(lora_path),
        use_lora=True,
        lora_name="my_lora",
        version=20000,
        alloc_mode=engine.allocation_mode,
    )
)

The backend constructs an HTTP POST to /load_lora_adapter via SGLangBackend.build_disk_weight_update_requests in areal/engine/sglang_remote.py.

Key Files Reference

Path	Role
areal/api/cli_args.py	Dataclasses for SGLangConfig, vLLMConfig, and CLI parsing
areal/infra/launcher/sglang_server.py	Process manager that spawns one or many SGLang server processes
areal/infra/launcher/vllm_server.py	Process manager for vLLM, mirrors the SGLang launch flow
areal/engine/sglang_remote.py	Remote inference engine implementation (RemoteSGLangEngine) and SGLangBackend
areal/engine/vllm_remote.py	Remote inference engine implementation (RemoteVLLMEngine)
areal/engine/vllm_ext/vllm_worker_extension.py	Custom vLLM worker extension that records AReaL-specific stats
areal/engine/vllm_ext/areal_vllm_server.py	Entrypoint used by vLLMConfig.build_cmd to start the server with the extension
areal/utils/network.py	Helper find_free_ports used by both launchers
areal/utils/name_resolve.py	Registers server addresses for the rollout controller

Summary

• AReaL abstracts SGLang and vLLM behind a unified InferenceEngine interface, enabling backend swaps via --allocation-mode.gen-backend.
• SGLang excels at single-node, low-latency generation with fine-grained scheduling and cross-node scaling via AREAL_SGLANG_MULTI_NODE_* environment variables.
• vLLM targets tensor-parallel scalability with built-in GPU memory utilization controls and custom worker extensions for metrics.
• Configuration differs in naming conventions (e.g., model_path vs model, random_seed vs seed) and available optimizations (CUDA graphs in SGLang, chunked prefill toggles in both).
• Production workflows require monitoring the _monitor_server_processes thread for clean failure modes and using name_resolve.add_subentry for controller discovery.

Frequently Asked Questions

How do I switch between SGLang and vLLM without changing my training script?

Set --allocation-mode.gen-backend to either sglang or vllm when launching the server. The AllocationMode dataclass in areal/api/cli_args.py routes to the appropriate wrapper (SGLangServerWrapper or vLLMServerWrapper), while your training controller continues to use the standard InferenceEngine interface for rollouts.

Can I scale vLLM across multiple nodes like SGLang?

No. As of the current implementation in areal/infra/launcher/vllm_server.py, vLLMServerWrapper.run raises NotImplementedError when attempting multi-node launches. SGLang supports cross-node scaling via the AREAL_SGLANG_MULTI_NODE_* environment variables, which configure node_rank, master_addr, and master_port for distributed initialization.

What is the difference between model_path and model in the configuration?

These fields refer to the same concept but follow each backend's native naming convention. SGLangConfig uses model_path to specify the HuggingFace model directory or name, while vLLMConfig uses model. Similarly, SGLang uses random_seed where vLLM uses seed. Always consult areal/api/cli_args.py for the exact field names when migrating configurations between backends.

How does AReaL handle LoRA updates in production?

Both backends support LoRA via enable_lora and max_lora_rank flags, but hot-swapping implementations differ. SGLang requires disk-based updates through RemoteSGLangEngine.update_weights_from_disk with weight_update_mode='disk', posting to /load_lora_adapter. vLLM supports both disk-based updates and runtime loading when VLLM_ALLOW_RUNTIME_LORA_UPDATING=True, offering more flexibility for dynamic adapter switching during training.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}