# Required Environment Variables for Configuring a Remote vLLM Inference Server in Chandra

> Configure your remote vLLM inference server in Chandra using essential environment variables VLLM_API_BASE VLLM_MODEL_NAME and VLLM_GPUS Learn about optional settings for secure and reliable deployments.

- Repository: [Datalab/chandra](https://github.com/datalab-to/chandra)
- Tags: how-to-guide
- Published: 2026-03-27

---

**Chandra requires three core environment variables—`VLLM_API_BASE`, `VLLM_MODEL_NAME`, and `VLLM_GPUS`—to establish a connection with a remote vLLM inference server, while `VLLM_API_KEY` and `MAX_VLLM_RETRIES` provide optional authentication and retry configuration.**

Chandra is an open-source document analysis framework that offloads large language model inference to external vLLM servers for optimized throughput. To route generation requests correctly, you must configure specific environment variables that define the server endpoint, model identifier, and GPU allocation as implemented in the `datalab-to/chandra` repository.

## Core Configuration Variables

The `Settings` class in [[`chandra/settings.py`](https://github.com/datalab-to/chandra/blob/main/chandra/settings.py)](https://github.com/datalab-to/chandra/blob/master/chandra/settings.py#L6-L23) defines five environment variables that control remote vLLM connectivity. The following three are **required** for any deployment:

### VLLM_API_BASE

**`VLLM_API_BASE`** specifies the base URL of the vLLM REST endpoint. The default value is `http://localhost:8000/v1`, but production deployments must override this to point to your reachable server.

```bash
export VLLM_API_BASE="http://my-vllm-host:8000/v1"

```

### VLLM_MODEL_NAME

**`VLLM_MODEL_NAME`** identifies the specific model loaded on your vLLM server. This value is injected into the OpenAI-compatible payload. The default is `chandra`, but it must match the `--model` parameter used when starting the vLLM server.

```bash
export VLLM_MODEL_NAME="my-chandra-model"

```

### VLLM_GPUS

**`VLLM_GPUS`** declares which GPU device IDs the server should utilize. This accepts a comma-separated list of integers. While the default is `0`, multi-GPU deployments require explicit enumeration.

```bash
export VLLM_GPUS="0,1,2"

```

## Optional Authentication and Retry Settings

### VLLM_API_KEY

**`VLLM_API_KEY`** enables authenticated access for vLLM deployments that enforce API key validation. The default is `EMPTY`, meaning no authentication header is sent unless explicitly configured.

```bash
export VLLM_API_KEY="my-secret-api-key"

```

### MAX_VLLM_RETRIES

**`MAX_VLLM_RETRIES`** controls the number of automatic retry attempts for transient network errors or rate limiting. The default value is `6`, but you can increase this for unreliable network conditions or decrease it for fail-fast scenarios.

```bash
export MAX_VLLM_RETRIES="10"

```

## Configuration Methods

Chandra supports two methods for loading these variables: direct environment variables or a `local.env` file. The `Settings` class uses Pydantic's `find_dotenv("local.env")` mechanism to automatically discover configuration files in the repository root.

### Method 1: Shell Environment Variables

Set the variables in your shell before invoking Chandra commands:

```bash
export VLLM_API_BASE="http://remote-server:8000/v1"
export VLLM_MODEL_NAME="chandra-70b"
export VLLM_GPUS="0,1"
export VLLM_API_KEY="sk-abc123"

chandra_vllm  # Docker container inherits these variables

```

### Method 2: local.env File

Create a **`local.env`** file in the project root for persistent configuration:

```dotenv

# Server endpoint configuration

VLLM_API_BASE=http://remote-server:8000/v1
VLLM_MODEL_NAME=chandra-70b
VLLM_GPUS=0,1

# Optional security and retry policies

VLLM_API_KEY=sk-abc123
MAX_VLLM_RETRIES=8

```

### Method 3: Programmatic Access

Access the validated settings object directly in Python code:

```python
from chandra.settings import settings

# Access validated configuration

endpoint = settings.VLLM_API_BASE
model = settings.VLLM_MODEL_NAME
gpus = settings.VLLM_GPUS

```

## Implementation Details

The configuration system is centralized in [[`chandra/settings.py`](https://github.com/datalab-to/chandra/blob/main/chandra/settings.py)](https://github.com/datalab-to/chandra/blob/master/chandra/settings.py), where the `Settings` Pydantic model validates types and defaults at runtime. The inference wrapper in [[`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py)](https://github.com/datalab-to/chandra/blob/master/chandra/model/vllm.py) consumes these values to construct OpenAI-compatible API requests. Additionally, the CLI helper in [[`chandra/scripts/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/scripts/vllm.py)](https://github.com/datalab-to/chandra/blob/master/chandra/scripts/vllm.py) ensures these variables are passed correctly into the Docker container runtime.

## Summary

- **Three variables are mandatory**: `VLLM_API_BASE`, `VLLM_MODEL_NAME`, and `VLLM_GPUS` define the server location, model identity, and GPU allocation.
- **Two variables are optional**: `VLLM_API_KEY` enables authentication when required, and `MAX_VLLM_RETRIES` adjusts error recovery behavior.
- **Configuration is flexible**: Values can be set via shell exports, a `local.env` file, or accessed programmatically through the `Settings` class.
- **Source of truth**: All variables are defined and validated in [`chandra/settings.py`](https://github.com/datalab-to/chandra/blob/main/chandra/settings.py) and consumed by the inference layer in [`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py).

## Frequently Asked Questions

### What happens if I don't set VLLM_API_BASE?

If `VLLM_API_BASE` is not configured, Chandra defaults to `http://localhost:8000/v1` as defined in [`chandra/settings.py`](https://github.com/datalab-to/chandra/blob/main/chandra/settings.py). This assumes a local vLLM server; remote deployments must override this value to avoid connection failures.

### Is VLLM_API_KEY required for all vLLM servers?

No. `VLLM_API_KEY` defaults to `EMPTY` and is only required when your vLLM deployment explicitly enables authentication via the `--api-key` server argument. Open deployments without authentication can omit this variable entirely.

### How do I configure multiple GPUs for the vLLM server?

Set `VLLM_GPUS` as a comma-separated string of device IDs. For example, `export VLLM_GPUS="0,1,2"` allocates three GPUs to the inference server, matching the tensor parallelism or pipeline parallelism configuration used when launching vLLM.

### Where should I place the local.env file?

Place `local.env` in the repository root directory. The `Settings.Config` class uses `find_dotenv("local.env")` to discover the file automatically, loading its contents before Chandra initializes the vLLM client in [`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py).