Required Environment Variables for Configuring a Remote vLLM Inference Server in Chandra

Chandra requires three core environment variables—VLLM_API_BASE, VLLM_MODEL_NAME, and VLLM_GPUS—to establish a connection with a remote vLLM inference server, while VLLM_API_KEY and MAX_VLLM_RETRIES provide optional authentication and retry configuration.

Chandra is an open-source document analysis framework that offloads large language model inference to external vLLM servers for optimized throughput. To route generation requests correctly, you must configure specific environment variables that define the server endpoint, model identifier, and GPU allocation as implemented in the datalab-to/chandra repository.

Core Configuration Variables

The Settings class in [chandra/settings.py](https://github.com/datalab-to/chandra/blob/master/chandra/settings.py#L6-L23) defines five environment variables that control remote vLLM connectivity. The following three are required for any deployment:

VLLM_API_BASE

VLLM_API_BASE specifies the base URL of the vLLM REST endpoint. The default value is http://localhost:8000/v1, but production deployments must override this to point to your reachable server.

export VLLM_API_BASE="http://my-vllm-host:8000/v1"

VLLM_MODEL_NAME

VLLM_MODEL_NAME identifies the specific model loaded on your vLLM server. This value is injected into the OpenAI-compatible payload. The default is chandra, but it must match the --model parameter used when starting the vLLM server.

export VLLM_MODEL_NAME="my-chandra-model"

VLLM_GPUS

VLLM_GPUS declares which GPU device IDs the server should utilize. This accepts a comma-separated list of integers. While the default is 0, multi-GPU deployments require explicit enumeration.

export VLLM_GPUS="0,1,2"

Optional Authentication and Retry Settings

VLLM_API_KEY

VLLM_API_KEY enables authenticated access for vLLM deployments that enforce API key validation. The default is EMPTY, meaning no authentication header is sent unless explicitly configured.

export VLLM_API_KEY="my-secret-api-key"

MAX_VLLM_RETRIES

MAX_VLLM_RETRIES controls the number of automatic retry attempts for transient network errors or rate limiting. The default value is 6, but you can increase this for unreliable network conditions or decrease it for fail-fast scenarios.

export MAX_VLLM_RETRIES="10"

Configuration Methods

Chandra supports two methods for loading these variables: direct environment variables or a local.env file. The Settings class uses Pydantic's find_dotenv("local.env") mechanism to automatically discover configuration files in the repository root.

Method 1: Shell Environment Variables

Set the variables in your shell before invoking Chandra commands:

export VLLM_API_BASE="http://remote-server:8000/v1"
export VLLM_MODEL_NAME="chandra-70b"
export VLLM_GPUS="0,1"
export VLLM_API_KEY="sk-abc123"

chandra_vllm  # Docker container inherits these variables

Method 2: local.env File

Create a local.env file in the project root for persistent configuration:

# Server endpoint configuration

VLLM_API_BASE=http://remote-server:8000/v1
VLLM_MODEL_NAME=chandra-70b
VLLM_GPUS=0,1

# Optional security and retry policies

VLLM_API_KEY=sk-abc123
MAX_VLLM_RETRIES=8

Method 3: Programmatic Access

Access the validated settings object directly in Python code:

from chandra.settings import settings

# Access validated configuration

endpoint = settings.VLLM_API_BASE
model = settings.VLLM_MODEL_NAME
gpus = settings.VLLM_GPUS

Implementation Details

The configuration system is centralized in [chandra/settings.py](https://github.com/datalab-to/chandra/blob/master/chandra/settings.py), where the Settings Pydantic model validates types and defaults at runtime. The inference wrapper in [chandra/model/vllm.py](https://github.com/datalab-to/chandra/blob/master/chandra/model/vllm.py) consumes these values to construct OpenAI-compatible API requests. Additionally, the CLI helper in [chandra/scripts/vllm.py](https://github.com/datalab-to/chandra/blob/master/chandra/scripts/vllm.py) ensures these variables are passed correctly into the Docker container runtime.

Summary

• Three variables are mandatory: VLLM_API_BASE, VLLM_MODEL_NAME, and VLLM_GPUS define the server location, model identity, and GPU allocation.
• Two variables are optional: VLLM_API_KEY enables authentication when required, and MAX_VLLM_RETRIES adjusts error recovery behavior.
• Configuration is flexible: Values can be set via shell exports, a local.env file, or accessed programmatically through the Settings class.
• Source of truth: All variables are defined and validated in chandra/settings.py and consumed by the inference layer in chandra/model/vllm.py.

Frequently Asked Questions

What happens if I don't set VLLM_API_BASE?

If VLLM_API_BASE is not configured, Chandra defaults to http://localhost:8000/v1 as defined in chandra/settings.py. This assumes a local vLLM server; remote deployments must override this value to avoid connection failures.

Is VLLM_API_KEY required for all vLLM servers?

No. VLLM_API_KEY defaults to EMPTY and is only required when your vLLM deployment explicitly enables authentication via the --api-key server argument. Open deployments without authentication can omit this variable entirely.

How do I configure multiple GPUs for the vLLM server?

Set VLLM_GPUS as a comma-separated string of device IDs. For example, export VLLM_GPUS="0,1,2" allocates three GPUs to the inference server, matching the tensor parallelism or pipeline parallelism configuration used when launching vLLM.

Where should I place the local.env file?

Place local.env in the repository root directory. The Settings.Config class uses find_dotenv("local.env") to discover the file automatically, loading its contents before Chandra initializes the vLLM client in chandra/model/vllm.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}