# How to Configure Custom OpenAI-Compatible Endpoints with vLLM in jcode

> Learn to configure custom OpenAI-compatible endpoints with vLLM using jcode. Set environment variables and log in to get started quickly. Integrate your LLM seamlessly.

- Repository: [Jeremy Huang/jcode](https://github.com/1jehuang/jcode)
- Tags: how-to-guide
- Published: 2026-04-30

---

**You can configure custom OpenAI-compatible endpoints with vLLM by setting the `JCODE_OPENAI_COMPAT_API_BASE` environment variable in `~/.config/jcode/openai-compatible.env` and running `jcode login --provider openai-compatible` to register the provider.**

The `jcode` CLI supports self-hosted vLLM instances through its OpenAI-compatible provider, allowing you to route LLM requests to any server implementing the OpenAI REST API. This configuration is driven entirely by environment variables stored in a provider-specific env file, eliminating the need for code changes when switching between cloud and local endpoints.

## Architecture Overview

The integration relies on several coordinated components across the `jcode` codebase:

- **Provider Catalog** – Registers the `openai-compatible` provider and assigns the environment file name. In [`crates/jcode-provider-metadata/src/lib.rs`](https://github.com/1jehuang/jcode/blob/main/crates/jcode-provider-metadata/src/lib.rs), the provider is defined with `id: "openai-compatible"` and `env_file: "openai-compatible.env"`.

- **Configuration Parser** – Reads environment variables from the file and constructs a `ProviderConfig` struct. The `type = "openai-compatible"` branch in [`src/config.rs`](https://github.com/1jehuang/jcode/blob/main/src/config.rs) handles parsing of the base URL and model settings.

- **CLI Interface** – Exposes the `--provider openai-compatible` flag in [`src/cli/args.rs`](https://github.com/1jehuang/jcode/blob/main/src/cli/args.rs), mapping user input to the provider ID.

- **Login UI** – Implements interactive prompts for API base URL and credentials. The commands in [`src/tui/app/auth_account_commands.rs`](https://github.com/1jehuang/jcode/blob/main/src/tui/app/auth_account_commands.rs) write values directly to the environment file.

- **Runtime Router** – Switches HTTP request paths based on the active provider. In [`src/provider/mod.rs`](https://github.com/1jehuang/jcode/blob/main/src/provider/mod.rs), the code selects the OpenAI-compatible API method when `JCODE_OPENAI_COMPAT_API_BASE` is present, forming requests against your vLLM server rather than the standard OpenAI endpoints.

## Step-by-Step Configuration Guide

### 1. Create the Environment Configuration File

On Linux and macOS, the default location is `~/.config/jcode/openai-compatible.env`. You can create this file manually or let `jcode` generate it automatically during the login process.

```bash
mkdir -p ~/.config/jcode
touch ~/.config/jcode/openai-compatible.env

```

### 2. Set Required Environment Variables

Populate the file with the following variables to point `jcode` at your vLLM server:

| Variable | Purpose | Example Value |
|----------|---------|---------------|
| `JCODE_OPENAI_COMPAT_API_BASE` | Base URL of your OpenAI-compatible server (must end with `/v1`) | `http://192.168.1.50:8000/v1` |
| `JCODE_OPENAI_COMPAT_DEFAULT_MODEL` | Default model identifier for requests | `Qwen/Qwen3-Coder-30B-A3B-Instruct` |
| `OPENAI_COMPAT_API_KEY` | *(Optional)* Bearer token for authenticated endpoints | `your-token-here` |

**Example configuration for a local vLLM instance:**

```bash
JCODE_OPENAI_COMPAT_API_BASE=http://127.0.0.1:8000/v1
JCODE_OPENAI_COMPAT_DEFAULT_MODEL=meta-llama/Meta-Llama-3-8B-Instruct

# Optional: only required if your vLLM server enforces authentication

OPENAI_COMPAT_API_KEY=sk-test-token

```

### 3. Register the Provider via CLI

Run the interactive login command to validate and save your configuration:

```bash
jcode login --provider openai-compatible

```

This command reads the environment file and prompts you to confirm or modify the `api-base`, `default-model`, and `api-key-name` values before registering the provider.

### 4. Verify Your Configuration

Confirm the settings were stored correctly using the account command:

```bash
/account openai-compatible settings

```

The TUI will display the current `api-base`, `default-model`, and any custom API key name you configured, sourced from [`src/tui/app/auth_account_commands.rs`](https://github.com/1jehuang/jcode/blob/main/src/tui/app/auth_account_commands.rs).

### 5. Route Requests to Your vLLM Server

Once configured, standard `jcode` commands automatically use your vLLM endpoint:

```bash
jcode run "Explain the borrow checker in Rust"

```

The request is sent to `http://127.0.0.1:8000/v1/chat/completions` (or your configured base URL) rather than the official OpenAI API.

## Practical Configuration Examples

### Manually Creating the Environment File

For automated deployments or CI/CD pipelines, create the configuration file programmatically:

```bash
cat > ~/.config/jcode/openai-compatible.env <<EOF
JCODE_OPENAI_COMPAT_API_BASE=http://192.168.1.100:8000/v1
JCODE_OPENAI_COMPAT_DEFAULT_MODEL=Qwen/Qwen3-Coder-30B-A3B-Instruct
OPENAI_COMPAT_API_KEY=your-token-here
EOF

```

### Updating Configuration from the CLI

Modify specific values without editing the file directly:

```bash

# Update the base URL for a new vLLM instance

/account openai-compatible api-base http://192.168.1.50:8000/v1

# Change the default model

/account openai-compatible default-model Qwen/Qwen3-Coder-30B-A3B-Instruct

```

These commands update the underlying environment file defined in [`crates/jcode-provider-metadata/src/lib.rs`](https://github.com/1jehuang/jcode/blob/main/crates/jcode-provider-metadata/src/lib.rs).

### Using a Custom API Key Variable

If your infrastructure requires a specific environment variable name for the API token, customize the key name:

```bash
/account openai-compatible api-key-name MY_VLLM_TOKEN
export MY_VLLM_TOKEN=super-secret-token

```

`jcode` will read `MY_VLLM_TOKEN` at runtime and include it as the `Authorization: Bearer …` header in requests.

## Summary

- **Environment-driven configuration**: All vLLM endpoint settings are controlled through `~/.config/jcode/openai-compatible.env` using variables like `JCODE_OPENAI_COMPAT_API_BASE`.
- **Provider registration**: Use `jcode login --provider openai-compatible` to activate the provider, which is cataloged in [`crates/jcode-provider-metadata/src/lib.rs`](https://github.com/1jehuang/jcode/blob/main/crates/jcode-provider-metadata/src/lib.rs).
- **Runtime routing**: The HTTP client in [`src/provider/mod.rs`](https://github.com/1jehuang/jcode/blob/main/src/provider/mod.rs) automatically directs requests to your custom endpoint when the OpenAI-compatible provider is active.
- **CLI management**: The `/account openai-compatible` commands in [`src/tui/app/auth_account_commands.rs`](https://github.com/1jehuang/jcode/blob/main/src/tui/app/auth_account_commands.rs) provide interactive editing of configuration values.

## Frequently Asked Questions

### What environment variables are required to configure a custom OpenAI-compatible endpoint?

You must set `JCODE_OPENAI_COMPAT_API_BASE` (the server URL ending in `/v1`) and `JCODE_OPENAI_COMPAT_DEFAULT_MODEL` (the model identifier). Optionally, set `OPENAI_COMPAT_API_KEY` if your vLLM server requires authentication. These are parsed by [`src/config.rs`](https://github.com/1jehuang/jcode/blob/main/src/config.rs) when the provider type is `openai-compatible`.

### Where does jcode store the OpenAI-compatible provider configuration?

The configuration is stored in `~/.config/jcode/openai-compatible.env` on Linux and macOS. This path is determined by the `env_file: "openai-compatible.env"` declaration in [`crates/jcode-provider-metadata/src/lib.rs`](https://github.com/1jehuang/jcode/blob/main/crates/jcode-provider-metadata/src/lib.rs), which associates the provider ID with its specific environment file.

### Can I use a custom API key environment variable name?

Yes. Run `/account openai-compatible api-key-name YOUR_CUSTOM_VAR` to instruct `jcode` to read from a different environment variable. This is useful when integrating with existing secret management systems that use specific variable names for bearer tokens.

### How do I verify my vLLM endpoint is configured correctly?

Execute `/account openai-compatible settings` in the `jcode` TUI to view the current `api-base`, `default-model`, and API key configuration. Then run a simple generation command like `jcode run "test"` to confirm requests are routing to your server and returning valid responses.