How to Configure Custom OpenAI-Compatible Endpoints with vLLM in jcode

You can configure custom OpenAI-compatible endpoints with vLLM by setting the JCODE_OPENAI_COMPAT_API_BASE environment variable in ~/.config/jcode/openai-compatible.env and running jcode login --provider openai-compatible to register the provider.

The jcode CLI supports self-hosted vLLM instances through its OpenAI-compatible provider, allowing you to route LLM requests to any server implementing the OpenAI REST API. This configuration is driven entirely by environment variables stored in a provider-specific env file, eliminating the need for code changes when switching between cloud and local endpoints.

Architecture Overview

The integration relies on several coordinated components across the jcode codebase:

• Provider Catalog – Registers the openai-compatible provider and assigns the environment file name. In crates/jcode-provider-metadata/src/lib.rs, the provider is defined with id: "openai-compatible" and env_file: "openai-compatible.env".

• Configuration Parser – Reads environment variables from the file and constructs a ProviderConfig struct. The type = "openai-compatible" branch in src/config.rs handles parsing of the base URL and model settings.

• CLI Interface – Exposes the --provider openai-compatible flag in src/cli/args.rs, mapping user input to the provider ID.

• Login UI – Implements interactive prompts for API base URL and credentials. The commands in src/tui/app/auth_account_commands.rs write values directly to the environment file.

• Runtime Router – Switches HTTP request paths based on the active provider. In src/provider/mod.rs, the code selects the OpenAI-compatible API method when JCODE_OPENAI_COMPAT_API_BASE is present, forming requests against your vLLM server rather than the standard OpenAI endpoints.

Step-by-Step Configuration Guide

1. Create the Environment Configuration File

On Linux and macOS, the default location is ~/.config/jcode/openai-compatible.env. You can create this file manually or let jcode generate it automatically during the login process.

mkdir -p ~/.config/jcode
touch ~/.config/jcode/openai-compatible.env

2. Set Required Environment Variables

Populate the file with the following variables to point jcode at your vLLM server:

Variable	Purpose	Example Value
JCODE_OPENAI_COMPAT_API_BASE	Base URL of your OpenAI-compatible server (must end with /v1)	http://192.168.1.50:8000/v1
JCODE_OPENAI_COMPAT_DEFAULT_MODEL	Default model identifier for requests	Qwen/Qwen3-Coder-30B-A3B-Instruct
OPENAI_COMPAT_API_KEY	(Optional) Bearer token for authenticated endpoints	your-token-here

Example configuration for a local vLLM instance:

JCODE_OPENAI_COMPAT_API_BASE=http://127.0.0.1:8000/v1
JCODE_OPENAI_COMPAT_DEFAULT_MODEL=meta-llama/Meta-Llama-3-8B-Instruct

# Optional: only required if your vLLM server enforces authentication

OPENAI_COMPAT_API_KEY=sk-test-token

3. Register the Provider via CLI

Run the interactive login command to validate and save your configuration:

jcode login --provider openai-compatible

This command reads the environment file and prompts you to confirm or modify the api-base, default-model, and api-key-name values before registering the provider.

4. Verify Your Configuration

Confirm the settings were stored correctly using the account command:

/account openai-compatible settings

The TUI will display the current api-base, default-model, and any custom API key name you configured, sourced from src/tui/app/auth_account_commands.rs.

5. Route Requests to Your vLLM Server

Once configured, standard jcode commands automatically use your vLLM endpoint:

jcode run "Explain the borrow checker in Rust"

The request is sent to http://127.0.0.1:8000/v1/chat/completions (or your configured base URL) rather than the official OpenAI API.

Practical Configuration Examples

Manually Creating the Environment File

For automated deployments or CI/CD pipelines, create the configuration file programmatically:

cat > ~/.config/jcode/openai-compatible.env <<EOF
JCODE_OPENAI_COMPAT_API_BASE=http://192.168.1.100:8000/v1
JCODE_OPENAI_COMPAT_DEFAULT_MODEL=Qwen/Qwen3-Coder-30B-A3B-Instruct
OPENAI_COMPAT_API_KEY=your-token-here
EOF

Updating Configuration from the CLI

Modify specific values without editing the file directly:

# Update the base URL for a new vLLM instance

/account openai-compatible api-base http://192.168.1.50:8000/v1

# Change the default model

/account openai-compatible default-model Qwen/Qwen3-Coder-30B-A3B-Instruct

These commands update the underlying environment file defined in crates/jcode-provider-metadata/src/lib.rs.

Using a Custom API Key Variable

If your infrastructure requires a specific environment variable name for the API token, customize the key name:

/account openai-compatible api-key-name MY_VLLM_TOKEN
export MY_VLLM_TOKEN=super-secret-token

jcode will read MY_VLLM_TOKEN at runtime and include it as the Authorization: Bearer … header in requests.

Summary

• Environment-driven configuration: All vLLM endpoint settings are controlled through ~/.config/jcode/openai-compatible.env using variables like JCODE_OPENAI_COMPAT_API_BASE.
• Provider registration: Use jcode login --provider openai-compatible to activate the provider, which is cataloged in crates/jcode-provider-metadata/src/lib.rs.
• Runtime routing: The HTTP client in src/provider/mod.rs automatically directs requests to your custom endpoint when the OpenAI-compatible provider is active.
• CLI management: The /account openai-compatible commands in src/tui/app/auth_account_commands.rs provide interactive editing of configuration values.

Frequently Asked Questions

What environment variables are required to configure a custom OpenAI-compatible endpoint?

You must set JCODE_OPENAI_COMPAT_API_BASE (the server URL ending in /v1) and JCODE_OPENAI_COMPAT_DEFAULT_MODEL (the model identifier). Optionally, set OPENAI_COMPAT_API_KEY if your vLLM server requires authentication. These are parsed by src/config.rs when the provider type is openai-compatible.

Where does jcode store the OpenAI-compatible provider configuration?

The configuration is stored in ~/.config/jcode/openai-compatible.env on Linux and macOS. This path is determined by the env_file: "openai-compatible.env" declaration in crates/jcode-provider-metadata/src/lib.rs, which associates the provider ID with its specific environment file.

Can I use a custom API key environment variable name?

Yes. Run /account openai-compatible api-key-name YOUR_CUSTOM_VAR to instruct jcode to read from a different environment variable. This is useful when integrating with existing secret management systems that use specific variable names for bearer tokens.

How do I verify my vLLM endpoint is configured correctly?

Execute /account openai-compatible settings in the jcode TUI to view the current api-base, default-model, and API key configuration. Then run a simple generation command like jcode run "test" to confirm requests are routing to your server and returning valid responses.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}