How to Configure LiteLLM for Different LLM Providers in Open Interpreter

Open Interpreter uses the LiteLLM library as a unified wrapper to route requests to OpenAI, Anthropic, AWS Bedrock, Ollama, and other providers using simple model name prefixes and environment variables.

When working with the openinterpreter/open-interpreter repository, you configure LiteLLM by setting the interpreter.llm.model string with a provider prefix (e.g., openai/gpt-4o, bedrock/anthropic.claude-3-sonnet, or ollama/phi) and supplying the corresponding API credentials. The system automatically handles provider-specific client initialization, token limit detection, and capability checking through the LiteLLM abstraction layer.

Understanding the LiteLLM Integration Architecture

The integration spans three core components that handle model resolution, credential validation, and declarative configuration.

The Core LLM Wrapper

In interpreter/core/llm/llm.py, the Llm class serves as the central interface. It stores the model name, API keys, and runtime options. When you assign a model string like bedrock/anthropic.claude-3-sonnet-20240229-v1:0, the wrapper uses LiteLLM's supports_function_calls() and supports_vision() utilities to detect capabilities automatically.

The wrapper reloads the provider-specific client dynamically when the model changes. For Ollama models (prefixed with ollama/), the load() method contacts the local daemon at http://localhost:11434 to pull the model if missing and extracts the context window size.

Interactive Validation

The interpreter/terminal_interface/validate_llm_settings.py module handles credential prompting. When you select an OpenAI model and the OPENAI_API_KEY environment variable is missing, this function interactively prompts for the key and assigns it to interpreter.llm.api_key. For AWS Bedrock and other providers, you must pre-export the environment variables, as the validation logic currently focuses on OpenAI authentication.

Profile-Based Configuration

Profile files in interpreter/terminal_interface/profiles/defaults/*.py provide a declarative way to bundle model settings. A profile is a Python script that runs on interpreter startup, setting attributes like interpreter.llm.model, interpreter.llm.api_key, and interpreter.llm.api_base. This allows you to switch between provider configurations by passing a single --profile argument.

Provider-Specific Configuration Patterns

Each provider requires a specific model string format and credential set.

OpenAI Configuration

Use the openai/ prefix or simply the model name for OpenAI models.

from interpreter import interpreter

# Both formats work identically

interpreter.llm.model = "openai/gpt-4o"

# or

interpreter.llm.model = "gpt-4o"

interpreter.llm.api_key = "sk-your-openai-key"

# Or set environment variable: export OPENAI_API_KEY="sk-..."

The base URL defaults to the official OpenAI endpoint. Override it with interpreter.llm.api_base for proxies or Azure deployments.

Anthropic via AWS Bedrock

For Claude models hosted on AWS Bedrock, use the bedrock/ prefix with the full model ID.

from interpreter import interpreter
import os

# Set AWS credentials in environment

os.environ["AWS_ACCESS_KEY_ID"] = "your-access-key"
os.environ["AWS_SECRET_ACCESS_KEY"] = "your-secret-key"
os.environ["AWS_REGION_NAME"] = "us-east-1"

# Enable tool use (computer API) for function calling

interpreter.computer.import_computer_api = True

# Model string format for Bedrock Anthropic

interpreter.llm.model = "bedrock/anthropic.claude-3-sonnet-20240229-v1:0"

The validate_llm_settings function does not prompt for AWS credentials automatically—you must export these environment variables before starting the interpreter.

Local Models with Ollama

For local inference, use the ollama/ prefix. No API key is required.

from interpreter import interpreter

interpreter.llm.model = "ollama/phi"

# Automatically contacts http://localhost:11434

# The wrapper pulls the model if not present and detects context window

Ensure the Ollama server is running locally. The system reads the OLLAMA_HOST environment variable if your server runs on a non-standard host or port.

Setting API Keys and Environment Variables

While you can assign interpreter.llm.api_key directly in Python, using environment variables is recommended for security.

# OpenAI

export OPENAI_API_KEY="sk-your-key"

# AWS Bedrock (for Anthropic, Amazon Titan, etc.)

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_REGION_NAME="us-east-1"

# Optional: Custom base URL for OpenAI-compatible endpoints

export OPENAI_API_BASE="https://your-proxy.com/v1"

When environment variables are set, the Llm class automatically picks them up via LiteLLM's internal resolution logic, requiring no additional code changes.

Summary

• LiteLLM Integration: Open Interpreter uses interpreter/core/llm/llm.py to wrap LiteLLM, enabling unified access to OpenAI, Anthropic, AWS Bedrock, and Ollama through model name prefixes.
• Configuration Method: Set interpreter.llm.model with provider prefixes (openai/, bedrock/, ollama/) and supply credentials via environment variables or direct assignment to interpreter.llm.api_key.
• Validation: The validate_llm_settings module handles interactive prompting for OpenAI keys, but AWS and other providers require pre-exported environment variables.
• Profiles: Declarative configuration files in interpreter/terminal_interface/profiles/defaults/ allow you to bundle provider settings and switch between them using the --profile CLI flag.

Frequently Asked Questions

How do I switch between OpenAI and Anthropic models without changing code?

Use the --profile command-line argument to load different configuration files. Create separate profile files in interpreter/terminal_interface/profiles/defaults/—one setting interpreter.llm.model to an OpenAI model and another to a Bedrock Anthropic model—then run interpreter --profile openai or interpreter --profile anthropic to switch instantly.

Why does Open Interpreter prompt for API keys only for OpenAI and not for AWS Bedrock?

The validate_llm_settings function in interpreter/terminal_interface/validate_llm_settings.py currently implements interactive prompting only for OpenAI models when the OPENAI_API_KEY environment variable is missing. For AWS Bedrock and other providers, you must export the required environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION_NAME) before starting the interpreter, as the validation logic does not yet handle credential prompting for these services.

Can I use a custom OpenAI-compatible API endpoint instead of the official OpenAI servers?

Yes. Set interpreter.llm.api_base to your custom endpoint URL, or export the OPENAI_API_BASE environment variable. The LiteLLM wrapper in interpreter/core/llm/llm.py passes this base URL to the underlying provider client, allowing you to use proxy servers, Azure OpenAI Service, or other OpenAI-compatible APIs while maintaining the same model interface.

How does Open Interpreter handle local models running on Ollama?

When you set interpreter.llm.model to a string prefixed with ollama/ (e.g., ollama/phi), the Llm class in interpreter/core/llm/llm.py automatically routes requests to the local Ollama server at http://localhost:11434 (or the host specified in the OLLAMA_HOST environment variable). The wrapper handles model pulling if the model is missing and automatically detects the context window size from the Ollama daemon, requiring no API key configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}