# How Graphify Selects and Switches Between LLM Backends: Claude, Gemini, OpenAI, and Ollama

> Discover how Graphify intelligently selects & switches LLM backends like Claude, Gemini, OpenAI, and Ollama. Learn about automatic detection and explicit backend control for seamless integration. Optimize your AI workflows today.

- Repository: [Graphify Labs/graphify](https://github.com/Graphify-Labs/graphify)
- Tags: internals
- Published: 2026-07-15

---

**Graphify automatically detects available LLM backends by checking environment variables in a fixed priority order, while also supporting explicit backend selection via the `backend=` parameter in API functions.**

Graphify, an open-source knowledge extraction toolkit from Graphify-Labs/graphify, implements a flexible two-stage system for LLM backend selection. Understanding how Graphify switches between providers like Anthropic Claude, Google Gemini, OpenAI, and local Ollama instances helps you configure the optimal setup for your extraction pipelines. This guide examines the source code implementation in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py) to explain the automatic detection logic, manual overrides, and custom provider support.

## Automatic Backend Detection in Graphify

When you run commands like `graphify extract` without specifying a backend, Graphify executes the `detect_backend()` function located at lines 2358-2384 in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py). This function inspects environment variables to identify which API keys are available, returning the first usable backend it finds.

### The Priority Order of Environment Variables

The detection algorithm checks providers in the following strict sequence:

1. **gemini** (checks `GEMINI_API_KEY` or `GOOGLE_API_KEY`)
2. **kimi**
3. **claude**
4. **openai**
5. **deepseek**
6. **azure**
7. **bedrock**
8. **ollama**

This hierarchy is hardcoded in the `detect_backend()` implementation. The function iterates through this list and stops at the first backend with a valid API key configured, immediately returning that provider's name as a string.

### Why the Order Matters

The priority ranking intentionally places paid cloud providers (Gemini, Claude, OpenAI) before local options like Ollama. According to the source code comments in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py), this prevents accidental "shadowing" where a locally running Ollama instance (detected via `OLLAMA_BASE_URL`) would mask a valid paid API key in your environment. By checking Ollama last, Graphify ensures you utilize your provisioned cloud credits when available, falling back to local inference only when no cloud keys exist.

## Explicit Backend Selection

While automatic detection provides convenience, Graphify allows forceful override through the `backend=` parameter available in most public API functions.

### Using the backend Parameter

Functions such as `extract_files_direct()`, `extract_corpus_parallel()`, and `label_communities()` accept an optional `backend` argument. When provided, this value bypasses the `detect_backend()` logic entirely, forcing Graphify to use your specified provider regardless of which environment variables are set.

```python
from graphify import llm

# Force Ollama usage even if OPENAI_API_KEY exists

result = llm.extract_files_direct(
    files=["document.pdf"],
    backend="ollama",
    root="/path/to/project"
)

```

### Configuration and Behavior Overrides

Once a backend is selected (either automatically or explicitly), Graphify determines several downstream behaviors through the `BACKENDS` dictionary defined at lines 59-77 in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py):

- **API endpoint and model**: Each entry specifies `base_url`, `default_model`, and the `env_key` containing the API key
- **Vision capabilities**: Backends with `vision: True` support image inputs; Ollama requires `GRAPHIFY_OLLAMA_VISION=1` to enable this (lines 88-89)
- **Request shaping**: The `_call_openai_compat()` function (lines 1031-1067) builds payload-specific parameters, adding `num_ctx` and `keep_alive` for Ollama while disabling thinking modes for Kimi when `GRAPHIFY_DISABLE_THINKING` is set

## Supported Backends and Configuration

The `BACKENDS` dictionary in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py) serves as the central registry for all LLM providers, defining their specific requirements and defaults.

### The BACKENDS Dictionary

Each backend entry contains:
- **env_key**: The environment variable(s) containing the API key
- **base_url**: Default API endpoint URL
- **default_model**: Fallback model if none specified
- **vision**: Boolean flag for multimodal support

For example, Gemini accepts either `GEMINI_API_KEY` or `GOOGLE_API_KEY` (lines 99-100), while Ollama uses `OLLAMA_BASE_URL` and optionally `OLLAMA_MODEL`.

### Vision Support and Special Cases

Ollama requires special handling for vision capabilities. Unlike cloud providers that report vision support natively, Ollama only enables image processing when you explicitly set the environment variable `GRAPHIFY_OLLAMA_VISION=1`. This safeguard prevents errors when running smaller local models that lack multimodal capabilities.

## Custom Provider Integration

Graphify supports arbitrary LLM providers through JSON configuration files. At lines 60-61 in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py), the system loads `~/.graphify/providers.json` (or a project-local file if `GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1` is set) and merges these entries into the `BACKENDS` dictionary.

Custom providers follow the same schema as built-in ones:

```json
{
  "my-local-vllm": {
    "base_url": "http://127.0.0.1:8000/v1",
    "env_key": "VLLM_API_KEY",
    "default_model": "my-model",
    "vision": false
  }
}

```

Once configured, `detect_backend()` recognizes these providers if their corresponding API keys are present, and you can reference them explicitly via `backend="my-local-vllm"`.

## Practical Implementation Examples

The following examples demonstrate the complete backend selection workflow:

```python
import os
from graphify import llm

# Example 1: Automatic detection priority

os.environ["OPENAI_API_KEY"] = "sk-my-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "sk-ant-api-key"

# Returns "claude" because it appears before "openai" in the priority list

detected = llm.detect_backend()
print(detected)  # Output: "claude"

# Example 2: Explicit Ollama configuration

os.environ["OLLAMA_BASE_URL"] = "http://localhost:11434/v1"
os.environ["GRAPHIFY_OLLAMA_NUM_CTX"] = "32768"

backend = "ollama"
model = llm._default_model_for_backend(backend)  # Respects OLLAMA_MODEL env var

print(f"Using {backend} with model {model}")

# Example 3: Custom provider detection

# File: ~/.graphify/providers.json contains vLLM configuration

os.environ["VLLM_API_KEY"] = "vllm-secret-key"
custom_backend = llm.detect_backend()
print(custom_backend)  # Output: "my-local-vllm"

```

## Summary

- **Automatic detection** uses `detect_backend()` in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py) to check environment variables in a fixed priority: Gemini → Kimi → Claude → OpenAI → DeepSeek → Azure → Bedrock → Ollama
- **Explicit selection** via the `backend=` parameter in functions like `extract_files_direct()` bypasses automatic detection completely
- **Configuration storage** happens in the `BACKENDS` dictionary (lines 59-77), which defines API endpoints, model defaults, and vision capabilities
- **Ollama special handling** requires `GRAPHIFY_OLLAMA_VISION=1` for image support and accepts `num_ctx` parameters via `GRAPHIFY_OLLAMA_NUM_CTX`
- **Custom providers** integrate through `~/.graphify/providers.json` and are merged at runtime, supporting the same selection mechanisms as built-in backends

## Frequently Asked Questions

### How does Graphify decide which LLM backend to use when I don't specify one?

Graphify runs the `detect_backend()` function, which inspects environment variables in a strict priority order: Gemini, Kimi, Claude, OpenAI, DeepSeek, Azure, Bedrock, and finally Ollama. It returns the first backend with a valid API key configured, ensuring paid cloud providers take precedence over local Ollama instances to prevent accidental shadowing of provisioned credentials.

### Can I force Graphify to use a specific backend even if I have multiple API keys configured?

Yes. Pass the `backend=` parameter to any major API function such as `extract_files_direct()`, `extract_corpus_parallel()`, or `label_communities()`. This explicit argument bypasses the automatic detection logic entirely, forcing Graphify to use your chosen provider regardless of which environment variables are present or their priority in the detection order.

### How do I enable vision capabilities for Ollama in Graphify?

Unlike cloud providers that enable vision automatically, Ollama requires you to set the environment variable `GRAPHIFY_OLLAMA_VISION=1`. This safeguards against errors when using smaller local models lacking multimodal support. Once set, Graphify treats Ollama as vision-capable and allows image inputs through the standard extraction pipeline.

### Can I add my own private LLM endpoint to Graphify?

Yes. Create a JSON file at `~/.graphify/providers.json` (or enable project-local providers with `GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1`) containing your custom backend configuration. Define fields like `base_url`, `env_key`, `default_model`, and `vision`. Graphify merges these entries into the `BACKENDS` dictionary at import time, allowing detection via `detect_backend()` or explicit selection via the `backend=` parameter.