How Graphify Selects and Switches Between LLM Backends: Claude, Gemini, OpenAI, and Ollama

Graphify automatically detects available LLM backends by checking environment variables in a fixed priority order, while also supporting explicit backend selection via the backend= parameter in API functions.

Graphify, an open-source knowledge extraction toolkit from Graphify-Labs/graphify, implements a flexible two-stage system for LLM backend selection. Understanding how Graphify switches between providers like Anthropic Claude, Google Gemini, OpenAI, and local Ollama instances helps you configure the optimal setup for your extraction pipelines. This guide examines the source code implementation in graphify/llm.py to explain the automatic detection logic, manual overrides, and custom provider support.

Automatic Backend Detection in Graphify

When you run commands like graphify extract without specifying a backend, Graphify executes the detect_backend() function located at lines 2358-2384 in graphify/llm.py. This function inspects environment variables to identify which API keys are available, returning the first usable backend it finds.

The Priority Order of Environment Variables

The detection algorithm checks providers in the following strict sequence:

1. gemini (checks GEMINI_API_KEY or GOOGLE_API_KEY)
2. kimi
3. claude
4. openai
5. deepseek
6. azure
7. bedrock
8. ollama

This hierarchy is hardcoded in the detect_backend() implementation. The function iterates through this list and stops at the first backend with a valid API key configured, immediately returning that provider's name as a string.

Why the Order Matters

The priority ranking intentionally places paid cloud providers (Gemini, Claude, OpenAI) before local options like Ollama. According to the source code comments in graphify/llm.py, this prevents accidental "shadowing" where a locally running Ollama instance (detected via OLLAMA_BASE_URL) would mask a valid paid API key in your environment. By checking Ollama last, Graphify ensures you utilize your provisioned cloud credits when available, falling back to local inference only when no cloud keys exist.

Explicit Backend Selection

While automatic detection provides convenience, Graphify allows forceful override through the backend= parameter available in most public API functions.

Using the backend Parameter

Functions such as extract_files_direct(), extract_corpus_parallel(), and label_communities() accept an optional backend argument. When provided, this value bypasses the detect_backend() logic entirely, forcing Graphify to use your specified provider regardless of which environment variables are set.

from graphify import llm

# Force Ollama usage even if OPENAI_API_KEY exists

result = llm.extract_files_direct(
    files=["document.pdf"],
    backend="ollama",
    root="/path/to/project"
)

Configuration and Behavior Overrides

Once a backend is selected (either automatically or explicitly), Graphify determines several downstream behaviors through the BACKENDS dictionary defined at lines 59-77 in graphify/llm.py:

• API endpoint and model: Each entry specifies base_url, default_model, and the env_key containing the API key
• Vision capabilities: Backends with vision: True support image inputs; Ollama requires GRAPHIFY_OLLAMA_VISION=1 to enable this (lines 88-89)
• Request shaping: The _call_openai_compat() function (lines 1031-1067) builds payload-specific parameters, adding num_ctx and keep_alive for Ollama while disabling thinking modes for Kimi when GRAPHIFY_DISABLE_THINKING is set

Supported Backends and Configuration

The BACKENDS dictionary in graphify/llm.py serves as the central registry for all LLM providers, defining their specific requirements and defaults.

The BACKENDS Dictionary

Each backend entry contains:

• env_key: The environment variable(s) containing the API key
• base_url: Default API endpoint URL
• default_model: Fallback model if none specified
• vision: Boolean flag for multimodal support

For example, Gemini accepts either GEMINI_API_KEY or GOOGLE_API_KEY (lines 99-100), while Ollama uses OLLAMA_BASE_URL and optionally OLLAMA_MODEL.

Vision Support and Special Cases

Ollama requires special handling for vision capabilities. Unlike cloud providers that report vision support natively, Ollama only enables image processing when you explicitly set the environment variable GRAPHIFY_OLLAMA_VISION=1. This safeguard prevents errors when running smaller local models that lack multimodal capabilities.

Custom Provider Integration

Graphify supports arbitrary LLM providers through JSON configuration files. At lines 60-61 in graphify/llm.py, the system loads ~/.graphify/providers.json (or a project-local file if GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1 is set) and merges these entries into the BACKENDS dictionary.

Custom providers follow the same schema as built-in ones:

{
  "my-local-vllm": {
    "base_url": "http://127.0.0.1:8000/v1",
    "env_key": "VLLM_API_KEY",
    "default_model": "my-model",
    "vision": false
  }
}

Once configured, detect_backend() recognizes these providers if their corresponding API keys are present, and you can reference them explicitly via backend="my-local-vllm".

Practical Implementation Examples

The following examples demonstrate the complete backend selection workflow:

import os
from graphify import llm

# Example 1: Automatic detection priority

os.environ["OPENAI_API_KEY"] = "sk-my-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "sk-ant-api-key"

# Returns "claude" because it appears before "openai" in the priority list

detected = llm.detect_backend()
print(detected)  # Output: "claude"

# Example 2: Explicit Ollama configuration

os.environ["OLLAMA_BASE_URL"] = "http://localhost:11434/v1"
os.environ["GRAPHIFY_OLLAMA_NUM_CTX"] = "32768"

backend = "ollama"
model = llm._default_model_for_backend(backend)  # Respects OLLAMA_MODEL env var

print(f"Using {backend} with model {model}")

# Example 3: Custom provider detection

# File: ~/.graphify/providers.json contains vLLM configuration

os.environ["VLLM_API_KEY"] = "vllm-secret-key"
custom_backend = llm.detect_backend()
print(custom_backend)  # Output: "my-local-vllm"

Summary

• Automatic detection uses detect_backend() in graphify/llm.py to check environment variables in a fixed priority: Gemini → Kimi → Claude → OpenAI → DeepSeek → Azure → Bedrock → Ollama
• Explicit selection via the backend= parameter in functions like extract_files_direct() bypasses automatic detection completely
• Configuration storage happens in the BACKENDS dictionary (lines 59-77), which defines API endpoints, model defaults, and vision capabilities
• Ollama special handling requires GRAPHIFY_OLLAMA_VISION=1 for image support and accepts num_ctx parameters via GRAPHIFY_OLLAMA_NUM_CTX
• Custom providers integrate through ~/.graphify/providers.json and are merged at runtime, supporting the same selection mechanisms as built-in backends

Frequently Asked Questions

How does Graphify decide which LLM backend to use when I don't specify one?

Graphify runs the detect_backend() function, which inspects environment variables in a strict priority order: Gemini, Kimi, Claude, OpenAI, DeepSeek, Azure, Bedrock, and finally Ollama. It returns the first backend with a valid API key configured, ensuring paid cloud providers take precedence over local Ollama instances to prevent accidental shadowing of provisioned credentials.

Can I force Graphify to use a specific backend even if I have multiple API keys configured?

Yes. Pass the backend= parameter to any major API function such as extract_files_direct(), extract_corpus_parallel(), or label_communities(). This explicit argument bypasses the automatic detection logic entirely, forcing Graphify to use your chosen provider regardless of which environment variables are present or their priority in the detection order.

How do I enable vision capabilities for Ollama in Graphify?

Unlike cloud providers that enable vision automatically, Ollama requires you to set the environment variable GRAPHIFY_OLLAMA_VISION=1. This safeguards against errors when using smaller local models lacking multimodal support. Once set, Graphify treats Ollama as vision-capable and allows image inputs through the standard extraction pipeline.

Can I add my own private LLM endpoint to Graphify?

Yes. Create a JSON file at ~/.graphify/providers.json (or enable project-local providers with GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1) containing your custom backend configuration. Define fields like base_url, env_key, default_model, and vision. Graphify merges these entries into the BACKENDS dictionary at import time, allowing detection via detect_backend() or explicit selection via the backend= parameter.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}