How to Configure Graphify for Different LLM Backends (Ollama, Claude, Gemini, OpenAI)

Graphify uses a backend registry in graphify/llm.py to abstract LLM interactions, allowing you to switch between Ollama, Claude, Gemini, OpenAI, and custom providers by setting environment variables or using the --backend flag.

Graphify is an open-source code analysis tool that supports multiple large language model providers through a unified abstraction layer. By configuring environment variables or CLI flags, you can route inference to local Ollama instances, cloud APIs like OpenAI and Anthropic, or custom OpenAI-compatible endpoints without modifying source code.

Backend Registry Architecture

The core of Graphify's multi-provider support is the BACKENDS dictionary defined in graphify/llm.py (lines 51-100). This registry maps provider names to configuration objects containing base URLs, default models, pricing metadata, token limits, and vision capabilities.

When you run graphify extract, the system consults this registry to determine how to structure API calls and authenticate requests.

Auto-Detection Logic

Graphify automatically selects a backend using the detect_backend() function (lines 1943-1969 in graphify/llm.py). The function checks for API keys in the following priority order:

1. gemini
2. kimi
3. claude
4. openai
5. deepseek
6. azure and bedrock
7. ollama (last)

Ollama is intentionally checked last to prevent accidental routing to local models when cloud API keys are present. The function returns the first backend with a valid API key configured, or None if no credentials are found.

Configuring Specific Backends

Claude (Anthropic)

To use Claude models, set the ANTHROPIC_API_KEY environment variable. Graphify defaults to https://api.anthropic.com as the base URL.

export ANTHROPIC_API_KEY=your-key-here
graphify extract ./src --backend claude

OpenAI

Configure OpenAI by setting OPENAI_API_KEY. You can override the default model using GRAPHIFY_OPENAI_MODEL.

export OPENAI_API_KEY=sk-...
export GRAPHIFY_OPENAI_MODEL=gpt-4o
graphify extract ./code --backend openai

Gemini (Google)

Gemini requires either GEMINI_API_KEY or GOOGLE_API_KEY. Override the model with GRAPHIFY_GEMINI_MODEL.

export GEMINI_API_KEY=your-key
graphify extract ./project

Gemini has the highest priority in auto-detection, so it will be selected automatically if this key is present.

Ollama (Local)

Ollama requires no API key by default but uses several environment variables for configuration:

• OLLAMA_BASE_URL: Server endpoint (default: http://localhost:11434/v1)
• OLLAMA_MODEL: Model tag (default: qwen2.5-coder:7b)
• GRAPHIFY_OLLAMA_VISION=1: Enable vision for supported models like llama3.2-vision
• GRAPHIFY_OLLAMA_NUM_CTX: Manual context window override
• GRAPHIFY_OLLAMA_KEEP_ALIVE: Keep-alive timeout (default: 30m)

export OLLAMA_BASE_URL=http://127.0.0.1:11434/v1
export OLLAMA_MODEL=llama3.1:8b
export GRAPHIFY_OLLAMA_NUM_CTX=32768
graphify extract ./src --backend ollama

The num_ctx and keep_alive parameters are passed via extra_body in the OpenAI-compatible call path (_call_openai_compat, lines 889-904).

Kimi (Moonshot)

Set MOONSHOT_API_KEY to use Moonshot AI models. The default base URL is https://api.moonshot.ai/v1.

export MOONSHOT_API_KEY=your-key
graphify extract ./src --backend kimi

Claude-CLI (No API Key)

If you have the Claude Code CLI installed, use the claude-cli backend to route through your subscription without managing API keys. Set GRAPHIFY_CLAUDE_CLI_MODEL to customize the model (default: claude-code-plan).

graphify extract ./src --backend claude-cli

The implementation resides in _call_claude_cli() (lines 1029-1150).

Custom Providers

Beyond built-in backends, Graphify loads user-defined providers from ~/.graphify/providers.json or project-local .graphify/providers.json. The _load_custom_providers() function (lines 1970-2025) validates base_url entries and merges them into the BACKENDS registry.

Create ~/.graphify/providers.json:

{
  "my-gateway": {
    "base_url": "http://localhost:8000/v1",
    "default_model": "custom-model",
    "pricing": {"input": 0, "output": 0}
  }
}

Enable with:

export GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1
graphify extract ./src --backend my-gateway

CLI Backend Selection Priority

The CLI entry point in graphify/__main__.py resolves backends through this hierarchy:

1. Explicit flag: --backend <name> overrides all auto-detection
2. Auto-detection: detect_backend() selects based on environment variables
3. Custom providers: Loaded from JSON files and selectable by name

Summary

• Graphify maintains a BACKENDS registry in graphify/llm.py supporting Ollama, Claude, Gemini, OpenAI, Kimi, and custom providers.
• Auto-detection prioritizes cloud APIs over local Ollama; override with --backend.
• Each provider requires specific environment variables (e.g., ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY).
• Ollama supports additional tuning via GRAPHIFY_OLLAMA_NUM_CTX and GRAPHIFY_OLLAMA_VISION.
• Custom OpenAI-compatible endpoints can be added via ~/.graphify/providers.json.

Frequently Asked Questions

How do I force Graphify to use Ollama instead of auto-detected Gemini?

Pass the --backend ollama flag explicitly. Auto-detection prioritizes Gemini when GEMINI_API_KEY is set, so explicit selection is required to override this behavior and route to your local instance.

Can I use Graphify without an API key?

Yes, if you use the Claude-CLI backend (requires the Claude Code CLI installed) or Ollama (local deployment). Both options do not require cloud API keys, though Ollama may use OLLAMA_API_KEY if your local instance requires authentication.

What is the default model for Ollama in Graphify?

The default model is qwen2.5-coder:7b. Override this by setting the OLLAMA_MODEL environment variable before running the extraction command.

How do I add a self-hosted LLM to Graphify?

Create a JSON file at ~/.graphify/providers.json with your endpoint configuration, including base_url and default_model. Set GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1 if using a project-local configuration file, then select your backend with --backend <name>.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}