# How to Configure Graphify for Different LLM Backends (Ollama, Claude, Gemini, OpenAI)

> Configure Graphify for Ollama, Claude, Gemini, and OpenAI backends easily. Learn to switch LLM providers via environment variables or the command line for seamless integration.

- Repository: [Safi/graphify](https://github.com/safishamsi/graphify)
- Tags: how-to-guide
- Published: 2026-06-15

---

**Graphify uses a backend registry in [`graphify/llm.py`](https://github.com/safishamsi/graphify/blob/main/graphify/llm.py) to abstract LLM interactions, allowing you to switch between Ollama, Claude, Gemini, OpenAI, and custom providers by setting environment variables or using the `--backend` flag.**

Graphify is an open-source code analysis tool that supports multiple large language model providers through a unified abstraction layer. By configuring environment variables or CLI flags, you can route inference to local Ollama instances, cloud APIs like OpenAI and Anthropic, or custom OpenAI-compatible endpoints without modifying source code.

## Backend Registry Architecture

The core of Graphify's multi-provider support is the `BACKENDS` dictionary defined in [`graphify/llm.py`](https://github.com/safishamsi/graphify/blob/main/graphify/llm.py) (lines 51-100). This registry maps provider names to configuration objects containing base URLs, default models, pricing metadata, token limits, and vision capabilities.

When you run `graphify extract`, the system consults this registry to determine how to structure API calls and authenticate requests.

## Auto-Detection Logic

Graphify automatically selects a backend using the `detect_backend()` function (lines 1943-1969 in [`graphify/llm.py`](https://github.com/safishamsi/graphify/blob/main/graphify/llm.py)). The function checks for API keys in the following priority order:

1. **gemini**
2. **kimi**
3. **claude**
4. **openai**
5. **deepseek**
6. **azure** and **bedrock**
7. **ollama** (last)

Ollama is intentionally checked last to prevent accidental routing to local models when cloud API keys are present. The function returns the first backend with a valid API key configured, or `None` if no credentials are found.

## Configuring Specific Backends

### Claude (Anthropic)

To use Claude models, set the `ANTHROPIC_API_KEY` environment variable. Graphify defaults to `https://api.anthropic.com` as the base URL.

```bash
export ANTHROPIC_API_KEY=your-key-here
graphify extract ./src --backend claude

```

### OpenAI

Configure OpenAI by setting `OPENAI_API_KEY`. You can override the default model using `GRAPHIFY_OPENAI_MODEL`.

```bash
export OPENAI_API_KEY=sk-...
export GRAPHIFY_OPENAI_MODEL=gpt-4o
graphify extract ./code --backend openai

```

### Gemini (Google)

Gemini requires either `GEMINI_API_KEY` or `GOOGLE_API_KEY`. Override the model with `GRAPHIFY_GEMINI_MODEL`.

```bash
export GEMINI_API_KEY=your-key
graphify extract ./project

```

Gemini has the highest priority in auto-detection, so it will be selected automatically if this key is present.

### Ollama (Local)

Ollama requires no API key by default but uses several environment variables for configuration:

- `OLLAMA_BASE_URL`: Server endpoint (default: `http://localhost:11434/v1`)
- `OLLAMA_MODEL`: Model tag (default: `qwen2.5-coder:7b`)
- `GRAPHIFY_OLLAMA_VISION=1`: Enable vision for supported models like `llama3.2-vision`
- `GRAPHIFY_OLLAMA_NUM_CTX`: Manual context window override
- `GRAPHIFY_OLLAMA_KEEP_ALIVE`: Keep-alive timeout (default: `30m`)

```bash
export OLLAMA_BASE_URL=http://127.0.0.1:11434/v1
export OLLAMA_MODEL=llama3.1:8b
export GRAPHIFY_OLLAMA_NUM_CTX=32768
graphify extract ./src --backend ollama

```

The `num_ctx` and `keep_alive` parameters are passed via `extra_body` in the OpenAI-compatible call path (`_call_openai_compat`, lines 889-904).

### Kimi (Moonshot)

Set `MOONSHOT_API_KEY` to use Moonshot AI models. The default base URL is `https://api.moonshot.ai/v1`.

```bash
export MOONSHOT_API_KEY=your-key
graphify extract ./src --backend kimi

```

### Claude-CLI (No API Key)

If you have the Claude Code CLI installed, use the `claude-cli` backend to route through your subscription without managing API keys. Set `GRAPHIFY_CLAUDE_CLI_MODEL` to customize the model (default: `claude-code-plan`).

```bash
graphify extract ./src --backend claude-cli

```

The implementation resides in `_call_claude_cli()` (lines 1029-1150).

## Custom Providers

Beyond built-in backends, Graphify loads user-defined providers from `~/.graphify/providers.json` or project-local [`.graphify/providers.json`](https://github.com/safishamsi/graphify/blob/main/.graphify/providers.json). The `_load_custom_providers()` function (lines 1970-2025) validates `base_url` entries and merges them into the `BACKENDS` registry.

Create `~/.graphify/providers.json`:

```json
{
  "my-gateway": {
    "base_url": "http://localhost:8000/v1",
    "default_model": "custom-model",
    "pricing": {"input": 0, "output": 0}
  }
}

```

Enable with:

```bash
export GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1
graphify extract ./src --backend my-gateway

```

## CLI Backend Selection Priority

The CLI entry point in [`graphify/__main__.py`](https://github.com/safishamsi/graphify/blob/main/graphify/__main__.py) resolves backends through this hierarchy:

1. **Explicit flag**: `--backend <name>` overrides all auto-detection
2. **Auto-detection**: `detect_backend()` selects based on environment variables
3. **Custom providers**: Loaded from JSON files and selectable by name

## Summary

- Graphify maintains a `BACKENDS` registry in [`graphify/llm.py`](https://github.com/safishamsi/graphify/blob/main/graphify/llm.py) supporting Ollama, Claude, Gemini, OpenAI, Kimi, and custom providers.
- Auto-detection prioritizes cloud APIs over local Ollama; override with `--backend`.
- Each provider requires specific environment variables (e.g., `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`).
- Ollama supports additional tuning via `GRAPHIFY_OLLAMA_NUM_CTX` and `GRAPHIFY_OLLAMA_VISION`.
- Custom OpenAI-compatible endpoints can be added via `~/.graphify/providers.json`.

## Frequently Asked Questions

### How do I force Graphify to use Ollama instead of auto-detected Gemini?

Pass the `--backend ollama` flag explicitly. Auto-detection prioritizes Gemini when `GEMINI_API_KEY` is set, so explicit selection is required to override this behavior and route to your local instance.

### Can I use Graphify without an API key?

Yes, if you use the **Claude-CLI** backend (requires the Claude Code CLI installed) or **Ollama** (local deployment). Both options do not require cloud API keys, though Ollama may use `OLLAMA_API_KEY` if your local instance requires authentication.

### What is the default model for Ollama in Graphify?

The default model is `qwen2.5-coder:7b`. Override this by setting the `OLLAMA_MODEL` environment variable before running the extraction command.

### How do I add a self-hosted LLM to Graphify?

Create a JSON file at `~/.graphify/providers.json` with your endpoint configuration, including `base_url` and `default_model`. Set `GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1` if using a project-local configuration file, then select your backend with `--backend <name>`.