# What LLM Models Can Be Used with Litho: Complete Provider Guide

> Explore LLM models compatible with Litho. Discover integrations with OpenAI, Anthropic, Gemini, Mistral, Ollama, and more via Litho's flexible provider system. Enhance your projects today.

- Repository: [Sopaco/deepwiki-rs](https://github.com/sopaco/deepwiki-rs)
- Tags: guide
- Published: 2026-02-16

---

**Litho supports any LLM model from OpenAI, Anthropic, Gemini, DeepSeek, Mistral, Moonshot, OpenRouter, and local Ollama instances through a provider-agnostic configuration system that accepts arbitrary model identifier strings.**

The Litho engine—part of the **deepwiki-rs** repository—decouples model selection from core logic, allowing you to integrate virtually any LLM available through supported providers. This architecture enables seamless switching between cloud APIs and local inference servers without modifying the underlying Rust source code.

## Understanding Litho's Provider-Agnostic LLM Architecture

Litho implements a flexible abstraction layer that treats model identifiers as configuration strings rather than hard-coded constants. This design allows the system to adapt immediately to new model releases from any supported provider.

### The LLMProvider Enum and Configuration Structure

The provider selection logic resides in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs), where the `LLMProvider` enum defines all supported backends. The `LLMConfig` struct (lines 36–52) captures two critical model identifiers:

- **`model_efficient`** – The low-cost, high-throughput model for routine inference tasks.
- **`model_powerful`** – The high-capability model for complex reasoning or fallback scenarios.

Litho automatically falls back from `model_efficient` to `model_powerful` if the former returns an error or generates a low-quality response. This dual-model strategy is implemented in the provider adapters located in [`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs).

## Supported LLM Providers and Compatible Models

Litho accepts any model identifier string recognized by the following providers. The specific provider implementations are defined in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) between lines 10–28.

### OpenAI Models

**Configuration location:** `src/config.rs#L10-L14`

Litho supports any model available through the OpenAI API, including:

- `gpt-4o-mini` (efficient tier)
- `gpt-4o` (powerful tier)
- `gpt-3.5-turbo`
- `o1-preview` and `o1-mini` (reasoning models)

### Anthropic Claude Models

**Configuration location:** `src/config.rs#L21-L23`

Compatible with the Claude 3 and 3.5 model families:

- `claude-3-5-haiku-20241022` (efficient)
- `claude-3-5-sonnet-20241022` (balanced)
- `claude-3-opus-20240229` (powerful)

### Google Gemini Models

**Configuration location:** `src/config.rs#L24-L26`

Supports Gemini 1.5 and future releases:

- `gemini-1.5-pro`
- `gemini-1.5-flash`

### DeepSeek Models

**Configuration location:** `src/config.rs#L15-L17`

Compatible with DeepSeek's chat and reasoning models:

- `deepseek-chat`
- `deepseek-reasoner`

### Local Ollama Instances

**Configuration location:** `src/config.rs#L26-L28`

Litho can connect to local Ollama servers for fully offline operation:

- `llama3.2`
- `llama3.1:8b`
- `llama3.1:70b`
- `qwen2.5-coder`
- Any custom model pulled into your Ollama instance

### Additional Providers

- **Moonshot** (`src/config.rs#L13-L15`): Moonshot-specific model IDs
- **Mistral** (`src/config.rs#L17-L19`): Mistral AI model family
- **OpenRouter** (`src/config.rs#L19-L21`): Access to hundreds of models through a unified API, including `meta-llama/llama-3.1-8b-instruct`

## Configuring Dual-Model Strategy in Litho

Litho's configuration system requires defining both an efficient and powerful model for each provider. This enables automatic fallback behavior when the primary model fails or returns insufficient results.

### Configuration Schema

The `LLMConfig` struct in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) validates the following fields:

- `provider`: The backend enum value
- `model_efficient`: String identifier for routine tasks
- `model_powerful`: String identifier for complex reasoning
- `api_key`: Authentication token (optional for Ollama)
- `api_base_url`: Endpoint override for custom deployments

### Fallback Mechanism

When `model_efficient` encounters an error or returns a low-confidence response, Litho automatically promotes the request to `model_powerful`. This logic is implemented in the provider client layer at [`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs), ensuring robust operation even when primary models experience degradation.

## Practical Configuration Examples

### OpenAI Configuration

```toml
[llm]
provider = "openai"
api_key = "${LITHO_LLM_API_KEY}"
api_base_url = "https://api.openai.com/v1"
model_efficient = "gpt-4o-mini"
model_powerful = "gpt-4o"
max_tokens = 4096
temperature = 0.1

```

*The `model_efficient` and `model_powerful` fields are read by [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) (see the `LLMConfig` struct) and injected into the OpenAI client implementation.*

### Local Ollama Setup

```toml
[llm]
provider = "ollama"
api_base_url = "http://localhost:11434/v1"
model_efficient = "llama3.1:8b"
model_powerful = "llama3.1:70b"

```

*Ollama runs locally, so no API key is required. The same `LLMClient` logic picks the `Ollama` provider ([`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs)).*

### CLI Overrides

```bash
litho --config litho.toml \
      --model-efficient deepseek-chat \
      --model-powerful deepseek-reasoner

```

*The CLI ([`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs)) maps the `--model-efficient` and `--model-powerful` flags directly to the `LLMConfig` fields.*

### Custom OpenRouter Routes

```toml
[llm]
provider = "openrouter"
api_key = "${OPENROUTER_API_KEY}"
api_base_url = "https://openrouter.ai/api/v1"
model_efficient = "meta-llama/llama-3.1-8b-instruct"
model_powerful = "meta-llama/llama-3.1-70b-instruct"

```

*No Rust code changes are required; the new model identifiers are just strings passed to the OpenRouter client.*

## Key Implementation Files

| File | Description | Link |
|------|-------------|------|
| [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) | `LLMProvider` enum and `LLMConfig` struct (model names, provider selection, API settings) | [src/config.rs](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) |
| [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) | Command-line parsing for `--model-efficient`, `--model-powerful`, and `--llm-provider` flags | [src/cli.rs](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) |
| [`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs) | Provider-specific adapters that forward configured model names to remote APIs or local Ollama servers | [src/llm/client/providers.rs](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs) |
| [`docs/en/5.Boundary-Interfaces.md`](https://github.com/sopaco/deepwiki-rs/blob/main/docs/en/5.Boundary-Interfaces.md) | Detailed configuration examples for all providers and model selections | [Boundary-Interfaces (EN)](https://github.com/sopaco/deepwiki-rs/blob/main/docs/en/5.Boundary-Interfaces.md) |
| [`litho-example.toml`](https://github.com/sopaco/deepwiki-rs/blob/main/litho-example.toml) | Reference configuration file demonstrating dual-model setups for various providers | [litho-example.toml](https://github.com/sopaco/deepwiki-rs/blob/main/litho-example.toml) |

These files demonstrate **how Litho decouples model identifiers from the core engine**, enabling you to use any LLM model that the chosen provider exposes—whether it is a cloud offering (OpenAI, Anthropic, Gemini, etc.) or a local inference server (Ollama).

## Summary

- **Litho supports any model identifier** accepted by OpenAI, Anthropic, Gemini, DeepSeek, Mistral, Moonshot, OpenRouter, or Ollama.
- **Dual-model configuration** requires defining `model_efficient` for routine tasks and `model_powerful` for complex reasoning or fallback scenarios.
- **Provider-agnostic architecture** means new models work immediately without code changes—just update your [`litho.toml`](https://github.com/sopaco/deepwiki-rs/blob/main/litho.toml) or CLI flags.
- **Local deployment** is fully supported via Ollama integration, enabling offline operation with models like Llama 3.1 or Qwen.

## Frequently Asked Questions

### Can I use GPT-4 with Litho?

Yes. Litho accepts any OpenAI model identifier, including `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`, and `gpt-3.5-turbo`. Configure them in [`litho.toml`](https://github.com/sopaco/deepwiki-rs/blob/main/litho.toml) under the `[llm]` section using the `model_efficient` and `model_powerful` fields, or override via CLI flags `--model-efficient` and `--model-powerful`.

### How do I switch between different LLM providers?

Switching providers requires changing the `provider` field in your [`litho.toml`](https://github.com/sopaco/deepwiki-rs/blob/main/litho.toml) configuration file to the desired enum value (e.g., `"openai"`, `"anthropic"`, `"ollama"`). The `LLMProvider` enum in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) defines all valid options. No code recompilation is necessary; the provider-specific adapter in [`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs) handles the transition automatically.

### What is the difference between model_efficient and model_powerful?

The `model_efficient` field specifies a low-cost, high-speed model for routine inference tasks, while `model_powerful` designates a higher-capability model for complex reasoning or as a fallback when the efficient model fails. This dual-model strategy is defined in the `LLMConfig` struct in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) and enables automatic failover without manual intervention.

### Can I use local models without an internet connection?

Yes. Litho supports local inference through Ollama integration. Configure `provider = "ollama"` and point `api_base_url` to your local server (typically `http://localhost:11434/v1`). You can use models like `llama3.1:8b`, `llama3.1:70b`, or `qwen2.5-coder` without requiring an API key or external network access.