# How to Configure Different LLM Providers for Litho: A Complete Guide

> Configure Litho with various LLM providers like OpenAI, Anthropic, Gemini, and Ollama using TOML, flags, or environment variables. Get started with this complete guide.

- Repository: [Sopaco/deepwiki-rs](https://github.com/sopaco/deepwiki-rs)
- Tags: how-to-guide
- Published: 2026-02-16

---

**Litho supports multiple LLM backends through a plugin architecture that allows you to configure providers via TOML configuration files, command-line flags, or environment variables, with automatic client instantiation for OpenAI, Anthropic, Gemini, Ollama, and others.**

Litho is a documentation generation tool from the sopaco/deepwiki-rs repository that analyzes codebases and produces structured documentation using large language models. When deploying Litho, you can configure different LLM providers to match your infrastructure requirements, whether you need cloud-based APIs or local inference engines.

## Understanding Litho's LLM Provider Architecture

Litho implements a flexible provider system centered around the `LLMProvider` enum defined in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs). This enum supports eight distinct backends, each mapping to a specific client implementation in the underlying `rig` framework. When Litho starts, the `ProviderClient::new` factory method in [`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs) (lines 35-80) instantiates the appropriate client based on your configuration, enabling seamless switching between providers without code changes.

## Configuration Methods for LLM Providers

You can specify which LLM provider Litho uses through three distinct mechanisms, listed here in order of precedence:

### Using the Configuration File (litho.toml)

The most persistent method involves creating a [`litho.toml`](https://github.com/sopaco/deepwiki-rs/blob/main/litho.toml) file in your project root. Under the `[llm]` table, specify the `provider` field with one of the supported identifiers:

```toml
[llm]
provider = "anthropic"
api_key = "${LITHO_LLM_API_KEY}"
api_base_url = "https://api.anthropic.com"
model_efficient = "claude-3-haiku-20240307"
model_powerful = "claude-3-opus-20240229"

```

According to the source code in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) (lines 138-144), the `LLMConfig` struct parses these values during startup, with the `provider` field deserializing into the `LLMProvider` enum.

### Using Command-Line Flags

For temporary overrides or CI/CD pipelines, pass the `--llm-provider` flag when invoking the binary. This overrides any value set in the configuration file:

```bash
deepwiki-rs \
  --llm-provider gemini \
  --llm-api-key "$GEMINI_API_KEY" \
  --model-efficient "gemini-1.5-flash" \
  --model-powerful "gemini-1.5-pro"

```

The CLI argument definitions reside in [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) (lines 78-80), with the override logic implemented in `Args::to_config` (lines 154-161).

### Using Environment Variables

When neither configuration files nor CLI flags specify a provider, Litho defaults to `OpenAI`. You can influence behavior through environment variables:

- `LITHO_LLM_API_KEY`: Sets the API key for the chosen provider
- Provider selection defaults to `openai` unless overridden by other methods

The default implementation in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) (lines 30-34) shows that `LLMProvider::default()` returns `OpenAI`, while `LLMConfig::default` (line 43) handles API key retrieval from the environment.

## Supported LLM Providers

Litho supports eight LLM backends through the `LLMProvider` enum defined in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) (lines 9-28):

```rust
pub enum LLMProvider {
    #[serde(rename = "openai")]
    OpenAI,
    #[serde(rename = "moonshot")]
    Moonshot,
    #[serde(rename = "deepseek")]
    DeepSeek,
    #[serde(rename = "mistral")]
    Mistral,
    #[serde(rename = "openrouter")]
    OpenRouter,
    #[serde(rename = "anthropic")]
    Anthropic,
    #[serde(rename = "gemini")]
    Gemini,
    #[serde(rename = "ollama")]
    Ollama,
}

```

Each variant maps to a corresponding client in the `rig` provider ecosystem. Note that `ollama` requires no API key and defaults to `http://localhost:11434` as implemented in [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) (lines 162-168).

## Practical Configuration Examples

### Configuring Anthropic Claude

To use Anthropic's Claude models, specify `anthropic` as the provider and provide your API key:

```toml
[llm]
provider = "anthropic"
api_key = "sk-ant-api03-..."
model_efficient = "claude-3-haiku-20240307"
model_powerful = "claude-3-opus-20240229"

```

### Configuring Google Gemini via CLI

For one-time documentation generation using Google's Gemini:

```bash
deepwiki-rs -p ./my-project -o ./docs \
  --llm-provider gemini \
  --llm-api-key "$GEMINI_API_KEY" \
  --model-efficient "gemini-1.5-flash" \
  --model-powerful "gemini-1.5-pro"

```

### Setting Up Local Ollama Models

To run Litho against local models via Ollama without cloud API costs:

```bash
deepwiki-rs -p . \
  --llm-provider ollama \
  --model-efficient "llama3.2" \
  --model-powerful "llama3.2:70b"

```

No API key is required. The client automatically connects to `http://localhost:11434` as specified in [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs).

### Programmatic Configuration in Rust

When embedding Litho as a library, construct the configuration programmatically:

```rust
use deepwiki_rs::config::{Config, LLMProvider, LLMConfig};

let mut cfg = Config::default();
cfg.llm = LLMConfig {
    provider: LLMProvider::OpenRouter,
    api_key: "my-openrouter-key".into(),
    api_base_url: "https://openrouter.ai/api/v1".into(),
    model_efficient: "meta-llama/Meta-Llama-3.1-8B-Instruct".into(),
    model_powerful: "meta-llama/Meta-Llama-3.1-70B-Instruct".into(),
    ..Default::default()
};

```

## Key Configuration Files and Implementation Details

Understanding the source structure helps when troubleshooting provider issues:

| File | Purpose | Key Components |
|------|---------|----------------|
| [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) | Core configuration structures | `LLMProvider` enum (lines 9-28), `LLMConfig` struct (lines 138-144), default implementations |
| [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) | Command-line interface | `--llm-provider` flag (lines 78-80), `Args::to_config` override logic (lines 154-161), Ollama URL fallback (lines 162-168) |
| [`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs) | Provider client factory | `ProviderClient::new` (lines 35-80) that instantiates `rig` clients |
| [`litho-example.toml`](https://github.com/sopaco/deepwiki-rs/blob/main/litho-example.toml) | Reference configuration | Complete example of `[llm]` table settings |

## Summary

- Litho supports **eight LLM providers** through a unified configuration system: OpenAI, Anthropic, Gemini, Mistral, DeepSeek, Moonshot, OpenRouter, and Ollama.
- You can **configure providers via three methods**: the [`litho.toml`](https://github.com/sopaco/deepwiki-rs/blob/main/litho.toml) configuration file (most persistent), command-line flags like `--llm-provider` (highest precedence), or environment variables (fallback defaults).
- The `LLMProvider` enum in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) defines the supported identifiers, while [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) handles CLI overrides and [`src/llm/client/providers.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/llm/client/providers.rs) manages client instantiation.
- **Ollama requires no API key** and automatically defaults to `http://localhost:11434`, making it ideal for local development or air-gapped environments.

## Frequently Asked Questions

### What is the default LLM provider if I don't specify one?

If you do not specify a provider in the configuration file, command-line arguments, or environment variables, Litho defaults to **OpenAI**. This behavior is defined in [`src/config.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/config.rs) where `LLMProvider::default()` returns the `OpenAI` variant (lines 30-34).

### Can I use Litho with local LLMs without an internet connection?

Yes. Litho supports **Ollama** as a provider, which runs models locally on your machine. When using `--llm-provider ollama`, you do not need to provide an API key, and the client automatically connects to `http://localhost:11434` as implemented in [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) (lines 162-168). This makes Ollama ideal for air-gapped environments or reducing API costs.

### How do I override the provider specified in my litho.toml file?

Command-line flags take precedence over configuration file settings. Pass the `--llm-provider` flag followed by the desired provider name when running the binary. For example: `deepwiki-rs --llm-provider gemini`. This override logic is handled in the `Args::to_config` method in [`src/cli.rs`](https://github.com/sopaco/deepwiki-rs/blob/main/src/cli.rs) (lines 154-161), where CLI arguments are merged with file-based configuration.

### Which model names should I use for the model_efficient and model_powerful settings?

The `model_efficient` and `model_powerful` fields accept free-form strings that correspond to model identifiers on your chosen provider's platform. For example, when using Anthropic, you might set `model_efficient = "claude-3-haiku-20240307"` and `model_powerful = "claude-3-opus-20240229"`. For OpenRouter, you would use names like `"meta-llama/Meta-Llama-3.1-8B-Instruct"`. These values are passed directly to the provider client without validation, so consult your provider's documentation for exact model names.