How to Configure Different LLM Providers for Litho: A Complete Guide

Litho supports multiple LLM backends through a plugin architecture that allows you to configure providers via TOML configuration files, command-line flags, or environment variables, with automatic client instantiation for OpenAI, Anthropic, Gemini, Ollama, and others.

Litho is a documentation generation tool from the sopaco/deepwiki-rs repository that analyzes codebases and produces structured documentation using large language models. When deploying Litho, you can configure different LLM providers to match your infrastructure requirements, whether you need cloud-based APIs or local inference engines.

Understanding Litho's LLM Provider Architecture

Litho implements a flexible provider system centered around the LLMProvider enum defined in src/config.rs. This enum supports eight distinct backends, each mapping to a specific client implementation in the underlying rig framework. When Litho starts, the ProviderClient::new factory method in src/llm/client/providers.rs (lines 35-80) instantiates the appropriate client based on your configuration, enabling seamless switching between providers without code changes.

Configuration Methods for LLM Providers

You can specify which LLM provider Litho uses through three distinct mechanisms, listed here in order of precedence:

Using the Configuration File (litho.toml)

The most persistent method involves creating a litho.toml file in your project root. Under the [llm] table, specify the provider field with one of the supported identifiers:

[llm]
provider = "anthropic"
api_key = "${LITHO_LLM_API_KEY}"
api_base_url = "https://api.anthropic.com"
model_efficient = "claude-3-haiku-20240307"
model_powerful = "claude-3-opus-20240229"

According to the source code in src/config.rs (lines 138-144), the LLMConfig struct parses these values during startup, with the provider field deserializing into the LLMProvider enum.

Using Command-Line Flags

For temporary overrides or CI/CD pipelines, pass the --llm-provider flag when invoking the binary. This overrides any value set in the configuration file:

deepwiki-rs \
  --llm-provider gemini \
  --llm-api-key "$GEMINI_API_KEY" \
  --model-efficient "gemini-1.5-flash" \
  --model-powerful "gemini-1.5-pro"

The CLI argument definitions reside in src/cli.rs (lines 78-80), with the override logic implemented in Args::to_config (lines 154-161).

Using Environment Variables

When neither configuration files nor CLI flags specify a provider, Litho defaults to OpenAI. You can influence behavior through environment variables:

• LITHO_LLM_API_KEY: Sets the API key for the chosen provider
• Provider selection defaults to openai unless overridden by other methods

The default implementation in src/config.rs (lines 30-34) shows that LLMProvider::default() returns OpenAI, while LLMConfig::default (line 43) handles API key retrieval from the environment.

Supported LLM Providers

Litho supports eight LLM backends through the LLMProvider enum defined in src/config.rs (lines 9-28):

pub enum LLMProvider {
    #[serde(rename = "openai")]
    OpenAI,
    #[serde(rename = "moonshot")]
    Moonshot,
    #[serde(rename = "deepseek")]
    DeepSeek,
    #[serde(rename = "mistral")]
    Mistral,
    #[serde(rename = "openrouter")]
    OpenRouter,
    #[serde(rename = "anthropic")]
    Anthropic,
    #[serde(rename = "gemini")]
    Gemini,
    #[serde(rename = "ollama")]
    Ollama,
}

Each variant maps to a corresponding client in the rig provider ecosystem. Note that ollama requires no API key and defaults to http://localhost:11434 as implemented in src/cli.rs (lines 162-168).

Practical Configuration Examples

Configuring Anthropic Claude

To use Anthropic's Claude models, specify anthropic as the provider and provide your API key:

[llm]
provider = "anthropic"
api_key = "sk-ant-api03-..."
model_efficient = "claude-3-haiku-20240307"
model_powerful = "claude-3-opus-20240229"

Configuring Google Gemini via CLI

For one-time documentation generation using Google's Gemini:

deepwiki-rs -p ./my-project -o ./docs \
  --llm-provider gemini \
  --llm-api-key "$GEMINI_API_KEY" \
  --model-efficient "gemini-1.5-flash" \
  --model-powerful "gemini-1.5-pro"

Setting Up Local Ollama Models

To run Litho against local models via Ollama without cloud API costs:

deepwiki-rs -p . \
  --llm-provider ollama \
  --model-efficient "llama3.2" \
  --model-powerful "llama3.2:70b"

No API key is required. The client automatically connects to http://localhost:11434 as specified in src/cli.rs.

Programmatic Configuration in Rust

When embedding Litho as a library, construct the configuration programmatically:

use deepwiki_rs::config::{Config, LLMProvider, LLMConfig};

let mut cfg = Config::default();
cfg.llm = LLMConfig {
    provider: LLMProvider::OpenRouter,
    api_key: "my-openrouter-key".into(),
    api_base_url: "https://openrouter.ai/api/v1".into(),
    model_efficient: "meta-llama/Meta-Llama-3.1-8B-Instruct".into(),
    model_powerful: "meta-llama/Meta-Llama-3.1-70B-Instruct".into(),
    ..Default::default()
};

Key Configuration Files and Implementation Details

Understanding the source structure helps when troubleshooting provider issues:

File	Purpose	Key Components
src/config.rs	Core configuration structures	LLMProvider enum (lines 9-28), LLMConfig struct (lines 138-144), default implementations
src/cli.rs	Command-line interface	--llm-provider flag (lines 78-80), Args::to_config override logic (lines 154-161), Ollama URL fallback (lines 162-168)
src/llm/client/providers.rs	Provider client factory	ProviderClient::new (lines 35-80) that instantiates rig clients
litho-example.toml	Reference configuration	Complete example of [llm] table settings

Summary

• Litho supports eight LLM providers through a unified configuration system: OpenAI, Anthropic, Gemini, Mistral, DeepSeek, Moonshot, OpenRouter, and Ollama.
• You can configure providers via three methods: the litho.toml configuration file (most persistent), command-line flags like --llm-provider (highest precedence), or environment variables (fallback defaults).
• The LLMProvider enum in src/config.rs defines the supported identifiers, while src/cli.rs handles CLI overrides and src/llm/client/providers.rs manages client instantiation.
• Ollama requires no API key and automatically defaults to http://localhost:11434, making it ideal for local development or air-gapped environments.

Frequently Asked Questions

What is the default LLM provider if I don't specify one?

If you do not specify a provider in the configuration file, command-line arguments, or environment variables, Litho defaults to OpenAI. This behavior is defined in src/config.rs where LLMProvider::default() returns the OpenAI variant (lines 30-34).

Can I use Litho with local LLMs without an internet connection?

Yes. Litho supports Ollama as a provider, which runs models locally on your machine. When using --llm-provider ollama, you do not need to provide an API key, and the client automatically connects to http://localhost:11434 as implemented in src/cli.rs (lines 162-168). This makes Ollama ideal for air-gapped environments or reducing API costs.

How do I override the provider specified in my litho.toml file?

Command-line flags take precedence over configuration file settings. Pass the --llm-provider flag followed by the desired provider name when running the binary. For example: deepwiki-rs --llm-provider gemini. This override logic is handled in the Args::to_config method in src/cli.rs (lines 154-161), where CLI arguments are merged with file-based configuration.

Which model names should I use for the model_efficient and model_powerful settings?

The model_efficient and model_powerful fields accept free-form strings that correspond to model identifiers on your chosen provider's platform. For example, when using Anthropic, you might set model_efficient = "claude-3-haiku-20240307" and model_powerful = "claude-3-opus-20240229". For OpenRouter, you would use names like "meta-llama/Meta-Llama-3.1-8B-Instruct". These values are passed directly to the provider client without validation, so consult your provider's documentation for exact model names.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}