What LLM Models Can Be Used with Litho: Complete Provider Guide

Litho supports any LLM model from OpenAI, Anthropic, Gemini, DeepSeek, Mistral, Moonshot, OpenRouter, and local Ollama instances through a provider-agnostic configuration system that accepts arbitrary model identifier strings.

The Litho engine—part of the deepwiki-rs repository—decouples model selection from core logic, allowing you to integrate virtually any LLM available through supported providers. This architecture enables seamless switching between cloud APIs and local inference servers without modifying the underlying Rust source code.

Understanding Litho's Provider-Agnostic LLM Architecture

Litho implements a flexible abstraction layer that treats model identifiers as configuration strings rather than hard-coded constants. This design allows the system to adapt immediately to new model releases from any supported provider.

The LLMProvider Enum and Configuration Structure

The provider selection logic resides in src/config.rs, where the LLMProvider enum defines all supported backends. The LLMConfig struct (lines 36–52) captures two critical model identifiers:

• model_efficient – The low-cost, high-throughput model for routine inference tasks.
• model_powerful – The high-capability model for complex reasoning or fallback scenarios.

Litho automatically falls back from model_efficient to model_powerful if the former returns an error or generates a low-quality response. This dual-model strategy is implemented in the provider adapters located in src/llm/client/providers.rs.

Supported LLM Providers and Compatible Models

Litho accepts any model identifier string recognized by the following providers. The specific provider implementations are defined in src/config.rs between lines 10–28.

OpenAI Models

Configuration location: src/config.rs#L10-L14

Litho supports any model available through the OpenAI API, including:

• gpt-4o-mini (efficient tier)
• gpt-4o (powerful tier)
• gpt-3.5-turbo
• o1-preview and o1-mini (reasoning models)

Anthropic Claude Models

Configuration location: src/config.rs#L21-L23

Compatible with the Claude 3 and 3.5 model families:

• claude-3-5-haiku-20241022 (efficient)
• claude-3-5-sonnet-20241022 (balanced)
• claude-3-opus-20240229 (powerful)

Google Gemini Models

Configuration location: src/config.rs#L24-L26

Supports Gemini 1.5 and future releases:

• gemini-1.5-pro
• gemini-1.5-flash

DeepSeek Models

Configuration location: src/config.rs#L15-L17

Compatible with DeepSeek's chat and reasoning models:

• deepseek-chat
• deepseek-reasoner

Local Ollama Instances

Configuration location: src/config.rs#L26-L28

Litho can connect to local Ollama servers for fully offline operation:

• llama3.2
• llama3.1:8b
• llama3.1:70b
• qwen2.5-coder
• Any custom model pulled into your Ollama instance

Additional Providers

• Moonshot (src/config.rs#L13-L15): Moonshot-specific model IDs
• Mistral (src/config.rs#L17-L19): Mistral AI model family
• OpenRouter (src/config.rs#L19-L21): Access to hundreds of models through a unified API, including meta-llama/llama-3.1-8b-instruct

Configuring Dual-Model Strategy in Litho

Litho's configuration system requires defining both an efficient and powerful model for each provider. This enables automatic fallback behavior when the primary model fails or returns insufficient results.

Configuration Schema

The LLMConfig struct in src/config.rs validates the following fields:

• provider: The backend enum value
• model_efficient: String identifier for routine tasks
• model_powerful: String identifier for complex reasoning
• api_key: Authentication token (optional for Ollama)
• api_base_url: Endpoint override for custom deployments

Fallback Mechanism

When model_efficient encounters an error or returns a low-confidence response, Litho automatically promotes the request to model_powerful. This logic is implemented in the provider client layer at src/llm/client/providers.rs, ensuring robust operation even when primary models experience degradation.

Practical Configuration Examples

OpenAI Configuration

[llm]
provider = "openai"
api_key = "${LITHO_LLM_API_KEY}"
api_base_url = "https://api.openai.com/v1"
model_efficient = "gpt-4o-mini"
model_powerful = "gpt-4o"
max_tokens = 4096
temperature = 0.1

The model_efficient and model_powerful fields are read by src/config.rs (see the LLMConfig struct) and injected into the OpenAI client implementation.

Local Ollama Setup

[llm]
provider = "ollama"
api_base_url = "http://localhost:11434/v1"
model_efficient = "llama3.1:8b"
model_powerful = "llama3.1:70b"

Ollama runs locally, so no API key is required. The same LLMClient logic picks the Ollama provider (src/llm/client/providers.rs).

CLI Overrides

litho --config litho.toml \
      --model-efficient deepseek-chat \
      --model-powerful deepseek-reasoner

The CLI (src/cli.rs) maps the --model-efficient and --model-powerful flags directly to the LLMConfig fields.

Custom OpenRouter Routes

[llm]
provider = "openrouter"
api_key = "${OPENROUTER_API_KEY}"
api_base_url = "https://openrouter.ai/api/v1"
model_efficient = "meta-llama/llama-3.1-8b-instruct"
model_powerful = "meta-llama/llama-3.1-70b-instruct"

No Rust code changes are required; the new model identifiers are just strings passed to the OpenRouter client.

Key Implementation Files

File	Description	Link
src/config.rs	LLMProvider enum and LLMConfig struct (model names, provider selection, API settings)	src/config.rs
src/cli.rs	Command-line parsing for --model-efficient, --model-powerful, and --llm-provider flags	src/cli.rs
src/llm/client/providers.rs	Provider-specific adapters that forward configured model names to remote APIs or local Ollama servers	src/llm/client/providers.rs
docs/en/5.Boundary-Interfaces.md	Detailed configuration examples for all providers and model selections	Boundary-Interfaces (EN)
litho-example.toml	Reference configuration file demonstrating dual-model setups for various providers	litho-example.toml

These files demonstrate how Litho decouples model identifiers from the core engine, enabling you to use any LLM model that the chosen provider exposes—whether it is a cloud offering (OpenAI, Anthropic, Gemini, etc.) or a local inference server (Ollama).

Summary

• Litho supports any model identifier accepted by OpenAI, Anthropic, Gemini, DeepSeek, Mistral, Moonshot, OpenRouter, or Ollama.
• Dual-model configuration requires defining model_efficient for routine tasks and model_powerful for complex reasoning or fallback scenarios.
• Provider-agnostic architecture means new models work immediately without code changes—just update your litho.toml or CLI flags.
• Local deployment is fully supported via Ollama integration, enabling offline operation with models like Llama 3.1 or Qwen.

Frequently Asked Questions

Can I use GPT-4 with Litho?

Yes. Litho accepts any OpenAI model identifier, including gpt-4o, gpt-4o-mini, gpt-4-turbo, and gpt-3.5-turbo. Configure them in litho.toml under the [llm] section using the model_efficient and model_powerful fields, or override via CLI flags --model-efficient and --model-powerful.

How do I switch between different LLM providers?

Switching providers requires changing the provider field in your litho.toml configuration file to the desired enum value (e.g., "openai", "anthropic", "ollama"). The LLMProvider enum in src/config.rs defines all valid options. No code recompilation is necessary; the provider-specific adapter in src/llm/client/providers.rs handles the transition automatically.

What is the difference between model_efficient and model_powerful?

The model_efficient field specifies a low-cost, high-speed model for routine inference tasks, while model_powerful designates a higher-capability model for complex reasoning or as a fallback when the efficient model fails. This dual-model strategy is defined in the LLMConfig struct in src/config.rs and enables automatic failover without manual intervention.

Can I use local models without an internet connection?

Yes. Litho supports local inference through Ollama integration. Configure provider = "ollama" and point api_base_url to your local server (typically http://localhost:11434/v1). You can use models like llama3.1:8b, llama3.1:70b, or qwen2.5-coder without requiring an API key or external network access.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}