# Integrating LLMs with GitButler AI Tooling: A Complete Guide to the but-llm Crate

> Explore the but-llm crate for seamless LLM integration with GitButler AI. Connect OpenAI, Anthropic, Ollama, and LMStudio using a provider-agnostic interface configured via Git settings.

- Repository: [GitButler/gitbutler](https://github.com/gitbutlerapp/gitbutler)
- Tags: tutorial
- Published: 2026-02-16

---

**GitButler provides a unified LLM abstraction in the `but-llm` crate that supports OpenAI, Anthropic, Ollama, and LMStudio through a provider-agnostic interface configured via Git settings.**

Integrating LLMs with GitButler AI tooling enables developers to automate commit generation, branch management, and interactive chat workflows through a provider-agnostic abstraction. The `gitbutlerapp/gitbutler` repository implements this integration in the `but-llm` crate, which exposes a unified `LLMProvider` interface that routes requests to OpenAI, Anthropic, Ollama, or LMStudio based on Git configuration.

## Architecture of the GitButler LLM Integration

### The LLMProvider Abstraction

The `LLMProvider` struct in [`crates/but-llm/src/lib.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/but-llm/src/lib.rs) serves as the central dispatch point for all LLM operations. It wraps concrete clients in an `LLMClientType` enum and exposes high-level methods like `tool_calling_loop` and `stream_response` that work identically regardless of the underlying provider.

```rust
// crates/but-llm/src/lib.rs
pub struct LLMProvider {
    client: LLMClientType,
}

enum LLMClientType {
    OpenAi(Arc<openai::OpenAiProvider>),
    Anthropic(Arc<anthropic::AnthropicProvider>),
    Ollama(Arc<ollama::OllamaProvider>),
    LmStudio(Arc<lmstudio::LmStudioProvider>),
}

```

### Provider Selection via Git Configuration

GitButler reads provider settings from Git configuration using the key `gitbutler.aiModelProvider`. The `from_git_config` function parses this value, matches it against supported providers, and delegates to provider-specific initialization logic that reads API keys and model names from the same config file or environment variables.

```rust
// crates/but-llm/src/lib.rs
pub fn from_git_config(config: &gix::config::File<'static>) -> Option<Self> {
    let provider_str = config.string(MODEL_PROVIDER).map(|v| v.to_string())?;
    let provider = LLMProviderKind::from_str(&provider_str);
    match provider {
        Some(LLMProviderKind::OpenAi) => {
            let client = openai::OpenAiProvider::from_git_config(config)?;
            Some(Self { client: LLMClientType::OpenAi(Arc::new(client)) })
        }
        // Similar blocks for Anthropic, Ollama, LMStudio
        _ => None,
    }
}

```

## Configuring LLM Providers in GitButler

### OpenAI, Anthropic, Ollama, and LMStudio Setup

Each provider requires specific configuration keys in your Git configuration file. Set the provider type globally or per-repository, then provide the corresponding API credentials.

**OpenAI Configuration:**

```bash
git config --global gitbutler.aiModelProvider openai
git config --global gitbutler.openAiApiKey "sk-..."
git config --global gitbutler.openAiModel "gpt-4o-mini"

```

**Anthropic Configuration:**

```bash
git config --global gitbutler.aiModelProvider anthropic
git config --global gitbutler.anthropicApiKey "sk-ant-..."
git config --global gitbutler.anthropicModel "claude-3-5-sonnet-20241022"

```

**Ollama (Local) Configuration:**

```bash
git config --global gitbutler.aiModelProvider ollama
git config --global gitbutler.ollamaUrl "http://localhost:11434"
git config --global gitbutler.ollamaModel "llama3.1"

```

**LMStudio (Local) Configuration:**

```bash
git config --global gitbutler.aiModelProvider lmstudio
git config --global gitbutler.lmStudioUrl "http://localhost:1234/v1"
git config --global gitbutler.lmStudioModel "local-model"

```

## Implementing Tool-Calling Workflows

### The tool_calling_loop Mechanism

The `tool_calling_loop` method in [`crates/but-llm/src/lib.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/but-llm/src/lib.rs) orchestrates conversations where the LLM can invoke GitButler's internal tools. The method sends the system prompt and user messages to the provider, parses any function-call responses, executes the requested tools, and feeds the results back to the LLM until a final answer is produced.

```rust
// crates/but-llm/src/lib.rs
pub fn tool_calling_loop(
    &self,
    system_message: &str,
    chat_messages: Vec<ChatMessage>,
    tool_set: &mut dyn ToolSet,
    model: &str,
) -> anyhow::Result<()> {
    match &self.client {
        LLMClientType::OpenAi(client) => {
            client.tool_calling_loop(system_message, chat_messages, tool_set, model)
        }
        LLMClientType::Anthropic(client) => {
            client.tool_calling_loop(system_message, chat_messages, tool_set, model)
        }
        // Ollama and LMStudio follow the same pattern
    }
}

```

### Registering Custom Tools

Tools implement the `Tool` trait defined in [`crates/but-tools/src/workspace/mod.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/but-tools/src/workspace/mod.rs). Each tool specifies its name, description, JSON schema for parameters, and an `execute` method. Register custom tools by implementing this trait and adding the tool to a `ToolSet` before invoking the LLM.

```rust
// Example custom tool implementation
use but_tools::workspace::Tool;
use serde_json::json;

pub struct TestRunner;

impl Tool for TestRunner {
    fn name(&self) -> &'static str { "run_tests" }

    fn description(&self) -> &'static str {
        "Execute the project's test suite and return a summary."
    }

    fn parameters(&self) -> serde_json::Value {
        json!({}) // No parameters required
    }

    fn execute(&self, _args: serde_json::Value) -> anyhow::Result<serde_json::Value> {
        let output = std::process::Command::new("cargo")
            .arg("test")
            .output()?;
        let summary = String::from_utf8_lossy(&output.stdout);
        Ok(json!({ "summary": summary }))
    }
}

// Usage in tool_calling_loop
let mut toolset = but_tools::workspace::commit_toolset(ctx);
toolset.register(Box::new(TestRunner));

```

## Real-World Integration Examples

### Tauri Chatbot Implementation

The desktop UI chatbot in [`crates/gitbutler-tauri/src/bot.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/gitbutler-tauri/src/bot.rs) demonstrates production usage of the LLM abstraction. It reads the global Git configuration, initializes the provider, and passes conversation history to the bot engine.

```rust
// crates/gitbutler-tauri/src/bot.rs
let git_config = gix::config::File::from_globals()?;
let llm = but_llm::LLMProvider::from_git_config(&git_config);
match llm {
    Some(llm) => but_bot::bot(project_id, message_id, emitter, &mut ctx, &llm, chat_messages),
    None => Err(Error::from(anyhow::anyhow!(
        "No valid credentials found for AI provider. Please configure your GitButler account credentials."
    ))),
}

```

### AI-Powered Branch Changes

The branch changes workflow in [`crates/but-action/src/branch_changes.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/but-action/src/branch_changes.rs) uses the tool-calling loop to automatically group file changes into logical commits. It serializes the project status and passes it to the LLM with the commit toolset.

```rust
// crates/but-action/src/branch_changes.rs
let mut toolset = commit_toolset(ctx);
let system_message = "You are an expert in grouping and committing file changes...";
let prompt = format!("... <project_status>{serialized_status}</project_status>");
llm.tool_calling_loop(system_message, vec![prompt.into()], &mut toolset, &model)?;

```

### Commit Message Rewording

The rewording feature in [`crates/but-action/src/reword.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/but-action/src/reword.rs) generates commit messages by invoking `generate::commit_message`, which internally uses the LLM provider to suggest improved wording based on diff content.

## Extending the LLM Layer

### Adding a New Provider

To support a new LLM backend, create a new file in `crates/but-llm/src/` (e.g., [`custom.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/custom.rs)) implementing the provider-specific `tool_calling_loop` and `from_git_config` methods. Update the `LLMClientType` enum in [`lib.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/lib.rs) to include the new variant, and add the corresponding match arm in `from_git_config` to parse the provider string from Git configuration.

### Overriding Models at Runtime

While Git configuration sets the default model, you can override it per-request by passing a different model string to the high-level methods:

```rust
let response = llm.stream_response(
    "You are a code reviewer.",
    vec![ChatMessage::User("Review this diff".into())],
    "claude-3.5-sonnet",  // Runtime override
)?;

```

This allows workflows to use lightweight models for simple tasks and powerful models for complex reasoning without changing global configuration.

## Summary

- **GitButler's LLM integration** centers on the `but-llm` crate, which abstracts OpenAI, Anthropic, Ollama, and LMStudio behind a unified `LLMProvider` interface.
- **Configuration** happens through standard Git config keys like `gitbutler.aiModelProvider` and provider-specific API keys, enabling seamless switching between local and cloud models.
- **Tool-calling workflows** use `tool_calling_loop` to let LLMs invoke GitButler's internal tools (commits, branch operations) through a structured `Tool` trait interface.
- **Real-world implementations** include the Tauri desktop chatbot ([`gitbutler-tauri/src/bot.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/gitbutler-tauri/src/bot.rs)), automated branch changes ([`but-action/src/branch_changes.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/but-action/src/branch_changes.rs)), and commit rewording ([`but-action/src/reword.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/but-action/src/reword.rs)).
- **Extensibility** allows developers to add new providers by implementing the client trait and registering new tools via the `ToolSet` interface.

## Frequently Asked Questions

### How do I configure GitButler to use a local LLM like Ollama instead of OpenAI?

Set the provider to `ollama` in your Git configuration and specify the local endpoint. Run `git config --global gitbutler.aiModelProvider ollama`, then set `gitbutler.ollamaUrl` to your local server (typically `http://localhost:11434`) and `gitbutler.ollamaModel` to your chosen model name. No API key is required for local deployments.

### What is the difference between `tool_calling_loop` and `stream_response` in the but-llm crate?

The `tool_calling_loop` method enables **function calling** workflows where the LLM can invoke GitButler tools (like creating commits) and receive results before generating a final response, looping until completion. The `stream_response` method provides a simpler **one-shot** conversation without tool execution, suitable for chat interfaces or text generation tasks that don't require repository mutations.

### Can I add custom tools that the LLM can invoke during a GitButler workflow?

Yes, by implementing the `Tool` trait from [`crates/but-tools/src/workspace/mod.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/but-tools/src/workspace/mod.rs) and registering your implementation with a `ToolSet` before calling `tool_calling_loop`. Your tool must define a name, description, JSON parameter schema, and an `execute` method that returns results the LLM can consume. This allows the LLM to trigger any custom automation, from running test suites to deploying code.

### How does GitButler handle missing or invalid LLM credentials?

When `LLMProvider::from_git_config` cannot find a valid configuration or encounters malformed credentials, it returns `None`. Callers like the Tauri chatbot in [`crates/gitbutler-tauri/src/bot.rs`](https://github.com/gitbutlerapp/gitbutler/blob/main/crates/gitbutler-tauri/src/bot.rs) explicitly handle this case by returning a user-friendly error message prompting the user to configure their GitButler account credentials, ensuring clear feedback rather than silent failures.