Integrating LLMs with GitButler AI Tooling: A Complete Guide to the but-llm Crate

GitButler provides a unified LLM abstraction in the but-llm crate that supports OpenAI, Anthropic, Ollama, and LMStudio through a provider-agnostic interface configured via Git settings.

Integrating LLMs with GitButler AI tooling enables developers to automate commit generation, branch management, and interactive chat workflows through a provider-agnostic abstraction. The gitbutlerapp/gitbutler repository implements this integration in the but-llm crate, which exposes a unified LLMProvider interface that routes requests to OpenAI, Anthropic, Ollama, or LMStudio based on Git configuration.

Architecture of the GitButler LLM Integration

The LLMProvider Abstraction

The LLMProvider struct in crates/but-llm/src/lib.rs serves as the central dispatch point for all LLM operations. It wraps concrete clients in an LLMClientType enum and exposes high-level methods like tool_calling_loop and stream_response that work identically regardless of the underlying provider.

// crates/but-llm/src/lib.rs
pub struct LLMProvider {
    client: LLMClientType,
}

enum LLMClientType {
    OpenAi(Arc<openai::OpenAiProvider>),
    Anthropic(Arc<anthropic::AnthropicProvider>),
    Ollama(Arc<ollama::OllamaProvider>),
    LmStudio(Arc<lmstudio::LmStudioProvider>),
}

Provider Selection via Git Configuration

GitButler reads provider settings from Git configuration using the key gitbutler.aiModelProvider. The from_git_config function parses this value, matches it against supported providers, and delegates to provider-specific initialization logic that reads API keys and model names from the same config file or environment variables.

// crates/but-llm/src/lib.rs
pub fn from_git_config(config: &gix::config::File<'static>) -> Option<Self> {
    let provider_str = config.string(MODEL_PROVIDER).map(|v| v.to_string())?;
    let provider = LLMProviderKind::from_str(&provider_str);
    match provider {
        Some(LLMProviderKind::OpenAi) => {
            let client = openai::OpenAiProvider::from_git_config(config)?;
            Some(Self { client: LLMClientType::OpenAi(Arc::new(client)) })
        }
        // Similar blocks for Anthropic, Ollama, LMStudio
        _ => None,
    }
}

Configuring LLM Providers in GitButler

OpenAI, Anthropic, Ollama, and LMStudio Setup

Each provider requires specific configuration keys in your Git configuration file. Set the provider type globally or per-repository, then provide the corresponding API credentials.

OpenAI Configuration:

git config --global gitbutler.aiModelProvider openai
git config --global gitbutler.openAiApiKey "sk-..."
git config --global gitbutler.openAiModel "gpt-4o-mini"

Anthropic Configuration:

git config --global gitbutler.aiModelProvider anthropic
git config --global gitbutler.anthropicApiKey "sk-ant-..."
git config --global gitbutler.anthropicModel "claude-3-5-sonnet-20241022"

Ollama (Local) Configuration:

git config --global gitbutler.aiModelProvider ollama
git config --global gitbutler.ollamaUrl "http://localhost:11434"
git config --global gitbutler.ollamaModel "llama3.1"

LMStudio (Local) Configuration:

git config --global gitbutler.aiModelProvider lmstudio
git config --global gitbutler.lmStudioUrl "http://localhost:1234/v1"
git config --global gitbutler.lmStudioModel "local-model"

Implementing Tool-Calling Workflows

The tool_calling_loop Mechanism

The tool_calling_loop method in crates/but-llm/src/lib.rs orchestrates conversations where the LLM can invoke GitButler's internal tools. The method sends the system prompt and user messages to the provider, parses any function-call responses, executes the requested tools, and feeds the results back to the LLM until a final answer is produced.

// crates/but-llm/src/lib.rs
pub fn tool_calling_loop(
    &self,
    system_message: &str,
    chat_messages: Vec<ChatMessage>,
    tool_set: &mut dyn ToolSet,
    model: &str,
) -> anyhow::Result<()> {
    match &self.client {
        LLMClientType::OpenAi(client) => {
            client.tool_calling_loop(system_message, chat_messages, tool_set, model)
        }
        LLMClientType::Anthropic(client) => {
            client.tool_calling_loop(system_message, chat_messages, tool_set, model)
        }
        // Ollama and LMStudio follow the same pattern
    }
}

Registering Custom Tools

Tools implement the Tool trait defined in crates/but-tools/src/workspace/mod.rs. Each tool specifies its name, description, JSON schema for parameters, and an execute method. Register custom tools by implementing this trait and adding the tool to a ToolSet before invoking the LLM.

// Example custom tool implementation
use but_tools::workspace::Tool;
use serde_json::json;

pub struct TestRunner;

impl Tool for TestRunner {
    fn name(&self) -> &'static str { "run_tests" }

    fn description(&self) -> &'static str {
        "Execute the project's test suite and return a summary."
    }

    fn parameters(&self) -> serde_json::Value {
        json!({}) // No parameters required
    }

    fn execute(&self, _args: serde_json::Value) -> anyhow::Result<serde_json::Value> {
        let output = std::process::Command::new("cargo")
            .arg("test")
            .output()?;
        let summary = String::from_utf8_lossy(&output.stdout);
        Ok(json!({ "summary": summary }))
    }
}

// Usage in tool_calling_loop
let mut toolset = but_tools::workspace::commit_toolset(ctx);
toolset.register(Box::new(TestRunner));

Real-World Integration Examples

Tauri Chatbot Implementation

The desktop UI chatbot in crates/gitbutler-tauri/src/bot.rs demonstrates production usage of the LLM abstraction. It reads the global Git configuration, initializes the provider, and passes conversation history to the bot engine.

// crates/gitbutler-tauri/src/bot.rs
let git_config = gix::config::File::from_globals()?;
let llm = but_llm::LLMProvider::from_git_config(&git_config);
match llm {
    Some(llm) => but_bot::bot(project_id, message_id, emitter, &mut ctx, &llm, chat_messages),
    None => Err(Error::from(anyhow::anyhow!(
        "No valid credentials found for AI provider. Please configure your GitButler account credentials."
    ))),
}

AI-Powered Branch Changes

The branch changes workflow in crates/but-action/src/branch_changes.rs uses the tool-calling loop to automatically group file changes into logical commits. It serializes the project status and passes it to the LLM with the commit toolset.

// crates/but-action/src/branch_changes.rs
let mut toolset = commit_toolset(ctx);
let system_message = "You are an expert in grouping and committing file changes...";
let prompt = format!("... <project_status>{serialized_status}</project_status>");
llm.tool_calling_loop(system_message, vec![prompt.into()], &mut toolset, &model)?;

Commit Message Rewording

The rewording feature in crates/but-action/src/reword.rs generates commit messages by invoking generate::commit_message, which internally uses the LLM provider to suggest improved wording based on diff content.

Extending the LLM Layer

Adding a New Provider

To support a new LLM backend, create a new file in crates/but-llm/src/ (e.g., custom.rs) implementing the provider-specific tool_calling_loop and from_git_config methods. Update the LLMClientType enum in lib.rs to include the new variant, and add the corresponding match arm in from_git_config to parse the provider string from Git configuration.

Overriding Models at Runtime

While Git configuration sets the default model, you can override it per-request by passing a different model string to the high-level methods:

let response = llm.stream_response(
    "You are a code reviewer.",
    vec![ChatMessage::User("Review this diff".into())],
    "claude-3.5-sonnet",  // Runtime override
)?;

This allows workflows to use lightweight models for simple tasks and powerful models for complex reasoning without changing global configuration.

Summary

• GitButler's LLM integration centers on the but-llm crate, which abstracts OpenAI, Anthropic, Ollama, and LMStudio behind a unified LLMProvider interface.
• Configuration happens through standard Git config keys like gitbutler.aiModelProvider and provider-specific API keys, enabling seamless switching between local and cloud models.
• Tool-calling workflows use tool_calling_loop to let LLMs invoke GitButler's internal tools (commits, branch operations) through a structured Tool trait interface.
• Real-world implementations include the Tauri desktop chatbot (gitbutler-tauri/src/bot.rs), automated branch changes (but-action/src/branch_changes.rs), and commit rewording (but-action/src/reword.rs).
• Extensibility allows developers to add new providers by implementing the client trait and registering new tools via the ToolSet interface.

Frequently Asked Questions

How do I configure GitButler to use a local LLM like Ollama instead of OpenAI?

Set the provider to ollama in your Git configuration and specify the local endpoint. Run git config --global gitbutler.aiModelProvider ollama, then set gitbutler.ollamaUrl to your local server (typically http://localhost:11434) and gitbutler.ollamaModel to your chosen model name. No API key is required for local deployments.

What is the difference between tool_calling_loop and stream_response in the but-llm crate?

The tool_calling_loop method enables function calling workflows where the LLM can invoke GitButler tools (like creating commits) and receive results before generating a final response, looping until completion. The stream_response method provides a simpler one-shot conversation without tool execution, suitable for chat interfaces or text generation tasks that don't require repository mutations.

Can I add custom tools that the LLM can invoke during a GitButler workflow?

Yes, by implementing the Tool trait from crates/but-tools/src/workspace/mod.rs and registering your implementation with a ToolSet before calling tool_calling_loop. Your tool must define a name, description, JSON parameter schema, and an execute method that returns results the LLM can consume. This allows the LLM to trigger any custom automation, from running test suites to deploying code.

How does GitButler handle missing or invalid LLM credentials?

When LLMProvider::from_git_config cannot find a valid configuration or encounters malformed credentials, it returns None. Callers like the Tauri chatbot in crates/gitbutler-tauri/src/bot.rs explicitly handle this case by returning a user-friendly error message prompting the user to configure their GitButler account credentials, ensuring clear feedback rather than silent failures.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}