# Configuration Options for LLM Providers in HugeGraph AI: A Complete Guide

> Explore HugeGraph AI's LLMConfig for seamless OpenAI, Ollama, and LiteLLM provider integration. Discover configuration options for chat, extraction, and text-to-Gremlin tasks.

- Repository: [The Apache Software Foundation/incubator-hugegraph-ai](https://github.com/apache/incubator-hugegraph-ai)
- Tags: how-to-guide
- Published: 2026-02-24

---

**HugeGraph AI abstracts LLM interactions through the `LLMConfig` data class, enabling seamless switching between OpenAI, Ollama, and LiteLLM providers via type selectors while maintaining separate configuration credentials and endpoints for chat, extraction, and text-to-Gremlin tasks.**

Apache HugeGraph AI provides a unified interface for integrating large language models with graph databases. The configuration system centers on the `LLMConfig` class defined in [`hugegraph-llm/src/hugegraph_llm/config/llm_config.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/hugegraph-llm/src/hugegraph_llm/config/llm_config.py), which supports distinct LLM instances for different operational modes including conversational AI, information extraction, and natural language-to-graph query translation.

## Core Provider Selection Fields

The `LLMConfig` data class declares provider type selectors at lines 25-36 that determine which backend client to instantiate for each operational mode. These fields accept specific string values that map to concrete implementations:

- **`chat_llm_type`**: Specifies the LLM for conversational interactions. Accepts `"openai"`, `"litellm"`, or `"ollama/local"`.
- **`extract_llm_type`**: Defines the LLM for information extraction tasks such as entity recognition and summarization. Accepts `"openai"`, `"litellm"`, or `"ollama/local"`.
- **`text2gql_llm_type`**: Configures the LLM for converting natural language to Gremlin or GraphQL queries. Accepts `"openai"`, `"litellm"`, or `"ollama/local"`.
- **`embedding_type`**: Sets the provider for vector embeddings used in RAG (Retrieval-Augmented Generation) pipelines. Accepts `"openai"`, `"litellm"`, or `"ollama/local"`.
- **`reranker_type`**: Selects the document reranking provider. Accepts `"cohere"` or `"siliconflow"`.
- **`keyword_extract_type`**: Determines the keyword extraction strategy. Accepts `"llm"`, `"textrank"`, or `"hybrid"`.

## Provider-Specific Configuration

Each LLM provider requires distinct connection parameters, credentials, and model identifiers. The `LLMConfig` class organizes these into dedicated field groups.

### OpenAI Configuration

OpenAI integration supports separate configurations for each LLM type (chat, extract, text2gql), allowing different models or endpoints for specific tasks. These fields are defined at lines 38-54:

- **Base URLs**: `openai_chat_api_base`, `openai_extract_api_base`, and `openai_text2gql_api_base` default to `https://api.openai.com/v1`.
- **API Keys**: `openai_chat_api_key`, `openai_extract_api_key`, and `openai_text2gql_api_key` fallback to the `OPENAI_API_KEY` environment variable if not explicitly set.
- **Model Names**: `openai_chat_language_model`, `openai_extract_language_model`, and `openai_text2gql_language_model` specify the model identifiers (e.g., `gpt-4`, `gpt-3.5-turbo`).
- **Token Limits**: `openai_chat_tokens`, `openai_extract_tokens`, and `openai_text2gql_tokens` control maximum context windows for each operation type.

### Ollama (Local) Configuration

For self-hosted Ollama instances, HugeGraph AI connects via host and port configurations defined at lines 58-70:

- **Network Settings**: `ollama_chat_host`, `ollama_chat_port`, `ollama_extract_host`, `ollama_extract_port`, `ollama_text2gql_host`, and `ollama_text2gql_port` default to `127.0.0.1:11434`.
- **Model Selection**: `ollama_chat_language_model`, `ollama_extract_language_model`, and `ollama_text2gql_language_model` specify the local model tags (e.g., `llama2:7b`, `mistral:latest`).

### LiteLLM Multi-Provider Configuration

LiteLLM acts as a unified proxy for multiple cloud providers, configured at lines 71-86:

- **Authentication**: `litellm_chat_api_key`, `litellm_extract_api_key`, and `litellm_text2gql_api_key` store provider-specific API credentials.
- **Endpoints**: `litellm_chat_api_base`, `litellm_extract_api_base`, and `litellm_text2gql_api_base` support custom base URLs for private deployments or proxy configurations.
- **Model Format**: `litellm_chat_language_model`, `litellm_extract_language_model`, and `litellm_text2gql_language_model` use the `provider/model` syntax (e.g., `openai/gpt-4.1-mini`, `anthropic/claude-3-opus`).
- **Context Limits**: `litellm_chat_tokens`, `litellm_extract_tokens`, and `litellm_text2gql_tokens` define token budgets per operation.

### Reranker Providers

Document reranking configuration appears at lines 54-58:

- **Cohere**: Uses `cohere_base_url` (default `https://api.cohere.com/v1/rerank`) along with `reranker_api_key` and `reranker_model`.
- **SiliconFlow**: Shares the same key fields (`reranker_api_key`, `reranker_model`) but routes requests to SiliconFlow's endpoints based on `reranker_type`.

## Runtime Configuration Consumption

Configuration instantiation follows a singleton pattern. The global `llm_settings = LLMConfig()` object is created at import time in [`hugegraph_llm/config/__init__.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/hugegraph_llm/config/__init__.py), making settings immediately available throughout the application.

Factory functions in [`hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py) consume these settings to construct appropriate clients:

- **`get_chat_llm(llm_settings)`**: Returns `OpenAIClient`, `OllamaClient`, or `LiteLLMClient` based on `chat_llm_type`.
- **`get_extract_llm(llm_settings)`**: Instantiates extraction-specific LLM instances using `extract_llm_type`.
- **`get_text2gql_llm(llm_settings)`**: Creates query generation LLMs according to `text2gql_llm_type`.

Each factory inspects the corresponding `*_llm_type` field and passes the matching provider-specific parameters to the client constructor.

## Configuration Examples

### Loading Configuration and Creating Clients

```python
from hugegraph_llm.config import llm_settings
from hugegraph_llm.models.llms.init_llm import get_chat_llm

# Inspect current provider settings

print(f"Active chat provider: {llm_settings.chat_llm_type}")
print(f"Chat model: {llm_settings.openai_chat_language_model}")

# Instantiate the configured client

chat_client = get_chat_llm(llm_settings)
response = chat_client.generate(prompt="Explain graph databases")
print(response)

```

### Environment Variable Overrides

HugeGraph AI supports runtime configuration via environment variables, enabling deployment-specific settings without code changes:

```bash
export CHAT_LLM_TYPE=ollama/local
export OLLAMA_CHAT_HOST=192.168.1.100
export OLLAMA_CHAT_PORT=11434
export OLLAMA_CHAT_LANGUAGE_MODEL=mistral:7b
export EXTRACT_LLM_TYPE=openai
export OPENAI_API_KEY=sk-...

```

When the application imports `llm_settings` after these exports, the factories automatically instantiate the correct clients:

```python
from hugegraph_llm.config import llm_settings
from hugegraph_llm.models.llms.init_llm import get_chat_llm, get_extract_llm

# Uses Ollama (chat) and OpenAI (extraction) simultaneously

chat = get_chat_llm(llm_settings)
extractor = get_extract_llm(llm_settings)

```

## Summary

- **Centralized Configuration**: All LLM provider settings reside in the `LLMConfig` class at [`hugegraph-llm/src/hugegraph_llm/config/llm_config.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/hugegraph-llm/src/hugegraph_llm/config/llm_config.py).
- **Per-Task Specialization**: HugeGraph AI supports different LLM providers for chat, extraction, and text-to-Gremlin operations through independent type selectors.
- **Three Native Providers**: OpenAI, Ollama (local), and LiteLLM are fully supported with distinct parameter sets for each.
- **Singleton Pattern**: The `llm_settings` object in [`hugegraph_llm/config/__init__.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/hugegraph_llm/config/__init__.py) provides global access to configuration values.
- **Factory Architecture**: [`init_llm.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/init_llm.py) maps configuration types to concrete client implementations without exposing provider details to business logic.

## Frequently Asked Questions

### How do I switch between OpenAI and Ollama in HugeGraph AI?

Set the `CHAT_LLM_TYPE` environment variable to `"openai"` or `"ollama/local"` before importing the configuration, or modify the `chat_llm_type` field in your `.env` file. The `get_chat_llm()` factory in [`hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py) automatically instantiates the corresponding client using the provider-specific credentials defined in `LLMConfig`.

### What is the difference between `chat_llm_type` and `text2gql_llm_type`?

`chat_llm_type` configures the LLM used for conversational responses and general question answering, while `text2gql_llm_type` specifically targets the model responsible for translating natural language into Gremlin or GraphQL queries. This separation allows optimization—using a fast, cheap model for chat and a more capable model for complex query generation.

### Can I use different LLM providers for different tasks simultaneously?

Yes. HugeGraph AI supports heterogeneous configurations where, for example, `chat_llm_type="ollama/local"` routes conversational tasks to a local Ollama instance while `extract_llm_type="openai"` sends extraction tasks to OpenAI's API. Each factory function (`get_chat_llm`, `get_extract_llm`, `get_text2gql_llm`) independently reads its corresponding type field and instantiates the appropriate client.

### How are API keys securely managed in HugeGraph AI?

API keys follow a hierarchy: explicit configuration fields (e.g., `openai_chat_api_key`) take precedence, followed by generic environment variables (e.g., `OPENAI_API_KEY`). For production deployments, the `generate_env()` utility in [`hugegraph-llm/src/hugegraph_llm/config/generate.py`](https://github.com/apache/incubator-hugegraph-ai/blob/main/hugegraph-llm/src/hugegraph_llm/config/generate.py) creates `.env` templates, though secrets should ultimately be injected via orchestration secrets or environment variables rather than committed configuration files.