Configuration Options for LLM Providers in HugeGraph AI: A Complete Guide

HugeGraph AI abstracts LLM interactions through the LLMConfig data class, enabling seamless switching between OpenAI, Ollama, and LiteLLM providers via type selectors while maintaining separate configuration credentials and endpoints for chat, extraction, and text-to-Gremlin tasks.

Apache HugeGraph AI provides a unified interface for integrating large language models with graph databases. The configuration system centers on the LLMConfig class defined in hugegraph-llm/src/hugegraph_llm/config/llm_config.py, which supports distinct LLM instances for different operational modes including conversational AI, information extraction, and natural language-to-graph query translation.

Core Provider Selection Fields

The LLMConfig data class declares provider type selectors at lines 25-36 that determine which backend client to instantiate for each operational mode. These fields accept specific string values that map to concrete implementations:

• chat_llm_type: Specifies the LLM for conversational interactions. Accepts "openai", "litellm", or "ollama/local".
• extract_llm_type: Defines the LLM for information extraction tasks such as entity recognition and summarization. Accepts "openai", "litellm", or "ollama/local".
• text2gql_llm_type: Configures the LLM for converting natural language to Gremlin or GraphQL queries. Accepts "openai", "litellm", or "ollama/local".
• embedding_type: Sets the provider for vector embeddings used in RAG (Retrieval-Augmented Generation) pipelines. Accepts "openai", "litellm", or "ollama/local".
• reranker_type: Selects the document reranking provider. Accepts "cohere" or "siliconflow".
• keyword_extract_type: Determines the keyword extraction strategy. Accepts "llm", "textrank", or "hybrid".

Provider-Specific Configuration

Each LLM provider requires distinct connection parameters, credentials, and model identifiers. The LLMConfig class organizes these into dedicated field groups.

OpenAI Configuration

OpenAI integration supports separate configurations for each LLM type (chat, extract, text2gql), allowing different models or endpoints for specific tasks. These fields are defined at lines 38-54:

• Base URLs: openai_chat_api_base, openai_extract_api_base, and openai_text2gql_api_base default to https://api.openai.com/v1.
• API Keys: openai_chat_api_key, openai_extract_api_key, and openai_text2gql_api_key fallback to the OPENAI_API_KEY environment variable if not explicitly set.
• Model Names: openai_chat_language_model, openai_extract_language_model, and openai_text2gql_language_model specify the model identifiers (e.g., gpt-4, gpt-3.5-turbo).
• Token Limits: openai_chat_tokens, openai_extract_tokens, and openai_text2gql_tokens control maximum context windows for each operation type.

Ollama (Local) Configuration

For self-hosted Ollama instances, HugeGraph AI connects via host and port configurations defined at lines 58-70:

• Network Settings: ollama_chat_host, ollama_chat_port, ollama_extract_host, ollama_extract_port, ollama_text2gql_host, and ollama_text2gql_port default to 127.0.0.1:11434.
• Model Selection: ollama_chat_language_model, ollama_extract_language_model, and ollama_text2gql_language_model specify the local model tags (e.g., llama2:7b, mistral:latest).

LiteLLM Multi-Provider Configuration

LiteLLM acts as a unified proxy for multiple cloud providers, configured at lines 71-86:

• Authentication: litellm_chat_api_key, litellm_extract_api_key, and litellm_text2gql_api_key store provider-specific API credentials.
• Endpoints: litellm_chat_api_base, litellm_extract_api_base, and litellm_text2gql_api_base support custom base URLs for private deployments or proxy configurations.
• Model Format: litellm_chat_language_model, litellm_extract_language_model, and litellm_text2gql_language_model use the provider/model syntax (e.g., openai/gpt-4.1-mini, anthropic/claude-3-opus).
• Context Limits: litellm_chat_tokens, litellm_extract_tokens, and litellm_text2gql_tokens define token budgets per operation.

Reranker Providers

Document reranking configuration appears at lines 54-58:

• Cohere: Uses cohere_base_url (default https://api.cohere.com/v1/rerank) along with reranker_api_key and reranker_model.
• SiliconFlow: Shares the same key fields (reranker_api_key, reranker_model) but routes requests to SiliconFlow's endpoints based on reranker_type.

Runtime Configuration Consumption

Configuration instantiation follows a singleton pattern. The global llm_settings = LLMConfig() object is created at import time in hugegraph_llm/config/__init__.py, making settings immediately available throughout the application.

Factory functions in hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py consume these settings to construct appropriate clients:

• get_chat_llm(llm_settings): Returns OpenAIClient, OllamaClient, or LiteLLMClient based on chat_llm_type.
• get_extract_llm(llm_settings): Instantiates extraction-specific LLM instances using extract_llm_type.
• get_text2gql_llm(llm_settings): Creates query generation LLMs according to text2gql_llm_type.

Each factory inspects the corresponding *_llm_type field and passes the matching provider-specific parameters to the client constructor.

Configuration Examples

Loading Configuration and Creating Clients

from hugegraph_llm.config import llm_settings
from hugegraph_llm.models.llms.init_llm import get_chat_llm

# Inspect current provider settings

print(f"Active chat provider: {llm_settings.chat_llm_type}")
print(f"Chat model: {llm_settings.openai_chat_language_model}")

# Instantiate the configured client

chat_client = get_chat_llm(llm_settings)
response = chat_client.generate(prompt="Explain graph databases")
print(response)

Environment Variable Overrides

HugeGraph AI supports runtime configuration via environment variables, enabling deployment-specific settings without code changes:

export CHAT_LLM_TYPE=ollama/local
export OLLAMA_CHAT_HOST=192.168.1.100
export OLLAMA_CHAT_PORT=11434
export OLLAMA_CHAT_LANGUAGE_MODEL=mistral:7b
export EXTRACT_LLM_TYPE=openai
export OPENAI_API_KEY=sk-...

When the application imports llm_settings after these exports, the factories automatically instantiate the correct clients:

from hugegraph_llm.config import llm_settings
from hugegraph_llm.models.llms.init_llm import get_chat_llm, get_extract_llm

# Uses Ollama (chat) and OpenAI (extraction) simultaneously

chat = get_chat_llm(llm_settings)
extractor = get_extract_llm(llm_settings)

Summary

• Centralized Configuration: All LLM provider settings reside in the LLMConfig class at hugegraph-llm/src/hugegraph_llm/config/llm_config.py.
• Per-Task Specialization: HugeGraph AI supports different LLM providers for chat, extraction, and text-to-Gremlin operations through independent type selectors.
• Three Native Providers: OpenAI, Ollama (local), and LiteLLM are fully supported with distinct parameter sets for each.
• Singleton Pattern: The llm_settings object in hugegraph_llm/config/__init__.py provides global access to configuration values.
• Factory Architecture: init_llm.py maps configuration types to concrete client implementations without exposing provider details to business logic.

Frequently Asked Questions

How do I switch between OpenAI and Ollama in HugeGraph AI?

Set the CHAT_LLM_TYPE environment variable to "openai" or "ollama/local" before importing the configuration, or modify the chat_llm_type field in your .env file. The get_chat_llm() factory in hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py automatically instantiates the corresponding client using the provider-specific credentials defined in LLMConfig.

What is the difference between chat_llm_type and text2gql_llm_type?

chat_llm_type configures the LLM used for conversational responses and general question answering, while text2gql_llm_type specifically targets the model responsible for translating natural language into Gremlin or GraphQL queries. This separation allows optimization—using a fast, cheap model for chat and a more capable model for complex query generation.

Can I use different LLM providers for different tasks simultaneously?

Yes. HugeGraph AI supports heterogeneous configurations where, for example, chat_llm_type="ollama/local" routes conversational tasks to a local Ollama instance while extract_llm_type="openai" sends extraction tasks to OpenAI's API. Each factory function (get_chat_llm, get_extract_llm, get_text2gql_llm) independently reads its corresponding type field and instantiates the appropriate client.

How are API keys securely managed in HugeGraph AI?

API keys follow a hierarchy: explicit configuration fields (e.g., openai_chat_api_key) take precedence, followed by generic environment variables (e.g., OPENAI_API_KEY). For production deployments, the generate_env() utility in hugegraph-llm/src/hugegraph_llm/config/generate.py creates .env templates, though secrets should ultimately be injected via orchestration secrets or environment variables rather than committed configuration files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}