How to Add Custom LLM Providers by Extending the Default LLM Info File in Neuro-SAN

You can add custom LLM providers to Neuro-SAN by creating a HOCON fragment that defines a new provider class and including it in registries/llm_config.hocon or your specific agent network configuration.

The cognizant-ai-lab/neuro-san-studio repository uses a declarative configuration system that lets you extend the default LLM provider list without modifying core source code. When you add custom LLM providers by extending the default LLM info file, you integrate private vendors, self-hosted models, or experimental cloud services into your agent networks using standard LangChain wrappers.

Understanding the Default LLM Info Architecture

Neuro-SAN maintains a default LLM info file (default_llm_info.hocon) that catalogs supported providers such as OpenAI, Azure-OpenAI, and Anthropic. The central registry at registries/llm_config.hocon includes this default file to make standard providers available globally.

To support a provider not listed in the default file, you create a custom LLM info fragment—a partial HOCON file that defines a unique provider class and references a LangChain LLM implementation. The system merges your fragment with the default configuration at runtime, allowing agents to instantiate your custom provider using the standard llm_config block.

Step 1 – Create a Custom LLM Info Fragment

Create a new HOCON file in the registries/ directory to define your provider. Name it descriptively, such as my_custom_llm.hocon.

# registries/my_custom_llm.hocon

{
    "myprovider": {
        "display_name": "My Private LLM",
        "llm_class": "mycompany.llms.MyLLM",
        "default_model_name": "my-model-v1",
        "temperature": 0.6,
        "max_tokens": 2048
    }
}

Defining the Provider Class and LangChain Wrapper

The llm_class field must specify a fully-qualified Python class that implements the langchain.llms.base.BaseLLM interface. Neuro-SAN uses this class path to instantiate the LLM via LangChain's factory pattern. Ensure your custom package is installed in the execution environment (e.g., listed in requirements.txt) so the import resolves at runtime.

Step 2 – Include the Fragment in Your LLM Configuration

You can register your custom provider globally or limit it to a specific agent network.

Global Configuration via registries/llm_config.hocon

To make the provider available across all agent networks, edit the central registry:

# registries/llm_config.hocon

{
    include "default_llm_info.hocon"
    include "my_custom_llm.hocon"
}

Network-Specific Configuration

To scope the provider to a single network, include the fragment directly in your agent network HOCON file:

# my_network.hocon

{
    include "registries/llm_config.hocon"
    include "registries/my_custom_llm.hocon"

    llm_config {
        class = "myprovider"
        model_name = "my-model-v2"
        temperature = 0.3
    }

    # ... agent network definition continues ...

}

Step 3 – Configure Runtime Credentials and Environment Variables

Most custom providers require authentication or endpoint configuration. Export the necessary environment variables before launching Neuro-SAN:

export MYPROVIDER_API_KEY="sk-your-private-key"
export MYPROVIDER_ENDPOINT="https://api.myprovider.com/v1"

Neuro-SAN passes these variables to the underlying LangChain wrapper, which typically expects specific naming conventions (e.g., {PROVIDER}_API_KEY). Consult your LangChain LLM class documentation to identify the exact variable names required.

Step 4 – Validate the Custom Provider Integration

Verify that Neuro-SAN can instantiate your provider by testing the LLM factory directly:

from neuro_san.internals.run_context.langchain.llms import LLMFactory

config = {
    "class": "myprovider",
    "model_name": "my-model-v1"
}

llm = LLMFactory.from_config(config)
print(f"Successfully instantiated: {llm}")

If the factory raises an ImportError, verify that your llm_class path is correct and that the package containing your LLM implementation is installed in the Python environment.

Complete Example – Adding an Ollama Local Model

The following example demonstrates adding an Ollama local LLM provider, which requires no API keys.

Create the fragment at registries/ollama_llm.hocon:

{
    "ollama": {
        "display_name": "Ollama",
        "llm_class": "langchain_community.llms.ollama.Ollama",
        "default_model_name": "llama3.1:8b",
        "temperature": 0.7,
        "max_tokens": 4096
    }
}

Include it in registries/llm_config.hocon:

{
    include "default_llm_info.hocon"
    include "ollama_llm.hocon"
}

Configure your network to use the local model:

llm_config {
    class = "ollama"
    model_name = "llama3.1:8b"
}

Ensure the Ollama daemon is running and the model is pulled locally (ollama pull llama3.1:8b). No environment variables are required for local Ollama instances.

Summary

• Create a HOCON fragment in registries/ that defines a unique provider class, llm_class (LangChain wrapper), and default parameters.
• Include the fragment in registries/llm_config.hocon for global availability, or directly in a specific agent network file for scoped use.
• Export environment variables for API keys and endpoints required by your custom LLM implementation.
• Validate the integration using LLMFactory.from_config() to ensure the LangChain class resolves correctly before deploying agents.

Frequently Asked Questions

What file format does Neuro-SAN use for LLM configuration?

Neuro-SAN uses HOCON (Human-Optimized Config Object Notation) for all configuration files, including LLM provider definitions. HOCON is a superset of JSON that supports comments, includes, and more flexible syntax, making it ideal for managing complex agent network configurations alongside LLM provider registries.

Can I add multiple custom LLM providers simultaneously?

Yes. You can define multiple providers in a single HOCON fragment or create separate fragment files for each provider. Simply include all fragment files in registries/llm_config.hocon or your network-specific configuration. Each provider must have a unique class identifier to prevent conflicts in the LLM registry.

How do I troubleshoot import errors when adding a custom provider?

If LLMFactory.from_config() raises an ImportError or ModuleNotFoundError, verify three things: first, ensure the llm_class path in your HOCON fragment matches the actual Python module structure; second, confirm the package containing your LLM implementation is installed in the execution environment (check pip list or requirements.txt); third, ensure any required environment variables that the LangChain wrapper expects are exported before runtime.

Is it possible to override default model parameters per agent?

Yes. While the LLM info fragment defines default parameters like temperature and max_tokens, you can override these at the network level within the llm_config block, or even per agent if the agent configuration supports an llm_config override. Simply specify the parameter name and new value in the HOCON configuration where the agent or network is defined, and these values take precedence over the defaults defined in the LLM info fragment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}