How to Create Custom LangChain LLM Factories for Unsupported Providers in Neuro-San

To integrate an unsupported LLM provider into Neuro-San, subclass LangChainLlmFactory, override the create_llm() method to return a LangChain-compatible BaseChatModel, and register the factory class in registries/llm_config.hocon.

Neuro-San uses a factory pattern to instantiate LangChain LLM objects for use by agents and tools. While the StandardLangChainLlmFactory handles built-in providers like OpenAI and Anthropic, the abstract base class in neuro_san/internals/run_context/langchain/llms/langchain_llm_factory.py allows you to inject any LLM that implements the LangChain protocol. This guide shows you how to create custom LangChain LLM factories for unsupported providers and register them for seamless dependency injection.

Understanding the Factory Architecture

Neuro-San delegates all LLM instantiation to classes implementing the LangChainLlmFactory interface. During startup, the platform reads registries/llm_config.hocon, dynamically loads factory classes using ResolverUtil.create_type(), and invokes create_llm() to obtain ready-to-use LangChain instances.

The default StandardLangChainLlmFactory knows how to build OpenAI, Anthropic, and Azure OpenAI models. When you need to integrate a provider that LangChain does not yet support natively, you create a custom factory that returns a thin wrapper around the provider's SDK, adapting it to the BaseChatModel interface.

Implementing a Custom LLM Factory

Step 1: Subclass the Abstract Base Factory

Create a new Python file in your package that inherits from LangChainLlmFactory. The base class defines the contract that Neuro-San expects: a single create_llm() method that returns a LangChain-compatible chat model.

# my_custom_llm_factory.py

from neuro_san.internals.run_context.langchain.llms.langchain_llm_factory import (
    LangChainLlmFactory,
)
from langchain_core.language_models import BaseChatModel

class MyCustomLlmFactory(LangChainLlmFactory):
    """Factory for a provider not natively supported by LangChain."""
    
    def __init__(self, api_key: str, model: str = "my-model"):
        self.api_key = api_key
        self.model = model

Step 2: Override create_llm() to Wrap the Provider SDK

Implement create_llm() to instantiate the provider's client and return a BaseChatModel subclass that translates LangChain method calls into the provider's API format.

    def create_llm(self) -> BaseChatModel:
        from my_provider_sdk import MyChatClient
        
        client = MyChatClient(api_key=self.api_key, model=self.model)
        
        class MyLangChainAdapter(BaseChatModel):
            async def _acall(self, messages, **kwargs):
                # Convert LangChain messages to provider format

                payload = [{"role": m.type, "content": m.content} for m in messages]
                response = await client.chat(payload)
                return response.content
        
        return MyLangChainAdapter()

Key requirements:

• Return type must be a subclass of BaseChatModel (or any class implementing LangChain's chat-model protocol).
• Async vs sync – If your provider's SDK is asynchronous, implement async _acall; otherwise, implement the synchronous invoke() method as shown in StandardLangChainLlmFactory.

Step 3: Register the Factory in llm_config.hocon

Add the fully-qualified import path to registries/llm_config.hocon. You can supply constructor arguments directly in the configuration.

# registries/llm_config.hocon

my_custom = {
  class = "my_pkg.my_custom_llm_factory.MyCustomLlmFactory"
  api_key = ${?MY_PROVIDER_API_KEY}
  model = "awesome-model"
  temperature = 0.7
}

Step 4: Reference in Tool Configurations

Use the registered name in any tool or agent definition. Neuro-San will resolve the factory, instantiate it with the configured parameters, and inject the resulting LLM into your CodedTool or Agent.

# registries/tools/custom_analysis.hocon

{
  tool_name = "custom_analysis"
  llm = "my_custom"
  description = "Analyzes data using the custom provider"
}

Complete Working Example

Here is a full implementation for a hypothetical "SuperLLM" provider that speaks an OpenAI-compatible API but requires a custom SDK client.

# super_llm_factory.py

from neuro_san.internals.run_context.langchain.llms.langchain_llm_factory import (
    LangChainLlmFactory,
)
from langchain_core.language_models import BaseChatModel
from super_llm import SuperClient  # Third-party SDK

class SuperLlmFactory(LangChainLlmFactory):
    """Factory for SuperLLM integration."""
    
    def __init__(self, api_key: str, model: str = "default", temperature: float = 0.7):
        self.api_key = api_key
        self.model = model
        self.temperature = temperature
    
    def create_llm(self) -> BaseChatModel:
        client = SuperClient(api_key=self.api_key, model=self.model)
        
        class SuperAdapter(BaseChatModel):
            async def _acall(self, messages, **kwargs):
                payload = [{"role": m.type, "content": m.content} for m in messages]
                resp = await client.chat(payload, temperature=self.temperature)
                return resp["message"]
        
        return SuperAdapter()

Configuration:

# registries/llm_config.hocon

super_llm = {
  class = "my_pkg.super_llm_factory.SuperLlmFactory"
  api_key = ${?SUPER_API_KEY}
  model = "giga-model"
  temperature = 0.6
}

Tool usage:

# registries/tools/super_search.hocon

{
  name = "super_search"
  llm = "super_llm"
}

When Neuro-San boots, it resolves my_pkg.super_llm_factory.SuperLlmFactory via ResolverUtil.create_type(), passes the configured api_key, model, and temperature to the constructor, and stores the resulting LLM instance for injection into the "super_search" tool.

Configuration and Runtime Behavior

Constructor injection – Neuro-San automatically passes any key-value pairs defined in the factory's .hocon entry to the class __init__ method. This allows you to externalize API keys, endpoints, and model parameters without hard-coding them.

Dependency injection – Once created, the LLM instance is injected into any CodedTool or Agent that declares an LLM dependency. Your custom provider becomes indistinguishable from built-in OpenAI or Anthropic models; tools call await llm.ainvoke(prompt) exactly the same way.

Testing – Verify your factory returns a working LangChain model by following the unit-test pattern in tests/coded_tools/tools/. Ensure the returned object properly implements invoke() or ainvoke() and handles message formatting correctly.

Summary

• Subclass LangChainLlmFactory from neuro_san/internals/run_context/langchain/llms/langchain_llm_factory.py to define how your unsupported provider should be instantiated.
• Override create_llm() to return a BaseChatModel that wraps the provider's SDK, implementing either async _acall or synchronous invoke() depending on the SDK's capabilities.
• Register in registries/llm_config.hocon using the fully-qualified class path and any required constructor arguments.
• Reference by name in tool configurations to enable dependency injection of your custom LLM into CodedTool and Agent instances.

Frequently Asked Questions

What interface must my custom LLM implement?

Your create_llm() method must return an instance of BaseChatModel from langchain_core.language_models, or any class that implements the LangChain chat-model protocol including invoke(), ainvoke(), and message handling. This ensures compatibility with the CodedTool and Agent calling conventions used throughout Neuro-San.

How do I handle asynchronous versus synchronous LLM calls?

If your provider's SDK uses asynchronous I/O, implement the async _acall() method in your BaseChatModel subclass. For synchronous SDKs, implement the invoke() method instead. The StandardLangChainLlmFactory demonstrates both patterns for reference.

Where do I specify API keys and other sensitive configuration?

Define constructor parameters in registries/llm_config.hocon using the HOCON object syntax. Reference environment variables with ${?ENV_VAR_NAME} syntax to keep secrets out of version control. Neuro-San passes these values directly to your factory's __init__ method during instantiation.

Can I register multiple custom factories for different providers?

Yes. Add separate entries to registries/llm_config.hocon for each factory class, using unique keys like provider_a and provider_b. Each factory operates independently, allowing you to mix custom providers with built-in OpenAI, Anthropic, or Azure models in the same Neuro-San deployment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}