How to Configure Custom LLM Providers in SkillSpector: A Complete Guide

Set the SKILLSPECTOR_PROVIDER environment variable to select built-in providers, or implement the LLMProvider Protocol in a new sub-package and register it in src/skillspector/providers/__init__.py to add completely custom backends.

NVIDIA SkillSpector provides a flexible provider architecture that allows you to integrate custom large language model backends beyond the default nv_build option. By leveraging the SKILLSPECTOR_PROVIDER environment variable and implementing the LLMProvider Protocol defined in src/skillspector/providers/base.py, you can connect proprietary APIs, local models, or specialized inference endpoints while maintaining full compatibility with SkillSpector's skill analysis pipeline.

Understanding the Provider Architecture

The Runtime Selector

The entry point for provider discovery resides in src/skillspector/providers/__init__.py. The _select_active_provider function (lines 76-98) reads the SKILLSPECTOR_PROVIDER environment variable and instantiates the corresponding provider class. If the variable is unset, the system falls back to the built-in nv_build provider.

The Provider Contract

Every provider must satisfy the LLMProvider Protocol defined in src/skillspector/providers/base.py (lines 37-84). This contract mandates implementations of three mix-in capabilities: ModelMetadataProvider for token budgets and model resolution, CredentialsProvider for API key management, and ChatModelProvider for LangChain model construction.

Model Registry Overrides

Token budget metadata loads through src/skillspector/providers/registry.py. The _resolve_path helper (lines 62-66) checks for the SKILLSPECTOR_MODEL_REGISTRY environment variable, allowing you to override the bundled model_registry.yaml without touching the source code.

Method 1: Switching to Built-in Providers

SkillSpector ships with multiple pre-configured providers including OpenAI, Claude CLI, and Gemini CLI. To activate any built-in provider, export the corresponding identifier:

export SKILLSPECTOR_PROVIDER=openai
skillspector scan ./my_skill --use-llm

Available provider names correspond to the hard-coded branches in _select_active_provider. If you specify an unknown name, the selector raises a ValueError with the supported options.

Method 2: Creating a Custom LLM Provider

For proprietary APIs or unsupported models, implement a new provider class following the protocol-based extension pattern.

Step 1: Create the Provider Package

Create a sub-package under src/skillspector/providers/ with this structure:

src/skillspector/providers/myprovider/
├── __init__.py
├── provider.py
└── model_registry.yaml  # Optional

Step 2: Implement the LLMProvider Protocol

In provider.py, create a class implementing all required methods from the protocol:

# src/skillspector/providers/myprovider/provider.py

import os
from skillspector.providers.base import LLMProvider

class MyProvider:
    DEFAULT_MODEL = "my-model-v1"
    SLOT_DEFAULTS = {"default": DEFAULT_MODEL, "code": "my-code-model"}

    def get_context_length(self, model: str) -> int | None:
        return 16384

    def get_max_output_tokens(self, model: str) -> int | None:
        return 4096

    def resolve_model(self, slot: str = "default") -> str:
        return self.SLOT_DEFAULTS.get(slot, self.DEFAULT_MODEL)

    def resolve_credentials(self) -> tuple[str, str | None] | None:
        api_key = os.getenv("MYPROVIDER_API_KEY")
        base_url = os.getenv("MYPROVIDER_BASE_URL")
        return (api_key, base_url) if api_key else None

    def create_chat_model(self, model: str, *, max_tokens: int, 
                          timeout: float | None = 120):
        from langchain_community.chat_models import MyChatModel
        api_key, base_url = self.resolve_credentials()
        return MyChatModel(
            model=model,
            api_key=api_key,
            base_url=base_url,
            max_tokens=max_tokens,
            timeout=timeout
        )

Step 3: Register in the Selector

Add a registration branch in src/skillspector/providers/__init__.py within _select_active_provider:

if name == "myprovider":
    from .myprovider import MyProvider
    return MyProvider()

Step 4: Configure Environment Variables

Export your provider credentials and selector:

export SKILLSPECTOR_PROVIDER=myprovider
export MYPROVIDER_API_KEY=sk-...
skillspector scan ./my_skill --use-llm

Method 3: Overriding Model Registry Configuration

To customize token limits for existing providers without writing code, create a custom model_registry.yaml and point to it:

export SKILLSPECTOR_MODEL_REGISTRY=/path/to/custom_registry.yaml

The registry loader in src/skillspector/providers/registry.py (lines 24-27) prioritizes this path over the bundled configuration, loading your custom context_length and max_output_tokens values.

Implementation Example: Minimal Custom Provider

Here is a complete example for a custom HTTP-based provider:

# src/skillspector/providers/custom_http/provider.py

import os
from skillspector.providers.base import LLMProvider

class CustomHTTPProvider:
    DEFAULT_MODEL = "custom-llm-v1"
    SLOT_DEFAULTS = {"default": DEFAULT_MODEL}

    def get_context_length(self, model: str) -> int | None:
        return None  # Falls back to generic defaults

    def get_max_output_tokens(self, model: str) -> int | None:
        return None

    def resolve_model(self, slot: str = "default") -> str:
        return self.SLOT_DEFAULTS.get(slot, self.DEFAULT_MODEL)

    def resolve_credentials(self):
        key = os.getenv("CUSTOM_API_KEY")
        return (key, None) if key else None

    def create_chat_model(self, model: str, *, max_tokens: int,
                          timeout: float | None = 120):
        from langchain_community.chat_models import ChatOpenAI
        api_key, _ = self.resolve_credentials()
        return ChatOpenAI(
            model_name=model,
            openai_api_key=api_key,
            max_tokens=max_tokens,
            request_timeout=timeout
        )

Register this in __init__.py as shown above, then activate with export SKILLSPECTOR_PROVIDER=custom_http.

Summary

• Environment-driven selection: Use SKILLSPECTOR_PROVIDER to switch between built-in backends or activate custom implementations.
• Protocol-based extension: Implement the LLMProvider Protocol in base.py to create compatible providers without modifying core logic.
• Registry customization: Override SKILLSPECTOR_MODEL_REGISTRY to supply custom token budgets via YAML configuration.
• Registration requirement: All providers must be registered in src/skillspector/providers/__init__.py within the _select_active_provider function.
• Optional CLI capability: Implement AgentCLICapable protocol for providers that require local CLI execution instead of HTTP APIs.

Frequently Asked Questions

What environment variables does SkillSpector use for provider configuration?

SkillSpector uses SKILLSPECTOR_PROVIDER to select the active provider at runtime, and SKILLSPECTOR_MODEL_REGISTRY to specify custom model metadata paths. Individual providers typically require their own credential variables, such as OPENAI_API_KEY or custom variables defined in your resolve_credentials implementation.

Do I need to modify SkillSpector's core code to add a new provider?

You must modify src/skillspector/providers/__init__.py to add a registration branch in _select_active_provider, but you do not need to change any other core components. The provider itself lives in a separate sub-package under src/skillspector/providers/, keeping your changes isolated and maintainable.

How does SkillSpector handle authentication for custom providers?

The CredentialsProvider protocol requires a resolve_credentials method that returns a tuple of (api_key, base_url) or None. SkillSpector calls this method at runtime to retrieve credentials, allowing you to implement custom logic for environment variables, configuration files, or secret managers.

Can I use local CLI-based models instead of HTTP APIs?

Yes. Implement the AgentCLICapable protocol (defined in base.py and demonstrated in _agent_cli_base.py) in addition to the standard LLMProvider methods. When has_cli_capability() returns True, SkillSpector routes requests through run_agent_cli in _agent_cli.py instead of constructing LangChain HTTP models.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}