How to Use Custom Provider Chains with FreeLLMAPI

Yes, FreeLLMAPI supports custom provider chains through named fallback profiles that you invoke using the auto:<chain-name> syntax in your model parameter.

FreeLLMAPI is an open-source LLM routing layer that allows you to define custom provider chains to control exactly which models handle your requests and in what priority order. Instead of hardcoding a single model, you can create named fallback profiles that automatically route across multiple providers based on availability, quotas, and scoring logic. This article explains the architecture behind these chains, how to configure them via the dashboard or API, and how to invoke them in your code.

Understanding Custom Provider Chains in FreeLLMAPI

What Are Provider Chains?

A custom provider chain (also called a named fallback chain) is an ordered list of models stored in the profile_models table. When you send a request with model=auto:mychain, the router loads this ordered list and attempts each model in sequence until one succeeds.

How the Router Resolves Chains

The router's entry point activeChainOrThrow validates the chain exists and contains at least one enabled model. If validation fails, it immediately returns a 400 error rather than falling back silently source.

Once validated, the routing engine iterates through the chain and instantiates each provider using getProvider from the providers registry source.

Creating Custom Provider Chains

You have two methods to define chains: the web dashboard or the REST API.

Using the Dashboard

Navigate to the Fallback page in the FreeLLMAPI dashboard. Create a new profile with a name containing only letters, digits, hyphens, and underscores. Manually add models in your preferred priority order. Disable Auto-include new models to keep the chain static.

Using the REST API

Programmatically define chains by sending a PUT request to /api/fallback. The payload must include the chain array, name, and the auto_include_new_models flag.

curl -X PUT https://api.free.llmapi.com/api/fallback \
  -H "Authorization: Bearer $FREE_LLMAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "chain": [
          { "model_id": "gpt-4o-mini", "priority": 1, "enabled": true },
          { "model_id": "gemini-1.5-flash", "priority": 2, "enabled": true }
        ],
        "name": "my-coding-chain",
        "auto_include_new_models": 0
      }'

Routing Requests Through Custom Chains

Once defined, invoke a chain using the auto:<name> syntax in the model parameter. The router resolves the chain from the profile_models table and applies quota, cooldown, and scoring logic per provider.

Example API Call with auto: Prefix

curl https://api.free.llmapi.com/v1/chat/completions \
  -H "Authorization: Bearer $FREE_LLMAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto:my-coding-chain",
        "messages": [{"role":"user","content":"Write a Python function to reverse a string"}],
        "max_tokens": 256
      }'

TypeScript Client Example

import { FreeLLMAPI } from "free-llmapi";

const client = new FreeLLMAPI({ apiKey: process.env.FREE_LLMAPI_KEY });

const response = await client.chat.completions.create({
  model: "auto:my-coding-chain",
  messages: [{ role: "user", content: "Explain memoization" }],
  max_tokens: 300,
});

console.log(response.choices[0].message.content);

Configuration and Behavior

The auto_include_new_models boolean flag (default 0) determines whether newly discovered models are automatically appended to your chain. Set this to 0 to maintain a curated, static list of providers source.

When building chains, consider that each additional model adds potential latency if earlier providers fail. The system walks the chain sequentially, applying cooldowns and quota checks at each step.

Summary

• FreeLLMAPI implements custom provider chains as named fallback profiles stored in the profile_models table.
• Activate any chain using the auto:<chain-name> syntax in your request's model field source.
• The activeChainOrThrow function validates chains before routing, returning a 400 error for invalid configurations.
• Create chains via the dashboard or programmatically via PUT /api/fallback.
• Control chain membership using the auto_include_new_models flag to prevent automatic expansion.

Frequently Asked Questions

What happens if my custom chain has no enabled models?

The router's activeChainOrThrow validation will detect the empty chain and return a 400 Bad Request error immediately. FreeLLMAPI does not silently skip to a global fallback, ensuring you always know when your chain configuration is invalid source.

Can I mix custom chains with standard model names?

No, the model parameter accepts either a specific model ID (e.g., gpt-4o-mini) or a chain reference (auto:mychain), but you cannot combine both in a single request. To use multiple specific models, define them explicitly within a custom chain.

How does FreeLLMAPI handle new models in custom chains?

By default, new models are not added to existing chains (auto_include_new_models: 0). If you enable this flag, the system automatically appends newly discovered providers to your chain with the lowest priority. Disable this to maintain strict control over your routing logic source.

Is there a limit to how many models I can include in a chain?

While there is no hardcoded limit in the router source code, practical limits apply based on request timeout configurations. Each additional model in the chain adds potential latency if earlier providers fail, so we recommend keeping chains under 10 models for optimal performance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}