How to Configure the Fallback Chain Priority for LLM Models in FreeLLMAPI

FreeLLMAPI stores the fallback chain in a SQLite table named fallback_config where lower integer values in the priority column indicate higher precedence, and you can configure this order either by loading a declarative JSON file or by calling the REST API endpoints.

FreeLLMAPI is an open-source multi-provider LLM gateway that routes requests through a prioritized fallback chain when primary models fail. To configure the fallback chain priority for LLM models, you manipulate the priority column in the SQLite database, either through environment-based JSON configuration or via the admin dashboard API. The router always orders active models by this priority value, ensuring deterministic failover behavior.

Understanding the Fallback Chain Storage

The fallback configuration persists in the fallback_config table inside the SQLite database. Each row links a specific model to its failover precedence using three key columns:

• model_db_id – Foreign key referencing the models table
• priority – Integer value determining the sort order (lowest numbers are attempted first)
• enabled – Boolean flag to temporarily deactivate a model without removing it

The router queries this table and joins it with the models table to ensure only active, enabled models are considered for routing.

Method 1: Declarative JSON Configuration

You can define the entire fallback chain in a JSON file and have FreeLLMAPI synchronize it to the database at startup. The server validates the configuration using Zod schema fallbackEntrySchema defined in server/src/services/declarative-config.ts (lines 65-70) and applies it via the applyFallback() function (lines 24-35).

Set the configuration file path:

export FREEAPI_CONFIG_PATH=/path/to/fallback.json

Or provide the JSON directly:

export FREEAPI_CONFIG_JSON='{"fallback": [{"platform": "openai", "modelId": "gpt-4", "priority": 1}]}'

Create a fallback.json file with the priority values explicitly set:

{
  "fallback": [
    { "platform": "openai", "modelId": "gpt-4", "priority": 1, "enabled": true },
    { "platform": "anthropic", "modelId": "claude-2", "priority": 2, "enabled": true },
    { "platform": "groq", "modelId": "mixtral-8x7b", "priority": 3, "enabled": true }
  ]
}

When the server starts, applyDeclarativeConfigFromEnv() parses this input and executes UPDATE fallback_config SET priority = ?, enabled = ? WHERE model_db_id = ? statements to synchronize the database state.

Method 2: REST API and Dashboard

For runtime updates without restarting the server, use the REST endpoints defined in server/src/routes/fallback.ts. The API exposes standard CRUD operations on the fallback_config table.

Patch a model's priority:

curl -X PATCH "http://localhost:3000/api/fallback/openai/gpt-4" \
  -H "Content-Type: application/json" \
  -d '{"priority": 1, "enabled": true}'

Retrieve the current chain:

curl http://localhost:3000/api/fallback

Programmatic update using Node.js:

async function setPriority(platform: string, modelId: string, priority: number) {
  const res = await fetch(`http://localhost:3000/api/fallback/${platform}/${modelId}`, {
    method: 'PATCH',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ priority, enabled: true })
  });
  if (!res.ok) throw new Error(`Failed to update: ${await res.text()}`);
}

await setPriority('openai', 'gpt-4', 1);
await setPriority('anthropic', 'claude-2', 2);

Implementation Details in the Router

The core routing logic resides in server/src/services/router.ts. The function getActiveChain() (lines 45-55) queries the database to fetch the ordered fallback chain:

const chain = db.prepare(`
  SELECT fc.model_db_id, fc.priority, fc.enabled,
         m.platform, m.model_id
  FROM fallback_config fc
  JOIN models m ON m.id = fc.model_db_id AND m.enabled = 1
  ORDER BY fc.priority ASC
`).all();

This query ensures that the router attempts models in ascending priority order. While the orderChain() function can apply additional routing-strategy weightings, the base order always respects the priority column you configured.

Step-by-Step Configuration Workflow

Follow this sequence to configure the fallback chain priority for LLM models:

1. Identify the model database ID by querying the models table:

   SELECT id FROM models WHERE platform = 'openai' AND model_id = 'gpt-4';

2. Set the priority using either the declarative JSON method (for initialization) or the REST API (for runtime updates). Lower integers indicate higher precedence.

3. Verify the configuration by querying the database or using the read endpoint:

   SELECT fc.priority, m.platform, m.model_id
   FROM fallback_config fc
   JOIN models m ON m.id = fc.model_db_id
   ORDER BY fc.priority ASC;

4. Confirm routing behavior by inspecting logs or testing a request that triggers failover, ensuring the router proceeds through your configured sequence.

Summary

• FreeLLMAPI stores fallback priorities in the SQLite fallback_config table with lower integer values indicating higher precedence.
• Configure priorities via declarative JSON (FREEAPI_CONFIG_PATH) using the schema in server/src/services/declarative-config.ts or via the REST API endpoints in server/src/routes/fallback.ts.
• The router fetches the ordered chain via getActiveChain() in server/src/services/router.ts using ORDER BY fc.priority ASC.
• The enabled column allows you to deactivate models without removing them from the configuration.
• Changes via the REST API take effect immediately, while declarative configuration requires a server restart to reload.

Frequently Asked Questions

How does the router determine which model to try first?

The router queries the fallback_config table and sorts results by the priority column in ascending order, as implemented in getActiveChain() within server/src/services/router.ts. The model with the lowest numeric priority value is attempted first, followed by the next lowest value if the request fails.

Can I update the fallback chain without restarting the FreeLLMAPI server?

Yes. Use the REST API endpoints defined in server/src/routes/fallback.ts to send PATCH requests that update the priority and enabled columns in real-time. The router reads these values fresh for each request, so changes apply immediately without requiring a restart.

What is the relationship between the models table and the fallback_config table?

The fallback_config table contains a foreign key model_db_id that references the id column in the models table. The router joins these tables to validate that fallback entries point to existing, enabled models before including them in the routing chain.

How do I temporarily disable a model without removing it from the chain?

Set the enabled column to false for the specific model entry in the fallback_config table. You can do this via the REST API by sending a PATCH request with {"enabled": false}, or by setting "enabled": false in your declarative JSON configuration. This keeps the priority value stored but excludes the model from the active failover sequence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}