How to Configure FreeLLMAPI Model Profiles: A Complete Guide to Fallback Chains

FreeLLMAPI lets you group individual model entries into named fallback-chain profiles that the router walks through in order, applying scoring logic until a request is satisfied.

FreeLLMAPI is an open-source LLM routing platform that supports intelligent model selection through configurable profiles. Learning how to configure FreeLLMAPI model profiles allows you to create curated fallback chains for specific use cases like vision tasks or coding assistance. This guide covers the database architecture, REST API endpoints, and practical implementation based on the current source code in the tashfeenahmed/freellmapi repository.

Understanding Model Profiles and Fallback Chains

A model profile in FreeLLMAPI is essentially a curated list of individual models grouped under a unique name. When you configure these profiles, you create prioritized fallback chains that the router evaluates sequentially.

The routing system applies its scoring logic—considering factors like rate limits, speed, and reliability—to each model in the profile until it finds one that can satisfy the request. This architecture ensures high availability even when specific providers experience downtime or rate limiting.

You can invoke profiles using two methods:

• auto – Uses the currently active profile stored in the database
• auto:<profile-name> – Bypasses the active profile and uses a specific profile directly

Database Architecture and Core Implementation

The profile system relies on a SQLite schema defined in server/src/services/profile-models.ts. This service manages three key database structures:

• profiles table – Stores named profile definitions with unique identifiers
• profile_models table – Links model entries to profiles with priority and enabled flags
• settings table – Stores the active_profile_id key to determine which profile serves as the default

The server/src/services/profile-models.ts file contains the core database helpers for creating profiles, adding models to profiles, and ensuring models appear in every profile that auto-includes new models. Each entry in the profile_models table includes a priority value for ordering and an enabled flag to toggle availability without removing the link.

Managing Profiles via the REST API

FreeLLMAPI exposes full CRUD operations for profiles through server/src/routes/profiles.ts. These endpoints allow programmatic configuration of your fallback chains.

Listing and Creating Profiles

Retrieve all existing profiles with the default profile listed first:

curl -X GET http://localhost:3000/api/profiles | jq .

Create a new profile by providing a unique name:

curl -X POST http://localhost:3000/api/profiles \
  -H "Content-Type: application/json" \
  -d '{"name":"coding"}' | jq .

Adding and Configuring Models

After creating a profile, populate it with models using the child endpoint. You must specify the model identifier, priority order, and enabled status:

curl -X POST http://localhost:3000/api/profiles/<PROFILE_ID>/models \
  -H "Content-Type: application/json" \
  -d '{"modelId":"openai:gpt-4o-mini","priority":1,"enabled":true}' | jq .

You can update profile metadata or remove profiles entirely using PATCH /api/profiles/:id and DELETE /api/profiles/:id respectively.

Setting the Active Profile

Designate which profile the router uses when requests specify model: "auto":

curl -X POST http://localhost:3000/api/profiles/active \
  -H "Content-Type: application/json" \
  -d '{"profileId":<PROFILE_ID>}' | jq .

The active profile ID persists in the settings table under the key active_profile_id.

Router Integration and Request Dispatching

The routing logic in server/src/services/router.ts handles profile resolution through the resolveRequestedIdForDispatch function. This implementation examines incoming requests for the auto:<profile-name> syntax and resolves the appropriate model list.

When processing requests, the router:

1. Parses the model identifier for the auto: prefix
2. Retrieves the specified profile or falls back to the active profile
3. Filters for enabled models in priority order
4. Applies scoring logic to select the optimal available model

This ensures that auto:vision routes through your vision-optimized profile while auto uses whatever profile is currently active.

Dashboard and CLI Integration

Beyond the REST API, FreeLLMAPI provides intuitive interfaces for profile management.

The web dashboard presents a Profiles tab where you can drag-and-drop models to reorder priority, toggle the enabled state for individual models, and set the active profile through a graphical interface. The dashboard consumes the same REST endpoints documented above.

For CLI users, the freellmapi command supports the --profile <NAME> flag when generating API keys. According to cli/README.md, this creates a profile entry in the dashboard and binds the generated key to that specific profile:

freellmapi key generate --profile coding

Practical Usage Examples

Complete Profile Setup Workflow

Configure a new "vision" profile from scratch:

# 1. Create the profile

PROFILE_ID=$(curl -X POST http://localhost:3000/api/profiles \
  -H "Content-Type: application/json" \
  -d '{"name":"vision"}' | jq -r '.id')

# 2. Add GPT-4o Mini with high priority

curl -X POST http://localhost:3000/api/profiles/$PROFILE_ID/models \
  -H "Content-Type: application/json" \
  -d '{"modelId":"openai:gpt-4o-mini","priority":1,"enabled":true}'

# 3. Add Claude 3 as fallback

curl -X POST http://localhost:3000/api/profiles/$PROFILE_ID/models \
  -H "Content-Type: application/json" \
  -d '{"modelId":"anthropic:claude-3-haiku","priority":2,"enabled":true}'

# 4. Activate the profile

curl -X POST http://localhost:3000/api/profiles/active \
  -H "Content-Type: application/json" \
  -d "{\"profileId\":$PROFILE_ID}"

Sending Requests with Profile Selection

Use the specific profile in your chat completion requests:

curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model":"auto:vision",
    "messages":[{"role":"user","content":"Describe this image"}]
  }' | jq .

Or rely on the active profile:

curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model":"auto",
    "messages":[{"role":"user","content":"Hello world"}]
  }' | jq .

Summary

• Model profiles are named fallback chains stored in the SQLite profiles table and linked via profile_models in server/src/services/profile-models.ts
• Priority and enabled flags control walker order and availability within each profile
• REST endpoints in server/src/routes/profiles.ts provide full CRUD operations for programmatic configuration
• Active profile selection persists in the settings table under active_profile_id and applies when requests use model: "auto"
• Per-request override syntax auto:<profile-name> routes through specific profiles regardless of the active setting, resolved by resolveRequestedIdForDispatch in server/src/services/router.ts
• CLI integration via --profile in cli/README.md binds generated keys to specific profiles

Frequently Asked Questions

What happens if no models in a profile are enabled?

If all models in a requested profile have enabled: false, the router cannot find a valid candidate and returns an error indicating no models are available for dispatch. The scoring logic skips disabled entries entirely, so ensure at least one high-priority model remains enabled for production profiles.

Can I specify multiple profiles in a single request?

No, the auto:<profile-name> syntax accepts only one profile identifier per request. However, you can nest fallback logic by ordering models within a single profile by priority. According to docs/api.md, unknown profile names trigger specific error responses, so verify profile names through the GET /api/profiles endpoint first.

How does the router prioritize models within a profile?

The router evaluates models in ascending order of the priority value defined in the profile_models table, as implemented in server/src/services/router.ts. Lower numbers indicate higher priority. Within the same priority level, the system applies dynamic scoring based on rate limit status, latency history, and reliability metrics to select the optimal provider.

Where is the active profile configuration stored?

The active profile ID is stored in the SQLite settings table under the key active_profile_id, as managed by the profile service in server/src/services/profile-models.ts. You can retrieve or modify this value through the GET and POST /api/profiles/active endpoints, or via the dashboard's profile selector interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}