How to Customize FreeLLMAPI Behavior Using a Config File

FreeLLMAPI loads a declarative JSON configuration at startup that controls API keys, model properties, fallback chains, and routing strategies through environment variables FREEAPI_CONFIG_PATH or FREEAPI_CONFIG_JSON.

To customize the API behavior using the config file in the tashfeenahmed/freellmapi repository, you declare your infrastructure and routing preferences in a JSON file and point the server to it via environment variables. The configuration system, implemented in server/src/services/declarative-config.ts, validates your input against a strict Zod schema and applies the settings to the SQLite database on boot. This makes your declared configuration instantly available to all API routes without requiring any code modifications.

How Config File Discovery Works

The server locates your configuration through the readConfigFromEnv function (lines 12–19 in server/src/services/declarative-config.ts). This function checks two environment variables in order of priority:

• FREEAPI_CONFIG_JSON – An inline JSON string containing the entire configuration.
• FREEAPI_CONFIG_PATH – A filesystem path to a JSON file on disk.

If neither variable is set, the server starts with default settings. When present, the function reads the content and passes it to the validation layer.

Configuration Schema: Five Customizable Sections

FreeLLMAPI validates every config file against a Zod schema defined in declarative-config.ts (lines 72–85). The schema recognizes five top-level sections that control distinct API behaviors:

keys – API credentials for built-in providers like OpenAI or Anthropic.

customProviders – Self-hosted LLM endpoints with custom base URLs and model lists.

models – Overrides for catalog model properties, including display names, intelligence rankings, speed rankings, token budgets, and enable flags.

fallback – Priority ordering and enable flags for the fallback chain used when primary models fail.

routing – Strategy selection (priority, balanced, smartest) and optional weight overrides for the routing algorithm.

How Configuration is Applied at Runtime

Once validated, the applyDeclarativeConfig function (lines 38–79) transforms the JSON into database state. This function inserts or updates rows for API keys, model definitions, and fallback entries, then sets the active routing strategy. The wrapper applyDeclarativeConfigFromEnv (lines 80–88) ties the discovery step to server startup, executing the configuration exactly once after the SQLite database opens.

This architecture ensures that when you run npm start or node server/src/index.js, the configuration is applied atomically before the API accepts requests.

Complete Configuration File Example

The following my-config.json demonstrates all five sections:

{
  "keys": [
    { "platform": "openai", "key": "sk-…", "label": "prod‑openai" }
  ],
  "customProviders": [
    {
      "baseUrl": "http://localhost:8000/v1",
      "apiKey": "my‑local‑key",
      "label": "Local Llama",
      "models": [
        "llama-7b",
        { "model": "llama-13b", "fallbackEnabled": false }
      ]
    }
  ],
  "models": [
    {
      "platform": "openai",
      "modelId": "gpt-4",
      "displayName": "GPT‑4 (high‑quality)",
      "intelligenceRank": 900,
      "speedRank": 300,
      "monthlyTokenBudget": "200M",
      "enabled": true
    }
  ],
  "fallback": [
    { "platform": "openai", "modelId": "gpt-4", "priority": 1 },
    { "platform": "custom", "modelId": "llama-7b", "priority": 2, "enabled": true }
  ],
  "routing": {
    "strategy": "smartest",
    "weights": { "reliability": 0.7, "speed": 0.2, "intelligence": 0.1 }
  }
}

In this example:

• The keys array registers a production OpenAI credential.
• customProviders adds a local Llama instance with two models, excluding the second from fallback.
• models overrides the GPT-4 catalog entry with specific rankings and a 200M token budget.
• fallback establishes a priority order with GPT-4 first.
• routing selects the smartest strategy with custom scoring weights.

Loading the Config: Environment Setup

To activate your configuration, create a .env file in the project root and specify the path:

FREEAPI_CONFIG_PATH=./my-config.json

Alternatively, embed the configuration directly as an inline string:

FREEAPI_CONFIG_JSON='{"keys":[{"platform":"openai","key":"sk-..."}]}'

When the server boots, the bootstrap code reads the environment variable, loads the JSON, validates it against the Zod schema, and applies the changes automatically.

Verifying Applied Configuration

You can confirm your settings loaded correctly through three methods:

1. Server logs – On startup, the console prints a summary: [config] applied ./my-config.json: 1 keys, 2 custom models, 1 model edits, 2 fallback rows, routing.

2. Settings endpoint – Query /settings to view the effective Fusion config stored in the database.

3. Fallback endpoint – Inspect /fallback to verify the current fallback chain order matches your config file.

Summary

• FreeLLMAPI discovers configuration via FREEAPI_CONFIG_PATH or FREEAPI_CONFIG_JSON environment variables, processed by readConfigFromEnv in declarative-config.ts.
• The config file supports five sections: keys, customProviders, models, fallback, and routing, validated against a Zod schema at lines 72–85.
• The applyDeclarativeConfig function (lines 38–79) persists settings to SQLite on startup, making them available to all routes immediately.
• Changes require a server restart to take effect, but no code modifications are necessary.

Frequently Asked Questions

Where does FreeLLMAPI look for the config file?

The server checks the FREEAPI_CONFIG_PATH environment variable for a file path, or FREEAPI_CONFIG_JSON for an inline JSON string. This logic resides in server/src/services/declarative-config.ts at lines 12–19. If neither variable is defined, the server starts with empty defaults.

What happens if the config file has invalid JSON?

The Zod schema validation in declarative-config.ts (lines 72–85) will throw an error during startup, preventing the server from booting with malformed configuration. Check the console output for specific validation errors indicating which field failed the schema check.

Can I add new custom providers without restarting the server?

No. The applyDeclarativeConfigFromEnv function runs exactly once during server initialization (lines 80–88). To add or modify providers, you must update the JSON file and restart the server process for the changes to be applied to the database.

How do I exclude a model from the fallback chain?

Set fallbackEnabled: false on the specific model entry within the customProviders or models section. This property is respected by the fallback initialization logic in applyDeclarativeConfig, ensuring the router skips that model when primary choices fail.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}