How to Configure Per-Provider Timeouts in FreeLLMAPI: A Complete Guide

FreeLLMAPI allows you to set custom HTTP timeouts for each LLM provider individually through constructor options or environment variables.

FreeLLMAPI is a lightweight, open-source unified API for multiple LLM providers. Controlling per-provider timeouts ensures your application fails fast when a backend is slow or unresponsive, without impacting other providers. This article walks through the implementation details and configuration options based on the current source code.

Understanding the Timeout Architecture

The timeout system in FreeLLMAPI follows a hierarchical resolution strategy. All providers ultimately rely on AbortSignal.timeout() to abort stalled requests, but the timeout value itself is determined through two layers of configuration.

Core Timeout Resolution Logic

The central helper providerTimeoutMs in server/src/lib/provider-timeout.ts handles all timeout lookups:

// server/src/lib/provider-timeout.ts
export function providerTimeoutMs(
  platform: string,
  fallbackMs: number,
): number {
  const envVar = `FREELLMAPI_TIMEOUT_${platform.toUpperCase()}`;
  const envTimeout = process.env[envVar];
  if (envTimeout) return Number(envTimeout);
  return fallbackMs;
}

This function implements a priority-based fallback:

1. Check FREELLMAPI_TIMEOUT_<PLATFORM> environment variable (uppercase)
2. Fall back to the timeoutMs value passed in constructor options
3. Use the hardcoded default if neither is defined

Method 1: Configure Timeouts in Code

Every built-in provider accepts a timeoutMs option during instantiation. The provider registration file server/src/providers/index.ts demonstrates the pattern used throughout the codebase:

// server/src/providers/index.ts
register(new GoogleProvider({ timeoutMs: 60_000 }));
register(new ZhipuProvider({ timeoutMs: 60_000 }));
register(new OllamaProvider({ timeoutMs: 120_000 }));

Individual providers consume this option through the providerTimeoutMs helper. For example, the OpenAI-compatible provider implementation in server/src/providers/openai-compat.ts shows the typical constructor pattern:

constructor(opts: { platform: string; timeoutMs?: number }) {
  this.timeoutMs = providerTimeoutMs(opts.platform,
                     opts.timeoutMs ?? 60_000);
}

Override a Built-in Provider Timeout

To increase Google Gemini's timeout to 90 seconds, modify the registration:

// server/src/providers/index.ts
import { GoogleProvider } from './google';

// Override default 60s timeout
register(new GoogleProvider({ timeoutMs: 90_000 }));

Add a Custom Provider with Configurable Timeout

When implementing custom providers, follow the same pattern using providerTimeoutMs:

// src/providers/mycustom.ts
import { BaseProvider } from '../lib/base-provider';
import { providerTimeoutMs } from '../lib/provider-timeout';

export class MyCustomProvider extends BaseProvider {
  private readonly timeoutMs: number;

  constructor(opts: { timeoutMs?: number } = {}) {
    super();
    // 'mycustom' becomes the PLATFORM identifier
    this.timeoutMs = providerTimeoutMs('mycustom', opts.timeoutMs ?? 45_000);
  }

  async chatCompletion(messages, model, opts) {
    const signal = AbortSignal.timeout(opts?.timeoutMs ?? this.timeoutMs);
    // fetch request with signal...
  }
}

// Register with 70 second timeout
register(new MyCustomProvider({ timeoutMs: 70_000 }));

Method 2: Configure Timeouts via Environment Variables

For runtime configuration without code changes, set FREELLMAPI_TIMEOUT_<PROVIDER> variables. The underscore-delimited provider name must be uppercase.

Environment Variable Examples

Add to your .env file, Docker environment, or CI configuration:

FREELLMAPI_TIMEOUT_GOOGLE=120000      # 2 minutes for Google/Gemini

FREELLMAPI_TIMEOUT_ZHIPU=50000        # 50 seconds for Zhipu

FREELLMAPI_TIMEOUT_OLLAMA=180000      # 3 minutes for Ollama

FREELLMAPI_TIMEOUT_OPENAI=30000       # 30 seconds for OpenAI

Environment variables take precedence over constructor options. This enables quick adjustments during incident response without redeployment.

Key Files and Their Roles

File	Purpose
server/src/lib/provider-timeout.ts	Core providerTimeoutMs(platform, fallbackMs) helper that resolves timeout values
server/src/providers/index.ts	Provider registry where default timeoutMs values are configured
server/src/providers/openai-compat.ts	Reference implementation showing timeout option consumption
.env.example	Template demonstrating FREELLMAPI_TIMEOUT_<PROVIDER> naming convention

Summary

• Per-provider timeouts in FreeLLMAPI are resolved through providerTimeoutMs() in server/src/lib/provider-timeout.ts
• Two configuration methods: constructor timeoutMs option or FREELLMAPI_TIMEOUT_<PLATFORM> environment variable
• Environment variables override code-level defaults for rapid runtime adjustment
• All providers use AbortSignal.timeout() for consistent request cancellation behavior
• Platform identifiers are case-insensitive in code but uppercased for environment variable names

Frequently Asked Questions

What happens if both environment variable and constructor option are set?

The environment variable wins. providerTimeoutMs() checks process.env[FREELLMAPI_TIMEOUT_<PLATFORM>] first and only falls back to the constructor timeoutMs if the environment variable is undefined.

How do I find the correct PLATFORM identifier for my provider?

Check the platform string passed to providerTimeoutMs() in the provider's constructor. For built-in providers, this matches the lowercase provider name: 'google', 'zhipu', 'ollama', etc. Your environment variable becomes FREELLMAPI_TIMEOUT_GOOGLE, FREELLMAPI_TIMEOUT_ZHIPU, etc.

What is the default timeout if I don't configure anything?

Built-in providers define their own defaults in server/src/providers/index.ts. Common defaults are 60,000 ms for OpenAI-compatible providers and 120,000 ms for Ollama. Check the registration code for your specific provider.

Can I disable timeouts entirely?

No. FreeLLMAPI requires a timeout value to prevent indefinite hangs. If you need effectively unlimited duration, set an extremely high value like 86400000 (24 hours) via environment variable or constructor option.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}