LLM Providers Supported by FreeLLMAPI: Complete Integration Guide

FreeLLMAPI aggregates free-tier models from 17+ major providers including Google Gemini, Groq, Mistral, and OpenRouter into a single OpenAI-compatible endpoint, automatically routing requests to healthy keys with available quota.

FreeLLMAPI is an open-source proxy server that unifies access to free-tier large language model APIs behind one standardized interface. The project implements provider-specific adapters and intelligent routing logic to eliminate the complexity of managing multiple API keys and endpoints. The complete list of supported providers is defined in the project documentation and implemented across the TypeScript adapter modules in the tashfeenahmed/freellmapi repository.

Supported LLM Providers

The repository supports a diverse ecosystem ranging from hyperscale cloud platforms to specialized inference services and community endpoints. Each provider integrates through a dedicated adapter that handles request translation, authentication, and response normalization.

Major Cloud Providers

• Google Gemini: Implemented in server/src/providers/google.ts, supporting Gemini 2.5 Flash and 3.x preview models.
• Groq: High-performance inference for Llama 3.3, Llama 4, GPT-OSS, and Qwen3 architectures.
• Cerebras: Access to Qwen3 235B through wafer-scale infrastructure.
• Mistral: Large 3, Medium 3.5, Codestral, and Devstral model families.
• Cohere: Command R+ and Command-A trial access via the adapter in server/src/providers/cohere.ts.

Aggregation Platforms

• OpenRouter: Aggregates 21 free-tier models from multiple sources, implemented in server/src/providers/openrouter.ts.
• OpenCode Zen: DeepSeek V4 Flash and promotional Nemotron access.
• GitHub Models: GPT-4.1 and GPT-4o for development and testing workflows.

Cloud and Edge Providers

• Cloudflare Workers AI: Kimi K2, GLM-4.7, GPT-OSS, and Granite 4 inference at the edge.
• OVH AI Endpoints: Qwen3.5 397B, GPT-OSS, and Llama 3.3 via anonymous routes.
• Z.ai (Zhipu): GLM-4.5 and GLM-4.7 Flash model access.

Developer and Community Routes

• HuggingFace Inference: DeepSeek V4, Kimi K2.6, and Qwen3 through the inference API.
• Ollama Cloud: GLM-4.7, Kimi K2, and GPT-OSS models.
• NVIDIA NIM: 40 RPM free evaluation tier for testing NIM deployments.
• Kilo Gateway: Anonymous free routes without authentication requirements.
• Pollinations: GPT-OSS 20B via anonymous access.
• LLM7: GPT-OSS, Llama 3.1, and GLM model access.

Custom OpenAI-Compatible Endpoints

The server/src/providers/openai-compat.ts adapter template enables integration with any OpenAI-compatible API, including local deployments such as llama.cpp, LM Studio, vLLM, and local Ollama instances.

Architecture and Routing

The system decouples provider-specific implementations from request handling through a clean adapter pattern and intelligent routing layer that maximizes free-tier utilization.

Provider Adapters

Each supported provider implements the abstract Provider class defined in server/src/providers/base.ts. Adapters handle translation of OpenAI request formats to provider-native APIs, response normalization, and authentication header management. Specific implementations include google.ts for Gemini, cohere.ts for Cohere, and openrouter.ts for OpenRouter aggregation.

Router Implementation

The server/src/services/router.ts module contains the core selection logic that evaluates provider health, quota availability, and priority to determine the optimal target for each request. When a provider returns a 429 or 5xx error, the router automatically fails over to the next available provider in the pool.

Rate Limiting and Health Monitoring

The server/src/services/ratelimit.ts maintains an in-memory ledger tracking RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), and TPD (tokens per day) counters per API key, with SQLite persistence for durability across restarts. The server/src/services/health.ts service runs periodic probes against each configured key to update availability status before routing decisions are made.

Usage Examples

Query any supported provider through the unified endpoint at http://localhost:<PORT>/v1.

Python example using the OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

cURL example:

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "hi"}]
  }'

Both examples automatically route to any available provider from the supported list, with fallback handling managed transparently by the router.

Summary

• FreeLLMAPI supports 17+ distinct LLM providers including Google Gemini, Groq, Cerebras, Mistral, OpenRouter, and Cloudflare Workers AI.
• Each provider integrates through dedicated TypeScript adapters in server/src/providers/ implementing the abstract Provider class from base.ts.
• The router in server/src/services/router.ts dynamically selects healthy providers with available quota, automatically failing over on 429 or 5xx errors.
• Rate limiting is enforced per-key via server/src/services/ratelimit.ts with persistent SQLite storage for RPM/RPD/TPM/TPD tracking.
• Custom OpenAI-compatible endpoints can be added using the template in server/src/providers/openai-compat.ts.

Frequently Asked Questions

How many LLM providers does FreeLLMAPI support?

FreeLLMAPI officially supports 17+ providers including major platforms like Google Gemini, Groq, Cerebras, and Mistral, plus aggregation services like OpenRouter and GitHub Models. The project also supports custom OpenAI-compatible endpoints through a configurable adapter template.

Can I use local LLMs with FreeLLMAPI?

Yes. The server/src/providers/openai-compat.ts adapter allows integration with any OpenAI-compatible local server including llama.cpp, LM Studio, vLLM, and local Ollama instances. Configure these as custom providers in your configuration file to include them in the routing pool.

How does FreeLLMAPI handle rate limits across providers?

The server/src/services/ratelimit.ts service tracks RPM, RPD, TPM, and TPD counters for each API key in an in-memory ledger with SQLite persistence. When a provider returns a 429 error or exceeds quota, the router automatically excludes that key from selection until the cooldown period expires, ensuring seamless failover to other providers.

Is there a specific adapter file for each provider?

Yes. The codebase follows a modular architecture where each provider has a dedicated adapter file in server/src/providers/, such as google.ts for Gemini, cohere.ts for Cohere, and openrouter.ts for OpenRouter. All adapters extend the abstract Provider class defined in server/src/providers/base.ts to ensure consistent interface implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}