What Are Unified Models in FreeLLMAPI?

TLDR: In FreeLLMAPI, unified models collapse multiple provider-specific implementations of the same logical model (such as GLM-4.7 on Cloudflare and Z.ai) into a single entry, enabling strict in-group failover between providers while exposing one canonical name in the /v1/models endpoint.

FreeLLMAPI simplifies access to free-tier LLMs by abstracting provider-specific endpoints into a single, cohesive interface called unified models. This architecture eliminates the need to manage separate API connections for every provider hosting the same underlying model. This guide explains how unified models work in the tashfeenahmed/freellmapi repository, referencing the actual implementation and source files.

Understanding Unified Models in FreeLLMAPI

Unified models solve the fragmentation problem of identical models appearing under different provider namespaces. According to docs/architecture.md (lines 50-52), the system handles this as follows:

"- Unified models — the same logical model served by several providers (say, GLM-4.7 on Cloudflare and Z.ai) collapses into one entry: one name in /v1/models, strict in-group failover between its providers, and merge/split overrides when the grouping guesses wrong."

When you query a unified model, you reference a single logical name like glm-4.7 rather than managing separate connections to cloudflare/glm-4.7 and z.ai/glm-4.7. The router transparently distributes requests across available providers and handles failures automatically.

How Unified Model Routing Works

Provider Grouping and Automatic Failover

The core routing logic resides in server/src/services/router.ts. When a request targets a unified model, the router selects a provider from the unified group based on health status and rate limit availability. If the selected provider fails, the system executes strict in-group failover, immediately retrying the request with the next available provider in the group without client intervention.

Override Controls for Edge Cases

The architecture acknowledges that automatic detection is not infallible. Administrators can apply merge/split overrides to force specific providers into or out of a unified model group. This ensures that when the heuristic grouping guesses wrong, you can manually correct the provider topology to match your requirements.

Using Unified Models in Practice

Listing Available Unified Models

Unified models appear in the standard OpenAI-compatible model listing endpoint. Use your unified key to see the collapsed model names:

curl http://localhost:3001/v1/models \
     -H "Authorization: Bearer freellmapi-your-unified-key"

The response contains unified entries like "glm-4.7" rather than separate provider-specific identifiers.

Making Requests with Automatic Routing

Reference the unified model name to leverage automatic provider selection and failover:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="glm-4.7",
    messages=[{"role": "user", "content": "Explain quantum tunnelling"}],
)
print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

The x-routed-via header indicates which specific provider served the request (e.g., cloudflare/glm-4.7 or z.ai/glm-4.7), providing transparency into the routing decision.

Pinning Specific Providers

To bypass automatic failover and target a specific backend, use the provider-prefixed format:

curl http://localhost:3001/v1/chat/completions \
     -H "Authorization: Bearer freellmapi-your-unified-key" \
     -H "Content-Type: application/json" \
     -d '{
           "model": "cloudflare/glm-4.7",
           "messages": [{"role":"user","content":"Summarise the plot of Inception"}]
         }'

Implementation Files

The unified model feature spans several critical files in the codebase:

• docs/architecture.md — Defines the conceptual framework and behavior of unified models (lines 50-52).
• server/src/services/router.ts — Implements provider selection and in-group failover logic.
• server/src/lib/system-prompt.ts — Defines the internal representation where unified models are marked with kind: 'unified'.
• server/src/routes/models-management.test.ts — Verifies that unified models appear correctly in /v1/models and that unified keys authenticate properly.

Summary

• Unified models collapse identical models from multiple providers into a single logical entry exposed in /v1/models.
• server/src/services/router.ts implements strict in-group failover between providers within a unified model group.
• Use simple model names (e.g., glm-4.7) for automatic routing or provider-prefixed names (e.g., cloudflare/glm-4.7) to pin specific backends.
• The x-routed-via header reveals which provider actually served a given request.
• Merge/split overrides allow manual correction of provider groupings when automatic detection is incorrect.

Frequently Asked Questions

What is the difference between a unified model and a provider-specific model?

A unified model represents the same logical model across multiple providers under a single name (like glm-4.7), while provider-specific models use qualified names (like cloudflare/glm-4.7) that target exactly one backend. Unified models enable automatic failover and load distribution, whereas provider-specific models bypass the router's grouping logic to pin requests to a single source.

How does FreeLLMAPI handle failover for unified models?

According to the implementation in server/src/services/router.ts, when a provider in a unified group returns an error or becomes unavailable, the router performs strict in-group failover. It automatically retries the request with the next healthy provider in the unified group without requiring client-side changes or exposing the failure to your application.

Can I override which providers are grouped into a unified model?

Yes. While FreeLLMAPI automatically detects when the same logical model appears on multiple providers, the architecture supports merge/split overrides. These allow administrators to manually correct the grouping when the automatic detection guesses wrong, ensuring providers are categorized according to your specific routing requirements.

How can I see which provider actually served my request?

Check the x-routed-via response header. When using a unified model, this header indicates the specific provider endpoint that handled the request (e.g., z.ai/glm-4.7), allowing you to track routing decisions, monitor provider performance, and diagnose provider-specific issues.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}