# Why FreeLLMAPI Uses Family-Based Failover for Embeddings and Avoids Cross-Model Routing

> FreeLLMAPI uses family-based failover for embeddings, avoiding cross-model routing to ensure vector compatibility for downstream vector stores and similarity searches.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: architecture
- Published: 2026-06-25

---

**FreeLLMAPI implements family-based failover for embeddings to prevent silent routing to incompatible vector spaces, ensuring that embedding vectors remain compatible with downstream vector stores and similarity searches.**

FreeLLMAPI, the open-source multi-provider LLM gateway hosted at `tashfeenahmed/freellmapi`, treats embedding requests with stricter consistency requirements than chat completions. While chat models can often substitute one another without breaking downstream logic, embeddings require **family-based failover** to maintain vector space integrity. This architectural decision prevents silent failures that would corrupt similarity searches and retrieval-augmented generation (RAG) pipelines.

## The Problem with Cross-Model Routing for Embeddings

Embedding models generate vectors that exist in specific, incompatible semantic spaces. Unlike text completions where different models produce semantically similar outputs, embedding vectors from different models differ in **dimensionality**, **scaling**, and **mathematical meaning**.

If FreeLLMAPI were to route an embedding request to a different model during failover, the resulting vectors would be unusable. Downstream systems—such as vector stores, similarity search algorithms, and RAG pipelines—expect consistent vector representations. A sudden shift in embedding space would break distance calculations and similarity metrics without any error indication.

## How Family-Based Failover Works

FreeLLMAPI isolates embedding routing to a single **family**—a combination of model identity and its output dimension (e.g., `gemini-embedding-001`). This approach guarantees that all failover attempts stay within the same vector space.

### Defining the Family Unit

The routing unit is a *family* rather than an individual provider endpoint. As documented in lines 1‑5 of [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts), embeddings "cannot fail over across models" because "vectors from different models live in incompatible spaces." Instead, the system uses the **family** as the routing boundary, ensuring that every provider in the failover chain serves identical dimensional output.

### The Failover Process

When `runEmbeddings` receives a request, it first resolves the target family using the internal `resolveFamily` function. The system then iterates only through providers configured for that specific family. For example, if you request `gemini-embedding-001`, the failover mechanism walks through providers serving that exact family—not alternative embedding models with different dimensions.

### Explicit Failure Over Silent Switching

If no provider for the requested family is healthy, FreeLLMAPI throws an `EmbeddingsError` rather than silently switching to an incompatible model. This deterministic failure mode protects vector store integrity and alerts developers immediately when their specified embedding space is unavailable.

## Implementation in the Source Code

The embedding service explicitly documents this architecture in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts). Lines 1‑5 state that the "routing unit is a 'family' … and failover only walks the providers serving that same family." This contrasts sharply with the generic routing logic found in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), which handles chat completions with broader substitution rules.

When using the default routing logic (`model: "auto"` or undefined), the system selects the **default family** (typically `gemini-embedding-001`) and provides cross-provider redundancy strictly within that family, as implemented in lines 7‑9 of the same file. The [`model-groups.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/model-groups.ts) file defines these families and their groupings, which the embedding service references to validate provider compatibility.

## Practical Usage Examples

The following TypeScript examples demonstrate how `runEmbeddings` enforces family consistency across different invocation patterns:

```typescript
import { runEmbeddings } from '@/server/src/services/embeddings';

// Use the default family (auto) – will pick any healthy provider that serves
// the default embedding family.
const resultAuto = await runEmbeddings(undefined, [
  'The quick brown fox jumps over the lazy dog',
]);

// Explicitly request a specific family – only providers for that family are tried.
const resultFamily = await runEmbeddings('gemini-embedding-001', [
  'Free LLMAPI guarantees consistent vector spaces.',
]);

// Supplying a provider‑specific model ID is also allowed; it resolves to its family.
const resultModelId = await runEmbeddings('openai-embedding-ada-002', [
  'Avoid cross‑model routing for embeddings.',
]);

```

In each scenario, the function resolves the **family** first, then attempts failover only across providers serving that exact family. If all providers fail, the function throws an `EmbeddingsError` rather than returning vectors from an alternative model.

## Summary

- **Family-based failover** restricts embedding requests to providers serving the exact same model family and dimension, preventing vector space contamination.
- FreeLLMAPI avoids cross-model routing for embeddings because vectors from different models live in incompatible semantic spaces with different dimensionalities.
- The system fails explicitly with an `EmbeddingsError` when no provider is available for a specific family, rather than silently switching to an incompatible embedding model.
- Implementation resides primarily in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts), with family definitions in [`server/src/services/model-groups.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/model-groups.ts), contrasting with the more flexible routing in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts).

## Frequently Asked Questions

### What is a "family" in FreeLLMAPI's embedding service?

A **family** is a routing unit that combines a specific model identity with its output dimension, such as `gemini-embedding-001`. FreeLLMAPI uses families to ensure that all failover attempts remain within the same vector space, preventing dimensional or semantic mismatches that would occur if requests routed to different embedding models.

### Why can't embedding models use the same routing logic as chat completions?

Chat completions produce human-readable text where different models can provide substitutable outputs, but **embedding vectors** from different models occupy incompatible mathematical spaces with varying dimensions and scaling. Cross-model routing would silently corrupt downstream vector stores and similarity searches, whereas chat substitutions typically remain functionally valid.

### What happens if no provider is available for a specific embedding family?

FreeLLMAPI throws an `EmbeddingsError` rather than routing to an alternative model family. This explicit failure mechanism ensures developers receive immediate notification when their required embedding space is unavailable, protecting the integrity of RAG pipelines and vector databases that depend on consistent vector representations.

### How does FreeLLMAPI handle provider-specific model IDs?

When you supply a provider-specific model ID like `openai-embedding-ada-002`, the `runEmbeddings` function uses the internal `resolveFamily` utility to map that ID to its corresponding family. The system then applies family-based failover across all providers serving that family, maintaining vector consistency even when using provider-specific identifiers.