# How FreeLLMAPI Embeddings Work and Why They Cannot Fail Over Across Different Models

> Learn how FreeLLMAPI embeddings work and why failover is restricted to the same vector space. Discover how this prevents downstream data corruption.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: deep-dive
- Published: 2026-06-29

---

**FreeLLMAPI treats embeddings as a family-scoped service where fail-over is restricted to providers within the same vector space, preventing the corruption of downstream vector stores that would occur if incompatible embedding models were mixed.**

FreeLLMAPI is an open-source LLM gateway that unifies multiple providers under a single API. Unlike chat completions, which can transparently fail over between different models, FreeLLMAPI embeddings operate under strict family-based constraints to ensure vector consistency. Understanding this architecture is critical for building reliable RAG pipelines and semantic search systems.

## The Core Constraint: No Cross-Model Failover

The fundamental reason FreeLLMAPI embeddings cannot fail over across different models is that **vectors from different models live in incompatible spaces**. Silently switching models would corrupt any vector store built on top of the API, as downstream algorithms assume a consistent metric space for similarity search and clustering.

This rule is explicitly enforced at the top of the embeddings service in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts):

```typescript
// Embeddings routing. Unlike chat, embeddings can NOT fail over across models:
// vectors from different models live in incompatible spaces, and silently
// switching models would corrupt any vector store built on top of us.

```

Mixing vectors from two different families—such as OpenAI `text-embedding-3-small` and Google `gemini-embedding-001`—would produce meaningless distance calculations, rendering semantic search results useless.

## Family-Based Routing Architecture

FreeLLMAPI implements a **family-scoped routing system** that groups providers serving the exact same vector space. This ensures that every embedding returned in a single request shares the same dimensionality and semantic encoding.

### Resolving the Target Family

When a request arrives, the `resolveFamily()` function determines which vector space to use. The client can pass `model: "auto"` (or omit the field), which resolves to the **default family** (e.g., `gemini-embedding-001`), or specify a concrete model identifier.

In [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts), the resolution logic works as follows:

```typescript
export function resolveFamily(model: string | undefined): string | null {
  if (!model || model === 'auto') return getDefaultFamily();          // ← default family
  const rows = listEmbeddingModels();
  if (rows.some(r => r.family === model)) return model;               // ← family name
  const byModelId = rows.find(r => r.model_id === model);
  return byModelId?.family ?? null;                                   // ← lookup by model id
}

```

If the provided string matches a family name directly, it returns that family; otherwise, it looks up the model ID to find its corresponding family.

### Building the Provider Chain

Once the family is resolved, `runEmbeddings()` constructs a **provider chain** containing only enabled rows belonging to that specific family, ordered by priority:

```typescript
const chain = getDb()
  .prepare('SELECT * FROM embedding_models WHERE family = ? AND enabled = 1 ORDER BY priority')
  .all(family) as EmbeddingModelRow[];

```

This SQL query ensures that fail-over candidates are strictly limited to providers within the same family, guaranteeing dimensional consistency.

### Executing the Failover Loop

The service iterates over the provider chain, attempting each credential in sequence. If a provider returns an error, the next provider **in the same family** is tried. No provider from a different family will ever be considered:

```typescript
for (const row of chain) {
  const credential = getProviderCredential(row);
  if (!credential) continue;                     // skip if no usable key
  try {
    const out = await callProvider(row, credential, inputs, dimensions);
    // ... processing ...
    return { /* EmbeddingsResult */ };
  } catch (err) {
    // remember the error then keep trying the next provider of the same family
  }
}

```

This implementation in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts) (lines 68-94) ensures that while individual providers may fail, the semantic integrity of the vector space remains intact.

## Practical Implementation Examples

### Using the HTTP API

The embeddings endpoint is exposed via `embeddingsRouter` in [`server/src/routes/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/embeddings.ts) and accepts requests with either a specific family or `"auto"`:

```http
POST /api/embeddings HTTP/1.1
Host: your-freellmapi-instance.com
Authorization: Bearer <your-api-key>
Content-Type: application/json

{
  "model": "auto",
  "input": [
    "What is the capital of France?",
    "Explain quantum entanglement in plain language."
  ]
}

```

The response includes the specific family used and the resulting vectors:

```json
{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.12, -0.05, ...] },
    { "object": "embedding", "index": 1, "embedding": [0.09, 0.33, ...] }
  ],
  "model": "gemini-embedding-001",
  "usage": { "prompt_tokens": 42 }
}

```

### Direct Service Invocation

For backend integrations, import `runEmbeddings()` directly:

```typescript
import { runEmbeddings } from './services/embeddings.js';

async function getEmbeddings() {
  const texts = [
    'Summarize the plot of "Hamlet".',
    'What is the boiling point of water?'
  ];

  // Resolves to default family (e.g., gemini-embedding-001)
  const result = await runEmbeddings('auto', texts);
  
  console.log('Family:', result.family);
  console.log('Dimensions:', result.dimensions);
  console.log('Vectors:', result.vectors);
}

```

The function returns an `EmbeddingsResult` object containing the **family**, **platform**, **modelId**, **dimensions**, **vectors**, and **token usage**, ensuring full traceability of which provider generated the embeddings.

## Summary

- **FreeLLMAPI embeddings are family-scoped**: Unlike chat completions, embeddings cannot fail over across different models because vectors from different families occupy incompatible semantic spaces.

- **Family resolution is strict**: The `resolveFamily()` function in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts) maps requests to specific vector spaces, supporting `"auto"` resolution or explicit family/model IDs.

- **Failover is intra-family only**: The provider chain built by `runEmbeddings()` queries `embedding_models` filtered by family, ensuring all fallback providers share identical dimensionality.

- **Vector store integrity is protected**: By preventing cross-model failover, FreeLLMAPI guarantees that all embeddings in a request share the same metric space, preserving the accuracy of similarity search and RAG pipelines.

## Frequently Asked Questions

### What happens if I request `"model": "auto"` for embeddings?

When you specify `"model": "auto"` or omit the model field entirely, FreeLLMAPI calls `getDefaultFamily()` to resolve the request to the configured default embedding family (typically `gemini-embedding-001`). This ensures consistent vector generation while allowing administrators to switch default providers globally without changing client code.

### Can I force FreeLLMAPI to use a specific embedding provider?

Yes, you can pass a specific model ID (e.g., `"text-embedding-3-small"`) or family name (e.g., `"gemini-embedding-001"`) in the request. The `resolveFamily()` function will validate this against the `embedding_models` table and restrict failover to other providers within that same family only.

### Why do embedding vectors from different models produce incompatible results?

Different embedding models are trained on distinct architectures and datasets, resulting in **different vector spaces** with unique dimensionalities and semantic mappings. A vector from OpenAI's `text-embedding-3-small` represents concepts in a fundamentally different coordinate system than Google's `gemini-embedding-001`. Mixing these in the same vector store would cause similarity calculations to return random distances, corrupting search results.

### How does FreeLLMAPI handle provider errors within the same family?

If a provider in the chain returns an error or lacks valid credentials, `runEmbeddings()` catches the exception and continues to the next provider in the priority-ordered chain. This intra-family failover mechanism ensures high availability while maintaining the strict constraint that all attempted providers belong to the same vector space, as enforced by the SQL query in [`server/src/services/embeddings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/embeddings.ts).