# How Vector Embedding Generation Works Across AI Providers in Open Notebook

> Learn how Open Notebook generates vector embeddings across AI providers like OpenAI and Anthropic. Discover its provider-agnostic pipeline for consistent results.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-29

---

**Open Notebook implements a provider-agnostic embedding pipeline that abstracts AI provider differences through the `ModelManager` class, unifies API calls in [`embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/embedding.py), and handles long documents via intelligent chunking with mean pooling to ensure consistent vector geometry across OpenAI, Anthropic, Grok, and other supported providers.**

Open Notebook creates vector embeddings using a flexible, provider-agnostic architecture that decouples the embedding logic from specific AI services. This system allows researchers to switch between embedding providers without modifying application code, while automatically handling token limits through intelligent batching and chunking strategies implemented in the `lfnovo/open-notebook` repository.

## Provider-Agnostic Model Resolution

The embedding pipeline begins in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), where the `ModelManager` class resolves the correct embedding model and provider configuration dynamically.

### How ModelManager Resolves Embedding Models

The `ModelManager.get_embedding_model()` method reads the default embedding model identifier from the database via `DefaultModels.default_embedding_model`. It then instantiates the concrete `EmbeddingModel` class from the Esperanto library, which provides a unified interface for multiple providers. This abstraction allows the same code to call any supported provider—whether OpenAI, Anthropic, or Grok—without changing the underlying embedding logic.

### Credential Handling and Environment Fallbacks

Credentials are extracted from the `Model` database record when available. If no credentials are stored in the record, the system falls back to environment variables. This dual approach ensures that sensitive API keys can be managed either through the database or via secure environment configuration, maintaining flexibility for different deployment scenarios.

## Unified Embedding Pipeline

All embedding calls route through [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py), which provides three core functions that standardize vector generation regardless of the underlying provider.

### Batch Processing with Retry Logic

The `generate_embeddings(texts: List[str])` function handles bulk embedding requests efficiently. It partitions the input list into batches sized according to the `EMBEDDING_BATCH_SIZE` configuration variable (defaulting to 50), configurable via the `OPEN_NOTEBOOK_EMBEDDING_BATCH_SIZE` environment variable. Each batch is sent to the provider's asynchronous `aembed` method. If a provider returns an error, the function implements retry logic using `EMBEDDING_MAX_RETRIES`, ensuring robustness against transient network failures or rate limiting.

### Token-Aware Chunking for Large Documents

For single-text inputs, `generate_embedding(text: str)` first checks the token count against `CHUNK_SIZE` (approximately 2,000 tokens by default). If the text exceeds this limit, the function calls `chunk_text` from [`open_notebook/utils/chunking.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/chunking.py), which splits the document according to its `ContentType` (PDF, HTML, or plain text). Each chunk is embedded separately, allowing the system to process documents of arbitrary length regardless of the provider's context window limitations.

### Mean Pooling for Consistent Vector Geometry

When chunking occurs, the resulting per-chunk vectors must be combined into a single document representation. The `mean_pool_embeddings` function normalizes each individual embedding to unit length, calculates the arithmetic mean across all chunks, and renormalizes the final vector. This technique guarantees that embeddings generated from large documents maintain the same geometric properties—specifically vector length and scale—as those generated from single-chunk embeddings, ensuring consistent similarity calculations in the SurrealDB vector store.

## Service Layer Integration

The front-end and CLI interfaces interact with the embedding system through `EmbeddingService` in [`api/embedding_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/embedding_service.py). This service layer provides a thin wrapper around the unified utilities, forwarding requests to the FastAPI client via `api_client.embed_content`. The service itself does not implement embedding logic; instead, it acts as a bridge between the API endpoints and the core embedding functions, maintaining separation of concerns between the transport layer and the AI provider logic.

## Complete Embedding Flow Across Providers

The end-to-end vector embedding generation follows this structured sequence:

1. **Model Selection** – `ModelManager` queries the default embedding model ID, loads the corresponding `Model` record, extracts credentials or falls back to environment variables, and constructs an Esperanto `EmbeddingModel` instance for the chosen provider.

2. **Input Preparation** – `generate_embedding` receives raw text and counts tokens. If the count exceeds `CHUNK_SIZE`, `chunk_text` splits the document into appropriate segments based on content type.

3. **Batch Embedding** – The text list (either original or chunked) passes to `generate_embeddings`, which creates batches of size `EMBEDDING_BATCH_SIZE`. Each batch is sent asynchronously via `embedding_model.aembed`, with automatic retry on failure up to `EMBEDDING_MAX_RETRIES`.

4. **Post-Processing** – For chunked inputs, `mean_pool_embeddings` combines per-chunk vectors through normalization, averaging, and renormalization, producing a single vector that represents the entire document.

5. **Storage** – The final normalized vector is stored in SurrealDB alongside the source record, enabling semantic search through the `vector search` endpoint.

Because the actual HTTP calls are encapsulated inside the Esperanto `EmbeddingModel`, adding a new provider requires only a new credential record or environment variable configuration; the remainder of the pipeline remains unchanged.

## Summary

- **Provider-agnostic architecture** – The `ModelManager` class abstracts provider-specific details, allowing seamless switching between OpenAI, Anthropic, Grok, and other providers without code changes.
- **Robust batch processing** – Built-in batching and retry logic in `generate_embeddings` protect against rate limits and transient failures.
- **Intelligent document handling** – Token-aware chunking and mean pooling ensure consistent embeddings regardless of document length or provider context limits.
- **Unified interface** – All embedding operations route through [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py), providing a single API surface for the entire application.

## Frequently Asked Questions

### How does Open Notebook handle different embedding dimensions across providers?

Open Notebook relies on the Esperanto library's `EmbeddingModel` interface to normalize provider-specific outputs. While different providers may return vectors of varying dimensions (e.g., 1,536 for OpenAI's `text-embedding-3-small` versus 1,024 for other models), the system stores these verbatim in SurrealDB. The vector search operates on the specific dimensionality of the active model, ensuring that all embeddings in a single collection use the same provider and dimension configuration.

### What happens if an embedding request fails temporarily?

The `generate_embeddings` function implements retry logic using `EMBEDDING_MAX_RETRIES`. When a provider returns an error—whether from rate limiting, network issues, or temporary service unavailability—the system waits briefly and retries the specific batch that failed. This protects long-running embedding jobs from failing completely due to transient provider issues.

### How does the system handle documents longer than the model's context window?

Documents exceeding the `CHUNK_SIZE` threshold (approximately 2,000 tokens) are automatically split using the `chunk_text` utility in [`open_notebook/utils/chunking.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/chunking.py). Each chunk is embedded separately, and the resulting vectors are combined using `mean_pool_embeddings`, which normalizes and averages the vectors to create a single document representation that maintains consistent geometric properties with shorter documents.

### Can I use a custom embedding provider not officially supported?

Yes, because the architecture uses the Esperanto library's abstraction layer, you can add support for new providers by implementing the appropriate `EmbeddingModel` interface in Esperanto and configuring the corresponding credentials in the `Model` record or environment variables. The `ModelManager` will resolve the new provider automatically, and the existing batching, chunking, and pooling logic in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py) will work without modification.