# How to Configure the Embedding Model for RAG Functionality in GPT Academic

> Configure the embedding model for RAG functionality in gpt_academic. Learn how to set EMBEDDING_MODEL in config py, use environment variables, or apply per-request customization via llm_kwargs.

- Repository: [binary-husky/gpt_academic](https://github.com/binary-husky/gpt_academic)
- Tags: how-to-guide
- Published: 2026-03-02

---

**Configure the embedding model for RAG functionality by setting the `EMBEDDING_MODEL` variable in [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py), overriding it via the `EMBEDDING_MODEL` environment variable, or passing `embed_model` in the `llm_kwargs` dictionary for per-request customization.**

The `binary-husky/gpt_academic` repository implements Retrieval-Augmented Generation (RAG) through modular workers that rely on embedding models to convert text into vector representations. Understanding how to configure the embedding model for RAG functionality ensures your vector stores use the correct dimensions and endpoints for optimal retrieval performance.

## Three Levels of Configuration for the Embedding Model

The system provides three distinct levels at which you can specify the embedding model, each offering different scopes of control.

### Global Default Configuration

The most persistent configuration resides in [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py), where the default embedding model is defined for all RAG operations.

In [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py) (line 52), the default is set as:

```python
EMBEDDING_MODEL = "text-embedding-3-small"

```

Changing this value affects all RAG workers—including `LlamaIndexRagWorker` and `MilvusWorker`—that rely on the global configuration when no override is present.

### Environment Variable Override

For containerized deployments or temporary changes, set the `EMBEDDING_MODEL` environment variable to supersede the [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py) value.

When the application initializes, `toolbox.get_conf("EMBEDDING_MODEL")` checks for environment variables before falling back to the configuration file. This allows the same codebase to run with different models across development, staging, and production environments without modifying source files.

### Per-Request Override

For granular control, pass the `embed_model` parameter within the `llm_kwargs` dictionary when initializing RAG workers programmatically.

This method enables a single conversation or plugin invocation to use a specific embedding model—such as `text-embedding-3-large` for higher accuracy—while the rest of the system maintains the global default.

## Architecture Flow: How the Embedding Model Propagates Through the System

Understanding the data flow helps debug configuration issues and ensures the correct model is active.

1. **Configuration Loading**: `toolbox.get_conf("EMBEDDING_MODEL")` retrieves the value from [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py) or the environment variable (source: [`toolbox.py`](https://github.com/binary-husky/gpt_academic/blob/main/toolbox.py), lines 10-16).

2. **Cookie Injection**: The `ArgsGeneralWrapper` decorator stores the embedding model name in request cookies under the key `embed_model`, making it available to downstream functions (source: [`toolbox.py`](https://github.com/binary-husky/gpt_academic/blob/main/toolbox.py), lines 109-118).

3. **Worker Initialization**: `LlamaIndexRagWorker` extracts `embed_model` from `llm_kwargs` and instantiates `OpenAiEmbeddingModel` with the specified model name (source: [`crazy_functions/rag_fns/llama_index_worker.py`](https://github.com/binary-husky/gpt_academic/blob/main/crazy_functions/rag_fns/llama_index_worker.py), lines 65-70).

4. **Embedding Computation**: `OpenAiEmbeddingModel.compute_embedding` selects the appropriate OpenAI endpoint using `embed_model_info` from [`bridge_all_embed.py`](https://github.com/binary-husky/gpt_academic/blob/main/bridge_all_embed.py) (source: [`request_llms/embed_models/openai_embed.py`](https://github.com/binary-husky/gpt_academic/blob/main/request_llms/embed_models/openai_embed.py), lines 42-55).

5. **Vector Store Alignment**: The `embedding_dimension()` method ensures the vector database (Milvus or LlamaIndex) allocates the correct vector size for the chosen model (source: [`request_llms/embed_models/openai_embed.py`](https://github.com/binary-husky/gpt_academic/blob/main/request_llms/embed_models/openai_embed.py), lines 74-82).

## Practical Code Examples

### Setting the Global Default in config.py

Modify the configuration file to change the default embedding model for all RAG operations:

```python

# config.py

EMBEDDING_MODEL = "text-embedding-3-large"  # Changed from text-embedding-3-small

```

This affects all workers that do not receive an explicit override.

### Overriding via Docker Compose Environment Variables

For containerized deployments, specify the model in your [`docker-compose.yml`](https://github.com/binary-husky/gpt_academic/blob/main/docker-compose.yml):

```yaml
services:
  gpt_academic:
    image: ghcr.io/binary-husky/gpt_academic:latest
    environment:
      - EMBEDDING_MODEL=text-embedding-3-large
      - API_KEY=sk-your-openai-key

```

The environment variable takes precedence over the value in [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py).

### Per-Request Configuration in Custom Scripts

When invoking RAG functionality programmatically, pass the embedding model explicitly:

```python
from crazy_functions.rag_fns.llama_index_worker import LlamaIndexRagWorker

# Configure parameters for this specific request

llm_kwargs = {
    "api_key": "sk-your-api-key",
    "llm_model": "gpt-4o-mini",
    "embed_model": "text-embedding-3-large",  # Specific model for this session

    "temperature": 0.7,
    "max_length": 4096,
}

# Initialize worker with custom embedding model

rag_worker = LlamaIndexRagWorker(user_name="user_123", llm_kwargs=llm_kwargs)

# Add documents and query

rag_worker.add_text_to_vector_store("Retrieval-Augmented Generation combines retrieval with generation.")
results = rag_worker.retrieve_from_store_with_query("What is RAG?")

```

This approach allows different conversations to use different embedding models without restarting the application.

### Debugging: Checking Vector Store Dimensions

Verify that your vector store matches the embedding model's output dimensions:

```python

# After initializing any RAG worker

dimension = rag_worker.embed_model.embedding_dimension()
print(f"Current embedding dimension: {dimension}")

# Output: Current embedding dimension: 3072  # for text-embedding-3-large

```

This helps diagnose dimension mismatches when switching between models like `text-embedding-3-small` (1536 dimensions) and `text-embedding-3-large` (3072 dimensions).

## Key Source Files for RAG Embedding Configuration

The following files govern how embedding models are selected and utilized:

- **[`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py)** – Defines the global `EMBEDDING_MODEL` default (line 52).
- **[`toolbox.py`](https://github.com/binary-husky/gpt_academic/blob/main/toolbox.py)** – Implements `get_conf()` for configuration retrieval and `ArgsGeneralWrapper` for cookie injection (lines 10-16, 109-118).
- **[`request_llms/embed_models/openai_embed.py`](https://github.com/binary-husky/gpt_academic/blob/main/request_llms/embed_models/openai_embed.py)** – Contains `OpenAiEmbeddingModel` class with `compute_embedding()` and `embedding_dimension()` methods (lines 42-55, 74-82).
- **[`request_llms/embed_models/bridge_all_embed.py`](https://github.com/binary-husky/gpt_academic/blob/main/request_llms/embed_models/bridge_all_embed.py)** – Maps model names to endpoint URLs and dimension metadata.
- **[`crazy_functions/rag_fns/llama_index_worker.py`](https://github.com/binary-husky/gpt_academic/blob/main/crazy_functions/rag_fns/llama_index_worker.py)** – RAG worker implementation that instantiates embedding models from `llm_kwargs` (lines 65-70).
- **[`crazy_functions/rag_fns/milvus_worker.py`](https://github.com/binary-husky/gpt_academic/blob/main/crazy_functions/rag_fns/milvus_worker.py)** – Alternative RAG backend supporting the same embedding configuration interface.

## Summary

- **Global configuration** sets the default embedding model via `EMBEDDING_MODEL` in [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py), affecting all RAG workers system-wide.
- **Environment variables** override the global setting at runtime, enabling deployment-specific models without code changes.
- **Per-request overrides** via `llm_kwargs` allow individual conversations or scripts to specify custom embedding models while the system maintains its default.
- **Architecture consistency** ensures that [`toolbox.py`](https://github.com/binary-husky/gpt_academic/blob/main/toolbox.py) propagates the model name through cookies to workers like `LlamaIndexRagWorker`, which instantiate `OpenAiEmbeddingModel` and validate dimensions through `embedding_dimension()`.

## Frequently Asked Questions

### What is the default embedding model in GPT Academic?

The default embedding model is `text-embedding-3-small`, defined in [`config.py`](https://github.com/binary-husky/gpt_academic/blob/main/config.py) at line 52. This model provides 1536-dimensional embeddings and serves as the fallback when no environment variable or per-request override is specified.

### Can I use different embedding models for different RAG workers simultaneously?

Yes. While the global configuration applies system-wide, you can instantiate individual workers with different `embed_model` values passed through `llm_kwargs`. For example, you can run `LlamaIndexRagWorker` with `text-embedding-3-large` for high-accuracy retrieval while keeping `MilvusWorker` on the default small model for other tasks.

### How do I verify which embedding model is currently active?

Check the embedding dimension using the `embedding_dimension()` method on the worker's embed_model instance, or inspect the `embed_model` cookie injected by `ArgsGeneralWrapper` in [`toolbox.py`](https://github.com/binary-husky/gpt_academic/blob/main/toolbox.py). The dimension will indicate the model: 1536 for `text-embedding-3-small`, 3072 for `text-embedding-3-large`, and 3072 for `text-embedding-ada-002`.

### Where are the embedding model endpoints and dimensions defined?

The mapping of model names to OpenAI API endpoints and vector dimensions resides in [`request_llms/embed_models/bridge_all_embed.py`](https://github.com/binary-husky/gpt_academic/blob/main/request_llms/embed_models/bridge_all_embed.py). This registry is consumed by [`openai_embed.py`](https://github.com/binary-husky/gpt_academic/blob/main/openai_embed.py) to instantiate the correct `OpenAiEmbeddingModel` with proper endpoint URLs and dimension metadata for vector store initialization.