How to Configure the Embedding Model for RAG Functionality in GPT Academic

Configure the embedding model for RAG functionality by setting the EMBEDDING_MODEL variable in config.py, overriding it via the EMBEDDING_MODEL environment variable, or passing embed_model in the llm_kwargs dictionary for per-request customization.

The binary-husky/gpt_academic repository implements Retrieval-Augmented Generation (RAG) through modular workers that rely on embedding models to convert text into vector representations. Understanding how to configure the embedding model for RAG functionality ensures your vector stores use the correct dimensions and endpoints for optimal retrieval performance.

Three Levels of Configuration for the Embedding Model

The system provides three distinct levels at which you can specify the embedding model, each offering different scopes of control.

Global Default Configuration

The most persistent configuration resides in config.py, where the default embedding model is defined for all RAG operations.

In config.py (line 52), the default is set as:

EMBEDDING_MODEL = "text-embedding-3-small"

Changing this value affects all RAG workers—including LlamaIndexRagWorker and MilvusWorker—that rely on the global configuration when no override is present.

Environment Variable Override

For containerized deployments or temporary changes, set the EMBEDDING_MODEL environment variable to supersede the config.py value.

When the application initializes, toolbox.get_conf("EMBEDDING_MODEL") checks for environment variables before falling back to the configuration file. This allows the same codebase to run with different models across development, staging, and production environments without modifying source files.

Per-Request Override

For granular control, pass the embed_model parameter within the llm_kwargs dictionary when initializing RAG workers programmatically.

This method enables a single conversation or plugin invocation to use a specific embedding model—such as text-embedding-3-large for higher accuracy—while the rest of the system maintains the global default.

Architecture Flow: How the Embedding Model Propagates Through the System

Understanding the data flow helps debug configuration issues and ensures the correct model is active.

1. Configuration Loading: toolbox.get_conf("EMBEDDING_MODEL") retrieves the value from config.py or the environment variable (source: toolbox.py, lines 10-16).

2. Cookie Injection: The ArgsGeneralWrapper decorator stores the embedding model name in request cookies under the key embed_model, making it available to downstream functions (source: toolbox.py, lines 109-118).

3. Worker Initialization: LlamaIndexRagWorker extracts embed_model from llm_kwargs and instantiates OpenAiEmbeddingModel with the specified model name (source: crazy_functions/rag_fns/llama_index_worker.py, lines 65-70).

4. Embedding Computation: OpenAiEmbeddingModel.compute_embedding selects the appropriate OpenAI endpoint using embed_model_info from bridge_all_embed.py (source: request_llms/embed_models/openai_embed.py, lines 42-55).

5. Vector Store Alignment: The embedding_dimension() method ensures the vector database (Milvus or LlamaIndex) allocates the correct vector size for the chosen model (source: request_llms/embed_models/openai_embed.py, lines 74-82).

Practical Code Examples

Setting the Global Default in config.py

Modify the configuration file to change the default embedding model for all RAG operations:

# config.py

EMBEDDING_MODEL = "text-embedding-3-large"  # Changed from text-embedding-3-small

This affects all workers that do not receive an explicit override.

Overriding via Docker Compose Environment Variables

For containerized deployments, specify the model in your docker-compose.yml:

services:
  gpt_academic:
    image: ghcr.io/binary-husky/gpt_academic:latest
    environment:
      - EMBEDDING_MODEL=text-embedding-3-large
      - API_KEY=sk-your-openai-key

The environment variable takes precedence over the value in config.py.

Per-Request Configuration in Custom Scripts

When invoking RAG functionality programmatically, pass the embedding model explicitly:

from crazy_functions.rag_fns.llama_index_worker import LlamaIndexRagWorker

# Configure parameters for this specific request

llm_kwargs = {
    "api_key": "sk-your-api-key",
    "llm_model": "gpt-4o-mini",
    "embed_model": "text-embedding-3-large",  # Specific model for this session

    "temperature": 0.7,
    "max_length": 4096,
}

# Initialize worker with custom embedding model

rag_worker = LlamaIndexRagWorker(user_name="user_123", llm_kwargs=llm_kwargs)

# Add documents and query

rag_worker.add_text_to_vector_store("Retrieval-Augmented Generation combines retrieval with generation.")
results = rag_worker.retrieve_from_store_with_query("What is RAG?")

This approach allows different conversations to use different embedding models without restarting the application.

Debugging: Checking Vector Store Dimensions

Verify that your vector store matches the embedding model's output dimensions:

# After initializing any RAG worker

dimension = rag_worker.embed_model.embedding_dimension()
print(f"Current embedding dimension: {dimension}")

# Output: Current embedding dimension: 3072  # for text-embedding-3-large

This helps diagnose dimension mismatches when switching between models like text-embedding-3-small (1536 dimensions) and text-embedding-3-large (3072 dimensions).

Key Source Files for RAG Embedding Configuration

The following files govern how embedding models are selected and utilized:

• config.py – Defines the global EMBEDDING_MODEL default (line 52).
• toolbox.py – Implements get_conf() for configuration retrieval and ArgsGeneralWrapper for cookie injection (lines 10-16, 109-118).
• request_llms/embed_models/openai_embed.py – Contains OpenAiEmbeddingModel class with compute_embedding() and embedding_dimension() methods (lines 42-55, 74-82).
• request_llms/embed_models/bridge_all_embed.py – Maps model names to endpoint URLs and dimension metadata.
• crazy_functions/rag_fns/llama_index_worker.py – RAG worker implementation that instantiates embedding models from llm_kwargs (lines 65-70).
• crazy_functions/rag_fns/milvus_worker.py – Alternative RAG backend supporting the same embedding configuration interface.

Summary

• Global configuration sets the default embedding model via EMBEDDING_MODEL in config.py, affecting all RAG workers system-wide.
• Environment variables override the global setting at runtime, enabling deployment-specific models without code changes.
• Per-request overrides via llm_kwargs allow individual conversations or scripts to specify custom embedding models while the system maintains its default.
• Architecture consistency ensures that toolbox.py propagates the model name through cookies to workers like LlamaIndexRagWorker, which instantiate OpenAiEmbeddingModel and validate dimensions through embedding_dimension().

Frequently Asked Questions

What is the default embedding model in GPT Academic?

The default embedding model is text-embedding-3-small, defined in config.py at line 52. This model provides 1536-dimensional embeddings and serves as the fallback when no environment variable or per-request override is specified.

Can I use different embedding models for different RAG workers simultaneously?

Yes. While the global configuration applies system-wide, you can instantiate individual workers with different embed_model values passed through llm_kwargs. For example, you can run LlamaIndexRagWorker with text-embedding-3-large for high-accuracy retrieval while keeping MilvusWorker on the default small model for other tasks.

How do I verify which embedding model is currently active?

Check the embedding dimension using the embedding_dimension() method on the worker's embed_model instance, or inspect the embed_model cookie injected by ArgsGeneralWrapper in toolbox.py. The dimension will indicate the model: 1536 for text-embedding-3-small, 3072 for text-embedding-3-large, and 3072 for text-embedding-ada-002.

Where are the embedding model endpoints and dimensions defined?

The mapping of model names to OpenAI API endpoints and vector dimensions resides in request_llms/embed_models/bridge_all_embed.py. This registry is consumed by openai_embed.py to instantiate the correct OpenAiEmbeddingModel with proper endpoint URLs and dimension metadata for vector store initialization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}