How to Set Up RAGAnything for Offline or Isolated Network Environments

Set up RAGAnything for offline use by caching tiktoken models locally and setting TIKTOKEN_CACHE_DIR before importing LightRAG.

RAGAnything builds on top of LightRAG, which relies on OpenAI's tiktoken library for text tokenization. By default, tiktoken downloads encoding files from external blob storage the first time it runs, causing failures in air-gapped networks. This guide shows you how to pre-cache these dependencies and configure the environment so RAGAnything initializes without any outbound network calls.

Understanding the Offline Challenge

The core issue lies in LightRAG's dependency on tiktoken, not in RAGAnything's code itself. When you import LightRAG, it calls tiktoken.get_encoding(), which attempts to fetch the cl100k_base model from openaipublic.blob.core.windows.net. In isolated environments, this produces an error:

Failed to initialize LightRAG instance: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443)...

RAGAnything solves this by ensuring the TIKTOKEN_CACHE_DIR environment variable is set before any LightRAG import occurs. The entry point at raganything/raganything.py handles this by calling load_dotenv() at lines 22-26, loading your .env file before the LightRAG dependency chain triggers the tiktoken initialization【/cache/repos/github.com/HKUDS/RAG-Anything/main/raganything/raganything.py#L22-L26】.

Step-by-Step Offline Setup

1. Cache tiktoken Models on an Online Machine

First, use a machine with internet access to download the required tokenizer files. Run the provided utility script:

uv run scripts/create_tiktoken_cache.py

This script (located at scripts/create_tiktoken_cache.py, lines 4-17) populates a local ./tiktoken_cache directory with the cl100k_base encoding files that LightRAG requires【/cache/repos/github.com/HKUDS/RAG-Anything/main/scripts/create_tiktoken_cache.py#L4-L17】. Once created, copy this entire directory to your offline environment.

2. Configure the Environment Variable

Create a .env file in your project root (or modify your shell environment) to point tiktoken to your cached files:

TIKTOKEN_CACHE_DIR=./tiktoken_cache

Critical: This variable must be set before any Python process imports LightRAG or tiktoken. The RAGAnything library automatically loads .env early in its initialization sequence, but if you're writing custom scripts, place load_dotenv() at the very top of your entry file.

3. Verify Early Loading

The raganything/raganything.py module automatically calls load_dotenv(dotenv_path=".env", override=False) before importing LightRAG components. This ensures that when tiktoken is eventually imported deeper in the dependency chain, it sees the cached directory and skips the network request【/cache/repos/github.com/HKUDS/RAG-Anything/main/raganything/raganything.py#L22-L26】.

Code Examples

Minimal Offline Initialization

This snippet demonstrates the minimal setup required to initialize LightRAG without network access:

# main_offline_demo.py

import os
from pathlib import Path
from dotenv import load_dotenv

# 1. Load .env early (must happen before LightRAG/tiktoken imports)

load_dotenv(dotenv_path=".env", override=False)

# 2. Import LightRAG – it will now read the cached tokenizer

from lightrag import LightRAG

# 3. Simple LightRAG initialization (no network needed)

rag = LightRAG(
    working_dir="./rag_storage",
    llm_model_func=lambda *a, **kw: None,   # placeholder – inject your own LLM later

    embedding_func=lambda *a, **kw: None,   # placeholder – inject your own embedder later

)

print("✅ LightRAG initialized offline!")

Full RAGAnything API in Offline Mode

For production use with the high-level RAGAnything API:

# 1. Ensure the cache exists (run once on an online machine)

uv run scripts/create_tiktoken_cache.py

# 2. Add the environment variable

echo "TIKTOKEN_CACHE_DIR=./tiktoken_cache" >> .env

# offline_rag.py

import asyncio
from dotenv import load_dotenv

# Load early – RAGAnything does this internally, but explicit is clearer

load_dotenv(".env", override=False)

from raganything import RAGAnything, RAGAnythingConfig
from lightrag.llm.openai import openai_complete_if_cache, openai_embed
from lightrag.utils import EmbeddingFunc

# Minimal LLM & embedding functions (replace with your own if needed)

api_key = "FAKE-KEY"          # not used offline, just a placeholder

def llm(prompt, **kw):
    return openai_complete_if_cache("gpt-4o-mini", prompt, api_key=api_key, **kw)

def embed(texts):
    return openai_embed.func(texts, model="text-embedding-3-large", api_key=api_key)

embedding = EmbeddingFunc(embedding_dim=3072, max_token_size=8192, func=embed)

async def run():
    cfg = RAGAnythingConfig(
        working_dir="./rag_storage",
        parser="mineru",
        parse_method="auto",
        enable_image_processing=False,
    )
    rag = RAGAnything(config=cfg, llm_model_func=llm, embedding_func=embedding)

    # Process a local PDF (no network required for tokenization)

    await rag.process_document_complete(
        file_path="data/document.pdf",
        output_dir="./output",
    )

    # Query the freshly built index

    answer = await rag.aquery("What does this document contain?", mode="hybrid")
    print("Answer:", answer)

if __name__ == "__main__":
    asyncio.run(run())

Running this script after the cache is present and the .env variable set will never attempt an external HTTP request during the tokenization phase.

Key Files Reference

• docs/offline_setup.md – Complete offline setup documentation with additional troubleshooting steps【/cache/repos/github.com/HKUDS/RAG-Anything/main/docs/offline_setup.md#L1-L78】
• scripts/create_tiktoken_cache.py – Utility to download and cache tiktoken encodings (specifically cl100k_base)【/cache/repos/github.com/HKUDS/RAG-Anything/main/scripts/create_tiktoken_cache.py#L4-L17】
• raganything/raganything.py – Entry point that ensures .env loading occurs before LightRAG imports (lines 22-26)【/cache/repos/github.com/HKUDS/RAG-Anything/main/raganything/raganything.py#L22-L26】
• .env.example – Template showing required variables including TIKTOKEN_CACHE_DIR

Summary

• Cache once: Run scripts/create_tiktoken_cache.py on an internet-connected machine to download cl100k_base tiktoken models.
• Set environment variable: Define TIKTOKEN_CACHE_DIR=./tiktoken_cache in your .env file.
• Import order matters: The cache variable must be loaded before importing LightRAG (handled automatically in raganything/raganything.py).
• Deploy anywhere: Once cached, RAGAnything runs on air-gapped servers, offline Docker containers, or secure clusters without outbound network requirements.

Frequently Asked Questions

How do I know if the offline setup is working correctly?

If the setup is correct, RAGAnything will initialize without raising HTTPSConnectionPool errors related to openaipublic.blob.core.windows.net. Check that your ./tiktoken_cache directory contains the cl100k_base.tiktoken file and that your .env file properly sets TIKTOKEN_CACHE_DIR to that directory's path.

Can I use RAGAnything offline with local LLMs only?

Yes. While tiktoken caching enables offline tokenization, you can also configure local LLM endpoints (such as Ollama or vLLM) by setting the appropriate llm_model_func and embedding_func in your RAGAnythingConfig. The offline setup handles the tokenizer dependency, while your local LLM configuration handles inference without cloud APIs.

What if I need to add RAGAnything to an existing project that already imports LightRAG?

Ensure load_dotenv() executes before any LightRAG imports in your entry point. If you cannot modify import order, set TIKTOKEN_CACHE_DIR as a system environment variable in your container or shell profile before launching Python. As long as the variable exists in the environment before tiktoken initializes, it will use the cached files.

Where can I find the official documentation for offline deployment?

The complete offline setup guide is located at docs/offline_setup.md in the repository【/cache/repos/github.com/HKUDS/RAG-Anything/main/docs/offline_setup.md#L1-L78】. This document contains additional details about network requirements, Docker deployment strategies, and validation steps for enterprise environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}