Purpose and Structure of the .adalflow Directory in DeepWiki-Open

The .adalflow directory serves as DeepWiki-Open's persistent storage layer, organizing cloned repositories, FAISS embedding indexes, and cached wiki pages under ~/.adalflow to ensure data survives container restarts.

DeepWiki-Open relies on the .adalflow directory to maintain long-living artifacts outside the container image. Located in the user's home directory at ~/.adalflow, this storage layer preserves cloned source code, vector databases, and generated wiki content across application restarts, as implemented in the AsyncFuncAI/deepwiki-open repository.

What Is the .adalflow Directory?

The .adalflow directory is the central data persistence mechanism for DeepWiki-Open. Rather than storing heavy artefacts inside the container image—which would be lost on restart—the application computes a default root path using get_adalflow_default_root_path() in api/api.py (lines 35-38). This function returns a path in the user's home directory, typically /home/username/.adalflow or /root/.adalflow in Docker contexts.

All processed repositories, their embedding indexes, and cached wiki outputs are organized beneath this root, creating a durable storage layer that persists independently of the application container lifecycle.

Directory Structure and Organization

The .adalflow directory organizes data into three top-level subfolders, each serving a distinct purpose in the DeepWiki-Open data pipeline. The _create_repo method in api/data_pipeline.py (lines 78-83) documents this layout and creates these folders automatically when processing a new repository.

repos/

The repos/ subdirectory stores the raw source code of each processed repository. When DeepWiki-Open clones a repository from GitHub, it stores the files under repos/owner_repo/ (for example, repos/asyncfuncai_deepwiki-open/). This enables the system to reuse previously cloned repositories without re-downloading, significantly speeding up subsequent wiki generation requests for the same codebase.

databases/

The databases/ subdirectory contains serialized FAISS (or other vector store) indexes that store embeddings for each repository. These files follow the naming pattern owner_repo.pkl and contain pre-computed vector representations of the source code. By persisting these indexes, DeepWiki-Open provides fast similarity search capabilities for RAG (Retrieval-Augmented Generation) and "Ask" features without recomputing embeddings on every query.

wikicache/

The wikicache/ subdirectory stores generated wiki structures and rendered pages as *.json files. When the wiki generation pipeline completes, it serializes the resulting documentation structure to this cache. This allows DeepWiki-Open to instantly load previously built wikis without regenerating them, improving response times for repeat requests and reducing API costs.

How the .adalflow Path Is Determined

The root path calculation is centralized in api/api.py through the get_adalflow_default_root_path() function. This helper constructs the path by joining the user's home directory with the .adalflow folder name:

from api.api import get_adalflow_default_root_path

# Returns something like "/home/your_user/.adalflow"

adal_root = get_adalflow_default_root_path()
print(adal_root)

This approach ensures consistency across the codebase, with both the data pipeline and API endpoints referencing the same root directory for all persistence operations.

Code Examples: Working with .adalflow Programmatically

Resolving a Repository's Storage Location

To locate where a specific repository's source code is stored, mirror the layout used in data_pipeline._create_repo:

import os
from api.api import get_adalflow_default_root_path

def repo_storage_path(owner: str, repo: str) -> str:
    # Mirrors the layout used in `data_pipeline._create_repo`

    base = get_adalflow_default_root_path()
    return os.path.join(base, "repos", f"{owner}_{repo}")

# Example: ~/.adalflow/repos/asyncfuncai_deepwiki-open

print(repo_storage_path("asyncfuncai", "deepwiki-open"))

Loading an Existing Embedding Index

To access pre-computed FAISS indexes for similarity search:

import os
import pickle
from api.api import get_adalflow_default_root_path

def load_embeddings(owner: str, repo: str):
    db_path = os.path.join(
        get_adalflow_default_root_path(),
        "databases",
        f"{owner}_{repo}.pkl",
    )
    with open(db_path, "rb") as f:
        index = pickle.load(f)   # typically a FAISS index

    return index

Accessing the Cached Wiki

To retrieve previously generated wiki structures from the cache:

import json
import os
from api.api import get_adalflow_default_root_path

def load_wiki_cache(repo_url: str):
    # The cache filename is derived from the repo URL inside the server implementation

    cache_dir = os.path.join(get_adalflow_default_root_path(), "wikicache")
    # Simplified example – the real implementation stores a JSON per repo

    cache_file = os.path.join(cache_dir, f"{repo_url.replace('/', '_')}.json")
    with open(cache_file, "r", encoding="utf-8") as f:
        return json.load(f)

Docker Persistence and Volume Mounts

When DeepWiki-Open runs in Docker, the host's ~/.adalflow directory is mounted into the container at /root/.adalflow. This mount is explicitly declared in the README.md (lines 474-477) and Docker Compose configuration, ensuring that the data persists even if the container is stopped or removed.

The volume mapping follows this pattern:

-v ~/.adalflow:/root/.adalflow

This approach guarantees that the three data types—raw repositories, embedding databases, and wiki caches—survive container restarts while remaining accessible to the application at the expected paths.

Summary

• The .adalflow directory acts as DeepWiki-Open's persistent storage layer, located at ~/.adalflow by default via get_adalflow_default_root_path() in api/api.py.
• It organizes data into three subfolders: repos/ for cloned source code, databases/ for serialized FAISS embedding indexes, and wikicache/ for generated wiki JSON files.
• The _create_repo method in api/data_pipeline.py automatically creates this structure when processing new repositories.
• Docker deployments mount the host's ~/.adalflow to /root/.adalflow to ensure data survives container restarts.

Frequently Asked Questions

Where does DeepWiki-Open store cloned repositories and embedding indexes?

DeepWiki-Open stores all persistent data under the ~/.adalflow directory in the user's home folder. Cloned repositories live in ~/.adalflow/repos/, while embedding indexes are serialized to ~/.adalflow/databases/. This location is computed by the get_adalflow_default_root_path() function in api/api.py.

How does DeepWiki-Open ensure data persists across Docker container restarts?

The application mounts the host's ~/.adalflow directory into the container at /root/.adalflow using a Docker volume mapping. This configuration, documented in the README.md, ensures that cloned repositories, vector databases, and wiki caches remain intact even when the container is stopped, removed, or updated.

What is the purpose of the wikicache folder in .adalflow?

The wikicache/ subdirectory stores generated wiki structures and rendered pages as JSON files. This caching mechanism allows DeepWiki-Open to instantly load previously built wikis without regenerating them, significantly improving response times for repeat requests and reducing computational overhead.

Which source files manage the creation and layout of the .adalflow directory?

The directory structure is defined and created by two key files: api/api.py contains the get_adalflow_default_root_path() function that determines the root location, while api/data_pipeline.py implements the _create_repo method that creates the repos/, databases/, and wikicache/ subfolders when processing a new repository.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}