How to Process Local Repositories with DeepWiki Without Remote Cloning

DeepWiki can analyze repositories that already exist on your local filesystem by using the /local_repo/structure endpoint and setting repo_type: "local" in RAG queries, bypassing the download_repo clone workflow entirely.

The AsyncFuncAI/deepwiki-open project provides a flexible RAG-based system for code analysis that works with both remote Git URLs and local directories. When you need to process local repositories with DeepWiki, you can point the API directly to an absolute filesystem path, eliminating network overhead and clone time while using the same embedding and querying pipeline as remote repositories.

Understanding the Local Repository Architecture

DeepWiki distinguishes between remote and local processing through two distinct code paths in api/data_pipeline.py. The download_repo function (lines 72-136) handles Git cloning from remote URLs, but when you provide a local filesystem path, DeepWiki invokes read_all_documents (lines 153-210) directly.

This local workflow performs recursive directory traversal, file filtering based on extension and exclusion rules defined in api/config.py, text splitting via TextSplitter, and embedding generation using your configured provider—all without requiring Git operations or network access.

Step-by-Step Local Repository Processing

Prepare Your Local Repository Path

Place your repository on the DeepWiki server at a known absolute path, such as /data/my-project. Ensure the DeepWiki service has read permissions for this directory and all subdirectories. The system supports standard project structures with nested folders, documentation files, and source code in multiple languages.

Validate Structure with the Local Repository Endpoint

Before processing, verify the repository is accessible using the /local_repo/structure endpoint defined in api/api.py (lines 275-315). This endpoint validates the path query parameter, walks the filesystem using os.walk, builds a file_tree representation, and extracts the first README.md content found.

Process and Index the Repository

Once validated, trigger the full RAG pipeline by calling the standard query endpoints with repo_type set to "local" and repo_url_or_path containing your absolute path. DeepWiki will invoke read_all_documents to recursively process files, apply the configured text splitter, generate embeddings using your selected provider (OpenAI, Ollama, Google, or Bedrock as configured in api/config.py), and store vectors for querying.

Code Examples for Local Repository Processing

cURL Request for Structure Validation

Test connectivity and view repository structure using a simple HTTP request:

curl -X GET \
  "http://localhost:8000/local_repo/structure?path=%2Fdata%2Fmy-project" \
  -H "Accept: application/json"

The response includes the file tree and README content:

{
  "file_tree": "src/__init__.py\nsrc/main.py\nREADME.md",
  "readme": "# My Project\n\nThis repository does ..."

}

Python Client Implementation

For programmatic access, use the requests library to validate and prepare the repository:

import requests

BASE_URL = "http://localhost:8000"
local_path = "/data/my-project"

resp = requests.get(
    f"{BASE_URL}/local_repo/structure",
    params={"path": local_path},
    headers={"Accept": "application/json"},
)

if resp.status_code == 200:
    data = resp.json()
    print("File tree:\n", data["file_tree"])
    print("\nREADME content:\n", data["readme"])
else:
    print("Error:", resp.json())

Querying Local Repositories with RAG

After validation, query the repository using the RAG endpoint:

curl -X POST "http://localhost:8000/rag" \
  -H "Content-Type: application/json" \
  -d '{
        "repo_type": "local",
        "repo_url_or_path": "/data/my-project",
        "question": "What is the purpose of the `main.py` module?"
      }'

DeepWiki executes read_all_documents on the specified path, embeds the content, retrieves relevant chunks, and generates a contextual answer based on your local codebase.

Key Implementation Details

The local repository functionality relies on specific components within the AsyncFuncAI/deepwiki-open codebase:

• api/api.py (lines 275-315): Implements the /local_repo/structure endpoint, handling path validation, directory traversal, and README extraction.

• api/data_pipeline.py (lines 153-210): Contains the read_all_documents function that recursively processes local directories, applies file extension filtering, exclusion rules (DEFAULT_EXCLUDED_DIRS and DEFAULT_EXCLUDED_FILES from api/config.py), text splitting, and embedding generation.

• api/config.py: Defines exclusion patterns and the get_embedder_type helper that determines whether to use OpenAI, Ollama, Google, or Bedrock embeddings for local repository processing.

• api/rag.py and api/websocket_wiki.py: Support querying local repositories through the standard RAG and WebSocket chat interfaces, accepting repo_type: "local" and repo_url_or_path parameters.

Summary

• DeepWiki processes local repositories by pointing directly to filesystem paths rather than cloning from remote URLs.
• Use the /local_repo/structure endpoint to validate accessibility and preview repository contents before indexing.
• The read_all_documents function in api/data_pipeline.py handles recursive file processing, filtering, and embedding for local paths.
• Configure exclusions and embedders via api/config.py to customize how local repositories are processed.
• Query indexed local repositories using standard RAG endpoints with repo_type: "local" and the absolute path specified in repo_url_or_path.

Frequently Asked Questions

Can DeepWiki process repositories on mounted network drives or external storage?

Yes, DeepWiki can process any directory accessible via the local filesystem, including NFS mounts, SMB shares, or external USB drives, provided the service has read permissions and the path is specified as an absolute path in the API calls.

Does processing local repositories skip the Git clone history and .git directory?

The read_all_documents function applies exclusion rules defined in api/config.py, which typically include .git directories in DEFAULT_EXCLUDED_DIRS. Therefore, Git metadata is ignored during processing, and only working directory files are indexed.

What file types are supported when processing local repositories?

DeepWiki processes text-based source code and documentation files based on configurable extension filters. The system reads files recursively from the local path, splits them using the configured TextSplitter, and embeds them regardless of whether they originated from a remote clone or local filesystem.

Can I switch between local and remote repositories without restarting DeepWiki?

Yes, the repo_type parameter in RAG and chat endpoints determines whether to treat the repo_url_or_path as a remote URL (triggering download_repo) or a local path (triggering read_all_documents). You can alternate between local and remote repositories on a per-request basis without service restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}