how-to-guide

How to Integrate DeepWiki with Private GitHub or GitLab Repositories Using Personal Access Tokens

February 16, 2026 asyncfuncai/deepwiki-open ↗

DeepWiki integrates with private GitHub, GitLab, and Bitbucket repositories by accepting a personal access token (PAT) in the API request, which it injects into Git clone URLs or HTTP Authorization headers while scrubbing all logs to prevent token leakage.

DeepWiki, the open-source RAG-based code intelligence platform from AsyncFuncAI/deepwiki-open, supports ingesting source code from private repositories. By supplying a personal access token in the ChatCompletionRequest payload, users can authenticate Git operations without exposing credentials in logs or persistent storage.

Understanding the Token-Based Authentication Flow

When you query a private repository, DeepWiki must authenticate either a full Git clone or a single-file fetch via the provider's REST API. The system propagates your PAT through three architectural layers: the FastAPI controller, the RAG orchestrator, and the DatabaseManager. At each layer, the token remains an in-memory string; it is never written to disk or logged in plain text.

Request Payload Structure

ChatCompletionRequest Model

The entry point for private repository access is the ChatCompletionRequest Pydantic model defined in api/simple_chat.py. The token field is optional and accepts the PAT for GitHub, GitLab, or Bitbucket.

class ChatCompletionRequest(BaseModel):
    repo_url: str = Field(..., description="URL of the repository to query")
    token: Optional[str] = Field(
        None,
        description="Personal access token for private repositories"
    )
    # … other fields omitted for brevity …

Source: [api/simple_chat.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py#L60-L64)

When calling the /chat/completions/stream endpoint, include the token in the JSON body alongside the repository URL.

Token Propagation Through the API Stack

The PAT flows through the system as follows:

Layer	Token Handling
FastAPI controller (`simple_chat.py`)	Extracts `request.token` and forwards it to `RAG.prepare_retriever` (line 14).
RAG class (`rag.py`)	`prepare_retriever` receives `access_token` and stores it; later calls `DatabaseManager.prepare_database` with the same token (line 45).
DatabaseManager (`data_pipeline.py`)	Uses the token for: • Cloning – `download_repo` builds an authenticated clone URL (lines 104‑122). • File fetching – `get_github_file_content`, `get_gitlab_file_content`, and `get_bitbucket_file_content` inject the token into HTTP headers (lines 497‑511, 569‑587, 640‑662).

Sources:

Token forwarding in controller – [api/simple_chat.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py#L14-L16)
RAG signature – [api/rag.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/rag.py#L45-L46)
Database manager entry – [api/data_pipeline.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L22-L27)

Authenticating Git Operations

Cloning Private Repositories

When DeepWiki requires a full repository clone, the download_repo function in api/data_pipeline.py constructs an authenticated URL by embedding the PAT directly into the Git remote URL.

def download_repo(repo_url: str, local_path: str,
                  repo_type: str = None, access_token: str = None) -> str:
    # …

    if access_token:
        parsed = urlparse(repo_url)
        encoded_token = quote(access_token, safe='')

        if repo_type == "github":
            clone_url = urlunparse(
                (parsed.scheme,
                 f"{encoded_token}@{parsed.netloc}",
                 parsed.path, '', '', '')
            )
        elif repo_type == "gitlab":
            clone_url = urlunparse(
                (parsed.scheme,
                 f"oauth2:{encoded_token}@{parsed.netloc}",
                 parsed.path, '', '', '')
            )
        # Bitbucket handled similarly …

Source: [api/data_pipeline.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L104-L122)

The token is URL‑encoded to handle special characters. GitHub uses TOKEN@host, while GitLab requires oauth2:TOKEN@host. The repository is cloned with git clone --depth=1 --single-branch to minimize data transfer.

Fetching Single Files via REST API

If the request specifies a filePath, DeepWiki bypasses Git cloning and uses the provider's REST API. The token is attached to HTTP headers:


# GitHub

if access_token:
    headers["Authorization"] = f"token {access_token}"

# GitLab

if access_token:
    project_headers["PRIVATE-TOKEN"] = access_token

# Bitbucket

if access_token:
    repo_headers["Authorization"] = f"Bearer {access_token}"

Sources:

GitHub helper – [api/data_pipeline.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L497-L511)
GitLab helper – [api/data_pipeline.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L569-L587)
Bitbucket helper – [api/data_pipeline.py](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L640-L662)

All error messages are sanitized to replace the literal token with ***TOKEN*** before logging or returning to the client.

End-to-End Integration Flow

Client sends a POST to /chat/completions/stream with repo_url, optional filePath, and token.
FastAPI (simple_chat.py) extracts the token and instantiates a RAG object.
RAG (rag.py) calls DatabaseManager.prepare_database, forwarding the token.
DatabaseManager (data_pipeline.py) decides whether to clone the whole repo (download_repo) or fetch a single file (get_file_content).
The token is embedded in the clone URL or Authorization header, the repository data is read, split, token‑counted, embedded, and stored in a FAISS index for retrieval.

Practical Code Examples

cURL Request for Private GitHub

curl -X POST https://your-deepwiki-instance.com/chat/completions/stream \
  -H "Content-Type: application/json" \
  -d '{
        "repo_url": "https://github.com/your-org/private-repo",
        "messages": [{"role":"user","content":"Explain the authentication module"}],
        "token": "ghp_XXXXXXXXXXXXXXXXXXXX",
        "type": "github"
      }'

Python Client for GitLab

import requests
import json

payload = {
    "repo_url": "https://gitlab.com/your-group/private-repo",
    "messages": [{"role": "user", "content": "Summarize the README"}],
    "token": "glpat-XXXXXXXXXXXXXXXXXXXX",
    "type": "gitlab"
}

resp = requests.post(
    "https://your-deepwiki-instance.com/chat/completions/stream",
    json=payload,
    stream=True
)

for chunk in resp.iter_content(chunk_size=8192):
    print(chunk.decode())

FastAPI Test Client

from fastapi.testclient import TestClient
from api.simple_chat import app

client = TestClient(app)

def test_private_repo():
    response = client.post(
        "/chat/completions/stream",
        json={
            "repo_url": "https://github.com/me/secret",
            "messages": [{"role":"user","content":"List the endpoints"}],
            "token": "ghp_fake",
            "type": "github"
        },
        timeout=30
    )
    assert response.status_code == 200

Key Implementation Files

File	Purpose	Direct Link
`api/simple_chat.py`	Request model & entry point (`ChatCompletionRequest`, token field)	View source
`api/rag.py`	`RAG.prepare_retriever` forwards token to DB manager	View source
`api/data_pipeline.py`	Core logic for cloning (`download_repo`) and file fetching with token handling	View source View source
`api/config.py`	Default exclusion lists for private repo processing	View source

Summary

DeepWiki accepts a personal access token via the token field in ChatCompletionRequest to authenticate private repository access.
The token propagates through three layers: FastAPI controller (simple_chat.py), RAG orchestrator (rag.py), and DatabaseManager (data_pipeline.py).
For full repository ingestion, the token is URL-encoded and embedded into the Git clone URL (TOKEN@host for GitHub, oauth2:TOKEN@host for GitLab).
For single-file requests, the token is attached to HTTP Authorization headers using provider-specific formats (token, PRIVATE-TOKEN, or Bearer).
All error messages and logs are sanitized to replace the literal token with ***TOKEN***, preventing accidental credential leakage.

Frequently Asked Questions

What permissions does my personal access token need?

Your token requires read access to code repositories. For GitHub, select the repo scope for private repositories. For GitLab, use the read_repository scope. Bitbucket tokens need Repositories: Read permissions. DeepWiki only performs read operations (cloning or file fetching), so write permissions are unnecessary and should not be granted for security.

Is my token stored permanently in DeepWiki?

No. DeepWiki handles your token as an in-memory transient value during the request lifecycle. The token is forwarded through the API stack (simple_chat.py → rag.py → data_pipeline.py) and used immediately for Git operations or HTTP requests. It is not persisted to the FAISS index, database, or log files—all error messages sanitize the token as ***TOKEN*** before output.

Can I use the same token for both GitHub and GitLab?

No, tokens are provider-specific. GitHub personal access tokens use the format ghp_... and authenticate via https://TOKEN@github.com/... or Authorization: token ... headers. GitLab tokens use glpat-... and require oauth2:TOKEN@gitlab.com/... for Git operations or PRIVATE-TOKEN: ... headers for API calls. DeepWiki detects the repository type from the URL and applies the appropriate authentication scheme automatically.

How does DeepWiki handle token security in logs?

DeepWiki implements aggressive log sanitization to prevent credential leakage. Before any error message or URL is logged, the code replaces literal token strings with the placeholder ***TOKEN***. This occurs in api/data_pipeline.py within the download_repo and file-fetching helper functions. Additionally, URL-encoded tokens in Git clone commands are constructed in-memory and never persisted to configuration files or environment variables beyond the scope of the single request.

Have a question about this repo?

These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:

Share the following with your agent to get started:

curl -s "https://instagit.com/install.md"

Add to your MCP client configuration:

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}

Ask your agent:

"Use Instagit MCP to understand how asyncfuncai/deepwiki-open works."

Works with

Claude Codex Cursor VS Code OpenClaw Any MCP Client

Maintain an open-source project? Get it listed too →