# How to Integrate DeepWiki with Private GitHub or GitLab Repositories Using Personal Access Tokens

> Securely integrate DeepWiki with private GitHub or GitLab repositories using personal access tokens. Learn how DeepWiki handles PATs safely to protect your code.

- Repository: [ASYNCFUNC/deepwiki-open](https://github.com/asyncfuncai/deepwiki-open)
- Tags: how-to-guide
- Published: 2026-02-16

---

**DeepWiki integrates with private GitHub, GitLab, and Bitbucket repositories by accepting a personal access token (PAT) in the API request, which it injects into Git clone URLs or HTTP Authorization headers while scrubbing all logs to prevent token leakage.**

DeepWiki, the open-source RAG-based code intelligence platform from [AsyncFuncAI/deepwiki-open](https://github.com/AsyncFuncAI/deepwiki-open), supports ingesting source code from private repositories. By supplying a personal access token in the `ChatCompletionRequest` payload, users can authenticate Git operations without exposing credentials in logs or persistent storage.

## Understanding the Token-Based Authentication Flow

When you query a private repository, DeepWiki must authenticate either a full Git clone or a single-file fetch via the provider's REST API. The system propagates your PAT through three architectural layers: the FastAPI controller, the RAG orchestrator, and the DatabaseManager. At each layer, the token remains an in-memory string; it is never written to disk or logged in plain text.

## Request Payload Structure

### ChatCompletionRequest Model

The entry point for private repository access is the `ChatCompletionRequest` Pydantic model defined in [`api/simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py). The `token` field is optional and accepts the PAT for GitHub, GitLab, or Bitbucket.

```python
class ChatCompletionRequest(BaseModel):
    repo_url: str = Field(..., description="URL of the repository to query")
    token: Optional[str] = Field(
        None,
        description="Personal access token for private repositories"
    )
    # … other fields omitted for brevity …

```

*Source*: [[`api/simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py#L60-L64)

When calling the `/chat/completions/stream` endpoint, include the token in the JSON body alongside the repository URL.

## Token Propagation Through the API Stack

The PAT flows through the system as follows:

| Layer | Token Handling |
|-------|----------------|
| **FastAPI controller** ([`simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/simple_chat.py)) | Extracts `request.token` and forwards it to `RAG.prepare_retriever` (line 14). |
| **RAG class** ([`rag.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/rag.py)) | `prepare_retriever` receives `access_token` and stores it; later calls `DatabaseManager.prepare_database` with the same token (line 45). |
| **DatabaseManager** ([`data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/data_pipeline.py)) | Uses the token for: <br>• **Cloning** – `download_repo` builds an authenticated clone URL (lines 104‑122). <br>• **File fetching** – `get_github_file_content`, `get_gitlab_file_content`, and `get_bitbucket_file_content` inject the token into HTTP headers (lines 497‑511, 569‑587, 640‑662). |

*Sources*:  
- Token forwarding in controller – [[`api/simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py#L14-L16)  
- RAG signature – [[`api/rag.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/rag.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/rag.py#L45-L46)  
- Database manager entry – [[`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L22-L27)

## Authenticating Git Operations

### Cloning Private Repositories

When DeepWiki requires a full repository clone, the `download_repo` function in [`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py) constructs an authenticated URL by embedding the PAT directly into the Git remote URL.

```python
def download_repo(repo_url: str, local_path: str,
                  repo_type: str = None, access_token: str = None) -> str:
    # …

    if access_token:
        parsed = urlparse(repo_url)
        encoded_token = quote(access_token, safe='')

        if repo_type == "github":
            clone_url = urlunparse(
                (parsed.scheme,
                 f"{encoded_token}@{parsed.netloc}",
                 parsed.path, '', '', '')
            )
        elif repo_type == "gitlab":
            clone_url = urlunparse(
                (parsed.scheme,
                 f"oauth2:{encoded_token}@{parsed.netloc}",
                 parsed.path, '', '', '')
            )
        # Bitbucket handled similarly …

```

*Source*: [[`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L104-L122)

The token is URL‑encoded to handle special characters. GitHub uses `TOKEN@host`, while GitLab requires `oauth2:TOKEN@host`. The repository is cloned with `git clone --depth=1 --single-branch` to minimize data transfer.

### Fetching Single Files via REST API

If the request specifies a `filePath`, DeepWiki bypasses Git cloning and uses the provider's REST API. The token is attached to HTTP headers:

```python

# GitHub

if access_token:
    headers["Authorization"] = f"token {access_token}"

# GitLab

if access_token:
    project_headers["PRIVATE-TOKEN"] = access_token

# Bitbucket

if access_token:
    repo_headers["Authorization"] = f"Bearer {access_token}"

```

*Sources*:  
- GitHub helper – [[`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L497-L511)  
- GitLab helper – [[`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L569-L587)  
- Bitbucket helper – [[`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py)](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L640-L662)

All error messages are sanitized to replace the literal token with `***TOKEN***` before logging or returning to the client.

## End-to-End Integration Flow

1. **Client** sends a POST to `/chat/completions/stream` with `repo_url`, optional `filePath`, and `token`.
2. **FastAPI** ([`simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/simple_chat.py)) extracts the token and instantiates a `RAG` object.
3. **RAG** ([`rag.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/rag.py)) calls `DatabaseManager.prepare_database`, forwarding the token.
4. **DatabaseManager** ([`data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/data_pipeline.py)) decides whether to **clone** the whole repo (`download_repo`) or **fetch a single file** (`get_file_content`).
5. The token is embedded in the clone URL or Authorization header, the repository data is read, split, token‑counted, embedded, and stored in a FAISS index for retrieval.

## Practical Code Examples

### cURL Request for Private GitHub

```bash
curl -X POST https://your-deepwiki-instance.com/chat/completions/stream \
  -H "Content-Type: application/json" \
  -d '{
        "repo_url": "https://github.com/your-org/private-repo",
        "messages": [{"role":"user","content":"Explain the authentication module"}],
        "token": "ghp_XXXXXXXXXXXXXXXXXXXX",
        "type": "github"
      }'

```

### Python Client for GitLab

```python
import requests
import json

payload = {
    "repo_url": "https://gitlab.com/your-group/private-repo",
    "messages": [{"role": "user", "content": "Summarize the README"}],
    "token": "glpat-XXXXXXXXXXXXXXXXXXXX",
    "type": "gitlab"
}

resp = requests.post(
    "https://your-deepwiki-instance.com/chat/completions/stream",
    json=payload,
    stream=True
)

for chunk in resp.iter_content(chunk_size=8192):
    print(chunk.decode())

```

### FastAPI Test Client

```python
from fastapi.testclient import TestClient
from api.simple_chat import app

client = TestClient(app)

def test_private_repo():
    response = client.post(
        "/chat/completions/stream",
        json={
            "repo_url": "https://github.com/me/secret",
            "messages": [{"role":"user","content":"List the endpoints"}],
            "token": "ghp_fake",
            "type": "github"
        },
        timeout=30
    )
    assert response.status_code == 200

```

## Key Implementation Files

| File | Purpose | Direct Link |
|------|---------|-------------|
| [`api/simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py) | Request model & entry point (`ChatCompletionRequest`, token field) | [View source](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/simple_chat.py) |
| [`api/rag.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/rag.py) | `RAG.prepare_retriever` forwards token to DB manager | [View source](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/rag.py#L45-L46) |
| [`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py) | Core logic for cloning (`download_repo`) and file fetching with token handling | [View source](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L104-L122) <br> [View source](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py#L497-L511) |
| [`api/config.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/config.py) | Default exclusion lists for private repo processing | [View source](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/config.py) |

## Summary

- DeepWiki accepts a **personal access token** via the `token` field in `ChatCompletionRequest` to authenticate private repository access.
- The token propagates through three layers: **FastAPI controller** ([`simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/simple_chat.py)), **RAG orchestrator** ([`rag.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/rag.py)), and **DatabaseManager** ([`data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/data_pipeline.py)).
- For full repository ingestion, the token is **URL-encoded and embedded** into the Git clone URL (`TOKEN@host` for GitHub, `oauth2:TOKEN@host` for GitLab).
- For single-file requests, the token is attached to **HTTP Authorization headers** using provider-specific formats (`token`, `PRIVATE-TOKEN`, or `Bearer`).
- All error messages and logs are **sanitized** to replace the literal token with `***TOKEN***`, preventing accidental credential leakage.

## Frequently Asked Questions

### What permissions does my personal access token need?

Your token requires **read access to code repositories**. For GitHub, select the `repo` scope for private repositories. For GitLab, use the `read_repository` scope. Bitbucket tokens need `Repositories: Read` permissions. DeepWiki only performs read operations (cloning or file fetching), so write permissions are unnecessary and should not be granted for security.

### Is my token stored permanently in DeepWiki?

**No.** DeepWiki handles your token as an **in-memory transient value** during the request lifecycle. The token is forwarded through the API stack ([`simple_chat.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/simple_chat.py) → [`rag.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/rag.py) → [`data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/data_pipeline.py)) and used immediately for Git operations or HTTP requests. It is not persisted to the FAISS index, database, or log files—all error messages sanitize the token as `***TOKEN***` before output.

### Can I use the same token for both GitHub and GitLab?

**No, tokens are provider-specific.** GitHub personal access tokens use the format `ghp_...` and authenticate via `https://TOKEN@github.com/...` or `Authorization: token ...` headers. GitLab tokens use `glpat-...` and require `oauth2:TOKEN@gitlab.com/...` for Git operations or `PRIVATE-TOKEN: ...` headers for API calls. DeepWiki detects the repository type from the URL and applies the appropriate authentication scheme automatically.

### How does DeepWiki handle token security in logs?

DeepWiki implements **aggressive log sanitization** to prevent credential leakage. Before any error message or URL is logged, the code replaces literal token strings with the placeholder `***TOKEN***`. This occurs in [`api/data_pipeline.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/data_pipeline.py) within the `download_repo` and file-fetching helper functions. Additionally, URL-encoded tokens in Git clone commands are constructed in-memory and never persisted to configuration files or environment variables beyond the scope of the single request.