# DeepWiki Server-Side Wiki Caching Mechanism: Implementation and Invalidation Strategies

> Explore DeepWiki's server-side caching: learn how it stores JSON wiki structures and handles invalidation via DELETE endpoints and POST overwrites. Enhance performance now.

- Repository: [ASYNCFUNC/deepwiki-open](https://github.com/asyncfuncai/deepwiki-open)
- Tags: internals
- Published: 2026-02-16

---

**DeepWiki implements a file-system-based server-side cache that stores generated wiki structures as JSON files under a dedicated directory, using deterministic filenames derived from repository metadata, with explicit invalidation via DELETE endpoints and implicit refresh through POST overwrites.**

The AsyncFuncAI/deepwiki-open repository uses this server-side wiki caching mechanism to eliminate redundant computation costs. By persisting generated wiki structures and pages to the local file system, the system avoids re-running expensive generation pipelines for previously processed repositories.

## How DeepWiki's Server-Side Cache Works

### Cache Directory Structure and Initialization

On API startup, DeepWiki automatically creates a dedicated cache directory at `<adalflow-root>/wikicache`. This path is defined as `WIKI_CACHE_DIR` in [`api/api.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/api.py) (lines 5-7). The directory is auto-created if missing, ensuring the cache is always available without manual intervention.

### Deterministic Cache Key Generation

Cache entries are addressed using deterministic filenames derived from repository metadata. The function `get_wiki_cache_path` in [`api/api.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/api.py) (lines 8-11) combines `owner`, `repo`, `repo_type`, and `language` into a filename formatted as:

```

<repo_type>_<owner>_<repo>_<language>.json

```

This deterministic mapping ensures that identical repository requests always resolve to the same cache file, while different languages or repository types receive separate cache entries.

### Cache Storage and Retrieval Endpoints

**Reading from cache:** The `GET /api/wiki_cache` endpoint (implemented in `get_cached_wiki`, lines 61-84) constructs the file path from query parameters and loads the JSON into a `WikiCacheData` Pydantic model. If the file is missing, it returns HTTP 200 with a `null` body—a behavior the frontend expects to trigger fresh generation.

**Writing to cache:** The `POST /api/wiki_cache` endpoint (`store_wiki_cache`, lines 86-100) receives a `WikiCacheRequest`, constructs a `WikiCacheData` payload, and writes it as pretty-printed JSON to the computed file path. This operation overwrites any existing file, effectively refreshing the cache.

## Cache Invalidation Strategies in DeepWiki

### Explicit Deletion via REST API

The primary invalidation mechanism uses the `DELETE /api/wiki_cache` endpoint (`delete_wiki_cache`, lines 104-135). Clients send a request with the same identifying query parameters used for retrieval (`owner`, `repo`, `repo_type`, `language`). The endpoint validates optional authentication (controlled by `DEEPWIKI_AUTH_MODE` and `DEEPWIKI_AUTH_CODE` in [`api/config.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/config.py)), then removes the corresponding cache file. Subsequent GET requests return `null`, forcing a full regeneration.

### Implicit Invalidation Through Overwrites

Cache entries are implicitly invalidated when a new generation completes. The `POST /api/wiki_cache` endpoint writes the new wiki data to the same deterministic file path, overwriting the previous JSON. This atomic replacement ensures that readers always see the most recent generation without requiring an explicit delete operation first.

### No TTL: Deterministic Cache Lifetime

Unlike many caching systems, DeepWiki **does not implement time-based expiration (TTL)**. Cache lifetime is controlled entirely through explicit deletion or overwrite operations. This deterministic approach keeps the implementation simple and ensures that cached data remains available indefinitely until actively invalidated, which is appropriate for wiki content that changes only when the underlying repository is updated.

## Practical API Usage Examples

### Retrieve a Cached Wiki

```bash
curl "http://localhost:8000/api/wiki_cache?owner=AsyncFuncAI&repo=deepwiki-open&repo_type=github&language=en"

```

**Cache hit response:**

```json
{
  "wiki_structure": {...},
  "generated_pages": [...],
  "repo": {"owner":"AsyncFuncAI","repo":"deepwiki-open","type":"github"},
  "provider":"openai",
  "model":"gpt-4"
}

```

**Cache miss response:**

```json
null

```

### Store a Newly Generated Wiki

```bash
curl -X POST "http://localhost:8000/api/wiki_cache" \
  -H "Content-Type: application/json" \
  -d '{
    "wiki_structure": {...},
    "generated_pages": [...],
    "repo": {"owner":"AsyncFuncAI","repo":"deepwiki-open","type":"github"},
    "language": "en",
    "provider": "openai",
    "model": "gpt-4"
  }'

```

**Response:**

```json
{ "message": "Wiki cache saved successfully" }

```

### Invalidate a Cache Entry

```bash
curl -X DELETE "http://localhost:8000/api/wiki_cache?owner=AsyncFuncAI&repo=deepwiki-open&repo_type=github&language=en"

```

If authentication is enabled, append `&authorization_code=YOUR_CODE` or set `DEEPWIKI_AUTH_MODE=False` in your environment.

**Response:**

```json
{ "message": "Wiki cache for AsyncFuncAI/deepwiki-open (en) deleted successfully" }

```

### List All Cached Projects

```bash
curl "http://localhost:8000/api/processed_projects"

```

**Response:**

```json
[
  {
    "id":"deepwiki_cache_github_AsyncFuncAI_deepwiki-open_en.json",
    "owner":"AsyncFuncAI",
    "repo":"deepwiki-open",
    "name":"AsyncFuncAI/deepwiki-open",
    "repo_type":"github",
    "submittedAt":1708099200000,
    "language":"en"
  }
]

```

## Key Source Files and Implementation Details

| File | Role | Key Components |
|------|------|----------------|
| [`api/api.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/api.py) | Core API implementation | `WIKI_CACHE_DIR` definition (lines 5-7), `get_wiki_cache_path()` (lines 8-11), `get_cached_wiki()` endpoint (lines 61-84), `store_wiki_cache()` endpoint (lines 86-100), `delete_wiki_cache()` endpoint (lines 104-135), `get_processed_projects()` endpoint (lines 76-115) |
| [`api/config.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/config.py) | Configuration and authentication | `DEEPWIKI_AUTH_MODE` flag, `DEEPWIKI_AUTH_CODE` for securing the delete endpoint |
| [`api/models.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/models.py) | Data models | `WikiCacheData` and `WikiCacheRequest` Pydantic models for JSON serialization/deserialization |
| [`api/utils.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/utils.py) | Path utilities | `get_adalflow_default_root_path()` used to locate the cache root directory |

These files implement a straightforward, file-system-backed cache that supports CRUD operations through REST endpoints, with deterministic addressing and explicit invalidation controls.

## Summary

- **DeepWiki's server-side caching mechanism** stores generated wiki structures as JSON files under `<adalflow-root>/wikicache` using deterministic filenames based on repository metadata.
- **Cache keys** are derived from `repo_type`, `owner`, `repo`, and `language`, producing filenames like [`github_AsyncFuncAI_deepwiki-open_en.json`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/github_AsyncFuncAI_deepwiki-open_en.json).
- **Cache invalidation** is handled explicitly via the `DELETE /api/wiki_cache` endpoint or implicitly when `POST /api/wiki_cache` overwrites existing entries.
- **No TTL mechanism** exists; cached data persists indefinitely until actively deleted or refreshed, simplifying the design and ensuring deterministic behavior.
- **Authentication** for destructive operations is controlled via `DEEPWIKI_AUTH_MODE` and `DEEPWIKI_AUTH_CODE` in [`api/config.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/config.py).

## Frequently Asked Questions

### How does DeepWiki determine the filename for a cached wiki?

DeepWiki constructs the cache filename by combining the `repo_type`, `owner`, `repo`, and `language` parameters into the format `<repo_type>_<owner>_<repo>_<language>.json`. This logic is implemented in the `get_wiki_cache_path` function within [`api/api.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/api.py) (lines 8-11). The deterministic naming ensures that identical repository requests always map to the same file path.

### What happens when a cached wiki entry is missing?

When the `GET /api/wiki_cache` endpoint cannot locate a file for the requested repository parameters, it returns HTTP status 200 with a `null` body. This behavior, implemented in `get_cached_wiki` ([`api/api.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/api.py), lines 61-84), signals to the frontend that no cached data exists and triggers the wiki generation pipeline to create a new entry.

### Is there an automatic expiration time for cached wiki files?

No, DeepWiki does not implement time-based expiration (TTL) for cached entries. The cache persists indefinitely until explicitly invalidated through the `DELETE /api/wiki_cache` endpoint or overwritten by a new `POST /api/wiki_cache` request. This design choice, documented in the `delete_wiki_cache` implementation ([`api/api.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/api.py), lines 104-135), provides deterministic cache behavior suitable for wiki content that changes only when source repositories are updated.

### How is the cache deletion endpoint secured?

The `DELETE /api/wiki_cache` endpoint supports optional authentication controlled by environment variables defined in [`api/config.py`](https://github.com/AsyncFuncAI/deepwiki-open/blob/main/api/config.py). When `DEEPWIKI_AUTH_MODE` is enabled, requests must include a valid `authorization_code` parameter matching `DEEPWIKI_AUTH_CODE`. If authentication fails, the endpoint returns HTTP 403. This security layer prevents unauthorized cache invalidation while allowing open access in trusted environments by setting `DEEPWIKI_AUTH_MODE=False`.