How CodeWiki Caches Wiki Content for High-Performance Access

CodeWiki stores generated wiki content as JSON files on disk and serves them through FastAPI endpoints, eliminating the need to re-run expensive repository analysis on subsequent requests.

The quangdungluong/codewiki project implements a file-based caching layer to deliver wiki content instantly without regenerating documentation from scratch. By persisting the complete wiki structure and generated pages as JSON, CodeWiki cache wiki content strategies reduce latency and server load significantly.

The File-Based Caching Architecture

Cache Storage Location and Structure

According to utils/constants.py, the system defines WIKI_CACHE_DIR pointing to ./.cache/wiki_cache/. Each cached repository receives a unique filename encoding the owner, repository name, and type:

# api/wiki_cache.py – get_wiki_cache_path()

filename = f"{owner}_{repo}_{repo_type}_wiki_cache.json"
cache_path = os.path.join(WIKI_CACHE_DIR, filename)

The WikiCacheData Model

The cache stores a WikiCacheData object containing the full wiki_structure and all generated_pages. This model ensures type-safe serialization and deserialization between the generation pipeline and the API layer.

How CodeWiki Writes to the Cache

When RepositoryStructureFetcher finishes building documentation, it invokes the private method _save_wiki_data_to_cache. This method constructs a JSON payload and posts it to the internal endpoint:

# utils/repository_structure.py – line 735-743

cache_url = f"{TARGET_SERVER_BASE_URL}/api/wiki_cache"
async with httpx.AsyncClient() as client:
    await client.post(cache_url, json=data_to_cache, ...)

The POST /api/wiki_cache handler in api/wiki_cache.py receives the WikiCacheRequest, instantiates WikiCacheData, and writes the serialized JSON to disk using the encoded filename pattern.

How CodeWiki Reads from the Cache

Subsequent requests bypass the expensive generation phase entirely. The GET /api/wiki_cache endpoint invokes read_wiki_cache_data, which checks for the file's existence and deserializes the content:

# api/wiki_cache.py – read_wiki_cache_data()

if os.path.exists(cache_path):
    with open(cache_path, "r", encoding="utf-8") as file:
        data = json.load(file)
        return WikiCacheData(**data)

If no cache file exists, the endpoint returns null, signaling the frontend to trigger a new generation job.

Listing Cached Repositories

The system exposes GET /api/processed_projects to enumerate all cached entries. Implemented in api/processed_projects.py, this endpoint scans WIKI_CACHE_DIR for files matching the *_wiki_cache.json pattern and returns a list of ProcessedProjectEntry records containing owner, repository, and submission timestamps.

Summary

• CodeWiki cache wiki content using a file-based JSON persistence layer stored in ./.cache/wiki_cache/.
• The RepositoryStructureFetcher writes cache entries via POST /api/wiki_cache after generation completes.
• Cache files follow the naming convention {owner}_{repo}_{repo_type}_wiki_cache.json.
• Read operations use GET /api/wiki_cache to deserialize WikiCacheData instantly without regeneration.
• The GET /api/processed_projects endpoint provides visibility into all cached repositories.

Frequently Asked Questions

Where does CodeWiki store cached wiki content?

CodeWiki stores cached content as JSON files in the ./.cache/wiki_cache/ directory, defined by the WIKI_CACHE_DIR constant in utils/constants.py. Each repository receives a unique filename encoding the owner, name, and repository type.

How does CodeWiki handle cache misses?

When GET /api/wiki_cache cannot locate a cache file for the requested repository, the read_wiki_cache_data function returns null. The frontend interprets this response as a signal to initiate a new wiki generation process via the generation pipeline.

Can I manually trigger a cache refresh in CodeWiki?

Yes. Since the cache is updated via a POST request to /api/wiki_cache, you can force a refresh by re-running the repository analysis. The RepositoryStructureFetcher automatically calls _save_wiki_data_to_cache after generation, overwriting any existing cache file with the new wiki structure and generated pages.

What data structure does CodeWiki use for caching?

CodeWiki uses the WikiCacheData Pydantic model to ensure type safety. This model contains the wiki_structure (a WikiStructureModel) and generated_pages (a dictionary mapping page IDs to content), which is serialized to JSON and stored on disk.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}