# How Graphify's Semantic Caching Strategy Minimizes LLM API Calls

> Graphify's semantic caching strategy minimizes LLM API calls by caching graph nodes and edges keyed to file content hashes. It checks the cache before each LLM invocation, only calling the API when source files change or cachin...

- Repository: [Graphify Labs/graphify](https://github.com/Graphify-Labs/graphify)
- Tags: performance
- Published: 2026-07-15

---

**Graphify reduces LLM API usage by caching extracted graph nodes and edges keyed to file content hashes, checking this semantic cache before every LLM invocation, and only calling the API when source files change or caching is explicitly bypassed.**

The **Graphify** open-source project (available at `Graphify-Labs/graphify`) implements an intelligent **semantic caching strategy** that eliminates redundant calls to large language model (LLM) APIs. By persisting the results of expensive semantic extraction operations—specifically the graph nodes and edges derived from source code—Graphify can reuse previous LLM outputs when processing unchanged files. This approach significantly reduces API costs, network latency, and rate-limit issues during iterative development workflows.

## The Semantic Cache Architecture

Graphify's caching mechanism operates on a simple but effective principle: if a file's content has not changed, the semantic graph derived from it remains identical. The system implements this through several coordinated components.

### Hash-Based Cache Keys

For every source file processed, Graphify computes a stable hash of the file's contents combined with its path. This hash serves as the primary key for the **semantic cache**. By using content-derived hashes rather than timestamps, Graphify ensures that even files moved or renamed without modification still match their cached entries. The cache data is stored in a JSON file ([`.graphify_cached.json`](https://github.com/Graphify-Labs/graphify/blob/main/.graphify_cached.json)) located in the specified output directory.

### Pre-Flight Cache Lookup

Before invoking any LLM API, Graphify calls `check_semantic_cache` (defined in [[`graphify/cache.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/cache.py) at line 531](https://github.com/Graphify-Labs/graphify/blob/v8/graphify/cache.py#L531)). This function checks whether the computed hash exists in the cache and returns the previously extracted nodes, edges, and metadata. If a match is found, the LLM call is skipped entirely, and the cached graph fragments are loaded directly into the processing pipeline.

## Implementation Deep Dive

The caching layer is tightly integrated into Graphify's extraction pipeline, ensuring minimal overhead and maximum reusability.

### Core Cache Functions in [`cache.py`](https://github.com/Graphify-Labs/graphify/blob/main/cache.py)

The [[`graphify/cache.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/cache.py)](https://github.com/Graphify-Labs/graphify/blob/v8/graphify/cache.py) module provides three primary functions that manage the cache lifecycle:

- **`check_semantic_cache`** (line 531): Validates cache hits by comparing file hashes against stored entries. Returns cached nodes, edges, and provenance data when available.
- **`save_semantic_cache`** (line 560): Persists newly extracted graph data to the cache file after a successful LLM invocation. This function serializes the nodes and edges alongside their content hashes.
- **`prune_semantic_cache`** (line 493): Removes cache entries for files that no longer exist in the current corpus, preventing unbounded cache growth.

### LLM Integration Layer

The LLM backend in [[`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py)](https://github.com/Graphify-Labs/graphify/blob/v8/graphify/llm.py) integrates directly with the cache utilities. Around line 1916, the code checks the semantic cache before constructing LLM prompts. When `check_semantic_cache` returns a hit, the LLM code path short-circuits, returning the cached result rather than invoking the external API. This integration ensures that expensive LLM calls only occur when the cache misses or when the user explicitly forces a refresh.

### Scoped Writes to Prevent Contamination

To avoid polluting the cache with stale data from unrelated files, Graphify implements scoped cache writes in [[`tools/skillgen/gen.py`](https://github.com/Graphify-Labs/graphify/blob/main/tools/skillgen/gen.py)](https://github.com/Graphify-Labs/graphify/blob/v8/tools/skillgen/gen.py) (lines 905-917). This logic ensures that `save_semantic_cache` only writes entries for files that were actually dispatched for extraction during the current run. By scoping writes to specific file sets, Graphify prevents scenarios where partial runs or targeted extractions might overwrite valid cache entries for other unchanged files.

## Cache Lifecycle Management

Maintaining cache integrity requires periodic cleanup of obsolete entries.

### Pruning Obsolete Entries

After processing, Graphify calls `prune_semantic_cache` (line 493 in [`cache.py`](https://github.com/Graphify-Labs/graphify/blob/main/cache.py)) to remove any cached items whose hashes no longer correspond to files in the current working directory. This garbage collection keeps the cache size bounded and ensures that only relevant, up-to-date semantic data persists across sessions. The pruning operation is idempotent and safe to run multiple times without affecting valid cache entries.

## Practical Usage Examples

Graphify's semantic caching operates transparently during normal extraction workflows, but developers can also interact with it programmatically.

### Automatic Cache Hits

When running Graphify repeatedly on the same codebase, the semantic cache automatically prevents redundant LLM calls:

```python
from graphify import extract
from pathlib import Path

# First run: LLM processes each file and caches results

graph = extract(["src/*.py"], cache_root=Path("output"))

# Second run: Instant cache retrieval, zero LLM calls

graph = extract(["src/*.py"], cache_root=Path("output"))

# Behind the scenes: check_semantic_cache returns cached nodes for unchanged files

```

### Bypassing the Cache

When source files change or you need fresh extractions, Graphify supports cache bypassing:

```python

# Force re-extraction regardless of cache status

graph = extract(["src/*.py"], cache_root=Path("output"), force=True)

# Or manually check cache status before extraction

from graphify.cache import check_semantic_cache

cached_nodes, edges, metadata, hit = check_semantic_cache(
    ["src/main.py"], root=Path("output")
)
if hit:
    print("Cache hit: Loading from semantic cache")
else:
    print("Cache miss: Will invoke LLM API")

```

## Summary

- **Graphify's semantic caching strategy** uses content hashing to cache LLM extraction results, eliminating redundant API calls for unchanged files.
- The **`check_semantic_cache`** function in [`graphify/cache.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/cache.py) (line 531) provides pre-flight lookups that short-circuit LLM invocations when valid cached data exists.
- **`save_semantic_cache`** (line 560) persists new extractions immediately after LLM calls, while **`prune_semantic_cache`** (line 493) garbage-collects stale entries.
- Scoped cache writes in [`tools/skillgen/gen.py`](https://github.com/Graphify-Labs/graphify/blob/main/tools/skillgen/gen.py) (lines 905-917) prevent cross-contamination between extraction runs.
- The integration layer in [`graphify/llm.py`](https://github.com/Graphify-Labs/graphify/blob/main/graphify/llm.py) (around line 1916) ensures seamless cache checking before every LLM operation.

## Frequently Asked Questions

### How does Graphify determine if a file's semantic data is still valid?

Graphify computes a cryptographic hash of each file's content combined with its path. This hash is stored alongside the extracted graph data in the semantic cache. When processing a file, Graphify recalculates the hash and compares it against the cache. If the hashes match, the cached semantic data is considered valid and returned immediately from `check_semantic_cache` without invoking the LLM.

### What happens when I modify a source file and run Graphify again?

When a file's content changes, its hash changes, causing a cache miss in `check_semantic_cache`. Graphify then invokes the LLM API to extract fresh nodes and edges, immediately persisting the new results via `save_semantic_cache` (line 560). The next run with the modified file will return these updated cached values, ensuring the semantic graph always reflects the current source code while still benefiting from caching for unchanged files.

### Can I disable semantic caching when running Graphify?

Yes. The extraction API accepts a `force` parameter that bypasses the cache lookup entirely. When `force=True`, Graphify skips `check_semantic_cache` and proceeds directly to LLM invocation, then overwrites any existing cache entries for the processed files with the new results. This is useful when updating LLM models or when you suspect cached data may be corrupted.

### Where does Graphify store the semantic cache data?

By default, Graphify writes semantic cache data to a file named [`.graphify_cached.json`](https://github.com/Graphify-Labs/graphify/blob/main/.graphify_cached.json) located in the `cache_root` directory (typically your specified output folder). This JSON file contains serialized nodes, edges, content hashes, and metadata. The `prune_semantic_cache` function manages this file's size by removing entries for deleted or modified files, ensuring the cache remains compact and relevant.