What Is the TTL for WhichLLM's Model Cache? A 6-Hour Technical Deep Dive

The TTL for WhichLLM's model cache is exactly 6 hours (21,600 seconds), enforced via the DEFAULT_TTL_SECONDS constant in src/whichllm/models/cache.py.

WhichLLM manages downloaded model metadata in a local JSON cache that automatically invalidates entries after a fixed time window. Understanding the TTL for WhichLLM's model cache is essential for predicting API call behavior and ensuring your application accesses fresh model data. The Andyyyy64/whichllm repository implements this expiration logic to minimize unnecessary network requests while preventing stale model information.

Where the TTL Is Defined in Source Code

The expiration logic centers on a single constant defined in the cache implementation file.

The DEFAULT_TTL_SECONDS Constant

In src/whichllm/models/cache.py at line 15, the cache module declares:

DEFAULT_TTL_SECONDS = 6 * 3600  # 21,600 seconds = 6 hours

This calculation explicitly sets the TTL for WhichLLM's model cache to six hours. The value is used throughout the ModelCache class to determine whether a cached entry remains valid or requires refresh from the remote API.

How Cache Expiration Works

When you query model data, the cache compares the current timestamp against the entry's stored timestamp. If the difference exceeds DEFAULT_TTL_SECONDS, the cache discards the JSON data and fetches fresh metadata.

To retrieve models while respecting the 6-hour TTL:

from whichllm.models.cache import ModelCache

# Initialize cache (uses default 6-hour TTL automatically)

cache = ModelCache()

# Returns cached data if < 6 hours old, otherwise fetches fresh

model_info = cache.get("gpt-4")
print(model_info)

Verifying the TTL Programmatically

You can inspect the exact TTL value by importing the constant directly:

from whichllm.models.cache import DEFAULT_TTL_SECONDS

hours = DEFAULT_TTL_SECONDS / 3600
print(f"Model cache TTL: {hours:.0f} hours")  # Output: 6 hours

This confirms the cache expires after exactly six hours as implemented in the source code.

Summary

• The TTL for WhichLLM's model cache is fixed at 6 hours (21,600 seconds)
• The constant DEFAULT_TTL_SECONDS is defined in src/whichllm/models/cache.py at line 15
• The ModelCache class automatically refreshes entries when they exceed this TTL
• This duration balances data freshness with reduced API call frequency during development sessions

Frequently Asked Questions

How long does WhichLLM cache model data before refreshing?

The cache retains model metadata for exactly 6 hours. After this period, the next access to that specific model entry triggers a fresh download from the remote API to ensure current information.

Where is the TTL for WhichLLM's model cache defined in the codebase?

The TTL is defined as DEFAULT_TTL_SECONDS = 6 * 3600 in src/whichllm/models/cache.py at line 15, according to the Andyyyy64/whichllm source code.

What happens when a cached model entry expires?

When an entry exceeds the 6-hour TTL, the ModelCache.get() method automatically discards the stale JSON file and performs a new API request. It then stores the updated metadata with a fresh timestamp.

Can I modify the default cache TTL?

The DEFAULT_TTL_SECONDS value is hardcoded in the source file. To change the TTL for WhichLLM's model cache, you would need to modify this constant in src/whichllm/models/cache.py or extend the ModelCache class to accept a custom TTL parameter.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}