How the TTL Cache Works in `ctx_fetch_and_index`: 24-Hour Cache Invalidation Logic

The ctx_fetch_and_index function in context-mode implements a 24-hour TTL cache that prevents refetching URLs indexed within the last 24 hours, automatically invalidating stale entries and allowing forced refreshes via the force: true parameter.

The ctx_fetch_and_index tool in the context-mode repository manages web content indexing with an efficient time-based caching mechanism. Understanding how this TTL cache determines content freshness helps optimize API usage and prevent unnecessary network requests when working with indexed sources.

Cache Lookup and Metadata Retrieval

When ctx_fetch_and_index executes, it first checks whether the caller explicitly requested a fresh fetch.

The force Parameter Check

In src/server.ts (lines 1416-1419), the function immediately bypasses the cache logic if force: true is provided in the arguments. This flag skips all TTL validation and proceeds directly to the network fetch path.

Retrieving Stored Metadata

For standard invocations, the function retrieves existing metadata using store.getSourceMeta(label), where the label is either the user-provided source identifier or the URL itself (lines 1425-1428). This metadata includes the critical indexedAt timestamp stored as UTC without timezone suffix.

TTL Calculation and Freshness Validation

The cache validation relies on comparing elapsed time against a hardcoded 24-hour threshold.

Age Calculation Logic

The implementation converts the indexedAt string to a JavaScript Date object and calculates ageMs as Date.now() - indexedAt.getTime() (lines 1429-1432). This millisecond value represents the exact time elapsed since the last successful indexing operation.

The 24-Hour Threshold

A constant TTL_MS = 24 * 60 * 60 * 1000 defines the cache lifetime. When ageMs < TTL_MS evaluates to true (lines 1431-1433), the entry is considered fresh and the function skips the network request entirely.

Cache Hit Behavior and Stale Entry Handling

The function distinguishes between fresh cache hits and expired entries with different execution paths.

Fresh Cache Response

For valid cache hits, ctx_fetch_and_index updates sessionStats.cacheHits and sessionStats.cacheBytesSaved (calculated as meta.chunkCount * 1600) before returning a concise preview message. This response informs the user of the source name, total chunk count, and cache age (e.g., "2h ago") without returning the actual content—the caller must invoke search() to query the indexed material (lines 1434-1446).

Stale Entry Re-Fetching

When the age exceeds 24 hours, execution falls through to the comment "Stale (>24h) — fall through to re-fetch silently" (lines 1448-1449). The function then proceeds with the normal fetch path, downloading fresh content, re-indexing it, and updating the stored metadata with a new indexedAt timestamp.

Practical Implementation Examples

The following TypeScript examples demonstrate the cache behavior in the context-mode runtime:

// Standard invocation - uses cache if indexed within 24 hours
await ctx_fetch_and_index({
  url: "https://developer.mozilla.org/en-US/docs/Web/JavaScript",
  source: "MDN JavaScript Docs"
});

// Force refresh - bypasses TTL cache regardless of age
await ctx_fetch_and_index({
  url: "https://developer.mozilla.org/en-US/docs/Web/JavaScript",
  source: "MDN JavaScript Docs",
  force: true  // Skips cache validation in src/server.ts lines 1416-1419
});

When the first example runs twice within a 24-hour window, the second call returns immediately from cache with no network activity. After 24 hours elapse, or when force: true is specified, the tool performs a complete re-fetch and re-indexing operation.

Summary

• 24-hour TTL: Cache entries expire automatically after 86,400,000 milliseconds (24 hours), controlled by the TTL_MS constant in src/server.ts.
• Label-based indexing: Cache lookups use store.getSourceMeta(label) where labels map to either explicit source names or raw URLs.
• Silent invalidation: Expired entries trigger automatic refetching without error states, falling through to the standard download logic.
• Explicit bypass: The force: true parameter unconditionally invalidates cache entries, useful for immediate content updates.
• Statistics tracking: Each cache hit updates sessionStats with hit counts and estimated bytes saved based on chunk count metrics.

Frequently Asked Questions

What happens when the TTL cache expires in ctx_fetch_and_index?

When a cached entry exceeds the 24-hour TTL threshold, ctx_fetch_and_index silently treats it as stale and proceeds to the normal fetch path. The function downloads the URL content fresh, indexes it, stores new metadata with an updated indexedAt timestamp, and returns the full preview rather than a cache hit message.

How does the force parameter interact with the TTL cache?

Setting force: true in the function arguments bypasses the entire cache validation block located at lines 1416-1419 in src/server.ts. This forces an immediate network request and re-indexing regardless of when the content was last cached, effectively treating every invocation as a cache miss.

What metadata is stored for cache validation?

The cache stores an indexedAt timestamp (UTC format without timezone suffix) and chunkCount via the store.getSourceMeta system. The timestamp enables TTL calculations while the chunk count feeds into sessionStats.cacheBytesSaved calculations, estimating 1600 bytes saved per chunk on cache hits.

Can I adjust the 24-hour TTL duration?

The current implementation hardcodes TTL_MS = 24 * 60 * 60 * 1000 as a constant in src/server.ts. Modifying this value requires changing the source code, as no runtime configuration parameter exists to adjust the cache duration dynamically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}