Pentagi Caching Strategies: In-Memory TTL, Negative Caching, and LRU Implementation

Pentagi employs lightweight in-process caching mechanisms including TTL-based sync.Map caches with negative caching for authentication data, and bounded LRU caches for GraphQL query parsing, all implemented directly in Go without external dependencies.

The vxcontrol/pentagi repository implements several strategic caching layers to minimize database load and accelerate request handling. These Pentagi caching strategies rely entirely on in-memory data structures rather than external cache servers, prioritizing low latency and operational simplicity. Every cache component is thread-safe and designed with explicit TTL (time-to-live) controls to balance performance against data freshness.

Authentication Caching with TTL and Negative Caching

Pentagi’s authentication layer uses two primary cache types to avoid repeated database lookups for user and token data. Both implementations reside in backend/pkg/server/auth/ and share a common design pattern: sync.Map storage, configurable TTL defaults, and explicit negative caching for missing records.

UserCache Implementation

The UserCache in backend/pkg/server/auth/users_cache.go stores user hashes and status flags with a default TTL of 5 minutes. It leverages Go’s sync.Map for concurrent safety without external locking overhead.

A distinctive feature is negative caching: when a user lookup fails, the cache stores a notFound:true flag instead of leaving the slot empty. Subsequent requests for that user ID return immediately from cache rather than hitting the database again.

// Initialise a user cache (uses the DB instance)
userCache := auth.NewUserCache(db)

// Optional: change the default TTL to 10 minutes
userCache.SetTTL(10 * time.Minute)

// Fetch a user hash – first hit hits the cache, later calls are O(1)
hash, status, err := userCache.GetUserHash(42)
if err != nil {
    // handle missing user or DB error
}

// Invalidate a specific entry (e.g., after a user’s password changes)
userCache.Invalidate(42)

TokenCache for API Authentication

Located in backend/pkg/server/auth/api_token_cache.go, the TokenCache stores API token status, privilege lists, and the same notFound negative-caching pattern. When a token is found, its role-privileges are loaded once and cached alongside the metadata.

Invalidation supports both single tokens (Invalidate) and bulk user revocation (InvalidateUser), the latter being called when an account is disabled.

tokenCache := auth.NewTokenCache(db)

// Retrieve token status and privileges (cached after first DB hit)
status, privs, err := tokenCache.GetStatus("abc123token")
if err != nil {
    // token not found or DB error
}

// Invalidate a single token (e.g., after revocation)
tokenCache.Invalidate("abc123token")

// Invalidate all tokens belonging to a user (called when a user is disabled)
tokenCache.InvalidateUser(userID)

Session and Cryptographic Key Caching

Pentagi avoids redundant cryptographic computation by caching derived keys used for session management.

Cookie Store and JWT Signing Keys

In backend/pkg/server/auth/session.go, two separate sync.Map instances—cookieStoreKeys and jwtSigningKeys—cache derived cryptographic material per salt value. This prevents repeated key-derivation work for every request while keeping sensitive material in application memory rather than serializing it to external storage.

GraphQL Query Caching with LRU

Parsing GraphQL queries is CPU-intensive. Pentagi implements bounded LRU (Least-Recently-Used) caches in backend/pkg/server/services/graphql.go to store parsed query structures and persisted query results.

Parsed Query Documents

The GraphQL service configures an LRU cache with capacity for 1,000 parsed query documents using lru.New[*ast.QueryDocument](1000). This cache is wired into the server via srv.SetQueryCache(), ensuring that identical queries skip the parser after their first execution.

Automatic Persisted Queries

For automatic persisted queries, Pentagi maintains a secondary LRU cache sized to 100 entries (lru.New[string](100)). This stores the string payloads of persisted queries, reducing network overhead and parsing time for repeated operations.

svc := services.NewGraphqlService(
    dbQueries,
    cfg,
    "/api",
    []string{"https://pentagi.example.com"},
    tokenCache,
    providersCtrl,
    flowCtrl,
    subsCtrl,
)

// The service internally caches parsed queries:
//   srv.SetQueryCache(lru.New[*ast.QueryDocument](1000))
//   AutomaticPersistedQuery uses a second LRU cache (size 100)

Additional Caching Patterns

TLS Certificate Caching

The test harness in backend/pkg/tools/proxy_test.go demonstrates a certCache pattern using sync.Map to store *tls.Certificate objects keyed by hostname. While primarily used in testing, this illustrates the repository’s consistent preference for simple, in-memory maps for expensive-to-generate resources.

Summary

• In-memory sync.Map provides thread-safe, low-latency storage for authentication and cryptographic data without external dependencies.
• TTL-based expiration defaults to 5 minutes for user and token caches, configurable per instance via SetTTL.
• Negative caching prevents database thrashing by storing explicit "not found" markers for missing users and tokens.
• LRU eviction caps memory usage for GraphQL parsing at 1,000 query documents and 100 persisted queries.
• Manual invalidation supports both single-entry and bulk-user eviction patterns for immediate security updates.

Frequently Asked Questions

What is the default TTL for Pentagi caches?

The default TTL is 5 minutes (300 seconds) for both UserCache and TokenCache. This value is configurable at runtime by calling SetTTL(duration) on the cache instance.

How does Pentagi handle cache invalidation?

Each cache exposes explicit invalidation methods. For example, userCache.Invalidate(id) removes a specific user, while tokenCache.InvalidateUser(userID) clears all tokens belonging to a user. The service layer in backend/pkg/server/services/api_tokens.go invokes these methods automatically when tokens are created, updated, or deleted.

Why does Pentagi use sync.Map instead of Redis?

Pentagi prioritizes operational simplicity and zero network latency for its caching layer. Using sync.Map keeps the cache in-process, eliminating external dependencies and network round-trips. This design suits single-instance deployments and avoids cache coherence complexity for authentication data that changes relatively infrequently.

What is negative caching and why does Pentagi use it?

Negative caching stores the result of a failed lookup (e.g., a missing user or invalid token) rather than leaving the cache empty. Pentagi marks these entries with notFound:true, preventing repeated database queries for non-existent records. This defends against cache stampedes during authentication attempts with invalid credentials or tokens.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}