How Sticky Session Routing Preserves Client Session Continuity in Grok

Sticky session routing in Grok binds each client to a specific backend account for a configurable time window by storing an affinity key-to-account mapping in either an in-memory or Redis-backed store, ensuring all requests from that client hit the same node until the binding expires.

Grok, the open-source API gateway implementation found in the chenyme/grok2api repository, uses sticky session routing to maintain stateful connections between clients and backend services. This mechanism prevents request scattering across multiple accounts, which is critical for applications requiring consistent session state. By leveraging a three-tier architecture of configuration, storage, and selection logic, Grok guarantees that a client's session remains pinned to a single backend account throughout the duration of the sticky time-to-live (TTL).

Core Components of Sticky Session Routing

The sticky session implementation relies on three tightly-coupled layers that work together to maintain client affinity.

Sticky Configuration

The routing behavior is governed by the StickyTTL field defined in backend/internal/infra/config/config.go at line 213. This configuration setting determines how long a client remains bound to a specific account before the binding expires and allows re-balancing. The default value is typically set to 1 hour (expressed as "1h" in JSON configuration payloads), though operators can adjust this based on their session requirements.

Sticky Store Implementations

Grok provides two production-ready backends for storing sticky bindings, both implementing the same interface to map affinity keys to account IDs with expiration timestamps:

In-Memory Store: Located in backend/internal/infra/runtime/memory/store.go (lines 187-205), this implementation uses a sharded map structure containing stickyBinding structs with accountID and expiresAt fields. The store actively prunes expired entries using pruneStickyBindingsLocked to prevent memory leaks.

Redis Store: Found in backend/internal/infra/runtime/redis/store.go (lines 610-686), this distributed implementation stores bindings as Redis keys with the format sticky:<key> and leverages Lua scripts (deleteStickyByAccountScript) to handle atomic operations and deduplication when the same account appears under multiple keys.

Selector and Egress Manager

The backend/internal/application/gateway/selector.go orchestrates the routing logic by calling sticky.Get to check for existing bindings and sticky.Bind to create new ones. Once a binding is established, the sticky flag propagates to the Lease object in backend/internal/infra/egress/manager.go (lines 1098 and 1159). This ensures the low-level HTTP client honors the sticky session when managing proxy connections and tunnel establishment.

How Sticky Session Routing Works

The workflow follows a deterministic lookup-and-bind pattern that executes on every incoming request:

1. Affinity Key Extraction: The HTTP handler extracts a unique identifier from the incoming request—typically a cookie value or authentication token—to serve as the affinity key.

2. Sticky Lookup: The selector queries the store using sticky.Get(ctx, key, now). If an unexpired binding exists, the stored accountID is returned immediately and the request routes to that specific account.

3. Dynamic Binding: When no valid binding exists, the selector applies standard load-balancing algorithms to choose an account, then persists the mapping via sticky.Bind(ctx, key, accountID, now, now+TTL), setting the expiration based on the configured stickyTTL.

4. Request Routing: Subsequent requests bearing the same affinity key hit the existing binding in step 2, ensuring continuity by routing to the identical backend node.

5. Automatic Expiration: When the current time surpasses the binding's expiresAt timestamp, the store evicts the entry (either through lazy deletion in memory or Redis TTL), forcing the next request to trigger a fresh load-balancing decision.

Implementation Examples

To enable sticky session routing in your Grok deployment, configure the TTL and inject the appropriate store implementation:

// Configuration structure for sticky routing
type SettingsDTO struct {
    Routing struct {
        StickyTTL string `json:"stickyTTL"` // e.g., "1h", "30m"
    } `json:"routing"`
}

// Initialize in-memory sticky store
stickyStore := memory.NewStickyStore()

// Alternative: Redis-backed store for distributed deployments
// stickyStore := redis.NewStickyStore(redisClient)

// Inject into the gateway selector
selector := gateway.NewSelector(
    accountRepo,
    concurrencyLimiter,
    stickyStore,          // Sticky session store
    registry,
    time.Hour,           // Request timeout
    time.Second,         // Poll interval
    time.Minute,         // Retry back-off
)

Inside your request handler, implement the sticky routing logic:

// Extract affinity identifier from request context
affinityKey := extractAffinityKey(r)

// Check for existing sticky binding
if accountID, found, err := stickyStore.Get(ctx, affinityKey, time.Now().UTC()); found && err == nil {
    // Route to existing bound account
    targetAccount = accountID
} else {
    // Create new binding for load-balanced account
    targetAccount = selector.PickAccount()
    expiry := time.Now().UTC().Add(config.Routing.StickyTTL)
    _ = stickyStore.Bind(ctx, affinityKey, targetAccount, time.Now().UTC(), expiry)
}

Configuration and Persistence Details

The stickyTTL value accepts Go duration strings (e.g., "1h", "30m") parsed by the settings handler in backend/internal/transport/http/settings/handler.go. When using the Redis implementation, bindings automatically inherit Redis's native expiration mechanisms, providing consistency across gateway restarts. The in-memory implementation requires careful consideration of shard sizing and pruning frequency to maintain performance under high concurrency, as implemented in the stickyShard structure with its bindings map.

Summary

• Sticky session routing maintains client continuity by binding affinity keys to specific backend accounts for a configurable duration.
• The system uses a three-tier architecture: configuration (StickyTTL), storage (in-memory or Redis), and selection logic (selector and egress manager).
• Source locations: Configuration resides in config.go (line 213), stores in memory/store.go (lines 187-205) and redis/store.go (lines 610-686), and routing logic in selector.go and manager.go.
• Default TTL is 1 hour, after which bindings expire and clients may be re-balanced to new accounts.
• The Redis implementation supports distributed deployments using Lua scripts for atomic operations, while the in-memory store offers lower latency for single-instance deployments.

Frequently Asked Questions

How does Grok handle sticky session expiration?

When a sticky binding reaches its expiration timestamp, the store automatically evicts the entry. In the in-memory implementation (memory/store.go), the pruneStickyBindingsLocked method removes expired entries during lookup operations. The Redis implementation relies on Redis's native TTL expiration on keys prefixed with sticky:. Once expired, subsequent requests with the same affinity key trigger a fresh account selection via the selector's load-balancing logic.

Can I use sticky sessions in a distributed Grok deployment?

Yes, but you must use the Redis-backed sticky store rather than the in-memory implementation. The Redis store (redis/store.go, lines 610-686) uses atomic Lua scripts to ensure consistent binding state across multiple Grok instances. Simply replace memory.NewStickyStore() with redis.NewStickyStore(redisClient) when initializing your selector to enable distributed sticky session support.

What happens if the bound backend account becomes unhealthy?

The selector checks account health before honoring a sticky binding. If the bound account fails health checks, the selector bypasses the sticky mapping and performs a fresh load-balanced selection, then updates the binding with the new healthy account via sticky.Bind. This ensures that sticky sessions do not trap clients on failed nodes while maintaining continuity during normal operations.

Where is the sticky flag propagated in the request lifecycle?

According to backend/internal/infra/egress/manager.go (lines 1098 and 1159), the sticky boolean is attached to the Lease object that represents the concrete backend connection. The manager detects sticky requirements by checking if the proxy URL contains application.ProxyAccountPlaceholder, then constructs the lease with the sticky flag enabled. This allows the HTTP client layer to maintain connection affinity even when managing proxy pools or establishing new tunnels.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}