# Prompt Cache Affinity in Grok2API: How It Works with Codex and Claude Code

> Discover prompt cache affinity in Grok2API. Learn how this session-binding mechanism ensures deterministic prompt reuse across Codex and Claude Code for improved performance.

- Repository: [Chenyme/grok2api](https://github.com/chenyme/grok2api)
- Tags: deep-dive
- Published: 2026-08-09

---

**Prompt cache affinity is a session-binding mechanism that ties a request's cached prompt to a specific backend account and egress node using a sticky store, ensuring deterministic reuse across Codex and Claude Code sessions.**

The `chenyme/grok2api` repository implements prompt cache affinity to maintain secure, stateful connections between client sessions and backend infrastructure. This mechanism ensures that cached prompts remain isolated to specific accounts while enabling efficient reuse across multiple turns in AI coding assistants.

## What Is Prompt Cache Affinity?

**Prompt cache affinity** refers to the architectural pattern of binding a cached prompt (or "turn") to a specific egress account and compute node for the duration of a session. Rather than treating each request as stateless, the system extracts a unique affinity key from incoming headers and uses it to route subsequent requests to the same backend resources. This guarantees that cached tokens remain available and are never shared across different user accounts or sessions.

## How Codex and Claude Code Supply the Affinity Key

Client applications provide the affinity key through HTTP headers that contain structured metadata about the current conversation turn.

### Codex Turn Metadata

For **Codex** sessions, the client sends the `X-Codex-Turn-Metadata` header containing a JSON payload. In [`backend/internal/infra/provider/cli/normalize.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/provider/cli/normalize.go), the API parses this header to extract the `prompt_cache_key` field, which serves as the primary affinity identifier.

```go
// Extracted from header parsing logic
turnMeta := request.Header.Get("X-Codex-Turn-Metadata")
var meta struct {
    PromptCacheKey string `json:"prompt_cache_key"`
    WindowID       string `json:"window_id"`
}
json.Unmarshal([]byte(turnMeta), &meta)
affinityKey := meta.PromptCacheKey

```

### Claude Code Session Headers

**Claude Code** follows an equivalent pattern, supplying the `prompt_cache_key` via a similar metadata header (e.g., `X-Claude-Turn-Metadata`). The parsing logic in [`backend/internal/infra/provider/cli/normalize.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/provider/cli/normalize.go) handles both variants, ensuring the affinity key is extracted regardless of which client initiates the request.

## The Sticky Store: Binding Keys to Accounts

Once extracted, the affinity key is stored in the **sticky store** implemented in [`backend/internal/infra/runtime/memory/store.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/runtime/memory/store.go). This component maintains the mapping between cache keys and backend accounts.

The `StickyStore.Bind` method records the association between the affinity key, the selected account ID, and an expiration timestamp:

```go
// Binding the affinity key to a specific account
if affinityKey != "" {
    sticky.Bind(ctx, affinityKey, accountID, time.Now(), expiresAt)
}

```

Subsequent requests carrying the same key trigger `StickyStore.Get`, which retrieves the bound account ID to ensure the cached prompt is accessed only from the original backend context:

```go
// Retrieving the bound account for cache reuse
boundID, ok, _ := sticky.Get(ctx, affinityKey, time.Now())
if ok {
    // Route to the same account to leverage cached tokens
}

```

## Deterministic Node Selection

The gateway selector enforces affinity at the node level through deterministic hashing. In [`backend/internal/application/gateway/selector.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/application/gateway/selector.go), the `selectNode` function hashes the affinity key using `sha256.Sum256` to choose a specific egress node from the available pool.

This deterministic selection ensures that all requests sharing the same `prompt_cache_key` route to the identical backend instance, maximizing cache hit rates while preventing distribution of sensitive cached data across multiple nodes.

## Security Boundaries and Fallback Handling

The system implements strict security controls to prevent cross-account cache leakage. As noted in [`adapter_test.go`](https://github.com/chenyme/grok2api/blob/main/adapter_test.go), the code explicitly avoids "fabricating a random conv-id without a session key" because doing so would break server affinity and force `cached_tokens` to zero.

When the affinity key is missing or the bound account has expired, the request falls back to the **soft identity** path. This preserves basic account affinity but does **not** reuse cached prompts, ensuring that stale or unauthorized cache entries are never served.

## Implementation Flow in Code

The following Go snippet illustrates the complete flow for a Codex request, from header extraction to sticky store binding:

```go
package main

import (
    "crypto/sha256"
    "encoding/json"
    "time"
)

// 1. Extract metadata from Codex header
turnMeta := request.Header.Get("X-Codex-Turn-Metadata")
var meta struct {
    PromptCacheKey string `json:"prompt_cache_key"`
}
json.Unmarshal([]byte(turnMeta), &meta)

// 2. Validate and bind affinity key
affinityKey := meta.PromptCacheKey
if affinityKey != "" {
    // backend/internal/infra/runtime/memory/store.go
    sticky.Bind(ctx, affinityKey, accountID, time.Now(), expiresAt)
    
    // 3. Deterministic node selection via selector.go
    hash := sha256.Sum256([]byte(affinityKey))
    node := selectNode(pool, hash)
}

```

This pattern is replicated for Claude Code in [`backend/internal/infra/provider/cli/adapter.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/provider/cli/adapter.go), where the affinity key is attached to outbound requests before routing.

## Summary

- **Prompt cache affinity** binds cached prompts to specific accounts and nodes using a `prompt_cache_key` extracted from client headers.
- **Codex** sends the key via `X-Codex-Turn-Metadata`, while **Claude Code** uses an equivalent header structure.
- The **sticky store** ([`backend/internal/infra/runtime/memory/store.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/runtime/memory/store.go)) persists key-to-account mappings with expiration timestamps via `Bind` and `Get` methods.
- **Deterministic node selection** uses SHA-256 hashing of the affinity key in [`backend/internal/application/gateway/selector.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/application/gateway/selector.go) to route requests consistently.
- **Security safeguards** prevent cache leakage by falling back to soft identity when keys are missing, as enforced in the adapter layer.

## Frequently Asked Questions

### What happens if the prompt_cache_key is missing from the request?

If the `prompt_cache_key` is absent or the sticky store entry has expired, the system falls back to the **soft identity** path. This maintains account-level routing but disables prompt cache reuse, ensuring `cached_tokens` remains at zero and preventing potential data leakage between sessions.

### How does prompt cache affinity improve performance?

By binding a `prompt_cache_key` to a specific account and node, subsequent requests with the same key route to the same backend instance where the prompt is already cached. This eliminates redundant processing and token generation, reducing latency for multi-turn conversations in Codex and Claude Code sessions.

### Is the affinity key shared between different users?

No. The sticky store explicitly binds each affinity key to a specific account ID in [`backend/internal/infra/runtime/memory/store.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/runtime/memory/store.go). The system guarantees isolation by checking the bound account before reusing cached tokens, ensuring that prompts from one user are never served to another, even if they somehow obtain the same cache key.

### Which files handle the header parsing for Codex?

The primary header parsing occurs in [`backend/internal/infra/provider/cli/normalize.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/provider/cli/normalize.go), which extracts the `X-Codex-Turn-Metadata` header and unmarshals the JSON to retrieve the `prompt_cache_key`. The adapter logic in [`backend/internal/infra/provider/cli/adapter.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/infra/provider/cli/adapter.go) subsequently attaches this key to outbound requests, while [`backend/internal/transport/http/inference/prompt_cache_test.go`](https://github.com/chenyme/grok2api/blob/main/backend/internal/transport/http/inference/prompt_cache_test.go) contains test cases validating the Codex turn behavior.