What Is Trace Context Tracking in CCX? A Technical Deep Dive

Trace context tracking in CCX is a mechanism that maintains "trace-affinity" mappings between users and API channels, enabling subsequent requests from the same conversation to reuse previously successful channels with 30-minute TTL expiration.

CCX implements intelligent request routing by preserving channel preferences per user and request type, ensuring low-latency retries and consistent performance across multi-channel AI deployments. This article examines the implementation details found in the BenedictKing/ccx repository, explaining how the TraceAffinityManager and ChannelScheduler collaborate to optimize API request distribution.

Core Components of Trace Context Tracking

TraceAffinityManager

The TraceAffinityManager defined in backend-go/internal/session/trace_affinity.go serves as the persistence layer for affinity data. It maintains a thread-safe map that stores channel preferences using composite keys formatted as <kind>:<userID>.

The manager handles four critical responsibilities:

• Mapping storage: Associates users with their preferred channel indices per request type (messages, responses, gemini, chat, images)
• TTL enforcement: Automatically expires entries after 30 minutes of inactivity
• Periodic cleanup: Runs a cleanupLoop every 5 minutes to purge stale records and prevent memory leaks
• Query interface: Provides GetPreferredChannel, SetPreferredChannel, and UpdateLastUsed methods for read/write operations

ChannelScheduler Integration

The ChannelScheduler in backend-go/internal/scheduler/channel_scheduler.go consumes the affinity data during the routing decision process. During initialization in backend-go/main.go (lines 29-35), the scheduler receives a singleton instance of TraceAffinityManager, ensuring consistent state across all request handling goroutines.

How Trace Affinity Works

Data Structure and Storage

Each affinity record uses the TraceAffinity struct:

type TraceAffinity struct {
    ChannelIndex int       // Selected channel index in the upstream pool
    LastUsedAt   time.Time // Last access timestamp for TTL calculation
}

The manager stores these in a map[string]*TraceAffinity where the key combines the channel kind and user identifier. This design allows separate affinity tracking across different API endpoints while maintaining user isolation.

TTL and Cleanup Mechanism

The system implements automatic resource reclamation through configurable expiration:

• Default TTL: 30 minutes (ttl: 30 * time.Minute in NewTraceAffinityManager)
• Cleanup interval: 5 minutes between garbage collection cycles
• Thread safety: All map operations use mutex locking to prevent race conditions during concurrent API requests

When UpdateTraceAffinity is called, the manager resets the LastUsedAt timestamp, extending the affinity window for active conversations.

The Selection Flow

During channel selection, SelectChannel evaluates trace affinity as a fallback mechanism. The decision logic in channel_scheduler.go follows this priority order:

1. Manual override: X-Channel headers specifying explicit channels
2. Promotional channels: High-priority routes that bypass health checks
3. Trace affinity: Reuse previous channel if promotional routes fail or are unavailable
4. Priority scheduling: Configured priority values from channel definitions
5. Fallback selection: Channels with lowest failure rates

The affinity check occurs only when:

if userID != "" {
    compositeKey := string(kind) + ":" + userID
    if preferredIdx, ok := s.traceAffinity.GetPreferredChannel(compositeKey); ok {
        // Verify channel health and availability before returning
        return &SelectionResult{
            Upstream:     upstream,
            ChannelIndex: preferredIdx,
            Reason:       "trace_affinity",
        }, nil
    }
}

The scheduler validates that the preferred channel remains active, healthy (no circuit breaker trips), and represents the best available option before committing to the affinity route.

Recording Successful Channels

After processing a successful request, business handlers invoke the affinity recording methods:

func (s *ChannelScheduler) SetTraceAffinity(userID string, channelIndex int, kind ChannelKind) {
    if userID != "" {
        compositeKey := string(kind) + ":" + userID
        s.traceAffinity.SetPreferredChannel(compositeKey, channelIndex)
    }
}

For ongoing conversation threads, handlers call UpdateTraceAffinity at request start to refresh the TTL timer:

func (s *ChannelScheduler) UpdateTraceAffinity(userID string, kind ChannelKind) {
    if userID != "" {
        compositeKey := string(kind) + ":" + userID
        s.traceAffinity.UpdateLastUsed(compositeKey)
    }
}

These invocations typically occur in handlers such as messages.Handler and responses.Handler, ensuring that multi-turn conversations maintain channel consistency.

Monitoring and Debugging

CCX exposes affinity statistics through the dashboard API endpoint /api/messages/channels/dashboard. The response includes:

{
    "traceAffinityCount": 12,
    "traceAffinityTTL": "30m0s"
}

The implementation resides in channel_scheduler.go via GetTraceAffinityManager and the HTTP handler in backend-go/internal/handlers/channel_dashboard.go. This visibility helps operators identify how many active affinity relationships exist in the system.

Practical Implementation Examples

Setting Affinity After Successful Requests

Business logic handlers update affinity records following successful API calls:

// After processing a successful message request
func afterSuccess(ctx context.Context, userID string, kind scheduler.ChannelKind, chIdx int) {
    // scheduler is the injected *ChannelScheduler instance from main.go
    scheduler.SetTraceAffinity(userID, chIdx, kind)
    scheduler.UpdateTraceAffinity(userID, kind) // Optional TTL renewal
}

Route Request With Affinity Check

The routing layer automatically evaluates affinity during channel selection:

func routeRequest(userID string, kind scheduler.ChannelKind) (*scheduler.SelectionResult, error) {
    failedChannels := make(map[int]bool)
    
    result, err := scheduler.SelectChannel(
        context.Background(),
        userID,
        failedChannels,
        kind,
        "", // model parameter
        "", // routePrefix
        "", // channelName
    )
    
    if err != nil {
        return nil, err
    }
    
    // Check if affinity routing was applied
    if result.Reason == "trace_affinity" {
        log.Printf("Routed via trace affinity to channel %d", result.ChannelIndex)
    }
    
    return result, nil
}

Why Trace Context Tracking Matters

Latency reduction: Subsequent requests bypass expensive health checks and priority calculations by reusing validated channels.

Success rate optimization: In multi-channel environments, specific channels often perform better for particular user contexts (token limits, model preferences). Affinity preserves these successful pairings.

Operational visibility: Requests routed via affinity include Reason: "trace_affinity" in logs and metrics, enabling rapid identification of traffic distribution patterns.

User experience consistency: Maintaining the same channel across conversation turns preserves context-specific optimizations and caching benefits.

Summary

• TraceAffinityManager in trace_affinity.go maintains user-to-channel mappings with 30-minute TTL expiration and 5-minute cleanup cycles.
• ChannelScheduler evaluates affinity as the third priority tier, after manual overrides and promotional channels but before generic priority scheduling.
• Composite keys using <kind>:<userID> format enable separate affinity tracking per API endpoint type (messages, chat, images).
• Recording methods SetTraceAffinity and UpdateTraceAffinity are invoked by business handlers after successful request processing.
• Dashboard endpoint /api/messages/channels/dashboard exposes real-time affinity statistics for operational monitoring.

Frequently Asked Questions

How long does CCX retain trace affinity data?

CCX retains trace affinity records for 30 minutes by default. The TraceAffinityManager initializes with ttl: 30 * time.Minute and runs a cleanup loop every 5 minutes to purge expired entries. Active requests refresh this timer via UpdateLastUsed, keeping the affinity window open for ongoing conversations.

When does the scheduler ignore trace affinity recommendations?

The scheduler bypasses trace affinity when: (1) no userID is present in the request, (2) the preferred channel has become unhealthy or inactive, (3) a higher-priority channel (manual override or promotional) is available, or (4) the affinity record has expired past the TTL threshold. The system always validates channel health before committing to an affinity route.

Can operators view current trace affinity mappings?

Yes. CCX exposes affinity statistics through the /api/messages/channels/dashboard endpoint, which returns the current count of active affinity relationships and the configured TTL duration. The underlying implementation in channel_dashboard.go and GetTraceAffinityManager provides this visibility without exposing sensitive user mapping details.

What happens when a preferred channel fails during affinity routing?

If an affinity-preferred channel fails health checks during SelectChannel, the scheduler automatically falls back to the standard priority-based selection logic. The failed channel is marked in the failedChannels map, and the request proceeds to the next available option. This ensures that affinity preferences do not compromise request success rates when channels degrade.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}