How Session Pinning Works in WorkWeave Router: Sticky Routing Architecture Explained

WorkWeave Router uses session pinning to persist routing decisions across multiple turns, enabling sticky provider-model selection, cache-aware token costing, and automatic failover without re-scoring.

Session pinning is the persistence layer that makes routing decisions sticky to a client session. In the workweave/router repository, this mechanism ensures that once a provider-model pair is selected for a session, subsequent turns reuse that decision while tracking usage metrics and handling errors gracefully.

The Core Contract: sessionpin Package

The architecture centers on a strict separation between business logic and storage. In internal/router/sessionpin/store.go, the package defines the abstract Store interface and the Pin data structure that captures a complete routing snapshot.

The Store interface (lines 17-38) specifies seven primary operations:

• Get – Retrieves the current pin for a session-key/role combination.
• Consume – Atomically removes a one-shot pin, used for /force-model continuations.
• Upsert – Inserts a new pin or updates mutable fields like PinnedUntil.
• UpdateUsage – Persists token consumption after a turn completes.
• IncrementUpstreamErrors / IncrementOverloadErrors – Tracks consecutive failures.
• DisableProvider – Adds a provider to the blocked list for the session.
• SweepExpired – Garbage collects stale pins based on PinnedUntil or LastSeenAt.

Any storage backend must satisfy this contract. The reference implementation uses PostgreSQL.

The Pin Data Structure

A sessionpin.Pin (lines 22-84 of store.go) is an immutable snapshot of a routing decision. It binds a session to a specific provider configuration:

• SessionKey – A 16-byte SHA-256 hash derived from the API key plus a per-session UUID.
• Role – Distinguishes between "default" and background turns (e.g., "background"), preventing low-priority requests from stealing high-priority pins.
• Provider / Model – The concrete binding that served the previous turn.
• PairedProvider / PairedModel – The runner-up from the scorer, retained for HMM-driven swaps.
• Strategy / PolicyGroup – The routing strategy that generated the pin.
• Usage counters – LastInputTokens, LastCachedReadTokens, and timestamps (LastTurnEndedAt, PinnedUntil) enable cache-warm detection.
• Error state – ConsecutiveUpstreamErrors, ConsecutiveOverloadErrors, and DisabledProviders manage eviction policies.

Postgres Adapter Implementation

The internal/postgres/session_pin_repo.go file contains postgres.SessionPinRepo, the SQL-backed implementation of the Store interface. Using SQLC-generated queries, the adapter maps Go structs to the router.session_pin table.

Key translation logic lives in the toSessionPin helper (lines 80-100+), which converts database rows into the canonical sessionpin.Pin type. The repository implements atomic operations:

• Upsert translates to an insert-on-conflict update (lines 60-76).
• Consume performs a delete-returning transaction (lines 43-57).
• UpdateUsage records token counts with a no-op fallback when pins are missing (lines 78-100).

Runtime Orchestration in the Turn Loop

During request processing, internal/proxy/turnloop.go orchestrates pin lifecycle management. The proxy.Service executes this workflow for every turn:

1. Fetch existing pin – Calls pinStore.Get(ctx, sessionKey, role) early in the loop.
2. Evaluate cache warmth – The cacheWarm(pin) function (lines 81-89) checks if LastTurnEndedAt falls within the provider's CacheTTLFor window.
3. Compute cache-share – cacheablePrefixTokens(pin, total, prefixBroken) (lines 17-34) calculates reusable tokens from prior cached reads.
4. Apply evidence – applyPinEvidence(&res, pin) (lines 96-100) populates observability fields like PinModel and PinProvider.
5. Persist usage – After streaming completes, pinStore.UpdateUsage records the turn's token consumption, enabling accurate cost accounting for the next request.

When no valid pin exists or the planner elects to switch models, the service calls pinStore.Upsert with a fresh Pin struct, resetting error counters and updating the PinnedUntil TTL.

Error Handling and Eviction Policies

The system implements a two-strike policy for upstream failures. In sessionpin/store.go (lines 29-33), IncrementUpstreamErrors bumps ConsecutiveUpstreamErrors; the turn loop evicts the pin after two non-retryable 4xx errors.

For provider overload (HTTP 529) scenarios, IncrementOverloadErrors tracks exhaustion events. When a threshold is crossed, DisableProvider adds the provider to the session's DisabledProviders list and resets overload strikes (lines 34-36).

Expired pins are garbage collected via runSessionPinSweep (invoked from cmd/router/main.go line 540), which periodically executes pinStore.SweepExpired to purge rows where PinnedUntil or LastSeenAt have passed.

Session Key Generation and Role Separation

Session affinity relies on the SessionKey, a 16-byte hash computed from the API key and a per-request UUID (derived in the auth middleware via sessionKeyFromContext). This ensures that multi-turn conversations maintain continuity even across load-balanced instances.

The Role field (defaulting to sessionpin.DefaultRole from line 18) enables parallel pinning tracks. A "main" turn and a "background" turn for the same session receive distinct pins, isolating failure domains and preventing background tasks from evicting production model bindings.

Summary

• Inner-ring contract – The sessionpin package defines pure data types and the Store interface in internal/router/sessionpin/store.go.
• Persistent storage – postgres.SessionPinRepo implements the contract with SQLC-generated queries in internal/postgres/session_pin_repo.go.
• Sticky routing – The turn loop in internal/proxy/turnloop.go reads pins to reuse provider-model pairs without re-scoring.
• Cache economics – Pins track LastCachedReadTokens to discount prompt costs for warm caches.
• Resilient eviction – Two upstream errors or provider overload triggers automatic pin invalidation via Store interface methods.
• Role isolation – Separate pins for "default" and "background" roles prevent cross-traffic interference.

Frequently Asked Questions

What happens when a session pin expires?

When a pin's PinnedUntil or LastSeenAt timestamp passes, the SweepExpired method removes it from the database during the periodic sweep initiated by runSessionPinSweep in cmd/router/main.go. On the next request, pinStore.Get returns found=false, forcing the planner to re-score and create a fresh pin via Upsert.

How does WorkWeave Router handle provider failures with session pins?

The router implements a two-strike policy for upstream errors. IncrementUpstreamErrors tracks consecutive failures; after two non-retryable 4xx errors, the turn loop treats the pin as stale. For overload errors (HTTP 529), IncrementOverloadErrors counts exhaustion events, and DisableProvider permanently blocks the provider for that session once a threshold is reached.

Can multiple roles share the same session pin?

No. The Role field (defaulting to "default" but also supporting "background") creates distinct namespaces within a session. Each role maintains its own pin in the router.session_pin table, preventing background tasks from interfering with high-priority main turn routing decisions.

How is cache warmth calculated for pinned sessions?

The cacheWarm function in internal/proxy/turnloop.go compares pin.LastTurnEndedAt against the current time using the provider-specific CacheTTLFor value. If the elapsed time is less than the TTL, the session is considered warm, and cacheablePrefixTokens calculates how many tokens can be discounted from the current prompt based on LastCachedReadTokens.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}