# How to Implement Conversation Affinity and Turn-Based Routing in Switchyard

> Implement conversation affinity and turn-based routing in Switchyard using AffinityRouter for sticky sessions and StageRouter for per-turn routing decisions. Learn how to enhance your conversational AI.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**To implement conversation affinity and turn-based routing in Switchyard, use `AffinityRouter` for sticky session routing and `StageRouter` for per-turn decision-making based on tool signals or LLM judges.**

Switchyard provides two orthogonal routing mechanisms that work together to optimize multi-turn LLM conversations. **Conversation affinity** ensures a user's session stays bound to a previously selected model, while **turn-based routing** dynamically chooses models for each turn based on tool-call outcomes or capability judgments. This guide walks through the implementation using the NVIDIA-NeMo/Switchyard source code.

## What Conversation Affinity Does

The `AffinityRouter` in [`crates/libsy/src/algorithms/util/affinity.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/affinity.rs) implements sticky routing by serving dual roles: as a **processor** that records the first routing decision, and as a **classifier** that returns that same decision on subsequent requests.

When enabled, the router:
- Derives a **routing identity** from `session_id` + `agent_id`, or from a hash of the first user message when `message_hash_fallback=True`
- Stores the selected model on the first **Decision** event
- Returns a **Score** with confidence `1.0` for the retained model on subsequent matching requests

If no usable identity exists, the router emits a one-time warning and abstains, letting the outer cascade decide normally.

## Enabling Session Affinity in LlmTaskClassifier

The `llm_task_classifier` Python binding automatically configures `AffinityRouter` when `session_affinity=True` in `TaskClassifierConfig`. In [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs) lines 608-618, the construction logic registers the router twice:

```python
from switchyard.libsy import llm_task_classifier, TaskClassifierConfig

classifier = llm_task_classifier(
    judge_target="judge",
    efficient_target="efficient",
    capable_target="capable",
    config=TaskClassifierConfig(
        base_threshold=0.5,
        session_affinity=True,          # Activates AffinityRouter

        message_hash_fallback=True,     # Hash first message if no session

    ),
)

```

The `AffinityRouter::new()` call in Rust creates the dual-registered processor/classifier pair via `route = route.with_processor(affinity.clone()).with_classifier(affinity)`. The identity lookup and 1.0 confidence scoring are implemented in [`affinity.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/affinity.rs) lines 163-175 and 188-198 respectively.

## How Turn-Based Routing Works

The `StageRouter` in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs) builds a **FallThrough** cascade that examines each turn independently. It decides whether to escalate to a capable tier, stay efficient, or invoke an optional capability judge.

The cascade order in `StageRouter::new` (lines 29-33) is:

1. `ToolSignalProcessor` — analyzes recent tool-call results (`recent_window` parameter)
2. `StageClassifier` — applies `confidence_threshold` to signal strength
3. `LlmFallback` (optional) — capability judge when signals are ambiguous
4. `DefaultTarget` — picker default tier as final fallback

Construct a stage router via the `stage_router` Python binding:

```python
from switchyard.libsy import stage_router, LlmFallback, TaskClassifierConfig

# Optional judge runs only when signals are undecided

judge = LlmFallback(
    judge_target="judge",
    config=TaskClassifierConfig(base_threshold=0.5)
)

router = stage_router(
    capable_target="strong",        # Capable tier (e.g., Codex-class model)

    efficient_target="weak",        # Efficient tier (e.g., smaller model)

    picker="efficient_first",       # Default tier when signals weak

    confidence_threshold=0.6,       # Signal confidence for capable tier

    recent_window=3,                # Tool results examined per turn

    escalation_note="Please investigate the failure.",  # Hand-off note

    classifier=judge,               # Optional capability judge

)

```

The `ToolSignalProcessor` writes a **Decision** with a `Signal` source when confidence exceeds the threshold (lines 172-179 in [`stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/stage.rs)). When this happens, the judge is **never invoked**—verified in the test "a decisive signal never reaches the judge" (lines 525-538).

If signals are ambiguous, the `LlmFallback` wrapped in a `SourceStamp` runs the capability judge (lines 180-192). The final `DefaultTarget` guarantees a routing decision (lines 194-198).

## Combining Both Mechanisms in a Pipeline

For production deployments, chain `AffinityRouter` outside `StageRouter` so session stickiness short-circuits before per-turn analysis:

```python
from switchyard.libsy import stage_router, llm_task_classifier, TaskClassifierConfig, LlmFallback

# Capability judge for ambiguous signals

judge = LlmFallback(
    judge_target="judge",
    config=TaskClassifierConfig(base_threshold=0.5, session_affinity=True)
)

# Per-turn routing decisions

stage = stage_router(
    capable_target="strong",
    efficient_target="weak",
    picker="efficient_first",
    confidence_threshold=0.7,
    recent_window=2,
    classifier=judge,
)

# Session affinity wrapper

affinity_classifier = llm_task_classifier(
    judge_target="judge",
    efficient_target="weak",
    capable_target="strong",
    config=TaskClassifierConfig(
        base_threshold=0.5,
        session_affinity=True,
        message_hash_fallback=True,
    ),
)

```

Runtime flow:

- **Affinity check first**: If the routing identity matches a stored assignment, return that model immediately with confidence 1.0
- **Proceed to stage router**: No affinity match means first turn or cleared session
- **Tool signal analysis**: `ToolSignalProcessor` examines the recent tool-call window
- **Threshold decision**: Signals above `confidence_threshold` → capable tier; decisive routing complete
- **Judge fallback**: Ambiguous signals trigger the `LlmFallback` capability judge
- **Default picker**: If all else abstains, use `picker` default tier

## Sub-Agent Affinity for Flexible Root Traffic

For agent architectures where root traffic should remain flexible but sub-agent conversations need stickiness, use `AffinityRouter.for_subagents()`:

```python
from switchyard.libsy import AffinityRouter

# Only retains assignments for child agents, not root sessions

sub_affinity = AffinityRouter.for_subagents()

# Register in custom FallThrough chain:

# route = FallThrough([...]).with_processor(sub_affinity).with_classifier(sub_affinity)

```

This keeps the main conversation free to reroute while ensuring delegated sub-tasks maintain model consistency.

## Customizing Turn-Based Behavior

Adjust `TaskClassifierConfig` parameters to control when the capability judge activates:

```python
from switchyard.libsy import LlmFallback, TaskClassifierConfig

custom_judge = LlmFallback(
    judge_target="custom_judge",
    config=TaskClassifierConfig(
        base_threshold=0.4,
        recent_turn_window=4,      # Judge sees last 4 turns for context

    ),
)

```

Lower `base_threshold` makes the judge more likely to render a verdict; higher `recent_turn_window` gives the judge more conversation history but increases token cost and latency.

## Key Source Files for Implementation

| File | Purpose |
|------|---------|
| [`crates/libsy/src/algorithms/util/affinity.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/affinity.rs) | `AffinityRouter` with dual processor/classifier implementation |
| [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs) | `LlmTaskClassifier` constructor; affinity integration |
| [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs) | `StageRouter` with `ToolSignalProcessor`, `StageClassifier`, `LlmFallback` |
| [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) | Python bindings for `stage_router`, `llm_task_classifier` |
| [`switchyard/libsy/__init__.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/__init__.py) | Public API exports |

## Summary

- **Conversation affinity** uses `AffinityRouter` to bind sessions to models after first selection, eliminating classifier overhead on subsequent turns
- **Turn-based routing** uses `StageRouter` to examine tool signals per turn, escalating to capable tiers when confidence thresholds are met
- **Enable affinity** with `session_affinity=True` in `TaskClassifierConfig` passed to `llm_task_classifier`
- **Configure turn routing** with `stage_router` parameters: `confidence_threshold`, `recent_window`, optional `LlmFallback` judge, and picker default
- **Combine both** by placing the affinity classifier outermost in the algorithm chain for short-circuit efficiency
- **Sub-agent variants** allow flexible root routing with sticky delegation via `AffinityRouter.for_subagents()`

## Frequently Asked Questions

### How does Switchyard handle the first request when no affinity record exists?

The `AffinityRouter` abstains and emits a one-time warning, allowing the downstream cascade (typically `StageRouter`) to make a normal routing decision. Once that decision is made, the processor component records it for future identity matches.

### What happens when tool signals are decisive versus ambiguous?

When `ToolSignalProcessor` confidence exceeds `confidence_threshold`, the `StageClassifier` immediately selects the capable tier and the optional `LlmFallback` judge never runs. When signals are below threshold, the cascade falls through to the judge if configured, then to the picker default.

### Can I use conversation affinity without turn-based routing?

Yes. The `AffinityRouter` is independently usable. Construct it via `llm_task_classifier` with `session_affinity=True` and any valid targets—the `judge_target` is ignored when affinity resolves successfully, so it can be a dummy value.

### How does `message_hash_fallback` work when no session ID is present?

Switchyard hashes the first user message content to generate a synthetic identity. This provides sticky routing for stateless clients or purely hash-based conversation grouping, implemented in [`affinity.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/affinity.rs) lines 163-175.