How Mid-Stream Behavior Controls User Message Flow During Active Agent Turns

Mid-stream behavior dictates whether user messages arriving during an active agent turn are queued for the next turn or injected into the current LLM stream to steer the ongoing response.

In the craft-ai-agents/craft-agents-oss framework, mid-stream behavior is the critical configuration switch that determines how the system handles user input that arrives while an agent is still generating a response. This mechanism ensures seamless interaction flows by either buffering messages for subsequent processing or dynamically redirecting the active language model output.

The Two Mid-Stream Modes: Queue vs. Steer

The midStreamBehavior setting, defined in packages/shared/src/config/llm-connections.ts, accepts one of two string values. Each value triggers a distinct code path in the server core that fundamentally changes how user messages interleave with agent processing.

Queue Mode

When configured to queue, the system treats incoming user messages as interrupts that must wait for the current turn to complete. The implementation follows this sequence:

1. Detection: SessionManager.sendMessage identifies that session.isProcessing() returns true, indicating an active generation.
2. Persistence: The incoming message is appended to a pending queue associated with the session and written to disk for durability.
3. Acknowledgment: The client receives an immediate ACK, confirming the message is saved without waiting for processing.
4. Redispatch: Once the agent finishes its current turn, the queued message is automatically re-dispatched via sendMessage as the next user input.

This approach ensures message integrity and strict turn-based ordering, making it ideal for providers that do not support mid-generation steering.

Steer Mode

When set to steer, the system treats incoming messages as real-time course corrections. According to the implementation in packages/server-core/src/sessions/SessionManager.ts (lines 5470-5493), the server calls agent.redirect(message) or a provider-specific steering API to inject the user content directly into the ongoing LLM stream.

In this mode, the model continues generating but adjusts its output to incorporate the new context immediately. The isProcessing() flag remains true throughout, and the user sees their message appear as part of the same continuous turn rather than triggering a new one.

How the Server Detects and Resolves Mid-Stream Messages

The flow control logic resides in the server core and follows a strict three-phase pipeline when handling user input during active turns.

Phase 1: Detection

The entry point is SessionManager.sendMessage, which checks the session state via session.isProcessing(). This boolean flag indicates whether the agent is currently emitting a response stream.

Phase 2: Resolution

The system calls resolveMidStreamBehavior, implemented in packages/shared/src/config/llm-connections.ts (lines 190-491). This function reads the midStreamBehavior field from the connection configuration and applies provider-specific defaults if the field is absent:

• Anthropic connections default to steer
• Pi connections default to queue

Phase 3: Application

Based on the resolved behavior, the server executes one of two mutually exclusive paths:

• queue path: Invokes queueMessage(sessionId, incoming) to persist the message to the pending queue.
• steer path: Invokes agent.redirect(incoming) to inject the message into the live LLM stream.

Configuration and Provider Defaults

The framework seeds default behaviors during connection initialization and persists user overrides to stable storage.

Default Seeding

When creating new LLM connections, packages/server-core/src/domain/connection-setup-logic.ts (line 242) sets the initial midStreamBehavior value based on the provider type. This ensures appropriate out-of-the-box behavior without requiring explicit configuration.

Persistent Storage

User modifications to the mid-stream behavior are written to packages/shared/src/config/storage.ts (line 2702). The system persists these settings alongside other connection parameters, ensuring consistency across server restarts.

Runtime Updates

Connections can be updated dynamically using the updateLlmConnection utility. As verified in packages/shared/src/config/__tests__/midstream-behavior.test.ts (lines 35-140), flipping a connection from steer to queue (or vice versa) takes effect immediately for subsequent mid-turn messages without requiring a server restart.

Practical Implementation Examples

The following code snippets demonstrate how to resolve, check, and update mid-stream behavior in production code:

// Resolve the effective behavior for a connection
import { resolveMidStreamBehavior } from '@craft-agent/shared/config';

const behavior = resolveMidStreamBehavior(connection);
// behavior is either 'queue' or 'steer'

// Inside SessionManager.sendMessage (simplified flow)
if (session.isProcessing()) {
  const behavior = resolveMidStreamBehavior(connection);
  
  if (behavior === 'queue') {
    // Persist the message and return early – it will be sent after the turn ends
    await queueMessage(sessionId, incoming);
  } else {
    // Inject the message into the live LLM stream
    await agent.redirect(incoming);
  }
}

// Updating a connection's mid-stream behavior (persisted to config)
await updateLlmConnection('my-pi-conn', { midStreamBehavior: 'steer' });

Summary

• Mid-stream behavior is the configuration switch in craft-agents-oss that controls user message flow during active agent turns.
• queue mode stores incoming messages in a pending queue, delivering them only after the current turn completes.
• steer mode injects messages directly into the ongoing LLM stream via agent.redirect(), allowing real-time course correction.
• The system resolves behavior via resolveMidStreamBehavior in packages/shared/src/config/llm-connections.ts, with Anthropic defaulting to steer and Pi defaulting to queue.
• Configuration changes persist through packages/shared/src/config/storage.ts and apply immediately to subsequent messages.

Frequently Asked Questions

What happens if a user sends multiple messages while the agent is still processing?

Each incoming message triggers the same mid-stream behavior check. In queue mode, all messages are appended to the session's pending queue in order and processed sequentially after the current turn ends. In steer mode, each message triggers a separate agent.redirect() call, potentially causing the LLM to adjust its generation multiple times within the same turn.

Can I change the mid-stream behavior without restarting the server?

Yes. The configuration is dynamic. When you call updateLlmConnection() to modify the midStreamBehavior field, the change is persisted to disk and takes effect immediately for the next mid-turn message, as confirmed by the test suite in midstream-behavior.test.ts.

Why do Anthropic and Pi have different default behaviors?

The defaults reflect provider capabilities. Anthropic's API supports mid-generation steering mechanisms, making steer the sensible default for responsive interactions. Pi's architecture or API constraints favor strict turn-based processing, so queue prevents message loss or ordering issues. The defaults are hardcoded in packages/server-core/src/domain/connection-setup-logic.ts (line 242) and packages/shared/src/config/llm-connections.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}