# SimStudio Agent Handler Architecture: Real-Time AI Workflow Execution

> Discover the SimStudio agent handler architecture for real-time AI workflow execution. Learn how its seven-phase pipeline orchestrates LLM responses and processes side effects efficiently.

- Repository: [Sim/sim](https://github.com/simstudioai/sim)
- Tags: architecture
- Published: 2026-05-02

---

**The SimStudio agent handler architecture uses a Socket.IO-based Realtime Engine to orchestrate AI workflows through a seven-phase pipeline that authenticates sessions, resolves variables, traverses execution graphs, streams LLM responses, and processes side effects.**

The `simstudioai/sim` repository implements a sophisticated real-time orchestration layer for AI agent execution. At its core, the **agent handler architecture** manages multi-step workflows through a modular system of Socket.IO handlers that coordinate graph traversal, variable resolution, and streamed AI responses.

## Architecture Overview

The SimStudio Realtime Engine operates as a bidirectional Socket.IO server that maintains persistent connections with clients while executing complex workflow graphs. The architecture centers on the **WorkflowExecutionContext**, an in-memory state container created per execution that tracks the block registry, variable stores, and execution metadata.

The system implements a layered handler pattern where specialized TypeScript modules manage distinct phases of the execution lifecycle. Each handler resides in `apps/realtime/src/handlers/` and communicates through a shared context object, enabling real-time updates while maintaining clean separation of concerns between authentication, graph traversal, and AI inference.

## The Seven-Phase Execution Pipeline

The agent handler architecture processes workflow execution through seven discrete phases, each handled by dedicated modules in the Realtime Engine.

### 1. Connection and Authentication ([`apps/realtime/src/auth.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/auth.ts))

When a client initiates a Socket.IO connection to `/socket.io`, the [`auth.ts`](https://github.com/simstudioai/sim/blob/main/auth.ts) middleware intercepts the request and validates the session token. Upon successful authentication, the middleware injects the user and workspace context into the socket instance, ensuring all subsequent operations execute within the correct authorization boundary.

### 2. Workflow Initialization ([`apps/realtime/src/handlers/connection.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/connection.ts))

The execution begins when the client emits a `workflow:start` event containing a serialized workflow definition. The [`connection.ts`](https://github.com/simstudioai/sim/blob/main/connection.ts) handler instantiates a **WorkflowExecutionContext**, generating a unique execution ID and initializing the block registry. This context object persists throughout the lifecycle of the workflow, serving as the central state repository for all subsequent operations.

### 3. Variable Resolution ([`apps/realtime/src/handlers/variables.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/variables.ts))

Before any block executes, the [`variables.ts`](https://github.com/simstudioai/sim/blob/main/variables.ts) handler resolves input variables from multiple sources, including user-provided prompts and previous tool outputs. The system maintains a per-execution in-memory variable store that makes resolved values available to downstream blocks through a unified access interface.

### 4. Graph Traversal ([`apps/realtime/src/handlers/subblocks.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/subblocks.ts))

Each workflow node represents a sub-block (starter, agent, function, or API call) within a directed acyclic graph (DAG). The [`subblocks.ts`](https://github.com/simstudioai/sim/blob/main/subblocks.ts) handler walks this graph, ensures all dependencies are satisfied before execution, and delegates each ready node to the appropriate executor. This phase implements the orchestration logic that determines execution order and parallelization opportunities.

### 5. Agent Block Execution ([`apps/realtime/src/handlers/workflow.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/workflow.ts))

When the traversal reaches a node of type **`agent`**, the [`workflow.ts`](https://github.com/simstudioai/sim/blob/main/workflow.ts) handler invokes the `runAgentBlock` function. This critical phase loads the agent's configuration—including the LLM model, system prompt, and tool catalog—and initiates a streaming request via the `requestJson` client from the SDK. The handler respects `selectedOutputs` and `stream` flags, emitting incremental `agent:output` events to the client for real-time display of partial responses.

### 6. Post-Execution Operations ([`apps/realtime/src/handlers/operations.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/operations.ts))

After the LLM returns a response, the [`operations.ts`](https://github.com/simstudioai/sim/blob/main/operations.ts) handler processes declared actions such as external API calls, database writes, or tool invocations. This phase updates the variable store with operation results and may trigger follow-up sub-blocks, enabling complex multi-step agent workflows with side effects.

### 7. State Synchronization and Cleanup ([`apps/realtime/src/handlers/presence.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/presence.ts))

Throughout execution, [`presence.ts`](https://github.com/simstudioai/sim/blob/main/presence.ts) maintains real-time synchronization of the "running" state indicator visible to clients. When the workflow completes successfully or encounters an error, [`connection.ts`](https://github.com/simstudioai/sim/blob/main/connection.ts) tears down the WorkflowExecutionContext and emits a final `workflow:finished` event, releasing memory resources and closing the execution lifecycle.

## Implementation Details and Code Examples

The following examples demonstrate the client-server interaction patterns and the core agent execution logic.

Client-side initiation:

```typescript
// Initiate workflow execution
socket.emit('workflow:start', {
  workflow: serializedWorkflow,   // JSON from the block editor
  variables: { prompt: 'Summarize the document' }
});

// Listen for streamed agent outputs
socket.on('agent:output', ({ blockId, chunk }) => {
  console.log(`Agent ${blockId} response:`, chunk);
});

```

Server-side agent execution:

```typescript
// Simplified runAgentBlock implementation from workflow.ts
async function runAgentBlock(ctx: WorkflowExecutionContext, block: AgentBlock) {
  const { model, systemPrompt, tools } = block.config;
  
  const response = await requestJson(
    llmContract,                     // Defined in SDK
    { 
      body: { 
        model, 
        systemPrompt, 
        messages: ctx.vars.input 
      } 
    },
    { signal: ctx.abortSignal, stream: true }
  );
  
  // Stream chunks back to client
  for await (const chunk of response) {
    ctx.socket.emit('agent:output', { 
      blockId: block.id, 
      chunk 
    });
  }
  
  return response.finalResult;
}

```

## Summary

- The **agent handler architecture** implements a seven-phase pipeline through the SimStudio Realtime Engine, using Socket.IO for bidirectional communication.
- Core execution logic resides in [`apps/realtime/src/handlers/workflow.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/workflow.ts), specifically within the `runAgentBlock` function that streams LLM responses via `requestJson`.
- Variable resolution occurs through [`apps/realtime/src/handlers/variables.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/variables.ts), maintaining per-execution state in an isolated in-memory store.
- The DAG traversal engine in [`apps/realtime/src/handlers/subblocks.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/subblocks.ts) coordinates execution order and dependency resolution across heterogeneous block types.
- Real-time streaming is achieved through incremental `agent:output` events emitted during the LLM inference phase.

## Frequently Asked Questions

### How does SimStudio authenticate agent execution requests?

SimStudio validates session tokens through the [`apps/realtime/src/auth.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/auth.ts) middleware before allowing Socket.IO connections. This injects user and workspace context into the execution environment, ensuring all agent operations occur within authorized boundaries.

### What component triggers AI agent block execution?

The [`workflow.ts`](https://github.com/simstudioai/sim/blob/main/workflow.ts) handler triggers agent execution when the DAG traversal identifies a block with type `agent`. The `runAgentBlock` function then orchestrates the LLM call using configuration from the block's metadata, including model selection and system prompts.

### How does the architecture support real-time response streaming?

The `runAgentBlock` function in [`apps/realtime/src/handlers/workflow.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/workflow.ts) utilizes the `stream: true` flag when calling `requestJson`. It iterates through the async response generator, emitting `agent:output` events for each chunk received from the LLM provider, enabling clients to display partial responses immediately.

### Where are workflow variables stored during execution?

Variables are stored in a per-execution in-memory store managed by [`apps/realtime/src/handlers/variables.ts`](https://github.com/simstudioai/sim/blob/main/apps/realtime/src/handlers/variables.ts). This store persists within the WorkflowExecutionContext for the duration of the workflow lifecycle, providing isolated state management that prevents cross-contamination between concurrent executions.