# Core Architecture of the Vane AI Answering Engine: A Technical Deep Dive

> Explore the core architecture of the Vane AI answering engine. Learn about its modular, streaming-first design, React frontend, Node.js backend, and real-time response capabilities.

- Repository: [Kushagra Srivastava/Vane](https://github.com/ItzCrazyKns/Vane)
- Tags: deep-dive
- Published: 2026-03-11

---

**Vane's answering engine employs a modular, streaming-first architecture that orchestrates React frontend components, a Node.js/Next.js backend, dynamic model providers, and an in-memory session manager to deliver real-time AI responses with integrated tool support.**

The Vane AI answering engine, developed in the open-source repository ItzCrazyKns/Vane, handles complex queries through a sophisticated pipeline designed for extensibility and performance. Unlike monolithic AI applications, Vane adopts a **streaming-first, event-driven design** that processes user requests through distinct classification, research, and synthesis phases. This article examines the core architecture of the Vane AI answering engine, detailing how its loosely coupled TypeScript components work together to generate contextual responses with citations and interactive widgets.

## Architecture Overview

Vane operates as an eight-stage pipeline that transforms user input into streamed output blocks. The system uses **Server-Sent Events (SSE)** to push incremental updates to the client, allowing the UI to render research progress, widget results, and final answers in real time.

The flow begins when the React UI invokes `sendMessage()` from the `useChat()` hook. This triggers a `POST` request to `/api/chat`, where the route handler validates the request and initializes the processing chain. The `ModelRegistry` loads the configured LLM and embedding providers, while a `SessionManager` creates an ephemeral, in-memory store for the conversation state. A **classifier LLM** then generates a `ClassifierOutput` that determines whether to execute widgets, perform web searches, or answer directly. If research is required, the `SearchAgent` spawns a `Researcher` to gather data, streaming **text blocks** back via RFC-6902 patches. Finally, the writer LLM synthesizes the gathered context into a response, with all blocks emitted as newline-delimited JSON through the SSE connection.

## Key Architectural Components

### Model Registry and Provider Abstraction

The **ModelRegistry** ([`src/lib/models/registry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/registry.ts)) centralizes provider initialization, exposing `loadChatModel()` and `loadEmbeddingModel()` functions. This abstraction allows the system to swap between OpenAI, Anthropic, or local models without modifying the business logic in [`src/app/api/chat/route.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/app/api/chat/route.ts). The registry reads server-side configuration and returns concrete instances that implement the `ChatTurnMessage` interface.

### Session Management and Streaming Infrastructure

At the heart of the streaming pipeline lies the **SessionManager** ([`src/lib/session.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/session.ts)), an in-memory, short-lived store that implements an EventEmitter-based pub/sub API. It maintains the state of `Block` objects and supports **RFC-6902 JSON patches** for incremental updates. When the `SearchAgent` or `WidgetExecutor` produces new data, they push blocks into the session; the API route consumes these events and writes them to the client's `EventSource` as structured JSON lines.

### The Classification Layer

Before executing any tools, Vane runs a **classifier** ([`src/lib/agents/search/classifier.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/agents/search/classifier.ts)) that analyzes the user query using a dedicated system prompt ([`src/lib/prompts/search/classifier.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/prompts/search/classifier.ts)). This LLM-driven component outputs a structured `ClassifierOutput` containing:
- A boolean indicating if search is required
- Flags for widget activation (weather, stock, calculator)
- A potentially rewritten query optimized for retrieval

This decision layer prevents unnecessary API calls and optimizes response latency.

### SearchAgent and Research Orchestration

The **SearchAgent** ([`src/lib/agents/search/index.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/agents/search/index.ts)) serves as the primary orchestrator. It coordinates the classifier, `WidgetExecutor`, and `Researcher` components while managing the lifecycle of the streaming session. Once research completes, the agent constructs the final writer prompt—combining search results, widget context, and system instructions—and streams the LLM's output chunks into a unified text block. It also persists the final message to the database using Drizzle ORM ([`src/lib/db/schema.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/db/schema.ts)).

### Widget Execution Engine

The **WidgetExecutor** runs applicable tools in parallel based on the classifier's output. Each widget implements a standard interface with `shouldExecute()` and `execute()` methods. Widgets stream their results as **widget blocks**, distinct from text blocks, allowing the frontend to render rich components (charts, weather cards, stock tickers) mid-conversation.

## Data Flow: From UI to Streamed Response

The following code illustrates how the frontend initiates the pipeline and consumes the SSE stream:

### Sending a Message from a Component

```tsx
import { useChat } from '@/lib/hooks/useChat';

export const MessageInput = () => {
  const { sendMessage, loading } = useChat();

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    const input = new FormData(e.currentTarget).get('msg') as string;
    if (input) sendMessage(input);
  };

  return (
    <form onSubmit={handleSubmit}>
      <input name="msg" disabled={loading} />
      <button type="submit" disabled={loading}>Ask Vane</button>
    </form>
  );
};

```

The `useChat()` hook (defined in [`src/lib/hooks/useChat.tsx`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/hooks/useChat.tsx)) handles model selection, payload construction, and state management, keeping UI components decoupled from transport logic.

### Consuming the SSE Stream

Inside `useChat`, the client consumes the backend's newline-delimited JSON stream:

```typescript
const res = await fetch('/api/chat', { method: 'POST', body: JSON.stringify(payload) });
const reader = res.body?.getReader();
const decoder = new TextDecoder();

while (true) {
  const { value, done } = await reader!.read();
  if (done) break;
  const chunk = decoder.decode(value, { stream: true });
  chunk.split('\n').forEach((line) => {
    if (!line.trim()) return;
    const event = JSON.parse(line);
    // Events: block, updateBlock, researchComplete, messageEnd, error
    // Forward to SessionManager for state updates
  });
}

```

The backend ([`src/app/api/chat/route.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/app/api/chat/route.ts)) emits events such as `block`, `updateBlock`, and `messageEnd`, which the hook parses to update the React `messages` state incrementally.

## Extending the Engine: Adding Custom Widgets

Vane's modular architecture supports custom widgets without modifying core agent logic. To add a weather widget, implement the widget interface and register it with the `WidgetExecutor`:

```typescript
// src/lib/agents/search/widgets/weather.ts
export const WeatherWidget = {
  type: 'weather',
  shouldExecute: (config) => config.showWeatherWidget,
  async execute({ llm, followUp }) {
    const forecast = await fetchWeatherAPI(followUp);
    const llmContext = await llm.generateText({
      messages: [{ role: 'system', content: `Here is the weather data:\n${forecast}` }],
    });
    return { type: 'weather', llmContext, data: forecast };
  },
};

```

The `WidgetExecutor` automatically streams the returned object as a `widget` block type, and the frontend renders it using the corresponding React component.

## Summary

- **Streaming-first design**: Vane uses Server-Sent Events and newline-delimited JSON to deliver real-time updates, ensuring low latency between research steps and final output.
- **Provider abstraction**: The `ModelRegistry` in [`src/lib/models/registry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/registry.ts) decouples the application from specific LLM vendors, enabling seamless model swaps.
- **Session-based state**: `SessionManager` ([`src/lib/session.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/session.ts)) provides an ephemeral, EventEmitter-driven pub/sub system with RFC-6902 patch support for incremental block updates.
- **Classifier-driven routing**: The classifier LLM optimizes execution paths by determining when to search, which widgets to run, and how to rewrite queries.
- **Agent orchestration**: `SearchAgent` ([`src/lib/agents/search/index.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/agents/search/index.ts)) coordinates research, widget execution, and response synthesis while persisting data via Drizzle ORM.
- **TypeScript interfaces**: Well-defined types (`ChatTurnMessage`, `Block`, `SearchAgentConfig`) ensure loose coupling between the React frontend, API routes, and agent logic.

## Frequently Asked Questions

### What makes Vane's architecture "streaming-first"?

Vane processes and transmits data continuously rather than waiting for complete generation. The backend writes newline-delimited JSON events immediately as the classifier decides, widgets execute, and the LLM generates tokens. This allows the frontend to render progress indicators, partial research results, and citations before the final answer completes, significantly improving perceived performance.

### How does Vane decide whether to perform a web search?

The **classifier** component ([`src/lib/agents/search/classifier.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/agents/search/classifier.ts)) evaluates each query using a dedicated system prompt. It generates a structured `ClassifierOutput` containing a boolean `needsSearch` flag. If the query requires current information, complex calculations beyond the LLM's training data, or specific widget data, the classifier triggers the `Researcher` agent; otherwise, it routes directly to the writer LLM for an immediate response.

### Can I replace the LLM provider without modifying the core logic?

Yes. The `ModelRegistry` ([`src/lib/models/registry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/registry.ts)) abstracts provider initialization through standardized interfaces. By implementing the required methods for your provider (e.g., Azure OpenAI, local Ollama instances) and updating the configuration, the system routes all LLM calls through your implementation without changes to the `SearchAgent`, classifier, or API routes.

### What database does Vane use for persistence?

Vane uses **Drizzle ORM** with a schema defined in [`src/lib/db/schema.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/db/schema.ts) to persist chats, messages, files, and source metadata. The `ensureChatExists` utility and `SearchAgent` write records to this store, enabling conversation history retrieval and cross-session context, while the streaming pipeline itself operates primarily in memory for performance.