How Headroom Handles Streaming Responses: A Technical Deep Dive

Headroom processes streaming LLM responses by forwarding the stream flag through its TypeScript SDK, parsing Server-Sent Events (SSE) from the proxy, and exposing them as an async generator that yields parsed JSON chunks in real-time.

The chopratejas/headroom repository provides a TypeScript SDK that supports real-time streaming for both OpenAI-style chat completions and Anthropic-style messages. Understanding how Headroom handles streaming responses reveals a clean pipeline that transforms low-level HTTP streams into developer-friendly async generators while maintaining compatibility with standard LLM APIs.

Understanding the Streaming Architecture

Headroom’s streaming implementation follows a five-stage pipeline: the user enables streaming via a boolean flag, the SDK forwards this to the proxy, the proxy returns SSE-formatted data, the SDK parses those events, and the consumer iterates over the resulting async generator.

Request Initialization in client.ts

The streaming flow begins in sdk/typescript/src/client.ts, where the create methods of both ChatCompletions and Messages classes accept a stream parameter. When params.stream is truthy, the method returns the result of parseSSE(response) rather than a complete JSON object.

// Located at sdk/typescript/src/client.ts (lines 84-108)
const response = await this.rawFetch(endpoint, {
  method: 'POST',
  body: JSON.stringify(params),
  stream: params.stream  // Forwarded to rawFetch
});

if (params.stream) {
  return parseSSE(response);  // Returns AsyncGenerator
}

The rawFetch helper (lines 99-106) constructs the full URL, applies default JSON headers, and critically passes the stream flag through to the underlying fetch call. This ensures the browser or Node.js runtime preserves the response body as a ReadableStream rather than buffering it entirely.

The SSE Parser in stream.ts

The core parsing logic resides in sdk/typescript/src/utils/stream.ts. The parseSSE function (lines 5-33) consumes the Response body using a ReadableStream reader and TextDecoder to handle byte-to-string conversion.

The implementation buffers incoming chunks, splits them on newlines, and identifies lines beginning with "data: ". Each data payload is parsed via JSON.parse() and yielded to the consumer. The generator terminates when encountering the [DONE] sentinel, which signals the end of the LLM response.

// Conceptual flow from sdk/typescript/src/utils/stream.ts (lines 5-33)
async function* parseSSE(response: Response) {
  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';
  
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split('\n');
    buffer = lines.pop() || '';  // Keep partial line
    
    for (const line of lines) {
      if (line.startsWith('data: ')) {
        const data = line.slice(6);
        if (data === '[DONE]') return;
        yield JSON.parse(data);
      }
    }
  }
}

Implementation Patterns

Headroom supports two major LLM API styles for streaming, both utilizing the same underlying SSE infrastructure.

OpenAI-Style Chat Completions

For OpenAI-compatible endpoints, you enable streaming by setting stream: true in the creation call. According to the source code in client.ts, this triggers the async generator return path.

import { HeadroomClient } from "headroom";

const client = new HeadroomClient({ baseUrl: "http://localhost:8787" });

const stream = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Explain streaming" }],
  stream: true,  // Enables AsyncGenerator return type
});

for await (const chunk of stream) {
  // Each chunk is a parsed JSON object from the SSE data line
  console.log(chunk.choices?.[0]?.delta?.content ?? "");
}

Anthropic-Style Messages

The same pattern applies to Anthropic-style message endpoints. The Messages class implements identical logic, checking the stream parameter and delegating to parseSSE when appropriate.

const stream = await client.messages.create({
  model: "claude-2.1",
  messages: [{ role: "user", content: "Summarize this" }],
  stream: true,
});

for await (const delta of stream) {
  console.log(delta.content?.text);
}

Working with Streamed Data

Once the SDK returns the async generator, you have two primary consumption patterns: real-time iteration or aggregation.

Consuming the Async Generator

The streaming response implements the AsyncIterable protocol, allowing direct use with for await...of loops. This approach minimizes memory usage by processing each fragment as it arrives from the Headroom proxy, without waiting for the complete response.

Aggregating Stream Results

When you need the full response as an array rather than individual chunks, the SDK provides the collectStream utility in sdk/typescript/src/utils/stream.ts (lines 41-49). This helper consumes the entire async generator and returns a promise that resolves to an array of all chunks.

import { collectStream } from "headroom/utils/stream";

const stream = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
  stream: true,
});

const allChunks = await collectStream(stream);
console.log(`Received ${allChunks.length} chunks total`);

Summary

• Headroom exposes streaming LLM responses through a unified stream: true parameter in both chat.completions.create() and messages.create() methods.
• The rawFetch helper in sdk/typescript/src/client.ts preserves the stream flag through to the HTTP fetch call, ensuring the response body remains a ReadableStream.
• Server-Sent Events are parsed by parseSSE in sdk/typescript/src/utils/stream.ts, which yields parsed JSON objects via an async generator until encountering the [DONE] sentinel.
• Developers consume streams using standard for await...of loops, or buffer them using the collectStream utility for non-streaming use cases.

Frequently Asked Questions

What data format does Headroom use for streaming?

Headroom uses Server-Sent Events (SSE) for streaming. The proxy returns a text stream where each line starts with data: followed by a JSON-encoded fragment of the LLM response. The SDK's parseSSE function extracts these lines, parses the JSON, and yields the resulting objects to your application.

How does Headroom signal the end of a stream?

The streaming parser stops when it encounters the [DONE] sentinel value in the SSE data line. As implemented in sdk/typescript/src/utils/stream.ts, the generator checks for this string after the data: prefix and returns early, effectively closing the async iterable and releasing the stream reader.

Can I use Headroom streaming with non-OpenAI models?

Yes. Headroom supports Anthropic-style message streaming through the same stream: true interface. The SDK's architecture in client.ts handles both OpenAI chat completions and Anthropic messages using the same underlying parseSSE infrastructure, making it compatible with any provider that returns SSE-formatted responses.

How do I buffer a streaming response into an array?

Import the collectStream helper from headroom/utils/stream. This function, located at sdk/typescript/src/utils/stream.ts lines 41-49, takes the async generator returned by a streaming request and accumulates all yielded chunks into a single array, returning a Promise<Chunk[]> that resolves when the stream completes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}