# How to Stream Generated Text Progressively in Transformers.js Text-Generation Pipelines

> Learn to stream generated text progressively in Transformers.js text-generation pipelines. This guide shows how to use TextStreamer for real-time incremental output.

- Repository: [Hugging Face/transformers.js](https://github.com/huggingface/transformers.js)
- Tags: how-to-guide
- Published: 2026-03-03

---

**The transformers.js library enables real-time text streaming by passing a `TextStreamer` instance to the generation pipeline, which invokes callback functions after each token batch to incrementally decode and emit partial outputs.**

The **huggingface/transformers.js** library supports **streaming generated text progressively in text-generation pipelines** through a specialized streaming architecture. By wiring a `TextStreamer` object into the generation loop, you can receive decoded words or raw tokens as they are produced rather than waiting for the full sequence to complete. This approach leverages specific callback hooks implemented in the model generation utilities to deliver low-latency, responsive output for CLI tools and web applications.

## How Streaming Works Under the Hood

### The TextStreamer Class Architecture

Located in [`packages/transformers/src/generation/streamers.js`](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/generation/streamers.js), the `TextStreamer` class (lines 38-68) serves as the primary interface for progressive output. It accumulates incoming token IDs in a cache, decodes them incrementally using the tokenizer, and forwards complete words to a configurable callback.

The constructor accepts several critical parameters:

- `skip_prompt` — Boolean to ignore the initial prompt tokens
- `skip_special_tokens` — Boolean to filter out special tokens like `<pad>` or `<eos>`
- `callback_function` — Handler for decoded text strings (defaults to `process.stdout.write`)
- `token_callback_function` — Handler for raw token ID arrays for advanced use cases

### Pipeline Integration

The `TextGenerationPipeline` in [`packages/transformers/src/pipelines/text-generation.js`](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/pipelines/text-generation.js) (lines 94-99) exposes the `streamer` option directly in its generation API. When you pass a streamer instance to the pipeline, it forwards this object to the underlying model's `generate()` method as part of the keyword arguments.

### Model Generation Loop Hooks

The actual streaming invocation occurs within [`packages/transformers/src/models/modeling_utils.js`](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/models/modeling_utils.js). After each forward pass during generation (lines 1095-1100), the model checks for a streamer and calls `streamer.put()` with the newly produced token IDs. When generation terminates (lines 1165-1170), the model invokes `streamer.end()` to flush any remaining cached tokens and finalize the output.

## Implementing Progressive Text Streaming

### Basic Console Output

The simplest implementation uses the default stdout callback to print words as they form:

```javascript
import { pipeline, TextStreamer } from '@huggingface/transformers';

async function demoConsole() {
  // Load a small causal LM (ONNX or TF.js back‑end)
  const generator = await pipeline(
    'text-generation',
    'onnx-community/SmolLM2-135M-ONNX',
  );

  // Create a streamer that prints each completed word
  const streamer = new TextStreamer(generator.tokenizer, {
    skip_prompt: true,           // don't echo the prompt
    skip_special_tokens: true,   // hide <eos>, <pad>, …
  });

  // Generate 30 new tokens, streaming to stdout as they appear
  await generator('Once upon a time', {
    max_new_tokens: 30,
    streamer,                      // <‑‑ hook into the generation loop
  });
}

demoConsole();

```

### Streaming to a Web UI

For React applications or browser environments, supply a custom `callback_function` to update state progressively:

```javascript
import { pipeline, TextStreamer } from '@huggingface/transformers';
import { useState } from 'react';

export default function TextGeneration() {
  const [output, setOutput] = useState('');

  async function run() {
    const generator = await pipeline('text-generation', 'onnx-community/Qwen3-0.6B-ONNX');

    const streamer = new TextStreamer(generator.tokenizer, {
      skip_prompt: true,
      skip_special_tokens: true,
      // Append each chunk to React state
      callback_function: (chunk) => setOutput((prev) => prev + chunk),
    });

    await generator('Explain transformers.js in one sentence', {
      max_new_tokens: 80,
      streamer,
    });
  }

  return (
    <>
      <button onClick={run}>Generate</button>
      <pre>{output}</pre>
    </>
  );
}

```

### Capturing Raw Token IDs

For analytics or custom decoding logic, use the `token_callback_function` to intercept token IDs before detokenization:

```javascript
import { pipeline, TextStreamer } from '@huggingface/transformers';

async function logTokenIds() {
  const generator = await pipeline('text-generation', 'meta-llama/Llama-2-7b-chat');

  const streamer = new TextStreamer(generator.tokenizer, {
    token_callback_function: (ids) => console.log('New token IDs:', ids),
    skip_prompt: true,
    skip_special_tokens: true,
  });

  await generator('What is the capital of France?', {
    max_new_tokens: 10,
    streamer,
  });
}

```

## Customizing Streamer Behavior

### Word Boundary Detection and Caching

The `put()` method implements intelligent buffering to avoid splitting words. When processing new tokens, it decodes the entire cache and applies the following heuristics:

- **Newline characters** — Immediately flush the entire cache
- **CJK characters** (Chinese, Japanese, Korean) — Emit immediately since they are not space-delimited
- **Standard text** — Print only up to the last space character, keeping partial words in the cache until complete

### Special Token Handling

The streamer maintains a set of special token IDs (`this.special_ids`) to handle control tokens. When a single special token arrives, the cache flushes first, then the token decodes separately. If `skip_special_tokens` is enabled, these are silently dropped rather than emitted to the callback.

## Summary

- The `TextStreamer` class in [`src/generation/streamers.js`](https://github.com/huggingface/transformers.js/blob/main/src/generation/streamers.js) provides the core infrastructure for progressive output
- Pass the streamer instance to `TextGenerationPipeline` via the `streamer` option in generation kwargs
- The model's generation loop in [`src/models/modeling_utils.js`](https://github.com/huggingface/transformers.js/blob/main/src/models/modeling_utils.js) invokes `put()` after each token and `end()` upon completion
- Configure `skip_prompt` and `skip_special_tokens` to control which tokens appear in the output stream
- Use `callback_function` for text updates and `token_callback_function` for raw ID access
- Only batch size 1 is supported; the streamer receives a 2-D array but expects single-batch input

## Frequently Asked Questions

### What is the difference between TextStreamer and WhisperTextStreamer?

`WhisperTextStreamer` extends `TextStreamer` to handle timestamp tokens specific to the Whisper speech-to-text model. While `TextStreamer` focuses on general text generation, the Whisper variant processes audio timestamp markers in addition to standard vocabulary tokens, located in the same [`src/generation/streamers.js`](https://github.com/huggingface/transformers.js/blob/main/src/generation/streamers.js) file (lines 71-92).

### How does transformers.js handle incomplete words during streaming?

The streamer maintains an internal token cache that buffers incomplete words. It only emits text up to the last detected space character (or newlines/CJK characters) to ensure words are not split mid-generation. Remaining tokens stay cached until the next `put()` call provides enough context to complete the word.

### Can I use streaming with batch generation?

No. The `TextStreamer` explicitly supports only batch size 1. The `put()` method receives a 2-D array with shape (batch × sequence) but expects the batch dimension to equal 1. Attempting to stream with larger batches will likely cause errors or undefined behavior in the current implementation.

### Which model files support the streamer parameter?

All text-generation models in transformers.js that utilize the standard `generate()` method in [`src/models/modeling_utils.js`](https://github.com/huggingface/transformers.js/blob/main/src/models/modeling_utils.js) support the streamer parameter. This includes ONNX and TensorFlow.js backends for models like Llama, Qwen, SmolLM, and other causal language models exposed through the `TextGenerationPipeline` in [`src/pipelines/text-generation.js`](https://github.com/huggingface/transformers.js/blob/main/src/pipelines/text-generation.js).