# How the Streaming Token Buffer Handles Markdown and Code Blocks in ML Intern

> Learn how ml interns streaming token buffer handles markdown and code blocks efficiently. Discover how fenced code blocks render atomically for a seamless user experience.

- Repository: [Hugging Face/ml-intern](https://github.com/huggingface/ml-intern)
- Tags: internals
- Published: 2026-04-24

---

**The streaming token buffer in ML Intern accumulates raw tokens paragraph-by-paragraph, ensuring fenced code blocks render atomically only after the closing backticks arrive.**

ML Intern, the open-source agent framework from Hugging Face, streams LLM output token-by-token to maintain a responsive terminal interface. To prevent fragmented code fences and broken Markdown structures during real-time display, the library implements a specialized **streaming token buffer** called `_StreamBuffer`. This stateful helper defined in [`agent/main.py`](https://github.com/huggingface/ml-intern/blob/main/agent/main.py) manages the delicate balance between low-latency output and structural integrity.

## Architecture of the Streaming Token Buffer

The `_StreamBuffer` class operates through a three-stage pipeline that processes incoming text incrementally. It balances immediate user feedback with the technical requirement to keep Markdown elements—especially fenced code blocks—intact until they are fully received.

### Accumulating Raw Chunks

Every token arriving from the LLM appends to an internal string buffer via the `add_chunk` method. This method simply concatenates new content to `self._buffer`, allowing the system to handle arbitrarily fragmented streaming responses without loss.

```python
def add_chunk(self, text: str):
    self._buffer += text          # ← adds streamed tokens

```

*(source: [[`agent/main.py`](https://github.com/huggingface/ml-intern/blob/main/agent/main.py) lines 22-23](https://github.com/huggingface/ml-intern/blob/main/agent/main.py#L22-L23))*

### Detecting Complete Markdown Blocks

Before emitting text, the buffer must determine whether a logical block is complete. A block is defined as text up to a paragraph break (`"\n\n"`), but with a critical safety constraint: fenced code sections must be closed.

The `_pop_block` method checks if the count of backtick triplets (```) is **even**. An odd count indicates an open code fence, so the method returns `None` and defers rendering until the closing delimiter arrives. Only when the fence is closed and a double newline appears does the buffer slice out the completed block for display.

```python
def _pop_block(self) -> str | None:
    if self._buffer.count("```") % 2 == 1:
        return None                # inside an open code fence → wait

    idx = self._buffer.find("\n\n")
    if idx == -1:
        return None                # no paragraph break yet

    block = self._buffer[:idx]
    self._buffer = self._buffer[idx + 2:]
    return block

```

*(source: [[`agent/main.py`](https://github.com/huggingface/ml-intern/blob/main/agent/main.py) lines 25-33](https://github.com/huggingface/ml-intern/blob/main/agent/main.py#L25-L33))*

### Flushing Ready Blocks Progressively

Once `_pop_block` identifies a complete block, `flush_ready` handles the actual rendering. This async method repeatedly extracts ready blocks and passes them to `print_markdown` (from [`agent/utils/terminal_display.py`](https://github.com/huggingface/ml-intern/blob/main/agent/utils/terminal_display.py)) for styled terminal output with a typewriter effect. Empty blocks are skipped to avoid unnecessary renders.

When the stream terminates, the `finish` method ensures any remaining content—including text that might not end with a double newline—is flushed to the terminal.

```python
async def flush_ready(...):
    while True:
        block = self._pop_block()
        if block is None: break
        if block.strip():
            await print_markdown(block, cancel_event=..., instant=...)

```

*(source: [[`agent/main.py`](https://github.com/huggingface/ml-intern/blob/main/agent/main.py) lines 40-48](https://github.com/huggingface/ml-intern/blob/main/agent/main.py#L40-L48))*

## Integration with the Event Loop

The **streaming token buffer** integrates directly into ML Intern's background `event_listener` inside [`agent/main.py`](https://github.com/huggingface/ml-intern/blob/main/agent/main.py). For each `assistant_chunk` event, incoming content appends to the buffer and immediately triggers `flush_ready`, while `assistant_stream_end` forces a final `finish` call to guarantee no text remains stranded in the buffer.

```python
elif event.event_type == "assistant_chunk":
    content = event.data.get("content", "")
    if content:
        stream_buf.add_chunk(content)
        shimmer.stop()
        await stream_buf.flush_ready(cancel_event=_cancel_event())
elif event.event_type == "assistant_stream_end":
    shimmer.stop()
    await stream_buf.finish(cancel_event=_cancel_event())

```

*(source: [[`agent/main.py`](https://github.com/huggingface/ml-intern/blob/main/agent/main.py) lines 300-311](https://github.com/huggingface/ml-intern/blob/main/agent/main.py#L300-L311))*

## Practical Usage Example

The following example demonstrates how the buffer behaves when simulating segmented LLM output. Notice that the code block only appears after the closing fence arrives, while regular paragraphs render immediately upon detecting the double newline.

```python
from agent.main import _StreamBuffer
from agent.utils.terminal_display import print_markdown
import asyncio

async def demo():
    console = ...                     # obtain a Rich console via get_console()

    buf = _StreamBuffer(console)

    # Simulate streamed tokens arriving in pieces

    buf.add_chunk("Here is some intro text.\n\n")
    await buf.flush_ready()           # prints the paragraph immediately

    # Start a fenced code block – nothing is shown yet

    buf.add_chunk("```python\nprint('Hello")
    await buf.flush_ready()           # no output because fence is open

    # Close the fence – now the whole block appears atomically

    buf.add_chunk(" world')\n```\n\n")
    await buf.flush_ready()           # prints the complete code block

    # End of stream – render any tail that didn't end with a double newline

    await buf.finish()

asyncio.run(demo())

```

## Summary

- **Paragraph-wise rendering** – Users see each paragraph as soon as it is complete, keeping the interface responsive during long generations.
- **Atomic code-block display** – Fenced code emits only after the closing ``` appears, preventing stray "half-code" fragments from cluttering the terminal.
- **Graceful termination** – The `finish` method ensures any leftover text is printed once the stream ends, regardless of whether it terminated with a paragraph break.
- **Event-driven integration** – The buffer plugs directly into ML Intern's event system, processing `assistant_chunk` events incrementally and finalizing on `assistant_stream_end`.

## Frequently Asked Questions

### Where is the streaming token buffer implemented in ML Intern?

The streaming token buffer is implemented as the `_StreamBuffer` class in `agent/main.py` according to the Hugging Face ML Intern source code. This private helper is instantiated once per interactive session inside the background `event_listener`.

### How does the buffer prevent broken code blocks from appearing?

The buffer counts occurrences of the backtick triplet (```) using `self._buffer.count("```") % 2 == 1`. An odd count indicates an open code fence, causing `_pop_block` to return `None` and defer rendering until the closing fence arrives, ensuring the entire block displays atomically.

### What triggers the final render of leftover text?

The `finish` method is invoked when the `assistant_stream_end` event fires. This method flushes any remaining content in the internal buffer that may not have ended with a double newline, ensuring the user sees the complete LLM output.

### Why does the buffer wait for paragraph breaks before rendering?

Waiting for `"\n\n"` paragraph breaks enables paragraph-wise streaming that maintains Markdown structural coherence. This approach allows the terminal to display content incrementally without breaking headers, lists, or code fences across multiple premature flushes.