# How to Enable Live Transcription During Voice Conversations with Hugging Face Speech-to-Speech

> Enable live transcription in Hugging Face Speech-to-Speech. Set enable_live_transcription=True to stream partial transcriptions in real-time during voice conversations.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-03

---

**Enable live transcription in Hugging Face Speech-to-Speech by setting `enable_live_transcription=True` in your `ModuleArguments`, which activates realtime VAD buffering and streams partial transcriptions through `TranscriptionNotifier` to your client.**

The Hugging Face **speech-to-speech** repository provides a production-ready pipeline for real-time voice conversations. By enabling live transcription, you can display subtitles while the user is still speaking, dramatically improving the conversational experience. This feature is implemented through three coordinated components that progressively process audio without waiting for speech completion.

## Understanding the Live Transcription Architecture

Live transcription in speech-to-speech works by intercepting audio at the **Voice Activity Detection (VAD)** layer and forwarding partial results before the final utterance ends. This requires tight coordination between the VAD handler, STT handler, and a dedicated notifier component.

The implementation spans four critical source files:

| File | Responsibility |
|------|---------------|
| [`src/speech_to_speech/arguments_classes/module_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/module_arguments.py) | Defines the `enable_live_transcription` CLI/API flag |
| [`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py) | Implements progressive audio yielding via `_process_realtime` |
| [`src/speech_to_speech/STT/transcription_notifier.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/transcription_notifier.py) | Bridges partial transcriptions to `text_output_queue` |
| [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py) | Wires components together and propagates configuration |

## Step 1: Configure Live Transcription via CLI

The fastest way to enable live transcription is through command-line arguments. In [`module_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/module_arguments.py) (line 61), the `ModuleArguments` dataclass exposes:

```python
from speech_to_speech.arguments_classes.module_arguments import ModuleArguments

# Or via CLI:

# python -m speech_to_speech.server --enable_live_transcription

```

**Recommended CLI invocation:**

```bash
python -m speech_to_speech.server \
  --enable_live_transcription \
  --live_transcription_update_interval 0.5 \
  --model_name gpt-4o

```

The `live_transcription_update_interval` parameter (default 0.5 seconds) controls how frequently progressive audio chunks are emitted. Lower values provide more responsive subtitles but increase STT computation.

## Step 2: Enable Live Transcription Programmatically

For embedded applications, configure `ModuleArguments` directly and pass to `S2SPipeline`:

```python
from speech_to_speech.arguments_classes.module_arguments import ModuleArguments
from speech_to_speech.s2s_pipeline import S2SPipeline

# Build configuration with live transcription enabled

module_args = ModuleArguments(
    enable_live_transcription=True,
    live_transcription_update_interval=0.3,  # 300ms updates

)

# Pipeline initializes VADHandler with realtime flag

pipeline = S2SPipeline(module_kwargs=module_args)
pipeline.run()

```

In [`s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/s2s_pipeline.py) (lines 579–585), the constructor propagates this flag:

```python
if module_kwargs.enable_live_transcription:
    vad_kw.enable_realtime_transcription = True
    vad_kw.realtime_processing_pause = module_kwargs.live_transcription_update_interval

```

## Step 3: How VADHandler Processes Realtime Audio

The `VADHandler` in [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py) (line 70) implements the core realtime logic. When `enable_realtime_transcription=True`, it switches to `_process_realtime` mode:

```python

# From vad_handler.py - _process_realtime excerpt

if (current_time - self.last_process_time) >= progressive_pause:
    yield VADAudio(
        audio=progressive_buffer,
        mode="progressive",  # Signals partial transcription expected

        timestamp=timestamp,
    )

```

Key behaviors of progressive mode:

- **`mode="progressive"`** — Tells downstream STT handlers this is intermediate audio
- **`SpeechStartedEvent` / `SpeechStoppedEvent`** — Keeps UI synchronized with speech state
- **Periodic yielding** — Controlled by `realtime_processing_pause` interval

## Step 4: Consuming Partial Transcription Events

The `TranscriptionNotifier` (line 42 in [`transcription_notifier.py`](https://github.com/huggingface/speech-to-speech/blob/main/transcription_notifier.py)) receives `PartialTranscription` objects from STT handlers and forwards them:

```python
self.text_output_queue.put(
    PartialTranscriptionEvent(delta=str(transcription.text), ...)
)

```

To consume live transcriptions in your client:

```python
import queue
from speech_to_speech.pipeline.queue_types import TextEventItem
from speech_to_speech.STT.transcription_notifier import (
    PartialTranscriptionEvent,
    TranscriptionCompletedEvent,
)

output_q: queue.Queue[TextEventItem] = pipeline.text_output_queue

while True:
    event = output_q.get()
    
    if isinstance(event, PartialTranscriptionEvent):
        # Update subtitle display in real time

        print(f"Live: {event.delta}", end="\r")
        
    elif isinstance(event, TranscriptionCompletedEvent):
        # Final transcription ready for LLM processing

        print("\nFinal:", event.transcript)
        break

```

This event-driven design decouples transcription display from LLM response generation—you get immediate visual feedback without blocking the conversation flow.

## Step 5: STT Handler Configuration for Progressive Mode

STT handlers must explicitly support progressive transcription. The `ParakeetTDTSTTHandler` (and `WhisperSTTHandler`) implement this via `enable_live_transcription`:

```python
from speech_to_speech.STT.parakeet_tdt_handler import ParakeetTDTSTTHandler

stt = ParakeetTDTSTTHandler(
    enable_live_transcription=True,
    # Additional: model_path, language_detection, etc.

)

pipeline = S2SPipeline(
    module_kwargs=module_args,
    stt_handler=stt,
)
pipeline.run()

```

When enabled, the handler's `process` method invokes `_show_progressive_transcription` for each chunk, yielding lightweight `PartialTranscription` objects rather than waiting for endpoint detection.

## Performance Considerations for Live Transcription

**Update interval trade-offs:**

| Interval | Latency | CPU Usage | Best For |
|----------|---------|-----------|----------|
| 0.2s | Lowest | Higher | Real-time subtitles, typing indicators |
| 0.5s (default) | Balanced | Moderate | General conversational UIs |
| 1.0s | Higher | Lower | Low-power devices, batch-like feel |

**Memory implications:** Progressive mode retains audio buffers longer than standard VAD. The `VADHandler` manages this via `progressive_buffer` accumulation, automatically flushed on `SpeechStoppedEvent`.

## Full Pipeline Flow Summary

1. **User speaks** → Microphone captures audio
2. **VADHandler** detects voice activity, yields `VADAudio(mode="progressive")` every N seconds
3. **STT Handler** (Parakeet/Whisper) processes chunk → emits `PartialTranscription`
4. **TranscriptionNotifier** wraps in `PartialTranscriptionEvent` → posts to `text_output_queue`
5. **Client UI** displays live subtitle from queue
6. **Speech ends** → `VADHandler` yields final `VADAudio`, STT emits full `Transcription`
7. **TranscriptionNotifier** sends `TranscriptionCompletedEvent`, then `GenerateResponseRequest` to LLM

## Summary

- **`enable_live_transcription`** in `ModuleArguments` activates the realtime code path
- **`VADHandler`** switches to `_process_realtime`, yielding progressive audio chunks
- **`TranscriptionNotifier`** bridges partial results to `text_output_queue` via `PartialTranscriptionEvent`
- **`live_transcription_update_interval`** tunes the subtitle refresh rate (0.2–1.0s typical)
- **Event-driven consumption** lets you display subtitles without blocking LLM response generation

## Frequently Asked Questions

### What happens if I enable live transcription but my STT handler doesn't support it?

The pipeline will fail gracefully with an attribute error or silently ignore progressive chunks depending on handler implementation. Always verify your chosen STT handler implements `enable_live_transcription`—`ParakeetTDTSTTHandler` and `WhisperSTTHandler` are confirmed compatible.

### Can I disable live transcription mid-conversation?

No. The `enable_live_transcription` flag is read at pipeline construction time in `S2SPipeline.__init__`. To change modes, you must reconstruct the pipeline with new `ModuleArguments`. Dynamic reconfiguration is not supported in the current architecture.

### How does `live_transcription_update_interval` affect STT accuracy?

Smaller intervals increase partial transcription noise—earlier chunks lack acoustic context. However, final transcriptions (sent to the LLM) use complete audio buffers, so interval choice only impacts subtitle quality, not downstream language model accuracy.

### Where should I connect my UI to receive live transcriptions?

Read from `pipeline.text_output_queue` and filter for `PartialTranscriptionEvent` instances. The queue is thread-safe and populated by `TranscriptionNotifier.process`. For web applications, wrap this in a WebSocket forwarder; for desktop apps, use a direct UI update callback.