# How to Implement Transcription for Multi-User Rooms with LiveKit Agents

> Implement multi-user room transcription using LiveKit Agents. Leverage MultiSpeakerAdapter and diarization plugins to automatically tag speaker IDs and manage speaker detection for clear audio.

- Repository: [LiveKit/agents](https://github.com/livekit/agents)
- Tags: how-to-guide
- Published: 2026-03-06

---

**Use the `MultiSpeakerAdapter` class to wrap any diarization-enabled STT plugin, which automatically tags transcript chunks with speaker IDs and manages primary/background speaker detection via RMS energy analysis.**

LiveKit Agents provides a production-ready transcription stack built around the abstract `STT` interface. When implementing transcription for multi-user rooms, you must identify who is speaking to maintain conversational context and prevent the agent from confusing overlapping dialogue. This guide walks you through the exact implementation using the `MultiSpeakerAdapter` and diarization-capable speech-to-text plugins.

## Prerequisites: Diarization-Enabled STT Plugins

Before processing multiple speakers, your underlying STT service must support **speaker diarization**. According to the source code in [`livekit-agents/livekit/agents/stt/stt.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/stt/stt.py), the STT implementation must set `STTCapabilities.diarization` to `True` and populate `SpeechData.speaker_id` in emitted events.

Compatible plugins include:
- **Speechmatics** – Set `enable_diarization=True` in the constructor
- **Deepgram** – Configure diarization parameters in the API request

The `MultiSpeakerAdapter` validates this capability during initialization and raises an error if the wrapped STT lacks diarization support (see validation logic at lines 19-30 in [`multi_speaker_adapter.py`](https://github.com/livekit/agents/blob/main/multi_speaker_adapter.py)).

## Understanding the MultiSpeakerAdapter Architecture

The transcription flow for multi-user rooms relies on three core components defined in [`livekit-agents/livekit/agents/stt/multi_speaker_adapter.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/stt/multi_speaker_adapter.py):

1. **`MultiSpeakerAdapter`** – Wraps the base STT and creates a `MultiSpeakerAdapterWrapper` stream for each recognition session.
2. **`MultiSpeakerAdapterWrapper`** – Forwards incoming audio frames to both the underlying STT recognizer and the internal `_PrimarySpeakerDetector` (see the `_run` implementation at lines 100-135).
3. **`_PrimarySpeakerDetector`** – Performs RMS-based energy analysis to rank speakers, determine the primary speaker, and format or suppress transcripts according to your configuration (see lines 44-158).

When audio flows through the adapter, the detector maintains rolling RMS buffers for each `speaker_id`. It switches the primary speaker designation based on energy levels, then applies custom formatting rules to distinguish foreground from background speech.

## Step-by-Step Implementation

### 1. Select a Diarization-Enabled STT Plugin

Initialize your base STT with diarization capabilities enabled. This example uses Speechmatics, but Deepgram and other providers follow similar patterns.

```python
from livekit.plugins import speechmatics

base_stt = speechmatics.STT(
    enable_diarization=True,
    end_of_utterance_silence_trigger=0.3
)

```

### 2. Wrap with MultiSpeakerAdapter

Import `MultiSpeakerAdapter` and configure it to detect the primary speaker and format transcript output.

```python
from livekit.agents.stt import MultiSpeakerAdapter

stt = MultiSpeakerAdapter(
    stt=base_stt,
    detect_primary_speaker=True,          # Enable RMS-based primary detection

    suppress_background_speaker=False,    # Set True to drop background speech entirely

    primary_format="<{speaker_id}>{text}</{speaker_id}>",
    background_format="<bg>{text}</bg>",
)

```

### 3. Configure AgentSession

Pass the adapted STT instance to your `AgentSession` configuration. The adapter transparently handles speaker tracking without modifying your agent logic.

```python
from livekit.agents import Agent, AgentSession, JobContext
from livekit.plugins import openai, silero

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You are a helpful assistant.")

async def entrypoint(ctx: JobContext) -> None:
    session = AgentSession(
        vad=silero.VAD.load(),
        llm=openai.LLM(),
        tts=openai.TTS(),
        stt=stt,  # MultiSpeakerAdapter instance

    )
    await session.start(room=ctx.room, agent=Assistant())

```

### 4. Deploy the Agent

Run your agent application using the LiveKit CLI. For a complete, runnable reference implementation, see [`examples/voice_agents/speaker_id_multi_speaker.py`](https://github.com/livekit/agents/blob/main/examples/voice_agents/speaker_id_multi_speaker.py) in the repository.

```bash
python agent.py start

```

## Complete Implementation Example

Here is the minimal code required to transcribe multi-user rooms with speaker identification and primary speaker formatting:

```python
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli
from livekit.plugins import speechmatics, openai, silero
from livekit.agents.stt import MultiSpeakerAdapter

# Step 1: Initialize diarization-enabled STT

base_stt = speechmatics.STT(enable_diarization=True)

# Step 2: Wrap with MultiSpeakerAdapter for primary speaker detection

stt = MultiSpeakerAdapter(
    stt=base_stt,
    detect_primary_speaker=True,
    suppress_background_speaker=False,
    primary_format="<{speaker_id}>{text}</{speaker_id}>",
    background_format="<bg>{text}</bg>",
)

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You are a friendly assistant.")

server = AgentServer()

@server.rtc_session()
async def entrypoint(ctx: JobContext) -> None:
    session = AgentSession(
        vad=silero.VAD.load(),
        llm=openai.LLM(),
        tts=openai.TTS(),
        stt=stt,
    )
    await session.start(room=ctx.room, agent=Assistant())

if __name__ == "__main__":
    cli.run_app(server)

```

## Customizing Primary Speaker Behavior

The `MultiSpeakerAdapter` provides fine-grained control over how the agent perceives different speakers:

- **`detect_primary_speaker`** – When `True`, the `_PrimarySpeakerDetector` calculates RMS energy for each speaker and designates the loudest voice as primary.
- **`suppress_background_speaker`** – When `True`, transcripts from non-primary speakers are dropped entirely rather than formatted.
- **`primary_format`** – A template string using `{speaker_id}` and `{text}` placeholders to markup primary speaker transcripts.
- **`background_format`** – A separate template for background speakers, useful for distinguishing side conversations.

Adjust these parameters based on your use case. For customer service bots, suppressing background speakers prevents interruptions, while meeting assistants might retain formatted background speech for completeness.

## Summary

- **Speaker diarization** requires an STT plugin with `STTCapabilities.diarization=True`, such as Speechmatics or Deepgram.
- **`MultiSpeakerAdapter`** wraps diarization-enabled STTs and is implemented in [`livekit-agents/livekit/agents/stt/multi_speaker_adapter.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/stt/multi_speaker_adapter.py).
- The adapter uses **`_PrimarySpeakerDetector`** to perform RMS-based energy analysis and identify the primary speaker in real-time.
- Configure **`primary_format`** and **`background_format`** to customize how speaker identities appear in transcripts.
- Set **`suppress_background_speaker=True`** to filter out non-primary speech entirely.
- Reference the full example at [`examples/voice_agents/speaker_id_multi_speaker.py`](https://github.com/livekit/agents/blob/main/examples/voice_agents/speaker_id_multi_speaker.py) for production deployment patterns.

## Frequently Asked Questions

### Does LiveKit Agents support real-time speaker diarization?

Yes. When you wrap a diarization-enabled STT plugin with `MultiSpeakerAdapter`, the system processes speaker identification in real-time as audio streams arrive. The `_PrimarySpeakerDetector` updates its RMS buffers continuously (see lines 44-158 in [`multi_speaker_adapter.py`](https://github.com/livekit/agents/blob/main/multi_speaker_adapter.py)), allowing immediate primary speaker switching without batch processing delays.

### Which STT plugins support speaker diarization?

The LiveKit Agents framework supports diarization through plugins that expose the `speaker_id` field in `SpeechData` events. As of the current release, **Speechmatics** and **Deepgram** officially support this capability. Verify diarization availability by checking the `STTCapabilities.diarization` flag on the plugin instance before wrapping it with `MultiSpeakerAdapter`.

### How does the primary speaker detector work?

The `_PrimarySpeakerDetector` class calculates **RMS (Root Mean Square) energy** for each speaker's audio stream over a sliding window. It ranks speakers by energy level and designates the highest-energy speaker as primary. When energy levels shift—such as when a new person starts speaking—the detector switches the primary designation and applies the appropriate formatting rules defined in your `MultiSpeakerAdapter` configuration.

### Can I suppress background speakers completely?

Yes. Set `suppress_background_speaker=True` when initializing `MultiSpeakerAdapter`. When this flag is enabled, the adapter drops all `SpeechEvent` objects from non-primary speakers rather than applying the `background_format` template. This ensures your agent processes only the dominant speaker's transcripts, which is ideal for high-noise environments or focused one-on-one interactions within group settings.