# How to Optimize Latency and Performance in LiveKit Agents: A Complete Guide

> Optimize your LiveKit agents latency and performance. Discover techniques like preemptive generation and balanced TTS modes to slash end-to-end delays significantly.

- Repository: [LiveKit/agents](https://github.com/livekit/agents)
- Tags: how-to-guide
- Published: 2026-03-06

---

**Enable preemptive generation, use balanced latency mode in TTS plugins, and tighten endpointing delays to reduce end-to-end latency from over 1 second to approximately 500ms.**

The LiveKit Agents framework powers real-time voice AI applications by orchestrating speech-to-text, LLM inference, and text-to-synthesis pipelines. To deliver conversational experiences that feel instantaneous, developers must optimize latency and performance at every stage of this pipeline. This guide examines the specific configuration options, source code locations, and implementation patterns that minimize round-trip delay in the LiveKit Agents repository.

## Understanding the LiveKit Agents Pipeline Architecture

LiveKit Agents processes audio through a sequential pipeline: audio capture → Voice Activity Detection (VAD) → Speech-to-Text (STT) → turn detection → Large Language Model (LLM) inference → Text-to-Speech (TTS) → network transport. Latency accumulates at each handoff point. The framework provides specific knobs in `AgentSession` and individual plugins to parallelize work, reduce waiting periods, and reuse connections.

## Key Strategies to Optimize Latency and Performance

### Enable Preemptive Generation

Preemptive generation overlaps LLM inference with user speech. When enabled, the model begins generating a response as soon as partial transcripts arrive, rather than waiting for the user to stop speaking. According to test suites in the repository, this reduces end-to-end latency from approximately 1.1 seconds to 0.8 seconds.

In [`livekit-agents/livekit/agents/voice/agent_session.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/voice/agent_session.py), set the `preemptive_generation` parameter:

```python
session = AgentSession(
    preemptive_generation=True,
)

```

The underlying implementation creates a `_preemptive_generation` object in `AgentActivity` at line 1437 of [`livekit-agents/livekit/agents/voice/agent_activity.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/voice/agent_activity.py), which is cancelled if the user speaks again (lines 975-978).

### Configure TTS Latency Modes

Streaming TTS plugins expose latency modes that trade quality for speed. Use `balanced` mode for the lowest round-trip time.

For FishAudio, configure `latency_mode` in [`livekit-plugins/livekit-plugins-fishaudio/livekit/plugins/fishaudio/tts.py`](https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-fishaudio/livekit/plugins/fishaudio/tts.py):

```python
from livekit.plugins.fishaudio import FishAudioTTS

tts = FishAudioTTS(
    api_key="YOUR_API_KEY",
    latency_mode="balanced",  # ~300ms round-trip

)

```

For ElevenLabs, enable `auto_mode` in [`livekit-plugins/livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/tts.py`](https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/tts.py) to disable chunk scheduling and synthesize sentence-by-sentence:

```python
from livekit.plugins.elevenlabs import ElevenLabsTTS

tts = ElevenLabsTTS(
    voice_id="YOUR_VOICE_ID",
    auto_mode=True,  # Reduces latency via sentence-level tokenization

    streaming_latency=4,  # Max latency optimization

)

```

### Optimize Turn Detection and Endpointing

Tightening endpointing parameters reduces the silence duration required before the system processes a turn.

In [`livekit-agents/livekit/agents/voice/agent_session.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/voice/agent_session.py), configure `AgentSessionOptions`:

```python
session = AgentSession(
    turn_detection="vad",  # Fast VAD-based detection

    min_endpointing_delay=0.2,  # Seconds of silence before ending turn

    max_endpointing_delay=2.0,  # Maximum wait time

    min_interruption_duration=0.1,  # Ignore very short pauses

)

```

If your LLM plugin supports it, use `turn_detection="realtime_llm"` to eliminate the separate turn-detection round-trip.

### Reuse Connections with APIConnectOptions

All plugins accept `APIConnectOptions` to enable connection pooling and keep-alive behavior, reducing handshake overhead.

Defined in [`livekit/agents/types.py`](https://github.com/livekit/agents/blob/main/livekit/agents/types.py) and used across plugins like [`elevenlabs/tts.py`](https://github.com/livekit/agents/blob/main/elevenlabs/tts.py):

```python
from livekit.agents import APIConnectOptions

conn_opts = APIConnectOptions(
    timeout=5.0,
    max_retries=2,
    keepalive=True,
)

# Reuse connection across synthesis calls

audio_stream = tts.stream(conn_options=conn_opts)

```

## Complete Implementation Examples

### Low-Latency AgentSession Configuration

Combine all optimizations into a single high-performance setup:

```python
from livekit.agents import AgentSession, APIConnectOptions
from livekit.plugins.fishaudio import FishAudioTTS
from livekit.plugins.deepgram import DeepgramSTT

# 1. Low-latency TTS with balanced mode

tts = FishAudioTTS(
    api_key="FISH_API_KEY",
    latency_mode="balanced",
)

# 2. Fast STT with tight endpointing

stt = DeepgramSTT(
    api_key="DEEPGRAM_KEY",
    endpointing_delay=0.2,
)

# 3. Session with preemptive generation and tight endpointing

session = AgentSession(
    turn_detection="vad",
    stt=stt,
    tts=tts,
    preemptive_generation=True,
    min_endpointing_delay=0.2,
    max_endpointing_delay=2.0,
    allow_interruptions=True,
    false_interruption_timeout=1.0,
)

# 4. Connect to room

session.connect(url="wss://my.livekit.server", token="my-token")

```

### ElevenLabs Auto-Mode Setup

For ElevenLabs-specific optimizations:

```python
from livekit.plugins.elevenlabs import ElevenLabsTTS
from livekit.agents import AgentSession

tts = ElevenLabsTTS(
    voice_id="EXAMPLE_VOICE",
    auto_mode=True,                # Sentence-level tokenization

    streaming_latency=4,           # Max latency optimization

    inactivity_timeout=60,           # Keep connection alive

)

session = AgentSession(
    tts=tts,
    preemptive_generation=True,
    turn_detection="realtime_llm",  # If LLM supports it

)

```

### Monitoring Latency with OpenTelemetry

Capture `e2e_latency` metrics for profiling:

```python
from livekit.agents.telemetry import tracer

@tracer.start_as_current_span("voice_pipeline")
def run_pipeline():
    # Your agent logic here

    pass

# After execution, inspect the span attributes in Jaeger or Tempo

# The 'e2e_latency' metric appears under span attributes

```

The telemetry implementation in [`livekit/agents/telemetry/traces.py`](https://github.com/livekit/agents/blob/main/livekit/agents/telemetry/traces.py) automatically attaches `e2e_latency` values to traces when available.

## Critical Source Files for Performance Tuning

- **[`livekit-agents/livekit/agents/voice/agent_session.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/voice/agent_session.py)** – Contains `AgentSessionOptions` including `preemptive_generation`, `turn_detection`, and endpointing delays.
- **[`livekit-agents/livekit/agents/voice/agent_activity.py`](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/voice/agent_activity.py)** – Implements the `_preemptive_generation` logic (creation at line 1437, cancellation at lines 975-978).
- **[`livekit-plugins/livekit-plugins-fishaudio/livekit/plugins/fishaudio/tts.py`](https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-fishaudio/livekit/plugins/fishaudio/tts.py)** – Exposes `latency_mode` parameter (balanced/normal).
- **[`livekit-plugins/livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/tts.py`](https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/tts.py)** – Provides `auto_mode` and `streaming_latency` configuration.
- **[`livekit-plugins/livekit-plugins-ultravox/livekit/plugins/ultravox/realtime/realtime_model.py`](https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-ultravox/livekit/plugins/ultravox/realtime/realtime_model.py)** – Demonstrates latency measurement patterns for realtime models.
- **[`livekit/agents/telemetry/traces.py`](https://github.com/livekit/agents/blob/main/livekit/agents/telemetry/traces.py)** – Emits `e2e_latency` metrics via OpenTelemetry.
- **[`livekit/agents/types.py`](https://github.com/livekit/agents/blob/main/livekit/agents/types.py)** – Defines `APIConnectOptions` for connection reuse across plugins.

## Summary

- **Enable preemptive generation** in `AgentSession` to overlap LLM inference with user speech, reducing latency by approximately 300ms.
- **Select balanced latency mode** in TTS plugins like FishAudio to achieve ~300ms round-trip times.
- **Activate auto_mode** for ElevenLabs to enable sentence-level tokenization and eliminate chunk scheduling delays.
- **Tighten endpointing parameters** (`min_endpointing_delay`, `max_endpointing_delay`) to reduce silence detection time.
- **Reuse connections** via `APIConnectOptions` to minimize handshake overhead across STT, LLM, and TTS calls.
- **Monitor `e2e_latency` traces** in [`livekit/agents/telemetry/traces.py`](https://github.com/livekit/agents/blob/main/livekit/agents/telemetry/traces.py) to identify bottlenecks iteratively.

## Frequently Asked Questions

### What is the typical latency reduction when enabling preemptive generation?

Enabling `preemptive_generation=True` typically reduces end-to-end latency from approximately 1.1 seconds to 0.8 seconds. This 300ms improvement occurs because the LLM begins generating responses while the user is still speaking, overlapping computation with audio capture.

### How does the balanced latency mode affect TTS quality?

The `latency_mode="balanced"` setting in plugins like FishAudio prioritizes speed over maximum quality, delivering audio in approximately 300ms compared to 500ms in normal mode. While this may slightly reduce audio fidelity in complex acoustic scenarios, the difference is typically imperceptible for conversational voice applications.

### Can I use realtime_llm turn detection with any LLM provider?

No, the `turn_detection="realtime_llm"` option requires specific support from your LLM plugin. This mode eliminates the separate turn-detection round-trip by having the LLM itself determine when the user has finished speaking. Check your specific plugin implementation in the `livekit-plugins` directory to verify compatibility.

### Where can I view e2e_latency metrics in production?

The `e2e_latency` metrics are emitted via OpenTelemetry traces defined in [`livekit/agents/telemetry/traces.py`](https://github.com/livekit/agents/blob/main/livekit/agents/telemetry/traces.py). Configure a trace collector such as Jaeger or Grafana Tempo in your deployment, then inspect the span attributes for `e2e_latency` values to profile your pipeline performance.