# Creating Voice Agents with LiveKit and Real-Time Speech Capabilities: A Complete Guide

> Build real-time voice agents with LiveKit and Gemini Live API. This guide shows how to use the LiveKit Agents framework and WebRTC for low-latency voice assistants.

- Repository: [Arindam Majumder /awesome-ai-apps](https://github.com/Arindam200/awesome-ai-apps)
- Tags: tutorial
- Published: 2026-05-06

---

**LiveKit combined with the Gemini Live API enables developers to build low-latency, real-time voice assistants using the LiveKit Agents framework and WebRTC-based media streaming.**

The **awesome-ai-apps** repository provides production-ready reference implementations demonstrating how to create sophisticated voice agents. These examples leverage LiveKit's WebRTC media layer integrated with Google's Gemini Live API to process continuous audio streams with minimal latency.

## Architecture Overview

Creating voice agents with LiveKit follows a consistent architectural pattern across all implementations. The system establishes a bidirectional audio pipeline where participants join a LiveKit room, and an `AgentSession` streams audio to a real-time LLM while returning synthesized responses.

The core components include:

- **LiveKit Room**: Handles WebRTC connections between clients and agents
- **AgentSession**: Manages the participant connection and audio routing
- **Real-time LLM**: Gemini's "flash-live-preview" model performs token-level inference on streaming audio
- **Function Tools**: Optional capabilities like web search that the LLM can invoke during conversations
- **Voice Configuration**: Pluggable voices such as "Zephyr" or "Puck" for audio synthesis

## Setting Up Your Development Environment

All voice agent implementations in the repository follow a standardized setup process using environment-based configuration.

Clone the repository and navigate to your chosen example:

```bash
git clone https://github.com/Arindam200/awesome-ai-apps.git
cd awesome-ai-apps/voice_agents/livekit_gemini_agents

```

Install dependencies using the recommended package manager:

```bash
uv sync

```

Configure your credentials by copying the example environment file:

```bash
cp .env.example .env

```

Edit the `.env` file to include your **LiveKit credentials** (`LIVEKIT_URL`, `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`) and **Google API key** (`GOOGLE_API_KEY`).

## Building a Basic Gemini Voice Agent

The foundation of real-time voice processing resides in [`voice_agents/livekit_gemini_agents/main.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/voice_agents/livekit_gemini_agents/main.py).

### Core Agent Configuration

The agent initialization defines the voice characteristics and session parameters. In [`main.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/main.py), the `VOICE` constant configures the speech synthesis output:

```python

# From voice_agents/livekit_gemini_agents/main.py

VOICE = "Zephyr"  # Alternative options include "Puck"

```

### Real-Time Model Integration

The integration point for streaming inference uses the `REALTIME_MODEL` constant to bind Gemini's live capabilities:

```python

# From voice_agents/livekit_gemini_agents/main.py

REALTIME_MODEL = "models/gemini-2.0-flash-live-preview"

```

Start the agent to begin listening for participants:

```bash
python main.py start

```

For development with auto-reload capabilities, use:

```bash
python main.py dev

```

## Adding Function Tools for Web Search

The [`voice_agents/livekit_web_search_agent/main.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/voice_agents/livekit_web_search_agent/main.py) file demonstrates extending voice agents with external capabilities. This implementation integrates **Olostep** for real-time web search functionality.

The `web_search` tool definition appears in the agent configuration:

```python

# From voice_agents/livekit_web_search_agent/main.py

def web_search(query: str) -> str:
    """Search the web for up-to-date information"""
    # Implementation details for Olostep integration

```

When Gemini requires current information, it automatically invokes this function during the conversation flow. Setup requires the additional `OLOSTEP_API_KEY` in your environment configuration.

Deploy this variant using:

```bash
cd ../livekit_web_search_agent
uv sync
cp .env.example .env
python main.py start

```

## Deploying Telephony Agents with SIP

The RSVP agent example in `voice_agents/livekit_rsvp_agent/` demonstrates telephony integration for outbound phone calls. This implementation combines multiple AI services: **Nebius** for LLM inference, **Deepgram** for Speech-to-Text, and **Cartesia** for Text-to-Speech.

### The Agent Worker

The [`agent.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/agent.py) file defines the worker process that handles audio processing and conversation state:

```bash
cd voice_agents/livekit_rsvp_agent
uv sync
cp .env.example .env  # Include SIP_OUTBOUND_TRUNK_ID, Nebius, Deepgram, and Cartesia keys

```

Launch the worker in development mode:

```bash
python agent.py dev

```

### Dispatching Calls

The [`dispatch.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/dispatch.py) script automates room creation and outbound SIP dialing. Located at [`voice_agents/livekit_rsvp_agent/dispatch.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/voice_agents/livekit_rsvp_agent/dispatch.py), this utility places calls to phone numbers and connects them to your voice agent.

Place outbound calls to all pending attendees:

```bash
python dispatch.py

```

For testing single recipients:

```bash
python dispatch.py --id A1

```

The dispatcher creates the LiveKit room, dispatches the agent worker, and initiates the SIP call sequence.

## Configuration and Environment Variables

Each implementation requires specific environment variables defined in `.env.example` files. Never commit actual API keys to version control.

**Required variables for basic agents:**
- `LIVEKIT_URL`
- `LIVEKIT_API_KEY`
- `LIVEKIT_API_SECRET`
- `GOOGLE_API_KEY`

**Additional variables for web search:**
- `OLOSTEP_API_KEY`

**Additional variables for telephony:**
- `SIP_OUTBOUND_TRUNK_ID`
- `NEBIUS_API_KEY`
- `DEEPGRAM_API_KEY`
- `CARTESIA_API_KEY`

Test your deployment using the LiveKit CLI to generate access tokens:

```bash
lk token create \
  --room gemini-agent \
  --identity me \
  --agent gemini-voice-agent \
  --join \
  --open console

```

This command creates a token that registers the agent and opens the LiveKit Agent Console for immediate voice interaction.

## Summary

- **LiveKit Agents framework** provides the WebRTC infrastructure for real-time voice communication between participants and AI models.
- **Gemini Live API** integration enables token-level streaming inference with synchronized audio input/output through constants like `REALTIME_MODEL`.
- **Function tools** extend agent capabilities beyond conversation, allowing web searches and database operations during active calls.
- **Telephony deployment** requires coordinating agent workers ([`agent.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/agent.py)) with dispatcher scripts ([`dispatch.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/dispatch.py)) for outbound SIP calls.
- **Environment isolation** via `.env` files separates credentials from source code, supporting safe deployment across development and production environments.

## Frequently Asked Questions

### What is the LiveKit Agents framework?

The LiveKit Agents framework is a Python SDK that manages WebRTC media streams between participants and AI services. It handles audio streaming, room management, and connection lifecycle, allowing developers to focus on conversation logic rather than low-level media handling.

### How does real-time speech processing work with Gemini Live?

Gemini Live accepts continuous audio streams through a persistent connection, processing speech token-by-token rather than waiting for complete utterances. The model returns synchronized audio responses with minimal latency, enabling natural turn-taking in conversations.

### Can I customize the voice used by LiveKit agents?

Yes. Voice agents support configurable speech synthesis parameters. In [`voice_agents/livekit_gemini_agents/main.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/voice_agents/livekit_gemini_agents/main.py), modify the `VOICE` constant to select alternatives like "Puck" or "Zephyr". For advanced customization, the RSVP agent demonstrates integration with Cartesia TTS for fine-grained voice control.

### How do I deploy a voice agent for production phone calls?

Production telephony deployment requires three components: a running agent worker (`python agent.py start`), the LiveKit SIP integration configured with your trunk ID, and a dispatch mechanism to initiate calls. The [`dispatch.py`](https://github.com/Arindam200/awesome-ai-apps/blob/main/dispatch.py) script in the RSVP example provides a reference implementation for automating outbound calls and room management.