# WebRTC Transport Setup with STUN/TURN Servers for Production Deployment in Speech-to-Speech

> Set up WebRTC transport with STUN/TURN servers for production speech-to-speech. Configure ICE servers via environment variables for robust peer-to-peer connections.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-03

---

**Configure STUN/TURN servers in the huggingface/speech-to-speech repository by setting the `SPEECH_TO_SPEECH_ICE_SERVERS` environment variable with a JSON list of RTCIceServer objects, then instantiate `WebRTCSession` with `rtc_configuration_from_env()` to enable production-grade peer-to-peer connectivity.**

The **huggingface/speech-to-speech** repository provides a self-hosted implementation of the OpenAI Realtime API supporting both WebSocket and WebRTC transports. For **WebRTC transport setup with STUN/TURN servers for production deployment**, the codebase uses **aiortc** to handle peer connections, RTP media streaming, and ICE negotiation. This guide walks through the configuration, negotiation flow, and operational considerations for running at scale.

---

## Core WebRTC Architecture

The transport layer centers on four key components defined in [`src/speech_to_speech/api/openai_realtime/webrtc_session.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/webrtc_session.py):

| Component | Role | Source Location |
|-----------|------|-----------------|
| **`WebRTCSession`** | Implements `SessionTransport` interface; manages `RTCPeerConnection`, audio pipelines, and lifecycle | Lines 1–250+ |
| **`PipelineAudioTrack`** | `MediaStreamTrack` that emits paced 20 ms, 48 kHz audio frames from server-side PCM buffers | Lines 99–152 |
| **`PcmResampler`** | Stateful wrapper around `av.AudioResampler` converting between 16 kHz pipeline and 48 kHz WebRTC formats | Lines 70–98 |
| **`"oai-events"` DataChannel** | Carries all OpenAI Realtime JSON messages (`response.created`, `session.update`, etc.) | Lines 94–98 |

---

## Configuring ICE Servers for Production

### Environment Variable Configuration

The `rtc_configuration_from_env()` helper (lines 50–67) reads ICE server configuration exclusively from the environment:

```bash
export SPEECH_TO_SPEECH_ICE_SERVERS='[
    {"urls": "stun:stun.l.google.com:19302"},
    {"urls": "turn:turn.example.com:3478",
     "username": "myuser",
     "credential": "mysecret"}
]'

```

**Critical requirements:**
- Must be valid JSON array of `RTCIceServer`-compatible dictionaries
- Supports multiple STUN and TURN entries
- Authentication via `username`/`credential` for TURN relay
- If absent or malformed, **aiortc falls back to defaults** (tested in [`tests/openai_realtime/test_webrtc.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/openai_realtime/test_webrtc.py))

### Runtime Instantiation

The router layer normally handles session creation. For custom deployments:

```python
from aiortc import RTCPeerConnection
from speech_to_speech.api.openai_realtime.webrtc_session import (
    WebRTCSession,
    rtc_configuration_from_env,
)

# Build configuration from environment

rtc_config = rtc_configuration_from_env()  # Returns RTCConfiguration or None

# Create peer connection with ICE servers

pc = RTCPeerConnection(rtc_config)

# Initialize session with callbacks

session = WebRTCSession(
    pc,
    on_client_event=handle_client_event,   # JSON messages from data-channel

    on_audio=handle_inbound_audio,         # PCM buffers from client mic

    on_open=handle_open,                   # Connection established

    on_closed=handle_closed,               # Cleanup and metrics

)
session.setup()  # Attaches track and data-channel handlers

```

---

## SDP Negotiation Flow

After session setup, complete the handshake with `session.negotiate()`:

```python

# Client (browser) generates offer; send to server

answer_sdp = await session.negotiate(client_offer_sdp)

# Return answer_sdp to client

```

The negotiation method performs four operations sequentially (lines 180–220):

1. **Set remote description** from client's SDP offer
2. **Create answer** with local media capabilities
3. **Gather ICE candidates** with `ICE_GATHERING_TIMEOUT_S` (5 s default)
4. **Return SDP answer** containing STUN/TURN candidates for client connectivity

---

## Production Reliability Mechanisms

### Connection Timeouts

| Timeout | Default | Purpose |
|---------|---------|---------|
| `ICE_GATHERING_TIMEOUT_S` | 5 s | Maximum wait for candidate collection; returns partial SDP if exceeded |
| `CONNECT_TIMEOUT_S` | 30 s | `_connect_watchdog` task monitors `RTCPeerConnection.state`; auto-closes if never reaches `"connected"` |

The watchdog prevents pipeline units from being held indefinitely when NAT traversal fails or clients disconnect mid-negotiation.

### Network Deployment Checklist

- **TLS termination**: Run behind Nginx or similar proxy forwarding `/v1/realtime/webrtc/calls`
- **UDP port access**: Open firewall ranges for TURN relay traffic (typically 3478, plus ephemeral ports)
- **Horizontally scaled instances**: Each server instance reads identical `SPEECH_TO_SPEECH_ICE_SERVERS`; TURN relay coordinates media routing across the pool

---

## Browser Client Integration

Match the server-side data-channel label exactly:

```javascript
const pc = new RTCPeerConnection({
  iceServers: [{ urls: "stun:stun.l.google.com:19302" }]
});

// Must use label "oai-events" — hardcoded in webrtc_session.py line 94
const dc = pc.createDataChannel("oai-events");
dc.onmessage = e => console.log(JSON.parse(e.data));

// Add audio transceiver for bidirectional streaming
pc.addTransceiver('audio', { direction: 'sendrecv' });

```

---

## Key Source Files Reference

| File | Purpose |
|------|---------|
| [`src/speech_to_speech/api/openai_realtime/webrtc_session.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/webrtc_session.py) | Core transport: ICE handling, audio resampling, watchdog logic |
| [`src/speech_to_speech/api/openai_realtime/transports.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/transports.py) | Abstract `SessionTransport` interface |
| [`src/speech_to_speech/api/openai_realtime/websocket_router.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/websocket_router.py) | HTTP routing and transport dispatch |
| [`src/speech_to_speech/api/openai_realtime/server.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/server.py) | FastAPI service entry point |
| [`demo/server.py`](https://github.com/huggingface/speech-to-speech/blob/main/demo/server.py) | Local development; contains `_webrtc_calls_url()` helper |
| [`tests/openai_realtime/test_webrtc.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/openai_realtime/test_webrtc.py) | Test coverage including invalid ICE config handling |

---

## Summary

- **ICE servers** are configured exclusively via the `SPEECH_TO_SPEECH_ICE_SERVERS` environment variable as JSON-serialized RTCIceServer arrays
- **WebRTCSession** wraps aiortc's `RTCPeerConnection` with production safeguards: ICE timeouts, connection watchdogs, and automatic cleanup
- **Bidirectional audio** flows over RTP tracks at 48 kHz while **JSON events** travel on the `"oai-events"` data-channel
- **Scaling** requires consistent ICE configuration across all instances and UDP port accessibility for TURN relay

---

## Frequently Asked Questions

### What happens if SPEECH_TO_SPEECH_ICE_SERVERS is malformed or missing?

The `rtc_configuration_from_env()` helper returns `None`, causing `RTCPeerConnection` to use aiortc's default STUN configuration. The test in [`tests/openai_realtime/test_webrtc.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/openai_realtime/test_webrtc.py) explicitly verifies this fallback behavior for invalid JSON inputs.

### Why does the audio resample from 16 kHz to 48 kHz?

The server-side speech pipeline operates at 16 kHz (optimized for ASR/TTS models), while WebRTC mandates 48 kHz for Opus codec compatibility. The `PcmResampler` class maintains stateful conversion using FFmpeg via `av.AudioResampler` to prevent frame boundary artifacts.

### How does the watchdog prevent resource leaks?

The `_connect_watchdog` coroutine (spawned in `setup()`) monitors `pc.connectionState` and triggers `session.close()` if `"connected"` is not reached within `CONNECT_TIMEOUT_S` (30 seconds). This automatically releases the pipeline unit back to the worker pool.

### Can I use a TURN server with TLS transport?

Yes. Include `turns:` (TLS) or `turn:` (UDP/TCP) URLs in the `SPEECH_TO_SPEECH_ICE_SERVERS` JSON. The aiortc library underlying `WebRTCSession` supports all standard RTCIceServer configurations including TLS-encrypted TURN relays on port 5349.