How to Configure ICE Servers for WebRTC in Production Deployments

Configure ICE servers for WebRTC by setting the SPEECH_TO_SPEECH_ICE_SERVERS environment variable to a JSON-encoded list of STUN/TURN servers, which the Speech-to-Speech server parses via rtc_configuration_from_env() in webrtc_session.py and exposes to browsers through the /api/config endpoint.

The huggingface/speech-to-speech repository uses aiortc to provide WebRTC transport compatible with the OpenAI Realtime API. In production deployments, you must configure Interactive Connectivity Establishment (ICE) servers to enable peers to establish connections across NATs and firewalls.

How ICE Server Configuration Works in Speech-to-Speech

The architecture separates server-side media handling from client-side connection setup, requiring coordinated ICE configuration across both environments.

Server-Side RTCConfiguration

In src/speech_to_speech/api/openai_realtime/webrtc_session.py, the WebRTCSession class initializes its RTCConfiguration by calling rtc_configuration_from_env() (lines 42-66). This function reads the SPEECH_TO_SPEECH_ICE_SERVERS environment variable, expecting a JSON-encoded list of dictionaries containing RTCIceServer keyword arguments. If the variable is missing or malformed, the system falls back to aiortc defaults (host candidates only), which typically fails in production NAT scenarios.

Client-Side ICE Distribution

The demo server in demo/server.py handles browser-facing configuration. It parses the RTC_ICE_SERVERS environment variable using the helper _parse_ice_servers() (lines 84-103) and returns the resulting array in the JSON response from the /api/config endpoint under the key "iceServers". The browser then instantiates RTCPeerConnection with these servers to perform ICE gathering.

Production Configuration Requirements

Successful WebRTC deployments require both STUN for NAT traversal discovery and TURN for relay when direct peer-to-peer fails.

Environment Variable Format

Set SPEECH_TO_SPEECH_ICE_SERVERS as a JSON array containing server objects with urls, and optionally username and credential for TURN authentication:

SPEECH_TO_SPEECH_ICE_SERVERS=[
  {"urls":"stun:stun.l.google.com:19302"},
  {"urls":"turn:turn.mycompany.com:3478","username":"s2s","credential":"s3cr3t"}
]

For the demo server (browser client), optionally set RTC_ICE_SERVERS with identical contents to ensure the client receives the same configuration.

Security and Scalability Considerations

Store TURN credentials exclusively in backend environment variables; they transmit to clients only within the SDP answer during ICE gathering, not through the config endpoint credentials. Deploy a dedicated TURN service (such as coturn) with a high-capacity UDP port range, using identical credentials across all server replicas to ensure any instance can respond to ICE connectivity checks. Maintain at least one public STUN server in the configuration to provide failover if TURN services become unavailable.

Connection Timeouts and Error Handling

The session enforces strict ICE gathering limits. In webrtc_session.py, the constant ICE_GATHERING_TIMEOUT_S = 5 (lines 42-45) defines the maximum seconds allowed for ICE candidate collection. If your TURN server requires longer to respond, connections abort with timeout errors. Verify TURN server latency stays well below this threshold in production monitoring.

Implementation Examples

Server-Side Python Configuration

For custom scripts or testing outside the standard environment variable flow, explicitly construct the configuration:

import os
import json
from aiortc import RTCConfiguration, RTCIceServer

def make_rtc_config() -> RTCConfiguration:
    raw = os.getenv("SPEECH_TO_SPEECH_ICE_SERVERS")
    if not raw:
        return RTCConfiguration()  # defaults: host candidates only

    entries = json.loads(raw)      # expects list of dicts

    servers = [RTCIceServer(**e) for e in entries]
    return RTCConfiguration(iceServers=servers)

rtc_cfg = make_rtc_config()

Client-Side JavaScript Integration

Fetch ICE servers from the backend before creating the peer connection:

async function getIceServers() {
  const resp = await fetch("/api/config");
  const cfg = await resp.json();
  return cfg.iceServers;   // array of {urls, username?, credential?}
}

async function startWebRTC() {
  const iceServers = await getIceServers();
  const pc = new RTCPeerConnection({ iceServers });
  // ... add tracks and negotiate
}

Docker Compose Configuration

Define the environment in your container orchestration:

services:
  speech-to-speech:
    image: huggingface/speech-to-speech
    environment:
      - SPEECH_TO_SPEECH_ICE_SERVERS=[{"urls":"stun:stun.l.google.com:19302"},{"urls":"turn:turn.example.com:3478","username":"user","credential":"pass"}]
  demo:
    image: huggingface/speech-to-speech-demo
    environment:
      - RTC_ICE_SERVERS=[{"urls":"stun:stun.l.google.com:19302"},{"urls":"turn:turn.example.com:3478","username":"user","credential":"pass"}]

Summary

• Set SPEECH_TO_SPEECH_ICE_SERVERS as a JSON array containing STUN and TURN server definitions for the backend media server.
• Configure RTC_ICE_SERVERS in the demo environment to propagate identical settings to browser clients via the /api/config endpoint.
• Implement TURN servers with static credentials across all replicas for horizontal scalability.
• Monitor ICE gathering times against the 5-second timeout defined in webrtc_session.py to prevent connection failures.
• Never expose TURN credentials in client-side code; rely on the backend-to-client configuration flow.

Frequently Asked Questions

What happens if I don't configure ICE servers in Speech-to-Speech?

Without the SPEECH_TO_SPEECH_ICE_SERVERS environment variable, the system defaults to aiortc's host-only candidates. This configuration works only for localhost testing or direct LAN connections; production deployments across NATs or corporate firewalls will fail to establish peer connections.

Why does Speech-to-Speech use separate environment variables for server and client?

The backend uses SPEECH_TO_SPEECH_ICE_SERVERS parsed by rtc_configuration_from_env() in webrtc_session.py, while the demo server uses RTC_ICE_SERVERS parsed by _parse_ice_servers() in demo/server.py. This separation allows different ICE strategies for media processing servers versus browser clients, though typically you configure identical values for symmetric connectivity.

How do I troubleshoot ICE gathering timeouts in production?

Check that your TURN server responds within the 5-second ICE_GATHERING_TIMEOUT_S limit defined in webrtc_session.py. Verify UDP ports are open between your Speech-to-Speech instances and the TURN server, and confirm credentials are correctly formatted as JSON without trailing commas or syntax errors that would cause parsing failures.

Can I use multiple TURN servers for redundancy?

Yes. The JSON array accepts multiple server objects. Configure multiple TURN endpoints with different hostnames or ports in the SPEECH_TO_SPEECH_ICE_SERVERS array; aiortc and browsers will attempt connectivity checks against all candidates, falling back to alternate servers if the primary becomes unreachable.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}