How to Use Voice Activity Detection (VAD) Models Like Silero VAD in Python

The ailia-models repository provides a production-ready Python implementation of Silero VAD that wraps ONNX inference, manages recurrent hidden states automatically, and exposes both batch processing via get_speech_timestamps and streaming detection via VADIterator.

Silero VAD is a lightweight, ONNX-based neural network that detects speech segments in audio streams with high accuracy and minimal latency. The ailia-models repository by axinc-ai packages this model in audio_processing/silero-vad/ with utilities that handle sampling-rate conversion, state management, and model versioning. This guide covers the architecture, API usage, and implementation patterns for integrating voice activity detection into your audio pipelines.

Architecture and Core Components

The implementation centers on three Python files that separate inference logic from high-level utilities.

OnnxWrapper (utils_vad.py)

The OnnxWrapper class in audio_processing/silero-vad/utils_vad.py (lines 9-90) normalizes input tensors, validates sampling rates, and manages the model's recurrent state across inference calls. It supports both legacy version 4 (which uses separate hidden and cell states _h and _c for its GRU) and newer versions 5-6 (which use a single _state context vector). The wrapper automatically resamples audio to 8 kHz or 16 kHz and validates that inputs are either 1-D waveforms or properly batched.

Batch Processing with get_speech_timestamps

For offline analysis of complete audio files, get_speech_timestamps in utils_vad.py (lines 29-124) implements a sliding-window approach. It processes audio in fixed chunks (512, 1024, or 1536 samples depending on the sampling rate), aggregates speech probabilities, and applies hysteresis thresholding using a primary threshold and a neg_threshold (set to threshold - 0.15). The function handles minimum speech duration, maximum speech length, and configurable padding (speech_pad_ms) to produce clean start/end timestamps.

Streaming with VADIterator

For real-time applications, the VADIterator class in utils_vad.py (lines 126-164) maintains internal state across chunks. It tracks whether speech is currently triggered, temporary end points, and the current sample position, yielding {'start': seconds} or {'end': seconds} dictionaries immediately upon detecting boundaries. This approach minimizes latency when processing live microphone input or chunked network streams.

Installation and Model Setup

The repository handles dependencies and model downloading automatically.

Install the required packages using the repository's requirements file:

pip install -r requirements.txt

This installs ailia, torch, librosa, soundfile, and onnxruntime. The helper function check_and_download_models in util/model_utils.py fetches the correct ONNX weights and prototxt files from Google Cloud Storage if they are not present locally. You can trigger this manually or let the CLI handle it:

python audio_processing/silero-vad/silero-vad.py --version 4 --onnx

Batch Processing Implementation

To process an entire audio file and extract speech segments, instantiate the wrapper, load your audio, and call get_speech_timestamps.

import torch
from ailia import Net
from audio_processing.silero_vad.utils_vad import (
    OnnxWrapper, get_speech_timestamps, read_audio, save_audio, collect_chunks
)
from audio_processing.silero_vad.silero_vad import check_and_download_models, REMOTE_PATH

# Download model files if missing

WEIGHT = "silero_vad.onnx"
PROTOTXT = "silero_vad.onnx.prototxt"
check_and_download_models(WEIGHT, PROTOTXT, REMOTE_PATH)

# Initialize ONNX session (ailia backend)

net = Net(PROTOTXT, WEIGHT)
model = OnnxWrapper(version="4")
model.session = net
model.ailia = True  # Set to False if using onnxruntime.InferenceSession

# Load mono 16kHz audio

wav = read_audio("en_example.wav", sampling_rate=16000)

# Detect speech intervals (returns list of {'start': int, 'end': int} in samples)

intervals = get_speech_timestamps(
    wav, 
    model, 
    sampling_rate=16000,
    threshold=0.5,
    speech_pad_ms=30
)

print(f"Detected {len(intervals)} speech segments")

# Extract and save only the speech portions

speech_only = collect_chunks(intervals, wav)
save_audio("extracted_speech.wav", speech_only, sampling_rate=16000)

Set visualize_probs=True in get_speech_timestamps to generate a matplotlib plot of speech probabilities across the waveform.

Real-Time Streaming Implementation

For chunked audio streams, use VADIterator to maintain state across calls.

from audio_processing.silero_vad.utils_vad import VADIterator

# Initialize iterator with same model instance

iterator = VADIterator(model, threshold=0.5, sampling_rate=16000)

# Process fixed-size chunks (e.g., 1536 samples at 16kHz = 96ms)

chunk_size = 1536
for i in range(0, len(wav), chunk_size):
    chunk = wav[i:i + chunk_size]
    if chunk.shape[0] < chunk_size:
        break
    
    result = iterator(chunk, return_seconds=True)
    if result:
        # Result contains {'start': float} or {'end': float}

        print(f"Speech boundary: {result}")

# Reset internal states when changing audio sources

iterator.reset_states()

The iterator buffers internal context, so you must call reset_states() when switching between different audio files or streams to prevent state contamination.

CLI Usage and Entry Points

The script audio_processing/silero-vad/silero-vad.py serves as the command-line entry point. It demonstrates three usage patterns:

1. Batch timestamps: Full-file analysis with get_speech_timestamps
2. Iterator mode: Chunked processing simulation
3. Raw probabilities: Direct model output without post-processing

Pass --onnx to force pure ONNX Runtime inference instead of the ailia SDK backend.

Summary

• OnnxWrapper in utils_vad.py abstracts ONNX session management and recurrent state tracking for Silero VAD versions 4-6.
• get_speech_timestamps provides batch processing with configurable thresholds, padding, and minimum duration filters.
• VADIterator enables real-time streaming with minimal latency by maintaining state across audio chunks.
• The repository auto-downloads model weights via check_and_download_models and supports both 8 kHz and 16 kHz sampling rates.
• collect_chunks concatenates detected speech segments for clean audio extraction.

Frequently Asked Questions

What audio formats and sampling rates does Silero VAD support?

Silero VAD accepts mono audio at 8 kHz or 16 kHz. The OnnxWrapper class validates input dimensions and can handle raw waveforms as 1-D tensors or batched 2-D arrays. While the wrapper performs basic validation, you should ensure your input is properly resampled to one of these target rates before inference to maintain detection accuracy.

Should I use batch processing or the streaming iterator?

Use get_speech_timestamps when analyzing recorded files where latency is not critical and you need the complete set of boundaries upfront. Use VADIterator for real-time applications such as microphone input or live network streams where you need immediate feedback when speech starts or ends. The iterator maintains internal state across chunks, making it suitable for continuous monitoring.

Can I run Silero VAD without installing the ailia SDK?

Yes. The code supports both ailia.Net and onnxruntime.InferenceSession backends. Set model.ailia = False when using an ONNX Runtime session, or pass --onnx to the CLI script. The OnnxWrapper handles the slight differences in tensor formatting between the two backends transparently.

What is the difference between model version 4 and newer versions?

Version 4 uses a GRU architecture with separate hidden (_h) and cell (_c) states that must be tracked independently. Versions 5 and 6 use a simplified state representation with a single context vector (_state). The OnnxWrapper detects the version string provided during initialization and manages the appropriate state tensors automatically, so your calling code remains identical regardless of which model version you load.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}